Kubernetes
Networking and Cilium Kubernetes 网络和 Cilium
An Instruction Manual for the Network Engineer 网络工程师指南
Contents 目录
About us ..... 04 关于我们..... 04
Preface ..... 05 前言 ..... 05
Expectations ..... 06 期望 ..... 06
What We Need The Network To Do ..... 07 我们需要网络做什么 ..... 07
Back To Basics ..... 08 回归基础 ..... 08
Kubernetes Networking Fundamentals ..... 10
Introducing Cilium ..... 10 介绍 Cilium ..... 10
Where is my CLI? ..... 11 我的 CLI 在哪里? ..... 11
How do I configure Kubernetes and Cilium? ..... 11 如何配置 Kubernetes 和 Cilium? ..... 11
Where is my DHCP Server? ..... 13 我的 DHCP 服务器在哪里? ..... 13
What is a Kubernetes namespace? ..... 14 Kubernetes 命名空间是什么? ..... 14
Where is my DNS Server? ..... 15 我的 DNS 服务器在哪里？..... 15
How Do Pods Talk To Each Other? ..... 15 Pod 之间如何通信？..... 15
Kubernetes Metadata ..... 17 Kubernetes 元数据..... 17
What's a Label? ..... 18 什么是标签？..... 18
What's an Annotation? ..... 18 什么是注释？..... 18
How do I secure my cluster? ..... 19 如何保护我的集群？..... 19
What's identity-based security? ..... 22 什么是基于身份的安全？..... 22
Contents 目录
Where's my Layer 7 firewall? ..... 22 我的第 7 层防火墙在哪里？..... 22
Where's my Load Balancer? ..... 24 我的负载均衡器在哪里？..... 24
How do I load balance traffic within the cluster? ..... 25 如何在集群内进行流量负载均衡？..... 25
How do I load balance traffic entering the cluster? ..... 26 如何负载均衡进入集群的流量？..... 26
Where's my web proxy? ..... 29 我的网络代理在哪里？..... 29
How can I connect my cluster to existing networks? ..... 32 如何将我的集群连接到现有网络？..... 32
I don't have any BGP-capable device in my existing network. Is ..... 34 我现有网络中没有任何支持 BGP 的设备。是 ..... 34
there another way for me to access my Kubernetes services? 有没有其他方式让我访问我的 Kubernetes 服务？
How do I connect my cluster with an external firewall? ..... 36 我如何将我的集群连接到外部防火墙？..... 36
How do internal Load Balancing and NAT work under the hood? ..... 37 内部负载均衡和 NAT 是如何工作的？..... 37
How do I manage and troubleshoot my network? ..... 40 如何管理和排除我的网络问题？..... 40
How can I monitor and visualize network traffic? ..... 41 如何监控和可视化网络流量？..... 41
How do I start securing my cluster? ..... 42 如何开始保护我的集群？..... 42
How do I encrypt the traffic in my cluster? ..... 44 如何在我的集群中加密流量？..... 44
How do we connect clusters together? ..... 45 如何将集群连接在一起？..... 45
Is IPv6 supported on Kubernetes? ..... 47 Kubernetes 支持 IPv6 吗？..... 47
Does the concept of Quality of Service (QoS) exist in Kubernetes? ..... 48 Kubernetes 中是否存在服务质量（QoS）的概念？..... 48
Which Kubernetes networking options are available ..... 50 可用的 Kubernetes 网络选项有哪些？..... 50
in managed Kubernetes services? 在托管的 Kubernetes 服务中有哪些选项？
Conclusion ..... 51 结论 ..... 51
About us 关于我们
About Isovalent 关于 Isovalent
Isovalent is the company founded by the creators and maintainers of Cilium and eBPF. We build open-source software and enterprise solutions to solve the networking, security, and observability needs of modern cloud-native infrastructure. Google (GKE, Anthos), Amazon (EKS-A), and Microsoft (AKS) have all adopted Cilium to provide networking and security for Kubernetes. Cilium is used by platform engineering teams such as Adobe, Bell Canada, ByteDance, Capital One, Datadog, IKEA, Schuberg Philis, and Sky. Isovalent 是由 Cilium 和 eBPF 的创始人和维护者创立的公司。我们构建开源软件和企业解决方案，以解决现代云原生基础设施的网络、安全和可观察性需求。Google（GKE、Anthos）、Amazon（EKS-A）和 Microsoft（AKS）都采用了 Cilium 为 Kubernetes 提供网络和安全性。Cilium 被 Adobe、Bell Canada、ByteDance、Capital One、Datadog、宜家、Schuberg Philis 和 Sky 等平台工程团队使用。
About the Author 关于作者
Nico Vibert is a Senior Staff Technical Marketing Engineer at Isovalent - the company behind the open-source cloud-native solution Cilium. Nico Vibert 是 Isovalent 的高级技术营销工程师，Isovalent 是开源云原生解决方案 Cilium 背后的公司。
Prior to joining Isovalent, Nico worked in many different roles operations and support, design and architecture, technical pre-sales - at companies such as HashiCorp, VMware, and Cisco. 在加入 Isovalent 之前，Nico 在许多不同的角色中工作，包括运营和支持、设计和架构、技术售前 - 在像 HashiCorp、VMware 和思科这样的公司。
In his current role, Nico focuses primarily on creating content to make networking a more approachable field and regularly speaks at events like KubeCon, VMworld, and Cisco Live. 在他目前的角色中，Nico 主要致力于创建内容，使网络成为一个更易接近的领域，并经常在 KubeCon、VMworld 和思科 Live 等活动中演讲。
Nico has held over 15 networking certifications, including the prestigious Cisco Certified Internetwork Expert CCIE (#22990). Nico is now the Lead Subject Matter Expert on the Cilium Certified Associate (CCA) certification. Nico 拥有超过 15 个网络认证，包括著名的思科认证网络专家 CCIE (#22990)。Nico 现在是 Cilium 认证助理 (CCA) 认证的首席专家。
Acknowledgements 致谢
I would like to thank everyone who reviewed this eBook: Thomas Graf, Jerry Hency, Liyi Huang, Dean Lewis, Filip Nikolic, Bill Mulligan, and Raphaël Pinson. Your feedback was extremely valuable; any remaining typos or imprecisions that may have slipped through the cracks are entirely my responsibility, 我想感谢所有审阅这本电子书的人：Thomas Graf、Jerry Hency、Liyi Huang、Dean Lewis、Filip Nikolic、Bill Mulligan 和 Raphaël Pinson。您的反馈非常宝贵；任何可能被忽略的错别字或不准确之处都完全是我的责任。
I would also like to thank Vadim Shchekoldin for the fantastic cover and PixelPoint for their help with the graphic design and editing, 我还要感谢 Vadim Shchekoldin 提供的精美封面和 PixelPoint 在平面设计和编辑方面的帮助。
Preface 序言
The dread of Kubernetes Networking is real. Kubernetes 网络的恐惧是真实的。
Most Kubernetes operators can confidently deploy applications onto their clusters, but connecting and securing them evokes a sense of apprehension. This fear also applies to experienced network engineers: even CCIEs would need help with the subtleties and nuances of Kubernetes networking. 大多数 Kubernetes 运维人员可以自信地将应用程序部署到他们的集群中，但连接和保护它们会引发一种忧虑感。这种恐惧也适用于经验丰富的网络工程师：即使是 CCIE 也需要在 Kubernetes 网络的微妙和细微之处寻求帮助。
I can relate: despite my CCIE and almost 20 years working in the networking industry, I found Kubernetes networking confusing. Frankly speaking, most content around Kubernetes Networking was not written with the network engineering community in mind. 我能理解：尽管我拥有 CCIE 资格并在网络行业工作了近 20 年，但我发现 Kubernetes 网络令人困惑。坦率地说，大多数关于 Kubernetes 网络的内容并非针对网络工程社区编写的。
In this eBook, we will fill this void. 在这本电子书中，我们将填补这一空白。
Given how ubiquitous Kubernetes has become, network engineers will increasingly need to understand Kubernetes and know how to configure, manage, and integrate clusters with the rest of the network. This eBook will focus on the Kubernetes networking aspects and on what's now become the de facto networking platform for Kubernetes: Cilium. 鉴于 Kubernetes 已经变得无处不在，网络工程师将越来越需要了解 Kubernetes，并知道如何配置、管理和将集群与网络的其余部分集成在一起。这本电子书将重点关注 Kubernetes 网络方面，以及现在已成为 Kubernetes 的事实标准网络平台的 Cilium。
It is written with you, network engineers, in mind, using words, references and terminology you will understand. 它是为您，网络工程师，而设计的，使用您能理解的词语，参考和术语。
Whether you are beginning your journey as a network administrator or are an established network architect, we hope you find this useful. 无论您是作为网络管理员开始您的旅程，还是已经是一名资深的网络架构师，我们希望您会发现这个有用。
Expectations 期望
The expected audience for this eBook is the network engineering community, but we hope it's accessible to anyone. The main technical expectations we expect the reader to be familiar with are concepts such as: 本电子书的预期受众是网络工程社区，但我们希望任何人都能理解。我们期望读者熟悉的主要技术概念包括：
TCP/IP
OSI Layers OSI 层级
Virtualization and containerization 虚拟化和容器化
Core networking building blocks such as: 核心网络构建模块，例如：
DNS
DHCP
Routing 路由
Switching 切换
Load-balancing 负载均衡
Firewalls 防火墙
This eBook should be accessible to most engineers with a CCNA (Cisco Certified Network Associate)-level skillset. 这本电子书应该对大多数具有 CCNA（思科认证网络工程师）级别技能的工程师都是可访问的。
Don't worry if you are not familiar with Kubernetes yet - we will explain some of the core building blocks before diving into its networking aspects. However, we expect the reader to be familiar with concepts such as virtualization and containerization and recommend the Kubernetes overview from the official Kubernetes documentation as pre-reading material. 如果您对 Kubernetes 还不熟悉，不用担心 - 我们将在深入探讨其网络方面之前解释一些核心构建模块。但是，我们希望读者熟悉虚拟化和容器化等概念，并建议从官方 Kubernetes 文档中阅读 Kubernetes 概述作为预读材料。
This document will not explain why Kubernetes has become so popular, why it's here to stay, or how becoming a Kubernetes networking expert could boost your career. 本文档不会解释为什么 Kubernetes 变得如此流行，为什么它会长存下去，或者成为 Kubernetes 网络专家如何能够促进您的职业发展。
What We Need The Network To Do 我们需要网络做什么
Regardless of the underlying computing abstractions - bare metal, virtual machine, container - there are common networking tenets that every platform should aspire to provide: 无论底层计算抽象是裸机、虚拟机还是容器，每个平台都应该努力提供的共同网络原则：
We need our applications to have accessible IP addresses 我们需要我们的应用程序具有可访问的 IP 地址
We need our applications to be able to communicate with other applications 我们需要我们的应用程序能够与其他应用程序进行通信
We need our applications to be able to access the outside world (outbound access) 我们需要我们的应用程序能够访问外部世界（出站访问）
We need our applications to be accessible from the outside world (inbound access) 我们需要使我们的应用程序可以从外部世界访问（入站访问）
We need our applications to be secured and our data protected 我们需要确保我们的应用程序安全，并保护我们的数据
We need our applications to be globally resilient and highly available 我们需要确保我们的应用程序在全球范围内具有弹性和高可用性
We may need to meet regulatory goals and requirements 我们可能需要达到监管目标和要求
We need to be able to operate and troubleshoot when applications or the infrastructure misbehave 当应用程序或基础设施出现故障时，我们需要能够操作和排除故障
In the "traditional" world, we would have needed a combination of networking and security appliances to address all these requirements. This includes routers, switches, DNS and DHCP servers, load balancers, firewalls, network monitoring appliances, VPN devices, etc... 在“传统”世界中，我们需要一组网络和安全设备来满足所有这些要求。这包括路由器、交换机、DNS 和 DHCP 服务器、负载均衡器、防火墙、网络监控设备、VPN 设备等...
Throughout this document, we will address how these guiding principles can be addressed in Kubernetes. 在本文档中，我们将讨论这些指导原则如何在 Kubernetes 中得到应用。
For the remainder of the eBook, we will refer to past networking practices as "traditional networking". We know this is an oversimplification, but please bear with us. 在本电子书的其余部分中，我们将把过去的网络实践称为“传统网络”。我们知道这是一个过度简化，但请谅解。
Back To Basics 回归基础
Let's start with the basics and the Kubernetes concepts you must understand before we start diving into networking, 让我们从基础知识和 Kubernetes 概念开始，这些概念在我们深入网络之前必须了解
One of the core premises behind the use of cloud-native technologies is the adoption of micro-services. The micro-services model provides benefits such as increased scalability, agility, and resilience compared to the monolith application architecture model. 云原生技术背后的核心前提之一是采用微服务。与单体应用程序架构模型相比，微服务模型提供了诸如增强的可伸缩性、灵活性和弹性等优势。
A monolith architecture consists of an entire application built as a single, tightly integrated unit. In contrast, a micro-services architecture deconstructs an application into a set of smaller, independent components - the micro-services. The applications running in micro-services can be updated independently, avoiding the development bottlenecks that can happen when developers work on the same codebase. 单体架构由整个应用程序作为一个单一、紧密集成的单元构建。相比之下，微服务架构将应用程序拆分为一组较小、独立的组件 - 微服务。在微服务中运行的应用程序可以独立更新，避免开发人员在相同代码库上工作时可能出现的开发瓶颈。
The micro-services could be written in different programming languages, providing developers more flexibility in adopting new frameworks than the monolith model. 微服务可以用不同的编程语言编写，为开发人员提供了比单体模型更灵活地采用新框架的可能性。
This brings us to our first Kubernetes concept: the Kubernetes pod. 这将引入我们的第一个 Kubernetes 概念：Kubernetes Pod。
A pod consists of one or more containers. A pod can represent an entire application, a single replica of a distributed application, or an individual service in a micro-service architecture. 一个 Pod 包含一个或多个容器。一个 Pod 可以代表整个应用程序，分布式应用程序的单个副本，或者微服务架构中的单个服务。
A pod will be running on a Kubernetes node. 一个 Pod 将在 Kubernetes 节点上运行。
Both virtual machines (VMs) and bare metal servers can be used as nodes in Kubernetes clusters. Typically, when you deploy Kubernetes clusters in the cloud, you would be leveraging VMs as nodes. When you deploy Kubernetes on-premises, the nodes may also be bare-metal servers. 虚拟机（VMs）和裸金属服务器都可以作为 Kubernetes 集群中的节点使用。通常，在云中部署 Kubernetes 集群时，您会利用 VM 作为节点。在本地部署 Kubernetes 时，节点也可以是裸金属服务器。
A group of nodes makes up a Kubernetes cluster. 一组节点组成一个 Kubernetes 集群。
Kubernetes is most well-known as a container orchestrator and scheduler platform. In other words, it decides on which node a pod will run. If you're familiar with VMware - and many network engineers operating data center networks will be - you might be familiar with VMware DRS (Dynamic Resource Scheduler). DRS dynamically balances computing resources across a cluster of VMware ESXi hosts. Similarly, Kubernetes monitors the resource usage (CPU, memory) of nodes and schedules pods onto a node based on available resources (among other factors that can be administratively controlled). Kubernetes 最为人熟知的是作为一个容器编排和调度平台。换句话说，它决定了一个 pod 将在哪个节点上运行。如果你熟悉 VMware - 许多操作数据中心网络的网络工程师会熟悉 - 你可能熟悉 VMware DRS（动态资源调度器）。DRS 动态平衡 VMware ESXi 主机集群中的计算资源。类似地，Kubernetes 监控节点的资源使用情况（CPU、内存）并根据可用资源（以及其他可以由管理员控制的因素）将 pods 调度到节点上。
The component responsible for making global decisions about the cluster (including scheduling) is the control plane. You are probably familiar with the idea of a control plane from configuring routers: the control plane in a network determines how data packets are forwarded. Similarly, the Kubernetes control plane determines where pods should run. 负责对集群做出全局决策（包括调度）的组件是控制平面。你可能熟悉通过配置路由器来了解控制平面的概念：网络中的控制平面决定数据包如何转发。同样，Kubernetes 控制平面确定 pods 应该在哪里运行。
Kubernetes Networking Fundamentals
Now that we understand pods, nodes, and clusters, we can introduce the Kubernetes network model. 现在我们了解了 pods、节点和集群，我们可以介绍 Kubernetes 网络模型。
Firstly, every pod in a cluster gets its unique cluster-wide IP address and can communicate with all other pods on any other node without using Network Address Translation (NAT). A practical benefit is that Kubernetes users do not need to create links between pods to connect them explicitly, 首先，集群中的每个 Pod 都会获得其独特的集群范围 IP 地址，并且可以与任何其他节点上的所有其他 Pod 进行通信，而无需使用网络地址转换（NAT）。一个实际的好处是 Kubernetes 用户不需要创建 Pod 之间的链接来显式连接它们。
Once you create pods, pods should receive a unique IP address and, assuming you have not configured some network segmentation policies (we will come to them later), be able to communicate directly with each other. 一旦您创建了 Pod，Pod 应该接收到一个唯一的 IP 地址，并且假设您没有配置一些网络分割策略（我们稍后会讨论它们），它们应该能够直接相互通信。
Note that containers within a pod all share the same IP address and MAC address. 请注意，Pod 内的容器都共享相同的 IP 地址和 MAC 地址。
Kubernetes requires a Container Network Interface (CNI) plugin to implement this model, Kubernetes 需要一个容器网络接口（CNI）插件来实现这个模型，
The CNI specifies how networking should be implemented in Kubernetes. Think of it as the IETF RFCS you may have consulted over the years. A CNI plugin is an actual implementation of said specification and is commonly referred to as a "CNI" (we will follow this terminology through the rest of this document). CNI 指定了在 Kubernetes 中应该如何实现网络。可以将其视为多年来您可能咨询过的 IETF RFCS。CNI 插件是所述规范的实际实现，通常被称为“CNI”（我们将在本文档的其余部分中遵循此术语）。
You must use a CNI compatible with your needs to meet your requirements. The feature set from one to another greatly varies. 您必须使用与您的需求兼容的 CNI 来满足您的要求。从一个 到另一个的功能集合差异很大。
Just like you, as a network architect, had to go through a network vendor selection (Cisco, Juniper, Arista, etc...) at the start of a networking project, operators and architects of Kubernetes clusters must select a CNI that provides the required networking, security, and observability features. 就像您一样，作为网络架构师，在网络项目开始时必须经历网络供应商选择（Cisco、Juniper、Arista 等...），Kubernetes 集群的运营商和架构师必须选择一个提供所需网络、安全和可观察性功能的 CNI。
The CNI that we will zoom in on this document - and the one that tends to win in most CNI evaluations - is the Cilium project. 我们将在本文中重点介绍的 CNI - 也是在大多数 CNI 评估中获胜的一个 - 是 Cilium 项目。
cilium
Introducing Cilium 介绍 Cilium
Cilium is a cloud native networking, security and observability platform. It provides most, if not all, connectivity, firewalling and network monitoring features required to operate and secure Kubernetes clusters. Cilium 是一个云原生的网络、安全和可观察性平台。它提供了操作和保护 Kubernetes 集群所需的大部分（如果不是全部）连接、防火墙和网络监控功能。
In words you may be more familiar with, it provides functions such routing, switching, load balancing, firewalling, VPN, web proxy, network monitoring, IP address management, Quality of Service and much more. Throughout the document, you will learn how these components are provided. 用你可能更熟悉的词汇来说，它提供了路由、交换、负载均衡、防火墙、VPN、Web 代理、网络监控、IP 地址管理、服务质量等功能。在本文档中，您将了解这些组件是如何提供的。
Cilium is an open-source solution and was donated to the Cloud Native Computing Foundation (CNCF) in 2021. It has been under the neutral stewardship of the CNCF ever since. Cilium 是一个开源解决方案，并于 2021 年捐赠给了云原生计算基金会（CNCF）。自那时起，它一直处于 CNCF 的中立监护之下。
Two years after its donation to the CNCF and after a thorough process involving due diligence reports and security audits, Cilium became and remains the first and only Graduated CNCF project in the cloud-native networking category, It has been adopted as the default CNI by most managed Kubernetes services and distributions. 在将 Cilium 捐赠给 CNCF 两年后，经过涉及尽职调查报告和安全审计的彻底过程，Cilium 成为并仍然是云原生网络类别中第一个也是唯一一个毕业的 CNCF 项目。它已被大多数托管的 Kubernetes 服务和发行版采用为默认的 CNI。
Isovalent, the creator of the Cilium project, offers a hardened and enterprise edition of Cilium that comes with support and additional enterprise features. We will highlight a couple of these features later in this document, but first, let's answer a question that most network engineers will have been pondering: Cilium 项目的创造者 Isovalent 提供了一个经过强化的企业版 Cilium，带有支持和额外的企业功能。我们将在本文档后面重点介绍其中一些功能，但首先，让我们回答大多数网络工程师一直在思考的一个问题：
Where is my CLI? 我的 CLI 在哪里？
Most network engineers begin their networking apprenticeship on the CLI (Command Line Interface) of a network device. While studying for Cisco certifications such as CCNA, CCNP, and CCIE, I spent countless hours on the IOS command line. It remains the primary tool to configure and administer a network. 大多数网络工程师在网络设备的 CLI（命令行界面）上开始他们的网络学徒生涯。在学习思科认证课程如 CCNA、CCNP 和 CCIE 时，我在 IOS 命令行上花费了无数个小时。它仍然是配置和管理网络的主要工具。
While the IOS CLI interacts directly with a router's operating system, interacting with Kubernetes is done by communicating with one of the components of the Kubernetes control plane: the Kubernetes API server. 虽然 IOS CLI 直接与路由器的操作系统交互，但与 Kubernetes 交互是通过与 Kubernetes 控制平面的一个组件通信来完成的：Kubernetes API 服务器。
It's typically done using the kubectI CLI - the Kubernetes equivalent of the IOS CLI. The kubectI CLI interacts with the Kubernetes API server to manage and control our Kubernetes cluster from the command line. 通常使用 kubectl CLI 来完成 - 这是 Kubernetes 中 IOS CLI 的等价物。kubectl CLI 与 Kubernetes API 服务器交互，从命令行管理和控制我们的 Kubernetes 集群。
When we run a command such as the following, kubectl will communicate with the Kubernetes API server and return an output. This command shows the pods in our cluster in the default namespace (we'll explain namespaces later on): 当我们运行类似以下的命令时，kubectl 将与 Kubernetes API 服务器通信并返回输出。此命令显示默认命名空间中我们集群中的 pods（稍后我们会解释命名空间）：
$ kubectl get pods
NAME
READY
STATUS
RESTARTS
AGE
pod
Running
0
pod -2
Running
0
This more advanced command shows the IPs of our pods alongside the nodes they are running on: 这个更高级的命令显示了我们的 Pod 的 IP 地址以及它们所在的节点：
$ kubectl
get pods
-o wide
NAME
READY
STATUS
RESTARTS
AGE
IP
NODE
NOMINATED NODE 被提名的节点
READINESS GATES 就绪门
pod
Running
0
10.0 .1 .179
kind-worker
pod-2
Running
0
98
10.0 .1 .241
kind-worker
How do I configure Kubernetes and Cilium? 我如何配置 Kubernetes 和 Cilium？
Network engineers have spent decades using conf (configure terminal) to enter the configuration mode on a Cisco IOS device. After configuring our router, we would save the configuration to permanent memory by running commands such as write memory or copy running-config startup-config. 网络工程师几十年来一直使用 conf （配置终端）在 Cisco IOS 设备上进入配置模式。配置路由器后，我们会通过运行命令如 write memory 或 copy running-config startup-config 将配置保存到永久内存中。
You won't configure individual nodes because Kubernetes is a distributed system with a centralized control plane. Instead, the cluster configuration is done centrally and stored in a data store called etcd, one of the components of the Kubernetes control plane. Just like you would back up your IOS configurations to a TFTP server, Kubernetes engineers should have a backup strategy in place in case etcd fails. 你不会配置单个节点，因为 Kubernetes 是一个具有集中控制平面的分布式系统。相反，集群配置是在中心位置完成的，并存储在称为 etcd 的数据存储中，这是 Kubernetes 控制平面的组件之一。就像你会将 IOS 配置备份到 TFTP 服务器一样，Kubernetes 工程师应该制定备份策略，以防 etcd 失效。
Configuring Kubernetes will require you to apply a configuration to the cluster, typically using the kubectl CLI. You will see some files in the YAML format throughout this eBook: they are manifests that represent your desired state. When you run a command such as kubectl apply -f manifest.yaml, the kubectl tool will push the configuration to the Kubernetes API server. The Kubernetes control plane will then create the corresponding object. 配置 Kubernetes 将需要您向集群应用配置，通常使用 kubectl CLI。在本电子书中，您将看到一些以 YAML 格式表示的文件：它们是代表您期望状态的清单。当您运行诸如 kubectl apply -f manifest.yaml 的命令时，kubectl 工具将配置推送到 Kubernetes API 服务器。Kubernetes 控制平面将随后创建相应的对象。
As the Kubernetes documentation explains, a Kubernetes object is a "record of intent". When you create an object, you're effectively telling the Kubernetes system what your cluster's workload should look like; this is your cluster's desired state. 正如 Kubernetes 文档所解释的那样，Kubernetes 对象是“意图记录”。当您创建一个对象时，实际上是在告诉 Kubernetes 系统您的集群工作负载应该是什么样子；这就是您集群的期望状态。
Note: If you're familiar with some of the software-defined networking concepts, then terms like "intent" and "desired state" should feel familiar. 注意：如果您熟悉一些软件定义网络概念，那么“意图”和“期望状态”这样的术语应该会让您感到熟悉。
Here is a sample manifest for a Pod named tiefighter. The Pod consists of a single container based on the docker.io/cilium/json-mock image. 这是一个名为 tiefighter 的 Pod 的示例清单。该 Pod 包含一个基于 docker.io/cilium/json-mock 镜像的单个容器。
Cilium's configuration is stored in two separate entities. The main Cilium configuration is stored in a ConfigMap, while the Cilium policies (for network security, BGP, etc...) leverage Custom Resource Definitions (CRDs). Let's explain what both entail. Cilium 的配置存储在两个单独的实体中。主要的 Cilium 配置存储在 ConfigMap 中，而 Cilium 策略（用于网络安全、BGP 等）利用自定义资源定义（CRD）。让我们解释一下它们分别是什么。
ConfigMaps are used within Kubernetes to store non-confidential information. A ConfigMap consists of key/value pairs. Cilium's configuration is stored in a ConfigMap called cilium-config. With the following lengthy command, you can check the Cilium configuration and filter whether IPv6 has been enabled. ConfigMaps 在 Kubernetes 中用于存储非机密信息。ConfigMap 由键/值对组成。Cilium 的配置存储在名为 cilium-config 的 ConfigMap 中。通过以下冗长的命令，您可以检查 Cilium 配置并过滤 IPv6 是否已启用。
To make things simpler, users can also use the Cilium CLI to install and manage Cilium. The Cilium CLI is a binary that helps users install, manage, and configure Cilium. The following Cilium command displays the Cilium configuration, with a filter on the IPv6 setting: 为了简化事务，用户还可以使用 Cilium CLI 来安装和管理 Cilium。Cilium CLI 是一个二进制工具，帮助用户安装、管理和配置 Cilium。以下 Cilium 命令显示了 Cilium 配置，过滤了 IPv6 设置：
This CLI tool will interface with the Kubernetes API as necessary, similar to that of kubectl. 这个 CLI 工具将根据需要与 Kubernetes API 进行交互，类似于 kubectl。
Cilium - and many other Kubernetes tools - also use Custom Resource Definitions (CRDs) to extend Kubernetes. These CRDs allow users to define and manage additional resources beyond the standard Kubernetes objects. Examples of Cilium CRDs include CiliumNetworkPolicies and CiliumBGPPeeringPolicies. We will review some of these CRDs later on. Cilium - 以及许多其他 Kubernetes 工具 - 也使用自定义资源定义（CRDs）来扩展 Kubernetes。这些 CRDs 允许用户定义和管理超出标准 Kubernetes 对象之外的其他资源。Cilium CRDs 的示例包括 CiliumNetworkPolicies 和 CiliumBGPPeeringPolicies。我们稍后将审查其中一些 CRDs。
Where is my DHCP Server? 我的 DHCP 服务器在哪里？
In traditional networking, a DHCP server allocates IP addresses to a server as it comes online (unless static addressing is used). 在传统网络中，DHCP 服务器会在服务器上线时为其分配 IP 地址（除非使用静态地址分配）。
While the nodes would typically receive their IP address over DHCP, we do not use it for pods. Instead, we refer to IPAM (IP Address Management), a term you might be familiar with if you've used platforms like Infoblox for your network. 节点通常会通过 DHCP 接收其 IP 地址，但我们不会将其用于 Pod。相反，我们使用 IPAM（IP 地址管理），如果您曾在网络中使用过像 Infoblox 这样的平台，您可能对此术语很熟悉。
In Kubernetes, the CNI is often responsible for assigning an IP address to your pod. Remember, though, that while a pod frequently consists of multiple containers, a pod only receives a single IP, which is shared by all containers in the pod. 在 Kubernetes 中，CNI 通常负责为您的 pod 分配 IP 地址。请记住，尽管一个 pod 经常由多个容器组成，但一个 pod 只接收一个 IP，该 IP 被所有容器共享。
The IP assigned to a pod comes from a subnet referred to as "PodCIDR". You might remember the concept of CIDR - Classless Inter-Domain Routing - from doing subnet calculations. In Kubernetes, just think of a PodCIDR as the subnet from which the pod receives an IP address. 分配给 pod 的 IP 来自一个称为 "PodCIDR" 的子网。您可能还记得 CIDR - 无类域间路由 - 的概念，用于进行子网计算。在 Kubernetes 中，只需将 PodCIDR 视为 pod 接收 IP 地址的子网。
Cilium supports multiple IPAM modes to address Kubernetes users' different requirements. Let's review three IPAM modes: Cilium 支持多种 IPAM 模式，以满足 Kubernetes 用户的不同需求。让我们回顾三种 IPAM 模式：
In Kubernetes-host scope mode (or per-Node PodCIDR), a single large cluster-wide prefix is assigned to a cluster, and Kubernetes carves out a subnet of that prefix for every node. Cilium would then assign IP addresses from each subnet. However, if you were to run out of IPs in your prefix, you would not be able to expand the cluster pool. 在 Kubernetes 主机范围模式（或每个节点 PodCIDR）中，为集群分配一个单个大的整个集群范围前缀，并且 Kubernetes 为每个节点划分出该前缀的一个子网。然后 Cilium 将从每个子网分配 IP 地址。但是，如果您的前缀中的 IP 地址用完了，您将无法扩展集群池。
In Cluster scope mode, Cilium is responsible for assigning CIDRs to each node, and as a bonus, it lets you add more prefixes to your cluster if you begin to run out of IPs. Just remember that pods on the same node would receive IP addresses from the same range in the first two modes. 在集群范围模式下，Cilium 负责为每个节点分配 CIDR，并且作为奖励，如果您开始用尽 IP 地址，它允许您向集群添加更多前缀。只需记住，在前两种模式中，同一节点上的 Pod 将从相同范围接收 IP 地址。
The Multi-pool mode is the most flexible one. It supports allocating PodCIDRs from multiple different IPAM pools, depending on properties of the workload defined by the user, e.g., annotations. Pods on the same node can receive IP addresses from various ranges. In addition, PodCIDRs can be dynamically added to a node as and when needed. 多池模式是最灵活的模式。它支持根据用户定义的工作负载的属性（例如注释）从多个不同的 IPAM 池中分配 PodCIDR。同一节点上的 Pod 可以从不同范围接收 IP 地址。此外，PodCIDR 可以根据需要动态地添加到节点中。
IPAM Mode
CIDR
configuration
Multiple CIDRs 多个 CIDR
per cluster
Multiple CIDRs 多个 CIDR
per node
Dynamic CIDR
allocation
Kubernetes Host Kubernetes 主机
Scope
Kubernetes
Cluster Scope 集群范围
Cilium
Multi-Pool Scope 多池范围
Cilium
What is a Kubernetes namespace? Kubernetes 命名空间是什么？
Kubernetes brings the concept of a namespace which serves to isolate groups of resources within a cluster. The segmentation is minimal: using multiple namespaces does not stop pods in a namespace from communicating with pods in other namespaces. Kubernetes 引入了命名空间的概念，用于在集群中隔离资源组。这种分割是最小的：使用多个命名空间不会阻止一个命名空间中的 Pod 与其他命名空间中的 Pod 进行通信。
The following command shows the active namespaces in our cluster: 以下命令显示了我们集群中活动的命名空间：
Operators would typically segregate an application or a tenant in its own namespace and then apply a quota. With a quota, you can limit the resources (CPU/Memory/Storage) used by pods in a specific namespace - quotas are to compute and storage resources what Quality of Service (QoS) is to networking (note: we will later discuss how to enforce networking quality of service in Kubernetes). Note that pods in different namespaces on the same node may pick up IP addresses from the same range. As you can see from the command below, a pod in the default namespace and one in the endor namespace receive IP addresses from the same PodCIDR and should be able to communicate with each other (unless network segmentation has been implemented). 运营商通常会将一个应用程序或租户隔离在自己的命名空间中，然后应用配额。通过配额，您可以限制特定命名空间中的 Pod 使用的资源（CPU/内存/存储）- 配额是计算和存储资源的限制，就像服务质量（QoS）是网络的限制一样（注意：我们稍后将讨论如何在 Kubernetes 中强制执行网络服务质量）。请注意，同一节点上不同命名空间中的 Pod 可能会从相同范围中获取 IP 地址。从下面的命令中可以看到，默认命名空间中的 Pod 和 endor 命名空间中的 Pod 从相同的 PodCIDR 接收 IP 地址，并且应该能够彼此通信（除非已实施网络分割）。
This is an essential concept in Kubernetes: IP addresses are irrelevant. While we used to remember the IP addresses of our favorite servers in networks of the past, they no longer represent an identity in the cloud-native space. 这是 Kubernetes 中一个基本概念：IP 地址是无关紧要的。虽然在过去的网络中，我们习惯记住我们喜爱的服务器的 IP 地址，但它们在云原生空间中不再代表一个身份。
Where is my DNS Server? 我的 DNS 服务器在哪里？
Now that we know how pods get an IP, let's look at how they get a DNS record. DNS is not part of the CNI's role - a Kubernetes cluster comes with a built-in DNS server (typically, coreDNS) that automatically assigns names based on the namespace and cluster name. 现在我们知道了 pod 如何获得 IP 地址，让我们看看它们如何获得 DNS 记录。DNS 不是 CNI 的职责范围 - Kubernetes 集群配备了一个内置的 DNS 服务器（通常是 coreDNS），它会根据命名空间和集群名称自动分配名称。
For example, a pod would get a name such as 10-244-1-234. default. pod.cluster. local, assuming its IP is 10.244.1.234, We will cover services later in the load balancing section, but know that a service name will have a format such as my-service.my-namespace.svc.my-cluster.local. By default, pods are assigned a DNS policy called ClusterFirst. When using this policy, any DNS query that does not match the configured cluster domain suffix is forwarded to an upstream nameserver by the DNS server. DNS policies can be set on a per-Pod basis. 例如，一个 Pod 的名称可能是 10-244-1-234.default.pod.cluster.local，假设其 IP 是 10.244.1.234。我们将在负载均衡部分讨论服务，但要知道，服务名称的格式可能是 my-service.my-namespace.svc.my-cluster.local。默认情况下，Pod 被分配了一个名为 ClusterFirst 的 DNS 策略。当使用此策略时，任何不匹配配置的集群域后缀的 DNS 查询都将被 DNS 服务器转发到上游名称服务器。DNS 策略可以根据每个 Pod 进行设置。
If you used to blame DNS for most networking issues, you might be somehow reassured that DNS is often the culprit in Kubernetes, too. 如果你过去大部分网络问题都归咎于 DNS，你可能会感到安慰，因为 DNS 在 Kubernetes 中也经常是问题的罪魁祸首。
How Do Pods Talk To Each Other? Pod 之间如何通信？
As a network engineer, you need to understand how endpoints communicate - clients and servers, bare metal servers and virtual machines, wireless clients and access points. 作为一名网络工程师，您需要了解端点之间的通信方式 - 客户端和服务器，裸机服务器和虚拟机，无线客户端和接入点。
You would have had to learn various layers of network abstractions (VLAN, VXLAN, GENEVE, GRE, MPLS) and multiple networking protocols (BGP, OSPF, EIGRP, RIP...). 您必须学习各种网络抽象层（VLAN、VXLAN、GENEVE、GRE、MPLS）和多种网络协议（BGP、OSPF、EIGRP、RIP...）。
Kubernetes and Cilium are no different. You will actually be able to re-use a lot of what you've already learned. Kubernetes 和 Cilium 也不例外。实际上，您可以重复使用您已经学过的很多知识。
Cilium provides two methods to connect pods on different nodes in the same cluster: encapsulation/overlay routing and native/direct routing. Cilium 提供了两种方法来连接同一集群中不同节点上的 Pod：封装/覆盖路由和本地/直连路由。
The default one and the one you are most likely to come across is the "encapsulation" mode, where a network overlay (typically VXLAN, but GENEVE is also an option) is created in the cluster, and a group of tunnels is created between all the nodes. 默认的方法，也是您最有可能遇到的是“封装”模式，在这种模式下，在集群中创建了一个网络覆盖（通常是 VXLAN，但 GENEVE 也是一个选择），并在所有节点之间创建了一组隧道。
This provides a set of benefits: 这提供了一系列好处：
No reliance on the underlying network - the network that connects the nodes does not need to be made aware of the Pod CIDRs. As long as the nodes can communicate, the pods should be able to communicate. 不依赖底层网络 - 连接节点的网络不需要知道 Pod CIDRs。只要节点能够通信，Pod 应该能够通信。
Overcoming addressing space challenges - given that there's no dependence on the underlying network, there is potentially a much larger set of IP addresses available to pods. 克服地址空间挑战 - 鉴于不依赖底层网络，可能有更多的 IP 地址可用于 Pod。
Auto-configuration and simplicity - new nodes joining the cluster will automatically be incorporated into the overlay, 自动配置和简单性 - 加入集群的新节点将自动并入覆盖层，
The only drawback is common to any platforms using network overlays - like Cisco ACI or VMware NSX; the overlay implies adding an encapsulation header. Fifty bytes per network packet for VXLAN for every 1500 bytes (assuming no jumbo frames) is marginal, but some organizations need their network to provide optimal network performance. For them, the native routing mode might be more appropriate. In native routing, the packets sent by the pods would be routed as if a local process had emitted the packet. As a result, the network connecting the nodes must be capable of routing the IP addresses allocated from the PodCIDRs. The Linux kernel on each node must be aware of how to forward packets to pods - in other words, they need to know how to route the packets! 唯一的缺点适用于使用网络覆盖的任何平台 - 如 Cisco ACI 或 VMware NSX; 覆盖意味着添加封装头。对于每个 1500 字节的 VXLAN 网络数据包，每个网络数据包需要五十个字节（假设没有巨幅帧），这是边缘的，但一些组织需要他们的网络提供最佳网络性能。对于他们来说，本地路由模式可能更合适。在本地路由中，由 Pod 发送的数据包将被路由，就好像本地进程发出了数据包一样。因此，连接节点的网络必须能够路由从 PodCIDRs 分配的 IP 地址。每个节点上的 Linux 内核必须知道如何将数据包转发到 Pod - 换句话说，它们需要知道如何路由数据包！
If all nodes share a single flat network, Cilium offers an option that tells every node about each PodCIDR. If not, the nodes will need to rely on an external router with that knowledge or will need to run an internal BGP daemon. 如果所有节点共享单个扁平 网络，Cilium 提供了一个选项，告诉每个节点关于每个 PodCIDR。如果不是，节点将需要依赖具有该知识的外部路由器，或者需要运行内部 BGP 守护程序。
Native Routing Mode 本地路由模式
Kubernetes Metadata Kubernetes 元数据
In our traditional world of networking, keeping track of assets connected to the network and the network itself is an ongoing challenge. Ideally, enterprise organizations might have up-to-date network diagrams or use IPAM tools like Infoblox but we know many folks still rely on Excel spreadsheets. 在我们传统的网络世界中，跟踪连接到网络的资产和网络本身是一个持续的挑战。理想情况下，企业组织可能拥有最新的网络图表或使用诸如 Infoblox 的 IPAM 工具，但我们知道许多人仍然依赖于 Excel 电子表格。
Understanding which device is connected to the network, which role the networked entity serves, and which application it is supporting is tremendously useful for IT operators but creating and managing the information and metadata relevant to each networked device is time consuming. 理解连接到网络的设备、网络实体扮演的角色以及它支持的应用程序对 IT 运营商非常有用，但创建和管理与每个网络设备相关的信息和元数据是耗时的。
In Kubernetes, we can assign metadata to our objects. While this doesn't constitute documentation, it can be used by operators to find out additional information about our applications. It can also be used for load balancing and security purposes, Let's review the two methods to assign metadata: 在 Kubernetes 中，我们可以为我们的对象分配元数据。虽然这并不构成文档，但运维人员可以使用它来查找关于我们应用程序的其他信息。它还可以用于负载平衡和安全目的，让我们回顾一下分配元数据的两种方法：
What's a Label? 什么是标签？
Kubernetes labels are ways to organize and group your resources. Of course, you would also find labels on your servers and cables in a well-maintained data center with a documented and clear color scheme and naming convention. A glance at the label should inform you where the cable goes or which workload it is running, Kubernetes 标签是组织和分组资源的方式。当然，在一个维护良好的数据中心中，您也会在服务器和电缆上找到带有记录和清晰颜色方案和命名约定的标签。看一眼标签应该告诉您电缆通向何处或者正在运行哪个工作负载。
Kubernetes labels are similar - they're a well-structured method to indicate to Kubernetes what a pod is and how to distinguish it from other objects. Kubernetes 标签类似-它们是一种良好结构的方法，用于指示 Kubernetes 一个 Pod 是什么，以及如何将其与其他对象区分开。
The following pod manifest has multiple labels to describe the application, the tier, and the environment where the pod is being deployed. 以下 Pod 清单具有多个标签，用于描述应用程序、层级和 Pod 部署的环境。
Unlike traditional networking, labels play an essential role in load balancing and security. We will review this shortly, 与传统网络不同，标签在负载平衡和安全性中发挥着重要作用。我们将很快进行审查。
What's an Annotation? 什么是注释？
While labels are a well-organized way to assign data, I refer to Kubernetes annotations as institutional knowledge about our resources. As mentioned, we, network engineers, all have used spreadsheets and Post-it notes to document our network - annotations play a similar role: they're unstructured information about pods that other humans or tools may want to consult or use. 虽然标签是一种组织数据的有效方式，但我将 Kubernetes 注释称为关于我们资源的机构知识。正如前面提到的，我们，网络工程师，都曾使用电子表格和便笺来记录我们的网络 - 注释发挥着类似的作用：它们是关于 pod 的非结构化信息，其他人类或工具可能希望查阅或使用。
Kubernetes annotations often provide documentation and context about Kubernetes objects like pods and services. As we will see in the Ingress routing and Quality of Service sections later, they can also be interpreted by external systems to add additional functionality to Kubernetes. Kubernetes 注释通常提供有关 Kubernetes 对象（如 pod 和服务）的文档和上下文。正如我们将在后面的 Ingress 路由和服务质量部分中看到的，它们也可以被外部系统解释，以为 Kubernetes 添加额外功能。
How do I secure my cluster? 我如何保护我的集群？
As you can imagine, Kubernetes Security is very complex and covers multiple areas such as data encryption at rest and in transit, protection of the control plane and workloads, auditing, Identity and Access Management - the list goes on. Some highly recommended books have been published and dedicated to Kubernetes Security, Let's focus on one aspect of security you would probably be interested in and answer the question: 正如你所想象的，Kubernetes 安全性非常复杂，涵盖多个领域，如数据加密、控制平面和工作负载的保护、审计、身份和访问管理等等。已经出版了一些高度推荐的专门针对 Kubernetes 安全性的书籍，让我们专注于你可能感兴趣的安全性方面，并回答这个问题：
What is the equivalent of firewall rules in Kubernetes? Kubernetes 中防火墙规则的等价物是什么？
The answer is Network Policies. 答案是网络策略。
Kubernetes Network Policies are implemented by the network plugin/CNI. Not all CNIs support network policies; some provide more advanced and granular network policies than most (we will review the benefits of using Cilium Network Policies shortly). Kubernetes 网络策略由网络插件/CNI 实现。并非所有的 CNI 都支持网络策略；一些提供比大多数更高级和精细的网络策略（我们将很快审查使用 Cilium 网络策略的好处）。
Standard Kubernetes Network Policies let you specify how a pod can communicate with various entities. They rely on Layer 3 and 4 elements and let you filter based on IP addresses or ports/protocols (such as TCP:80 for HTTP or TCP:22 for SSH). 标准的 Kubernetes 网络策略允许您指定一个 pod 如何与各种实体通信。它们依赖于第 3 层和第 4 层元素，并允许您基于 IP 地址或端口/协议（如 TCP:80 用于 HTTP 或 TCP:22 用于 SSH）进行过滤。
Layer 3 第 3 层
Layer 4 第 4 层
We'll now review a sample Network Policy. 现在我们将审查一个示例网络策略。
It should look familiar to most network engineers, but you might find some configuration aspects new. 对大多数网络工程师来说，这应该看起来很熟悉，但你可能会发现一些配置方面是新的。
In the networking world, you would be familiar with an extended Access List (ACL) for example: 在网络世界中，你可能熟悉扩展访问列表（ACL）的例子：
It denies all TCP traffic over port 23 (Telnet) and allows anything else. 它拒绝所有通过端口 23（Telnet）的 TCP 流量，并允许其他任何流量。
Note that the ACL applies to traffic entering (in) the Ethernet0 interface - in other words, it's our selector: we select the traffic to which this particular filter applies. 注意 ACL 适用于进入（in）Ethernet0 接口的流量 - 换句话说，它是我们的选择器：我们选择此特定过滤器适用的流量。
In Network Policies, we use a podSelector to determine who the filters will apply to. An empty one would select all the pods in the namespace. A standard method is to use labels to identify the pods the policy applies to, 在网络策略中，我们使用 podSelector 来确定筛选器适用于谁。一个空的 podSelector 将选择命名空间中的所有 pod。常用的方法是使用标签来标识策略适用的 pod，
In the ACL example above, we used the keyword in to indicate the rules would apply to traffic entering the interface. In Kubernetes Network Policies, we would use the ingress block: 在上述 ACL 示例中，我们使用关键字 in 来指示规则适用于进入接口的流量。在 Kubernetes 网络策略中，我们将使用 ingress 块：
Consider the example above - you can see how we are allowing traffic from the 172.17.0.0/16 block, except the 172.17.1.0/24 range. 考虑上面的例子-你可以看到我们是如何允许来自 172.17.0.0/16 网段的流量，除了 172.17.1.0/24 范围之外。
In a Cisco , the equivalent would be something like this: 在 Cisco 中，相当于这样的东西：
But we can also combine it with selectors - we can allow traffic only it comes from particular blocks, from a specific namespace, and from a specific pod: 但我们也可以将其与选择器结合使用-我们可以只允许来自特定网段、特定命名空间和特定 pod 的流量。
Why would we do this when IP addresses were all we needed in Cisco IOS ACL? As mentioned earlier, IP addresses are primarily irrelevant in Kubernetes. Pods are started and destroyed frequently. One of the advantages of Kubernetes is its ability to scale up and down an application by creating or deleting multiple replicas of a pod across the cluster. The IP addresses of pods are unpredictable. Traditional IP-based firewalling approaches are going to be mostly ineffective. 为什么在 Cisco IOS ACL 中只需要 IP 地址时我们要这样做？如前所述，在 Kubernetes 中 IP 地址主要是无关紧要的。Pod 经常启动和销毁。Kubernetes 的一个优势是通过在集群中创建或删除 Pod 的多个副本来扩展或缩小应用程序。Pod 的 IP 地址是不可预测的。传统基于 IP 的防火墙方法将大多无效。
What's identity-based security? 什么是基于身份的安全？
Cilium's Network Policies are an advanced method to secure clusters. For the reasons highlighted above, Cilium's approach to security is not based on IP addresses but on identities, Identities are derived from labels and other metadata and are assigned to endpoints (Cilium refers to Kubernetes pods as endpoints). Instead of creating a rule allowing traffic from the frontend pod's IP address to the backend pod's IP address, you would create a rule to allow an endpoint with the Kubernetes label role=frontend to connect to any endpoint with the Kubernetes label role=backend. Cilium 的网络策略是一种保护集群的高级方法。基于上述原因，Cilium 的安全方法不是基于 IP 地址，而是基于身份，身份是从标签和其他元数据派生的，并分配给端点（Cilium 将 Kubernetes Pod 称为端点）。您不会创建一个允许从前端 Pod 的 IP 地址连接到后端 Pod 的 IP 地址的规则，而是创建一个允许具有 Kubernetes 标签 role=frontend 的端点连接到具有 Kubernetes 标签 role=backend 的任何端点的规则。
Whenever replica pods are created with such labels, the network policy will automatically apply to such pods, meaning you will not have to update the existing network policy with the IP addresses of the new pods. 每当使用这些标签创建副本 Pods 时，网络策略将自动应用于此类 Pods，这意味着您不必使用新 Pods 的 IP 地址更新现有网络策略。
Another benefit provided by Cilium Network Policies is Layer 7 filtering. Cilium 网络策略提供的另一个好处是第 7 层过滤。
Where's my Layer 7 firewall? 我的第 7 层防火墙在哪里？
Cilium Network Policies also provide Layer 7 application visibility and control. You will be familiar with this concept if you use any web application firewalls (WAFs). Cilium Network Policies let you filter based on application (Layer 7) parameters, like HTTP path, method, or domain names. Cilium 网络策略还提供第 7 层应用程序的可见性和控制。如果您使用任何 Web 应用程序防火墙（WAF），您将熟悉这个概念。Cilium 网络策略允许您根据应用程序（第 7 层）参数进行过滤，如 HTTP 路径、方法或域名。
Let's review an example: 让我们来看一个例子：
While Kubernetes Network Policies use podSelector, Cilium Network Policies use endpointSelector to determine which endpoints this applies to. The principle remains the same the policy above will only apply to pods with the labels org: empire and class: deathstar. 虽然 Kubernetes 网络策略使用 podSelector，但 Cilium 网络策略使用 endpointSelector 来确定适用于哪些端点。原则仍然相同，上面的策略只适用于具有标签 org:empire 和 class:deathstar 的 pod。
The policy will only permit HTTP traffic from pods with the org: empire label with the HTTP POST method and to the path /v1/request-landing. 该策略仅允许带有 org: empire 标签的 pod 使用 HTTP POST 方法通过 /v1/request-landing 路径发送 HTTP 流量。
While the CiliumNetworkPolicies offer advanced Layer 7 filtering, they do not replace a web-facing firewall or an advanced IDS/IPS, You will learn in a later section how to integrate Kubernetes with your firewalls. 虽然 CiliumNetworkPolicies 提供了先进的第 7 层过滤功能，但它们并不能取代面向网络的防火墙或高级 IDS/IPS。您将在后面的部分中学习如何将 Kubernetes 与您的防火墙集成。
Where's my Load Balancer? 我的负载均衡器在哪里？
As a network engineer, you probably have deployed and used load balancers from the likes of F5 or Citrix. Load-balancing is, of course, an essential aspect of Kubernetes. 作为一名网络工程师，您可能已经部署并使用过来自 F5 或 Citrix 等公司的负载均衡器。负载均衡当然是 Kubernetes 的一个重要方面。
A load balancer helps you achieve resilience and scale by making an application accessible through a single IP (often referred to as a "virtual IP" of a "virtual server") and by distributing the traffic across multiple servers (which we sometimes refer to as "real servers") while monitoring the health of servers through health checks. 负载均衡器通过使应用程序通过单个 IP（通常称为“虚拟服务器”的“虚拟 IP”）可访问，并通过将流量分布到多个服务器（有时我们称为“真实服务器”）来帮助您实现弹性和扩展，同时通过健康检查监视服务器的健康状况。
In the traditional world, you have to manually add the IPs of the "real servers" fronted by the virtual server. As mentioned earlier, this model wouldn't work in Kubernetes - IPs are meaningless, and the churn of pods is so high that a manual approach would cancel out Kubernetes' auto-scaling benefits. 在传统世界中，您必须手动添加由虚拟服务器前端的“真实服务器”的 IP。正如前面提到的，这种模式在 Kubernetes 中行不通 - IP 是没有意义的，而且 pod 的变动如此之快，以至于手动方法会抵消 Kubernetes 的自动扩展优势。
Kubernetes Services are primarily used for load balancing. Think of it as your virtual server, fronting your pods. Kubernetes 服务主要用于负载均衡。 将它视为您的虚拟服务器，面向您的 pod。
Here is a sample Service configuration: 这里是一个示例服务配置：
Note how we are using selectors again - remember when we talked about labels earlier and how we suggested they can help us identify a group of applications? Instead of manually adding the IPs of all the echoserver pods to our pool of virtual servers, all pods with the app: echoserver label will be automatically included, and traffic will be distributed across all pods. 请注意我们再次使用选择器 - 还记得我们之前讨论标签时建议它们如何帮助我们识别一组应用程序吗？ 与手动将所有 echoserver pod 的 IP 添加到我们的虚拟服务器池不同，所有具有 app: echoserver 标签的 pod 将自动包含在内，并且流量将分布在所有 pod 之间。
Your data center probably has internal load balancers (to load balance between internal clients and internal servers) and external-facing load balancers (to proxy and load balance traffic from external clients to internal servers). 您的数据中心可能有内部负载均衡器（用于在内部客户端和内部服务器之间进行负载均衡）和面向外部的负载均衡器（用于代理和负载均衡来自外部客户端到内部服务器的流量）。
In Kubernetes, we use different types of Services to distinguish between internal and external use cases. 在 Kubernetes 中，我们使用不同类型的服务来区分内部和外部用例。
How do I load balance traffic within the cluster? 如何在集群内部负载均衡流量？
ClusterIP Services are used to expose the Service only within the cluster. As you can see in the kubectl command output below, the ClusterIP service receives a cluster-scoped virtual IP address. ClusterIP 服务用于仅在集群内部公开服务。正如您在下面的 kubectl 命令输出中所看到的，ClusterIP 服务接收一个集群范围的虚拟 IP 地址。
Note that while the is responsible for assigning IPs to pods, the Kubernetes control plane is responsible for assigning virtual IPs to Services. 请注意，虽然 负责为 pod 分配 IP 地址，但 Kubernetes 控制平面负责为服务分配虚拟 IP 地址。
ClusterIP is the default Service behavior if you don't specify the type in your Service definition. Once the ClusterIP Service is deployed, traffic will automatically be distributed across the pods in the cluster that match the selector (which is based on the labels of a pod). 如果您在服务定义中不指定类型，则 ClusterIP 是默认的服务行为。一旦部署了 ClusterIP 服务，流量将自动分布到与选择器匹配的集群中的 pod 上（这是基于 pod 的标签）。
Cluster 集群
How do I load balance traffic entering the cluster? 我如何负载均衡进入集群的流量？
This might be one of the most complex aspects of Kubernetes networking and, to do it justice, we would probably need to double the size of this eBook. To keep it brief, let's focus on the core Kubernetes Services types that are designed to expose a Service into an external IP address: NodePort and LoadBalancer. 这可能是 Kubernetes 网络中最复杂的方面之一，为了做到公正，我们可能需要将这本电子书的大小加倍。为了简洁起见，让我们专注于核心 Kubernetes 服务类型，这些类型旨在将服务暴露到外部 IP 地址：NodePort 和 LoadBalancer。
NodePort Service NodePort 服务
A NodePort Service lets users access your internal services by opening the same port on all nodes (with that port typically in the 30000-32767 range). The nodes will act as NAT gateways and forward traffic to the pods. NodePort 服务允许用户通过在所有节点上打开相同的端口（通常在 30000-32767 范围内）来访问您的内部服务。节点将充当 NAT 网关并将流量转发到 Pod。
To make the node port available, Kubernetes sets up a cluster IP address, the same as if you had requested a ClusterIP Service. The difference is that NodePort also exposes a port on every node towards this IP. 要使节点端口可用，Kubernetes 设置一个集群 IP 地址，就像您请求 ClusterIP 服务一样。不同之处在于 NodePort 还会向此 IP 对应的每个节点公开一个端口。
Take a look at this output of a NodePort Service: 查看 NodePort 服务的输出：
The node-port Service is deployed in the default namespace and uses the same selector as above (app=echoserver). NodePort 服务部署在默认命名空间中，并使用与上述相同的选择器（app=echoserver）。
This particular Service is IPv4-only but note that Kubernetes and Kubernetes Services also support Dual Stack and IPv6-only (there's a section on IPv6 later in this eBook). 这个特定的服务仅支持 IPv4，但请注意 Kubernetes 和 Kubernetes 服务也支持双栈和仅 IPv6（本电子书后面有关于 IPv6 的部分）。
All nodes in the cluster will be listening on port 30618 for this particular service. 集群中的所有节点将监听端口 30618，用于此特定服务。
Traffic arriving on the node on this port will be NATed to the healthy pods listening on the targetPort specified (80), 到达节点的此端口的流量将被 NAT 到指定的目标端口（80）上监听的健康 Pod。
Let's review a connectivity test from my external client into the cluster: 让我们回顾一下从我的外部客户端到集群的连接测试：
The first kubectl command returns the IP addresses of the nodes. The subsequent curl commands are executed to http://$NODE_IP:$NODE_PORT and only return the HTTP return code (with 200 highlighting a successful connection). The external client was able to access the internal service successfully. 第一个 kubectl 命令返回节点的 IP 地址。接下来的 curl 命令将执行到 http://$NODE_IP:$NODE_PORT，并且仅返回 HTTP 返回码（200 表示连接成功）。外部客户端成功访问了内部服务。
NodePort is simple but has limitations: users often run into issues with the constrained 30000-32767 port range, and NodePort does not natively offer load-balancing capabilities. Instead (or rather, alongside it), Kubernetes engineers tend to rely on services of the type LoadBalancer. NodePort 这种方式简单但有一些限制：用户经常在 30000-32767 的端口范围内遇到问题，并且 NodePort 本身不提供负载均衡功能。作为替代（或者说与之并行），Kubernetes 工程师往往依赖于 LoadBalancer 类型的服务。
LoadBalancer Service LoadBalancer 服务
The LoadBalancer Service builds on top of the two previously mentioned Services. 负载均衡器服务是在前面提到的两种服务的基础上构建的。
When you create a LoadBalancer service, Kubernetes will communicate with the underlying provider's API to create an external load balancer (like AWS ELB, or Google Cloud Load Balancer), which is outside of the Kubernetes cluster. 当您创建负载均衡器服务时，Kubernetes 将与底层提供商的 API 进行通信，以创建一个外部负载均衡器（如 AWS ELB 或 Google Cloud 负载均衡器），该负载均衡器位于 Kubernetes 集群之外。
The cloud provider will also allocate an external IP address and/or DNS name, which will be used by external clients to reach your internal service. The load balancer will then forward and distribute incoming traffic across the nodes. 云服务提供商还将分配一个外部 IP 地址和/或 DNS 名称，这将被外部客户端用于访问您的内部服务。然后，负载均衡器将转发和分发传入的流量到节点。
To implement a LoadBalancer Service, Kubernetes typically starts off by making changes that are equivalent to you requesting a NodePort Service by exposing the same ports across all cluster nodes. The load balancer will then forward and distribute incoming traffic across the nodes (helping you overcome the inherent NodePort limitations). 要实现 LoadBalancer 服务，Kubernetes 通常会通过进行等同于您请求 NodePort 服务的更改来开始，通过在所有集群节点上公开相同的端口。然后负载均衡器将转发和分发流入的流量到节点（帮助您克服固有的 NodePort 限制）。
But the load balancer remains external to the cluster. In other words, deploying a LoadBalancer Service in Kubernetes is like sending an email to your hosting provider that says: 但负载均衡器仍然是集群外部的。换句话说，在 Kubernetes 中部署 LoadBalancer 服务就像给您的托管提供商发送一封电子邮件，内容是：
I would like to expose an internal application running on these servers. Could you please assign a public IP for this application and configure your external load balancer accordingly? 我想要公开运行在这些服务器上的内部应用程序。您能否为此应用程序分配一个公共 IP，并相应地配置您的外部负载均衡器？
When using managed Kubernetes services, the cloud providers will deploy and configure that load balancing for you. 当使用托管的 Kubernetes 服务时，云提供商将为您部署和配置负载均衡。
If you're managing your own clusters on-premises, you can use Cilium instead to act as a load balancer network provider. Cilium can assign an external IP address (with the Load Balancer IP Address Management [LB-IPAM] feature) and then make this IP address accessible by BGP or ARP as you will learn in upcoming sections. 如果您在本地管理自己的集群，您可以使用 Cilium 作为负载均衡网络提供者。Cilium 可以分配外部 IP 地址（使用负载均衡 IP 地址管理 [LB-IPAM] 功能），然后通过 BGP 或 ARP 使此 IP 地址可访问，您将在接下来的部分中了解到。
If you're interested in how load-balancing actually works in Kubernetes, the section "How do Load Balancing and NAT work under the hood?" will provide you with more information. 如果您对 Kubernetes 中负载均衡的实际工作原理感兴趣，"负载均衡和 NAT 在幕后是如何工作的？" 部分将为您提供更多信息。
Where's my web proxy? 我的网络代理在哪里？
As a network engineer, you may have used web proxies, Layer 7 load balancers, and SSL VPN gateways devices that let you load balance, route, and control HTTP/HTTPS traffic as it enters your network. In Kubernetes, this role is often referred to as Ingress and was initially supported by the Kubernetes Ingress API and, most recently, by the Kubernetes Gateway API. 作为一名网络工程师，您可能已经使用过网络代理、第 7 层负载均衡器和 SSL VPN 网关设备，这些设备可以让您在进入网络时负载平衡、路由和控制 HTTP/HTTPS 流量。在 Kubernetes 中，这个角色通常被称为 Ingress，最初由 Kubernetes Ingress API 支持，最近由 Kubernetes Gateway API 支持。
Ingress exposes HTTP and HTTPS routes from outside the cluster to services within the cluster. Traffic routing is controlled by rules defined on the Ingress resource. Ingress 将集群外部的 HTTP 和 HTTPS 路由暴露给集群内的服务。流量路由由在 Ingress 资源上定义的规则控制。
In order for the Ingress resource to work, the cluster must have an Ingress Controller running, 为了使 Ingress 资源正常工作，集群必须运行 Ingress 控制器。
One of the benefits of Cilium is that it is the only that can also act as an ingress controller. This means you don't need to install another tool to support the Ingress functionality. Let's review a sample Cilium Ingress configuration. Cilium 的一个好处是它是唯一一个也可以充当 Ingress 控制器的 。这意味着您不需要安装另一个工具来支持 Ingress 功能。让我们来看一个示例 Cilium Ingress 配置。
By applying a manifest such as the one above, the Cilium ingress controller would create a Service of type LoadBalancer. 通过应用类似上面的清单，Cilium Ingress 控制器将创建一个类型为 LoadBalancer 的 Service。
The route above would direct HTTP requests for the path / details to the details Service, and / to the productpage Service. Assuming $HTTP_INGRESS is the IP address assigned to the Service, you can access the details Service from outside the cluster, as shown in the output below: 上面的路由将把路径 /details 的 HTTP 请求定向到 details 服务，将 / 的请求定向到 productpage 服务。假设 $HTTP_INGRESS 是分配给服务的 IP 地址，您可以从集群外部访问 details 服务，如下面的输出所示：
You can see from this non-exhaustive list on the Kubernetes Ingress Controllers page that many implementations of the Ingress API exist. The challenge with Ingress API was that every Ingress implementation became different from one to another. It made changing Ingresses very difficult: migrating from one Ingress to another would be like copying your Juniper configuration onto your Cisco router and expecting good results. 您可以从 Kubernetes Ingress Controllers 页面的这个非详尽列表中看到，存在许多 Ingress API 的实现。Ingress API 的挑战在于每个 Ingress 实现都不同。这使得更改 Ingress 变得非常困难：从一个 Ingress 迁移到另一个就像将您的 Juniper 配置复制到 Cisco 路由器并期望获得良好的结果一样。
To address this shortcoming, the Ingress API is being superseded by Gateway API. Gateway API is portable and inter-compatible, meaning that you can easily migrate from one Gateway API implementation to another. 为了解决这个缺点，Ingress API 正在被 Gateway API 取代。Gateway API 是可移植和互操作的，这意味着您可以轻松地从一个 Gateway API 实现迁移到另一个。
Again, Cilium natively supports Gateway API; removing the need to install another tool for ingress routing, The equivalent Gateway API configuration to the one above would be the one below: 再次，Cilium 原生支持 Gateway API；无需安装另一个工具进行入口路由，与上面的配置相当的 Gateway API 配置将是下面的配置：
One of the other advantages of using Gateway API is that it expands beyond just the HTTP path-based routing use case that Ingress API was based upon. 使用 Gateway API 的另一个优势是，它不仅仅局限于 Ingress API 基于 HTTP 路径的路由用例。
Gateway API supports a multitude of routing capabilities, including traffic splitting, URL redirection, path rewrites, mirroring, HTTP request/response manipulation, TLS termination and passthrough, and other capabilities that were only possible in Ingress through custom annotations. Gateway API 支持多种路由功能，包括流量分流、URL 重定向、路径重写、镜像、HTTP 请求/响应操作、TLS 终止和透传，以及 Ingress 中只能通过自定义注释实现的其他功能。
Gateway API Use Cases 网关 API 使用案例
How can I connect my cluster to existing networks? 我如何将我的集群连接到现有网络？
There are multiple ways to connect your cluster to your existing network, which might depend on the routing mode selected ("native" or "encapsulation"), how the network is configured, which level of resilience and traffic engineering is expected, etc. 有多种方法可以将您的集群连接到现有网络，这可能取决于所选择的路由模式（“本地”或“封装”），网络的配置方式，期望的弹性和流量工程水平等。
Network engineers might be pleased to know one of the options to connect Kubernetes clusters to the rest of the network is through our beloved Border Gateway Protocol (BGP). 网络工程师可能会高兴地得知，连接 Kubernetes 集群与网络其余部分的选项之一是通过我们所喜爱的边界网关协议（BGP）。
BGP support in Kubernetes comes in different forms. While it is possible to install and manage your own BGP daemons in your cluster to connect your cluster to the rest of the network, this presents additional complexity. The more elegant method is to use Cilium's built-in support for BGP. Kubernetes 中的 BGP 支持以不同形式呈现。虽然可以在集群中安装和管理自己的 BGP 守护程序以将集群连接到网络的其余部分，但这会带来额外的复杂性。更加优雅的方法是使用 Cilium 内置的 BGP 支持。
Cilium lets you build a BGP peering session over IPv4 or IPv6 and advertises the LoadBalancer Service IPs or the PodCIDR ranges to the rest of the network. Peering sessions are typically established with a Top-of-Rack (ToR) device. Cilium 允许您在 IPv4 或 IPv6 上建立 BGP 对等会话，并向网络的其余部分广播负载均衡器服务 IP 或 PodCIDR 范围。对等会话通常与顶部交换机（ToR）设备建立。
Cilium supports the most commonly used BGP features - MD5-based session authentication, customization of BGP timers, communities, local preferences, graceful restart, etc... Cilium 支持最常用的 BGP 功能-基于 MD5 的会话认证、自定义 BGP 计时器、社区、本地优先级、优雅重启等...
Configuring it is simple - simply apply a CiliumBGPPeeringPolicy like the following: 配置它很简单-只需应用一个类似以下的 CiliumBGPPeeringPolicy：
Assuming your peer is a Cisco device, it may have a BGP configuration such as: 假设您的对等方是思科设备，它可能有类似以下的 BGP 配置：
Once the peering session is established and routes advertised and exchanged, the rest of the network can access applications hosted in the Kubernetes cluster. 一旦对等会话建立并广告和交换路由，网络的其余部分就可以访问托管在 Kubernetes 集群中的应用程序。
You can even use the Cilium CLI to verify that peering sessions have been established and that routes are being advertised. Here is another example: 您甚至可以使用 Cilium CLI 来验证对等会话是否已建立并且路由是否正在被广告。这里是另一个例子：
Node
Local AS
Peer AS
Peer Address
Session State 会话状态
Uptime
Family
Received
Advertised
kind-control-plane
65001
65000
fd00:10:0:1::1
established
ipv4/unicast
3
1
ipv6/unicast
3
1
kind-worker
65002
65000
fd00:10:0:2::1
established
ipv4/unicast
3
1
ipv6/unicast
3
1
kind-worker2
65003
65000
fd00:10:0:3::1
established
ipv4/unicast
3
1
ipv6/unicast
3
1
I don't have any BGP-capable device in my existing network. Is there another way for me to access my Kubernetes services? 我的现有网络中没有任何支持 BGP 的设备。是否有其他方法让我访问我的 Kubernetes 服务？
If you would rather not use BGP or if you don't have any BGP-capable device in your network, you can advertise the networks locally, over Layer 2, using another protocol known to all network engineers: Address Resolution Protocol (ARP). 如果您不想使用 BGP，或者您的网络中没有任何支持 BGP 的设备，您可以使用另一种众所周知的协议——地址解析协议（ARP）通过第 2 层在本地广播网络。
If you have local clients on the network that need access to a Service, they need to know which MAC address to send their traffic to. Cilium leverages ARP to announce the MAC address associated with a particular IP. 如果您的网络上有需要访问服务的本地客户端，它们需要知道要发送流量的 MAC 地址。Cilium 利用 ARP 来公布与特定 IP 关联的 MAC 地址。
Let's walk through an example. In the image below, our client is on the same network as the Kubernetes LoadBalancer Service. In this case, you don't need BGP; the client simply needs to understand which MAC address is associated with the IP address of our service so that it knows to which host it needs to send the traffic, 让我们通过一个例子来解释。在下面的图像中，我们的客户端与 Kubernetes LoadBalancer 服务位于同一网络上。在这种情况下，你不需要 BGP；客户端只需要了解与我们服务的 IP 地址相关联的 MAC 地址，这样它就知道需要将流量发送到哪个主机，
Cilium can reply to ARP requests from clients for LoadBalancer IPS and enable clients to access their services. In this instance, the client (with an IP of 12.0.0.1) wants to access a Service at 12.0.0.101 and sends an ARP request as a broadcast asking, "Who has 12.0.0.101? Tell 12.0.0.1". Cilium 可以回复客户端的 LoadBalancer IPS 的 ARP 请求，并使客户端能够访问它们的服务。在这种情况下，客户端（具有 IP 地址 12.0.0.1）希望访问 12.0.0.101 的服务，并发送 ARP 请求作为广播，问，“谁有 12.0.0.101？告诉 12.0.0.1”。
The Cilium node will reply to the client with its MAC address ("12,0.0.101 is at aa:c1🆎05:ed:67") and the client will be able to access the service. Cilium 节点将用其 MAC 地址回复客户端（“12.0.0.101 在 aa:c1🆎05:ed:67”），客户端将能够访问该服务。
The feature is often referred to as "Layer 2 (L2) Announcements" and is typically for very small environments or home labs as BGP's flexibility and scalability characteristics are better suited for data center networks. 该功能通常被称为“第 2 层（L2）公告”，通常适用于非常小的环境或家庭实验室，因为 BGP 的灵活性和可扩展性特征更适合数据中心网络。
How do I connect my cluster with an external firewall? 我如何将我的集群连接到外部防火墙？
The applications running on your Kubernetes cluster will eventually need to access external resources, including data on the Internet. Of course, you will want to ensure your applications are protected and that your Internet or external-facing firewall controls Internet-bound traffic from the cluster. 运行在您的 Kubernetes 集群上的应用程序最终将需要访问外部资源，包括互联网上的数据。当然，您希望确保您的应用程序受到保护，并且您的互联网或外部防火墙控制来自集群的面向互联网的流量。
However, the challenge when connecting your traditional firewall (from vendors such as Palo Alto Networks, CheckPoint, or Cisco) is that they typically have little to no awareness of Kubernetes objects like namespace or labels. When a packet leaves the cluster, its source IP is typically masqueraded (or source NATed) to that of the node it was on. 然而，连接传统防火墙（如 Palo Alto Networks、CheckPoint 或 Cisco 等供应商）时的挑战在于它们通常对 Kubernetes 对象（如命名空间或标签）几乎没有意识。当数据包离开集群时，其源 IP 通常会伪装（或源 NAT）为其所在节点的 IP。
Your firewall will see that traffic coming in but will have no idea where the traffic originates - if it's from a particular pod or the node itself. To address this, CNIs like Cilium support a feature called Egress 您的防火墙将看到流入的流量，但不知道流量的来源 - 它是来自特定的 Pod 还是节点本身。为了解决这个问题，像 Cilium 这样的 CNI 支持一个名为 Egress Gateway 的功能。
Gateway. 。
Egress Gateway is essentially Kubernetes-Aware Source-NAT: for example, for pods in a particular namespace, we can force the traffic to exit via a certain interface and to be NATed with a specific IP, enabling the firewall to apply a rule accordingly. 出口网关本质上是 Kubernetes-Aware 源地址转换：例如，对于特定命名空间中的 Pod，我们可以强制流量通过特定接口退出，并使用特定 IP 进行地址转换，从而使防火墙能够相应地应用规则。
Let's take a look at a sample CiliumEgressGatewayPolicy: 让我们看一个示例 CiliumEgressGatewayPolicy：
What the policy above achieves is the following: when pods with the org: empire label and located in the default namespace send traffic outside the cluster, the traffic will exit via a node with a specific label, and the packets will be source-NAT with 10.168.60.100, which should be an IP associated with a network interface on this particular node. 上面的策略实现了以下功能：当具有 org: empire 标签并位于默认命名空间中的 Pod 发送流量到集群外部时，流量将通过具有特定标签的节点退出，并且数据包将使用 10.168.60.100 进行源地址转换，这应该是与此特定节点上的网络接口关联的 IP。
You now know traffic originating from your org: empire pods will have the source IP 10.168.60.100, and you can configure the appropriate security rules in your firewalls based on that IP. 您现在知道来自您的组织：帝国 pods 的流量将具有源 IP 10.168.60.100，并且您可以根据该 IP 在防火墙中配置适当的安全规则。
The only drawback with this model is that the Egress node becomes a single point of failure. For this reason, Isovalent's enterprise edition of Cilium includes Egress Gateway High Availability (HA), which supports multiple egress nodes. The nodes acting as egress gateways will then load-balance traffic in a round-robin fashion and provide fallback nodes in case one or more egress nodes fail. 这种模型的唯一缺点是 Egress 节点成为单点故障。因此，Isovalent 的 Cilium 企业版包括 Egress Gateway 高可用性（HA），支持多个 egress 节点。充当 egress 网关的节点将以循环方式负载平衡流量，并在一个或多个 egress 节点失败时提供备用节点。
How do internal Load Balancing and NAT work under the hood? 内部负载均衡和 NAT 是如何在内部工作的？
In the traditional networking world, performance is dictated by many factors - the configuration and the choice of network protocols, the network interface cards, the underlying hardware, etc... 在传统的网络世界中，性能受许多因素的影响 - 配置和选择网络协议、网络接口卡、底层硬件等...
In Kubernetes, all these factors still matter (the software has to run on some hardware, after all), but one decision engineers must make as their Kubernetes environment grows is whether to use kube-proxy. 在 Kubernetes 中，所有这些因素仍然很重要（毕竟软件必须在某些硬件上运行），但随着 Kubernetes 环境的增长，工程师必须做出的一个决定是是否使用 kube-proxy。
By default, kube-proxy is installed on each node, ensuring network traffic flows from services to the appropriate pods. When a new service or endpoint is created, it abstracts the underlying pod IPS, allowing other services to communicate using the service's virtual IP address. 默认情况下，kube-proxy 安装在每个节点上，确保网络流量从服务流向适当的 pod。当创建新的服务或端点时，它会抽象出底层的 pod IP 地址，允许其他服务使用服务的虚拟 IP 地址进行通信。
Despite what its name might suggest, kube-proxy is not an actual proxy - rather, it watches the Kubernetes control plane to then ensure that rules on the underlying operating system packet filtering are up-to-date. It manages network rules necessary for routing traffic, including Network Address Translation (NAT) when required, to ensure packets reach their intended destinations. The problem is that kube-proxy was originally based on a 25 -year-old technology (iptables), which wasn't designed for the churn of Kubernetes. 尽管其名称可能暗示了什么，kube-proxy 实际上并不是一个真正的代理 - 相反，它监视 Kubernetes 控制平面，然后确保底层操作系统数据包过滤的规则是最新的。它管理路由流量所需的网络规则，包括在需要时进行网络地址转换（NAT），以确保数据包到达其预期目的地。问题在于，kube-proxy 最初基于一个 25 年历史的技术（iptables），这并不是为 Kubernetes 的变化而设计的。
The best comparison I can come up with is this: imagine walking into a data center today and installing a Cisco PIX to secure your Al workloads. 我能想到的最好的比较是这样的：想象一下，今天走进一个数据中心，安装一个思科 PIX 来保护您的 Al 工作负载。
Yikes. 哎呀。
The main challenge with iptables' performance is that it relies on a sequential algorithm. Iptables goes through each rule individually in the table to match rules against observed traffic. This means that the time taken scales linearly with each added rule, and performance strain quickly becomes apparent as more services and endpoints are added. A packet has to traverse each rule to find a match, introducing latency and instability. iptables 的主要挑战在于它依赖于顺序算法。Iptables 会逐个规则地在表中匹配观察到的流量。这意味着随着每个添加的规则，所花费的时间会线性增加，随着添加更多服务和端点，性能压力很快就会显现出来。数据包必须遍历每个规则才能找到匹配项，这会引入延迟和不稳定性。
While Kubernetes users can now use kube-proxy in the better-performing ipvs mode, it is still reliant on netfilter. The alternative is to use Cilium's eBPF-based kube-proxy replacement. 虽然 Kubernetes 用户现在可以在性能更好的 ipvs 模式下使用 kube-proxy，但它仍然依赖于 netfilter。另一种选择是使用 Cilium 基于 eBPF 的 kube-proxy 替代方案。
eBPF is a revolutionary Linux technology that you may have come across. You don't have to know about eBPF to install and operate Cilium - after all, most of us network engineers operate networks without understanding how ASICS are programmed - but a brief primer doesn't hurt. eBPF 是一项革命性的 Linux 技术，你可能已经听说过。你不必了解 eBPF 就能安装和操作 Cilium - 毕竟，我们大多数网络工程师在不了解 ASIC 如何编程的情况下操作网络 - 但简短的入门也无妨。
eBPF is best known for the ability to safely run virtual networking and security programs within the Linux kernel. It resembles the virtual network function (VNF) concept in Telco networks or service insertion for VMware NSX or Cisco ACI users. eBPF 最为人所知的是能够在 Linux 内核中安全运行虚拟网络和安全程序。它类似于电信网络中的虚拟网络功能（VNF）概念，或者适用于 VMware NSX 或 Cisco ACI 用户的服务插入。
Here is another comparison: our beloved Cisco Catalyst 6500 switches were initially used purely for switching packets. Eventually we started inserting dedicated line cards in the chassis to support routing, security, and load-balancing functionalities. You can think of eBPF-based tools like Cilium as service modules you can insert into your Linux kernel. 这里是另一个比较：我们心爱的 Cisco Catalyst 6500 交换机最初仅用于交换数据包。最终，我们开始在机箱中插入专用线卡，以支持路由、安全和负载均衡功能。您可以将基于 eBPF 的工具如 Cilium 想象成可以插入到 Linux 内核中的服务模块。
Cilium provides a kube-proxy replacement that relies on efficient eBPF hash tables. eBPF maps are the mechanism in the Linux kernel used by eBPF programs to store and retrieve data efficiently. Cilium 提供了一个依赖于高效 eBPF 哈希表的 kube-proxy 替代方案。eBPF 映射是 Linux 内核中由 eBPF 程序用于高效存储和检索数据的机制。
Cilium's eBPF kube-proxy replacement benefits are particularly evident at scale. While iptables struggles with latency as the number of microservices grows, you can see the efficiency of eBPF in the performance benchmarking graphic below. Cilium 的 eBPF kube-proxy 替代方案在规模上特别明显。当微服务数量增加时，iptables 在延迟方面存在问题，您可以在下面的性能基准图表中看到 eBPF 的效率。
HTTP request latency via k8s service (usec) 通过 k8s 服务的 HTTP 请求延迟（微秒）
How do I manage and troubleshoot my network? 我如何管理和排除网络问题？
In the traditional networking world, network engineers have relied on trusted tools and systems to troubleshoot. Here is a non-exhaustive list of tools you may have in your toolbox and their equivalent for Kubernetes and Cilium users: 在传统的网络世界中，网络工程师一直依赖于可靠的工具和系统进行故障排除。以下是您可能在工具箱中拥有的工具的非详尽列表，以及它们在 Kubernetes 和 Cilium 用户中的等效物：
Use case
Traditional Networking 传统网络
Kubernetes Networking Kubernetes 网络
check TCP/IP connectivity 检查 TCP/IP 连通性
ping/traceroute
ping
check HTTP connectivity 检查 HTTP 连通性
curl
curl
check the status of the network 检查网络状态
IOS CLI
kubectl or cilium CLI kubectl 或 cilium 命令行界面
capture logs from network 从网络捕获日志
IOS CLI and Syslog IOS CLI 和 Syslog
kubectl logs
capture traffic patterns and bandwidth usage 捕获流量模式和带宽使用情况
NetFlow/sFlow
Hubble
analyse network traffic 分析网络流量
tcpdump/Wireshark
tcpdump/Wireshark/Hubble
generate traffic for performance testing 为性能测试生成流量
iperf
iperf
In your network, you may have run some of the connectivity tooling commands from your routers and devices or straight from your terminal. In your Kubernetes cluster, you may need to deploy a monitoring pod for troubleshooting purposes. I recommend the Kubernetes networking troubleshooting container netshoot as it includes tools such as tcpdump and iperf amongst many others. 在您的网络中，您可能已经从路由器和设备或直接从终端运行了一些连接工具命令。在您的 Kubernetes 集群中，您可能需要部署一个监控 pod 用于故障排除目的。我推荐使用 Kubernetes 网络故障排除容器 netshoot，因为它包含诸如 tcpdump 和 iperf 在内的许多工具。
Even better, you can use kubectl debug to deploy an ephemeral container in an existing pod to debug networking issues (remember they share the same networking stack): 更好的是，您可以使用 kubectl debug 在现有的 pod 中部署一个临时容器来调试网络问题（请记住它们共享相同的网络堆栈）：
How can I monitor and visualize network traffic? 我如何监视和可视化网络流量？
In your traditional network, you may use Netflow/sFlow to collect and export network flows in the following format: 在传统网络中，您可以使用 Netflow/sFlow 以以下格式收集和导出网络流量：
Source IP
Destination IP 目的地 IP
Packets
Bytes
192.0.2.2
198.51 .100 .5
20
500
203.0.113.129
198.51 .100 .3
50
1250
192.0 .2 .6
203.0 .113 .68
10
1000
Your NetFlow collector might then be able to represent the flows in a graphical interface and build a visual representation of who's talking to whom. To do the same in Kubernetes, you'd typically use Hubble. 您的 NetFlow 收集器可能能够在图形界面中表示流量，并构建谁与谁通信的可视化表示。在 Kubernetes 中执行相同操作时，通常会使用 Hubble。
Hubble is the network observability tool that comes with Cilium - think of it as a combination of NetFlow and Wireshark. By using eBPF, Hubble can hook into the network and extract networking data. Hubble comes with its own CLI and UI. Hubble 是随 Cilium 一起提供的网络可观测性工具 - 将其视为 NetFlow 和 Wireshark 的组合。通过使用 eBPF，Hubble 可以钩入网络并提取网络数据。Hubble 自带其自己的 CLI 和 UI。
You can use the CLI to observe all the flows in the cluster, with the option to use filters to, for example, only see traffic from the tiefighter pod in the endor namespace. 您可以使用 CLI 观察集群中的所有流量，还可以使用过滤器选项，例如，仅查看来自 endor 命名空间中 tiefighter pod 的流量。
The Hubble UI lets you visualize traffic in a Kubernetes cluster as a service map. The example below shows three pod identities in the endor namespace: Hubble UI 允许您将 Kubernetes 集群中的流量可视化为服务地图。下面的示例显示了 endor 命名空间中三个 pod 的身份：
Both xwing and tiefighter pods are trying to access the Deathstar. Note the granularity of the flow information you get from Hubble. You can see that the traffic was HTTP over the /v1/request-landing HTTP path, using the POST method. xwing 和 tiefighter pod 都试图访问 Deathstar。请注意从 Hubble 获取的流量信息的细粒度。您可以看到流量是通过 /v1/request-landing HTTP 路径进行的 HTTP 传输，使用的是 POST 方法。
While the open source edition of Cilium includes a standard version of Hubble, the enterprise edition of Cilium includes an advanced enterprise edition of Hubble, with multi-tenant self-service access, historical flow and analytics data, and a built-in network policy editor (described in more details at the end of the next section). 尽管 Cilium 的开源版本包含标准版本的 Hubble，但 Cilium 的企业版包含高级企业版的 Hubble，具有多租户自助访问、历史流量和分析数据，以及内置网络策略编辑器（在下一节末尾更详细地描述）。
How do I start securing my cluster? 我如何开始保护我的集群？
Clusters are typically deployed without network policies: as is the case in traditional networking, we are all often guilty of bolting security on later. 集群通常在没有网络策略的情况下部署：就像传统网络中的情况一样，我们经常会在后期添加安全性。
Deploying network policies without disrupting the applications running on the cluster presents a couple of challenges for operators: 1) How do I deploy network policies in a transparent manner without breaking the cluster? and 2) How do I even know which security rules to enforce? 在不影响集群中运行的应用程序的情况下部署网络策略对运维人员提出了一些挑战：1）如何在透明的方式下部署网络策略而不破坏集群？2）我该如何知道要执行哪些安全规则？
The first challenge can be addressed by selecting the Cilium Network Policy enforcement mode. In the default enforcement mode, endpoints start without restriction but as soon as a rule matches, traffic has to be explicitly allowed or traffic will be dropped. In the never enforcement mode, all traffic is permitted. This is an audit-only mode you can run to see what the policy might catch before enforcing it. 第一个挑战可以通过选择 Cilium 网络策略执行模式来解决。在默认执行模式下，端点在没有限制的情况下启动，但一旦规则匹配，流量必须明确允许，否则流量将被丢弃。在从不执行模式下，所有流量都被允许。这是一个仅用于审计的模式，您可以在执行之前运行以查看策略可能捕获到的内容。
The more security-savvy network engineers that want strict, or zero trust, network security from the start could also set the policy enforcement mode to always in which Cilium will block all traffic unless there is a Network Policy explicitly allowing it. 更注重安全的网络工程师可以从一开始就将策略执行模式设置为始终，这样 Cilium 将阻止所有流量，除非有网络策略明确允许。
The second challenge can be tackled in various manners. You can rely on the developers to tell you how the micro-services are inter-connected, over which ports they are communicating, which API they're using, etc... The problem is that it can be difficult to understand the developer's intent, Let's face it most developers don't always know how micro-services are interconnected! 第二个挑战可以用各种方式解决。您可以依赖开发人员告诉您微服务是如何相互连接的，它们通过哪些端口进行通信，它们使用的是哪个 API 等等... 问题在于理解开发人员的意图可能会很困难，让我们面对现实，大多数开发人员并不总是知道微服务是如何相互连接的！
Alternatively, you could deploy Hubble, observe the traffic, and create network policies based on the traffic. This works fine although it can be time-consuming. 或者，您可以部署 Hubble，观察流量，并根据流量创建网络策略。这样做虽然有效，但可能会耗费时间。
This is an area where Isovalent is here to help. Firstly, Isovalent provides a free, online Network Policy Editor designed to ease the cognitive overhead of developing network policies. It lets you visualize policies, allowing users to easily create policies before downloading them in YAML files, 这是 Isovalent 可以提供帮助的领域。首先，Isovalent 提供了一个免费的在线网络策略编辑器，旨在减轻开发网络策略时的认知负担。它让您可视化策略，使用户能够轻松创建策略，然后将其下载为 YAML 文件。
Additionally, Isovalent's Enterprise Edition of Cilium includes an advanced enterprise Hubble view that is designed to greatly simplify and accelerate network policy generation. 另外，Isovalent 的 Cilium 企业版还包括一个先进的企业 Hubble 视图，旨在极大简化和加速网络策略生成。
It lists all the flows in the cluster (which can be filtered by namespace) and by clicking on a particular flow, you can automatically create a Cilium Network Policy based on this particular flow. The Network Policy can then be downloaded as a YAML file and applied to the cluster. 它列出了集群中的所有流量（可以按命名空间进行过滤），通过点击特定流量，您可以基于该特定流量自动创建 Cilium 网络策略。然后，可以将网络策略下载为 YAML 文件并应用于集群。
How do I encrypt the traffic in my cluster? 如何在我的集群中加密流量？
Encrypting data in transit is a common requirement. Whether over public networks or even within your data centers, you probably have considered how to protect your data and ensure you meet certain regulatory requirements. 在传输中加密数据是一个常见的要求。无论是在公共网络上还是在您的数据中心内部，您可能已经考虑过如何保护您的数据并确保符合某些监管要求。
Over the Internet, between data centers, or even within a network, you may have used IPsec-based VPNs to encrypt the traffic between two different sites. Those who can afford the specialized crypto hardware may have used technologies such as MACsec (defined by the IEEE 802.1AE standard) to encrypt traffic on Ethernet links. 在互联网上，在数据中心之间，甚至在网络内部，您可能已经使用基于 IPsec 的 VPN 来加密两个不同站点之间的流量。那些能够负担得起专门的加密硬件的人可能已经使用诸如 MACsec（由 IEEE 802.1AE 标准定义）这样的技术来加密以太网链路上的流量。
The requirement can also exist in Kubernetes and might be even more common, given Kubernetes clusters are often deployed on shared hardware in cloud data centers. Cilium's Transparent Encryption feature set offers two options for encrypting traffic between pods on different nodes (or even between clusters): IPsec and WireGuard. 在 Kubernetes 中也可能存在这种要求，而且可能更常见，因为 Kubernetes 集群通常部署在云数据中心的共享硬件上。Cilium 的透明加密功能集提供了两种选项，用于加密不同节点上（甚至在集群之间）的 Pod 之间的流量：IPsec 和 WireGuard。
The IPsec method lets you choose the algorithm and cipher and manage the keys (just as you would do when setting IPsec between two VPN endpoints in your network), while the WireGuard option is opinionated: it leverages very robust cryptography and does not let the user choose ciphers and protocols. IPsec 方法允许您选择算法和密码并管理密钥（就像在网络中设置两个 VPN 端点之间的 IPsec 时所做的那样），而 WireGuard 选项是有主见的：它利用非常强大的加密技术，不允许用户选择密码和协议。
Both methods ensure that traffic between pods is encrypted as long as pods are located on different nodes. 只要 Pod 位于不同的节点上，这两种方法都可以确保 Pod 之间的流量是加密的。
How do we connect clusters together? 我们如何将集群连接在一起？
As a network engineer, you have probably been tasked to design resilient and highly resilient solutions across multiple sites. You may also have had to connect both networks together, often over a Layer 3 network, and use an intersite control plane such as BGP EVPN VXLAN to distribute network information. You may have also implemented some form of load balancing across multiple sites. You would have had to consider how to enforce consistent security policy across multiple sites. 作为一名网络工程师，您可能被要求设计跨多个站点的弹性和高度弹性解决方案。您可能还必须连接两个网络，通常是通过第 3 层网络，并使用诸如 BGP EVPN VXLAN 之类的站点间控制平面来分发网络信息。您可能还实施了某种形式的跨多个站点的负载均衡。您必须考虑如何在多个站点之间执行一致的安全策略。
In Kubernetes, these challenges remain - in a multi-cluster topology, you have multiple clusters located across different zones for resiliency and high availability. While it might be relatively trivial to set up IP connectivity between the clusters, they would have no awareness of each other's objects, such as pods, services, namespaces, labels, and other metadata. 在 Kubernetes 中，这些挑战仍然存在 - 在多集群拓扑中，您有多个集群位于不同区域以实现弹性和高可用性。虽然在集群之间建立 IP 连接可能相对简单，但它们不会意识到彼此的对象，如 pod、服务、命名空间、标签和其他元数据。
As we've seen, network policies and load balancing often rely on such metadata. Therefore, load-balancing across clusters or applying security policies across multiple clusters is not possible in a standard Kubernetes cluster implementation. 正如我们所看到的，网络策略和负载均衡通常依赖于这些元数据。因此，在标准 Kubernetes 集群实现中，跨集群负载均衡或应用安全策略跨多个集群是不可能的。
This is where you might need a multi-cluster solution like Cilium Cluster Mesh. 这是您可能需要像 Cilium Cluster Mesh 这样的多集群解决方案的地方。
Cilium Cluster Mesh addresses these requirements by federating multiple Cilium instances on different clusters. Cluster Mesh automatically discovers services across all meshed clusters and load balances the traffic effectively, Cilium Cluster Mesh 通过在不同集群上联合多个 Cilium 实例来满足这些要求。 Cluster Mesh 会自动发现所有网格化集群中的服务，并有效地负载均衡流量，
One of the main use cases for using Cluster Mesh is High Availability. This use case includes operating Kubernetes clusters in multiple regions or availability zones and running the replicas of the same services in each cluster. Traffic will by default be load-balanced to all endpoints across clusters. When all endpoints in a given cluster fail, traffic will be forwarded to the remaining endpoints in other clusters, 使用 Cluster Mesh 的主要用例之一是高可用性。该用例包括在多个地区或可用性区域中操作 Kubernetes 集群，并在每个集群中运行相同服务的副本。流量将默认负载平衡到跨集群的所有终端点。当给定集群中的所有终端点全部失败时，流量将被转发到其他集群中的剩余终端点，
With Cilium Cluster Mesh, you can enforce network policy to workloads spanning multiple clusters. 使用 Cilium 集群网格，您可以强制执行跨多个集群的工作负载的网络策略。
In the example above, the frontend pods on cluster- 1 are allowed to connect to backend pods running in cluster-2. However, frontend pods on cluster-2 are not allowed to connect to backend pods running in cluster-1. 在上面的示例中，允许集群-1 上的前端 pod 连接到运行在集群-2 中的后端 pod。然而，不允许集群-2 上的前端 pod 连接到运行在集群-1 中的后端 pod。
Below is an example of a Cilium Network Policy that can be applied in such a configuration. 以下是一个可以在这种配置中应用的 Cilium 网络策略示例。
Is IPv6 supported on Kubernetes? IPv6 是否在 Kubernetes 上受支持？
Yes. Kubernetes has supported IPv6-only clusters since Kubernetes 1.18 and Dual Stack IPv4/IPv6 reached General Availability (GA) in Kubernetes 1.23. 是的。Kubernetes 从版本 1.18 开始支持 IPv6-only 集群，而双栈 IPv4/IPv6 在 Kubernetes 1.23 中达到了一般可用性(GA)。
Like network policies, this depends entirely on whether the CNI plugin supports this feature: not all CNIs support IPv6-only or Dual Stack. The good news: Cilium supports IPv4-only, IPv6-only, and Dual Stack IPv4/IPv6 clusters. 像网络策略一样，这完全取决于 CNI 插件是否支持此功能：并非所有的 CNIs 都支持 IPv6-only 或双栈。好消息是：Cilium 支持仅 IPv4、仅 IPv6 和双栈 IPv4/IPv6 集群。
However, the challenge isn't necessarily whether your pod picks up an IPv6 address at deployment; it's whether you can operate and troubleshoot an IPv6 cluster. Thankfully, Hubble - mentioned earlier greatly simplifies IPv6 operations. 然而，挑战并不在于您的 Pod 在部署时是否获取了 IPv6 地址；而在于您是否能够操作和排除故障 IPv6 集群。值得庆幸的是，前面提到的 Hubble 大大简化了 IPv6 操作。
You can, for example, easily find all IPv6 traffic from a particular pod: 例如，您可以轻松找到特定 Pod 的所有 IPv6 流量：
You can also visualize IPv6 service connectivity, using the Hubble UI that comes with Cilium OSS: 您还可以使用附带 Cilium OSS 的 Hubble UI 可视化 IPv6 服务连接性：
Does the concept of Quality of Service (QoS) exist in Kubernetes? Kubernetes 中是否存在服务质量（QoS）的概念？
Let's distinguish between compute QoS and networking QoS. While you're probably thinking of networking QoS, the Kubernetes docs refer to Quality of Service (QoS) for pods. Kubernetes operators can assign a QoS class, which can be used to determine whether to evict pods when node resources are exceeded. The focus of this feature is to prevent CPU and memory starvation. 让我们区分计算 QoS 和网络 QoS。虽然您可能在考虑网络 QoS，但 Kubernetes 文档中提到了用于 Pod 的服务质量（QoS）。Kubernetes 操作员可以分配一个 QoS 类别，用于确定在节点资源超出时是否要驱逐 Pod。此功能的重点是防止 CPU 和内存饥饿。
As a network engineer, you would have used networking QoS to prevent bandwidth starvation. Networking QoS encompasses various requirements, such as: 作为网络工程师，您可能已经使用网络 QoS 来防止带宽饥饿。网络 QoS 包括各种要求，例如：
Marking: how a network device labels packets with a specific priority marker to ensure they receive preferential treatment throughout the network 标记：网络设备如何使用特定优先级标记对数据包进行标记，以确保它们在整个网络中获得优先处理
Congestion management: how a network device manages packets as they wait to exit a device 拥塞管理：网络设备如何管理数据包在等待离开设备时的情况
Congestion avoidance: how a network device decides if and when to drop packets as a system becomes congested 拥塞避免：网络设备如何在系统拥塞时决定是否以及何时丢弃数据包
Shaping/Policing: how a network device delays or drops packets to ensure that the throughput does not exceed a defined rate 塑形/调度: 网络设备延迟或丢弃数据包，以确保吞吐量不超过定义的速率
As the Kubernetes platform will be running on a physical network, you may want to enforce some level of quality of service within the physical network (out of scope for this document). Within Kubernetes, you should ensure a fair distribution of the node's network interface bandwidth. 由于 Kubernetes 平台将在物理网络上运行，您可能希望在物理网络内执行某种程度的服务质量保证（本文档范围之外）。在 Kubernetes 内部，您应确保节点的网络接口带宽公平分配。
Consider that all pods on a node will typically share a single network interface. If a pod consumes an unfair share of the available bandwidth, the other pods on the node might suffer. 请考虑一个节点上的所有 pod 通常将共享单个网络接口。如果一个 pod 占用了可用带宽的不公平份额，该节点上的其他 pod 可能会受到影响。
With Cilium's Bandwidth Manager, you can mark a Pod by adding an annotation to its specification and shape traffic as it egresses the pod. See an example below: 使用 Cilium 的带宽管理器，您可以通过向其规范添加注释来标记 Pod，并在其出口流量形状。请参见下面的示例：
When running a throughput performance test, you would see the egress traffic limited down to just under 10 Mbps: 运行吞吐量性能测试时，您会看到出口流量限制在接近 10 Mbps 以下：
Which Kubernetes networking options are available in managed Kubernetes services? 托管 Kubernetes 服务中有哪些可用的 Kubernetes 网络选项？
With most organizations following a cloud-first approach, it is no surprise that many Kubernetes clusters are hosted in cloud providers such as AWS, Azure, and Google Cloud. Some choose to build and manage clusters themselves on platforms such as AWS laaS service EC2. Still, most organizations tend to gravitate towards the managed Kubernetes services offerings, such as Azure Kubernetes Service (AKS), AWS Elastic Kubernetes Service (EKS) and Google Kubernetes Service (GKE). In these services, the Kubernetes control plane components are managed on your behalf - you get access to your worker nodes and consume your pool of compute resources as you see fit. 随着大多数组织采用云优先的方法，许多 Kubernetes 集群托管在 AWS、Azure 和 Google Cloud 等云提供商中并不奇怪。一些人选择在诸如 AWS 的 IaaS 服务 EC2 等平台上自行构建和管理集群。然而，大多数组织倾向于使用托管的 Kubernetes 服务，如 Azure Kubernetes Service (AKS)、AWS Elastic Kubernetes Service (EKS) 和 Google Kubernetes Service (GKE)。在这些服务中，Kubernetes 控制平面组件由您代为管理 - 您可以访问您的工作节点并根据需要使用您的计算资源池。
We would need a separate white paper to cover the diversity of Kubernetes networking options in cloud offerings and given how quickly it evolves, it would soon become outdated. However, the majority of the concepts highlighted in this eBook apply. Cilium is either the default networking layer or often recommended by managed service providers and Kubernetes distribution maintainers. 我们需要一份单独的白皮书来涵盖云服务中 Kubernetes 网络选项的多样性，考虑到它的快速演进，它很快就会过时。然而，本电子书中强调的大多数概念仍然适用。Cilium 要么是默认的网络层，要么经常被托管服务提供商和 Kubernetes 发行版维护者推荐。
Take AKS and GKE, for example - GKE's datapath v2 is based on Cilium, while the preferred option for large-scale clusters on AKS is the "Azure CNI powered by Cilium" mode. 以 AKS 和 GKE 为例 - GKE 的 datapath v2 基于 Cilium，而在 AKS 上用于大规模集群的首选选项是“由 Cilium 提供支持的 Azure CNI” 模式。
Given that these cloud-hosted Kubernetes clusters are managed services, you don't get to choose the Cilium version and the features available. 鉴于这些托管在云中的 Kubernetes 集群是托管服务，您无法选择 Cilium 版本和可用功能。
If you'd like access to all Cilium features, you can, for example, use AKS's "Bring-Your-Own-CNI" option. It lets you deploy a cluster without a CNI and install the one of your choice. While this model enables you to install and customize Cilium fully, it comes with the caveat that Microsoft support couldn't help with -related issues. 如果您想要访问所有 Cilium 功能，例如，您可以使用 AKS 的“Bring-Your-Own-CNI”选项。它允许您部署一个没有 CNI 的集群，并安装您选择的 CNI。虽然这种模式使您能够完全安装和定制 Cilium，但缺点是 Microsoft 支持无法帮助处理与 相关的问题。
Head to https://isovalent.com/partners/ for up-to-date information on how the various cloud providers and technology partners are standardizing on Cilium. 前往 https://isovalent.com/partners/ 获取有关各种云提供商和技术合作伙伴如何在 Cilium 上进行标准化的最新信息。
Conclusion 结论
I hope this book has helped you understand the core building blocks of Kubernetes networking, how pods can communicate with each other and with the outside world, how you can secure the applications running in your cluster, and how you can operate and manage it. 我希望这本书能帮助您了解 Kubernetes 网络的核心构建模块，以及 pod 如何相互通信以及与外部世界通信，如何保护运行在您的集群中的应用程序，以及如何操作和管理它。
But there are many intricacies to Kubernetes Networking and Cilium. To keep this eBook brief, we made editorial decisions that experienced Kubernetes architects might question: we decided to omit concepts and tools like headless services, networking namespaces, service meshes, multicast, etc...We also recognize we barely scratched the surface of what eBPF is capable and didn't even mention Tetragon. 但是 Kubernetes 网络和 Cilium 有许多复杂之处。为了使本电子书简洁，我们做出了一些编辑决策，有经验的 Kubernetes 架构师可能会质疑：我们决定省略诸如无头服务、网络命名空间、服务网格、组播等概念和工具...我们也意识到我们几乎没有深入探讨 eBPF 的潜力，甚至没有提到 Tetragon。
We will save them for another volume. 我们将它们留到另一卷中。
We greatly welcome your feedback and comments. Feel free to get in touch on social media to share your thoughts. You can find me on Linkedln at @nicolasvibert. 我们非常欢迎您的反馈和评论。随时通过社交媒体与我们联系，分享您的想法。您可以在 Linkedln 上找到我，用户名为 @nicolasvibert。
Learn More with the Isovalent Hands-On Labs 通过 Isovalent 亲身实验室了解更多
As useful as this eBook may have been to grasp the key concepts of Kubernetes networking and the many Cilium use cases, nothing beats practical experience. To that effect, Isovalent introduced a program of free, online, hands-on labs. 尽管这本电子书对于掌握 Kubernetes 网络的关键概念和众多 Cilium 使用案例可能很有用，但没有什么能比得上实际经验。为此，Isovalent 推出了一项免费在线实验室的计划。
By undertaking these labs, you will learn about different features and use cases of Cilium, Hubble, Tetragon, and their Isovalent Enterprise editions. You will also earn digital badges: in late 2022, we started an accreditation program to recognize the efforts of the individuals who had completed the labs. 通过参与这些实验室，您将了解 Cilium、Hubble、Tetragon 及其 Isovalent 企业版本的不同功能和使用案例。您还将获得数字徽章：2022 年底，我们启动了一个认证计划，以表彰完成实验室的个人的努力。
In the past 12 months alone, over 37,000 labs have been played by over 12,000 engineers. 仅在过去的 12 个月中，超过 12,000 名工程师玩了超过 37,000 个实验室。
The labs are constantly maintained and updated to the latest Cilium version. At the time of writing, there are 33 publicly available labs. 实验室不断维护和更新至最新的 Cilium 版本。在撰写本文时，有 33 个公开可用的实验室。
Here are some of the labs that cover functionalities highlighted in this eBook. 以下是一些覆盖本电子书中突出功能的实验室。
Getting Started with Cilium 使用 Cilium 入门
In this hands-on lab we provide you a fully fledged Cilium installation and a few challenges to solve. See for yourself how Cilium works by securing a moon-sized battlestation in a "Star Wars"-inspired challenge. 在这个实践实验中，我们为您提供了一个完整的 Cilium 安装和一些需要解决的挑战。通过在一个受《星球大战》启发的挑战中保护一个巨大的月球大小的战斗站，亲自体验 Cilium 的工作原理。
Getting Started with Cilium Lab 使用 Cilium 入门实验
Cilium Network Policies Cilium 网络策略
Achieving zero-trust network connectivity via Kubernetes Network Policy is complex. Enter Isovalent Enterprise for Cilium: it provides tooling to simplify and automate the creation of Network Policy. 通过 Kubernetes 网络策略实现零信任网络连接是复杂的。Isovalent 企业为 Cilium 提供了工具，用于简化和自动化网络策略的创建。
In order to connect Kubernetes and the existing network infrastructure, BGP is needed. Cilium offers native support for BGP, exposing Kubernetes to the outside and all the while simplifying users' deployments. 为了连接 Kubernetes 和现有的网络基础设施，需要使用 BGP。Cilium 提供对 BGP 的原生支持，将 Kubernetes 暴露给外部，并同时简化用户的部署。
(P) BGP on Cilium Lab (P) Cilium 上的 BGP 实验室
Learn More with the Isovalent Hands-On Labs 通过 Isovalent 实践实验室了解更多
Cilium Gateway API Cilium 网关 API
In this lab, you will learn how you can use the Cilium Gateway API functionality to route HTTP and HTTPS traffic into your Kubernetes-hosted application. 在这个实验室中，您将学习如何使用 Cilium 网关 API 功能将 HTTP 和 HTTPS 流量路由到您的 Kubernetes 托管应用程序中。
(S) Cilium Gateway API Lab (S) Cilium 网关 API 实验室
Cilium Transparent Encryption Cilium 透明加密
Cilium provides two options to encrypt traffic: IPsec and WireGuard. In this lab, you will be installing and testing both features and will get to experience how easy it is to encrypt data in transit with Cilium. Cilium 提供两种加密流量的选项：IPsec 和 WireGuard。在这个实验中，您将安装和测试这两个功能，并体验使用 Cilium 在传输过程中加密数据的简便性。
Cilium Transparent Encryption with IPSec and WireGuard Lab 使用 IPSec 和 WireGuard 实现 Cilium 透明加密实验室
Cilium Egress Gateway Cilium 出口网关
Cilium's Egress Gateway feature allows you to specify which nodes should be used by a pod in order to reach the outside world. Cilium 的出口网关功能允许您指定哪些节点应该被一个 Pod 使用，以便访问外部世界。
(1) Cilium Egress Gateway Lab (1) Cilium 出口网关实验室
IPv6 Networking and Observability IPv6 网络和可观测性
This lab will walk you through how to deploy a IPv4/IPv6 Dual Stack Kubernetes cluster and install Cilium and Hubble to benefit from their networking and observability capabilities. 本实验室将指导您如何部署 IPv4/IPv6 双栈 Kubernetes 集群，并安装 Cilium 和 Hubble 以从它们的网络和可观测性功能中受益。
Cilium IPv6 Networking and Observability Lab Cilium IPv6 网络和可观测性实验室
IPv6 OBSERVABILITY IPv6 可观测性
Introducing Cilium, Hubble, and Tetragon 介绍 Cilium、Hubble 和 Tetragon
What is Cilium? 什么是 Cilium？
Cilium is an open source, cloud native solution for providing, securing, and observing network connectivity between workloads. Cilium was created by Isovalent and is currently the only CNCF graduated networking plugin project. Cilium 是一个开源的、云原生的解决方案，用于提供、保护和观察工作负载之间的网络连接。Cilium 是由 Isovalent 创建的，目前是唯一一个 CNCF 毕业的网络插件项目。
Cilium is best known as a CNI (Container Network Interface) and provides secure and observable connectivity at the network and service mesh level inside Kubernetes and beyond. It is fueled by the revolutionary Kernel technology eBPF. Google (GKE, Anthos), Amazon (EKS-A), and Microsoft (AKS) have all adopted Cilium to provide networking and security for Kubernetes. Cilium is used by platform engineering teams including Adobe, Bell Canada, ByteDance, Capital One, Datadog, IKEA, Schuberg Philis, and Sky. Cilium 最为人所知的是作为 CNI（容器网络接口），在 Kubernetes 内部及更广泛的网络和服务网格层面提供安全和可观察的连接。它利用了革命性的内核技术 eBPF。Google（GKE、Anthos）、Amazon（EKS-A）和 Microsoft（AKS）都采用了 Cilium 来为 Kubernetes 提供网络和安全性。Cilium 被 Adobe、Bell Canada、ByteDance、Capital One、Datadog、IKEA、Schuberg Philis 和 Sky 等平台工程团队所使用。
Figure 1: Overview of Cilium Suite Cilium 套件概述
What is Hubble? 什么是 Hubble?
Hubble is a fully distributed networking and security observability platform built on top of Cilium and eBPF. It enables deep visibility into the communication and behaviour of services and networking infrastructure in a completely transparent manner. Hubble 是一个完全分布式的网络和安全可观测平台，构建在 Cilium 和 eBPF 之上。它以完全透明的方式深入了解服务和网络基础设施的通信和行为。
What is Tetragon? 什么是 Tetragon？
Tetragon is a flexible, low overhead eBPF-based security observability and runtime enforcement agent. Tetragon is Kubernetes-aware and able to detect and react to security-significant events directly in the kernel, monitoring all processes executed in the system, sensitive files opened, egress network connections made and Linux credentials and namespaces that were changed. Tetragon 是一个灵活、低开销的基于 eBPF 的安全可观察性和运行时执行代理。Tetragon 具有 Kubernetes 意识，并能够直接在内核中检测和响应安全重要事件，监视系统中执行的所有进程、打开的敏感文件、出站网络连接以及更改的 Linux 凭据和命名空间。
