On

the top Kubernetes log monitoring tool

Kubernetes 6 metrics that the cluster needs to focus on

how to gracefully limit the number of pods running on a Kubernetes node

what exactly is the difference and connection between containers and pods ？

How to debug network latency issues
in a Kubernetes cluster

Kubernetes architecture that newbies must know

< a href="https://mp.weixin.qq.com/s?__biz=MzIxMTE0ODU5NQ==&mid=2650248445&idx=1&sn=2cce59c7a9b521c897b4d864f2c8ac08&scene=21#wechat_redirect" > how to improve Flink K8s cluster resource utilization?

Directory:

    one target:

  • container operations
  • The four layers of services in two places, three centers

    • and

  • four layers found
  • five kinds of pods to share resources
  • and six CNI commonly used plug-ins
  • Seven-layer load balancing
  • has eight isolation dimensions
  • and nine network model principles

    • One goal: container operations; Three centers in two places; Four-tier service discovery; Five types of pods share resources; Six CNI plugins; Seven-layer load balancing; Eight isolation dimensions; Nine network model principles; Ten categories of IP addresses; 100-level product line; thousand-level physical machine; 10,000-level containers; Like no billion, K8s has 100 million: hundreds of millions of daily service times.

    One goal: Container Operations

    Kubernetes (k8s) is an open-source platform for automating container operations. These container operations include: deployment, scheduling, and inter-node scaling.

    Specific features:

    • automates container deployment and replication.
    • Elastically shrink container size in real time.
    • Containers are grouped and provide load balancing between containers.

    Scheduling: On which machine the container runs.

    Composition

    :

    • kubectl: client command line tool, as the operation entry point of the entire system.
    • kube-apiserver: Provides an interface in the form of a REST API service as a control entry point for the entire system.
    • kube-controller-manager: Executes background tasks of the entire system, including node status, number of pods, and association of pods and services.
    • kube-scheduler: responsible for node resource management, receiving pods tasks from kube-apiserver and assigning them to a node.
    • etcd: Responsible for service discovery and configuration sharing between nodes.
    • kube-proxy: Runs on each compute node and is responsible for the pod network proxy. Periodically obtain service information from etcd to make corresponding strategies.
    • kubelet: runs on each computing node, acts as an agent, receives pods tasks assigned to the node and manages containers, periodically obtains the container status, and feeds back to the kube-apiserver.
    • DNS: An optional DNS service that creates DNS records for each Service object so that all pods can access the service through DNS.

    The following is the architecture topology diagram of K8s:

    > two places and three centers

    The three centers in the two places include local production centers, local disaster recovery centers and off-site disaster recovery centers.

    An important problem to be solved by the three centers in two places is the problem of data consistency. K8S uses the etcd component as a highly available, strongly consistent service to discover storage repositories. Used to configure sharing and service discovery.

    It was spawned as a project inspired by Zookeeper and doozer. In addition to all their features, they also have the following 4 features

    :

    • simple: the http+json-based API makes it easy to use with curl commands.
    • Secure: Optional SSL client authentication mechanism.
    • Fast: Each instance supports 1,000 write operations per second.
    • Trusted: Fully distributed using the Raft algorithm.

    Layer 4 service discovery

    First explain the network layer 7 protocol in a diagram:

    K8S provides two ways to do service discovery :

    Environment variables: When creating a pod, kubelet will inject the relevant environment variables of all services in the cluster into the pod. It should be noted that in order to inject a service’s environment variable into a pod, the service must be created before the pod. This almost makes service discovery unusable in this way.

    For example, if a Service with a ServiceName of redis-master corresponds to ClusterIP:Port of 10.0.0.11:6379, the corresponding environment variable is:

    DNS: You can easily create KubeDNS by cluster add-on to discover services in the cluster.

    The above two ways, one is based on tcp, as we all know, DNS is based on UDP, they are built on the four-layer protocol.

    Five kinds of pods shared resource

    pod

    is the most basic operation unit of K8s, containing one or more closely related containers, a pod can be regarded as a “logical host” of the application layer by a containerized environment; Multiple container applications in a pod are usually tightly coupled, and pods are created, started, or destroyed on the node. Each pod runs a special mounted volume called Volume, so communication and data exchange between them is more efficient, and we can take advantage of this feature when designing to put a set of closely related service processes into the same pod.

    Containers in the same pod can communicate with each other only through localhost.

    Application containers in a pod share five resources

    :

    • PID namespace: different applications in the pod can see the process IDs of other applications.
    • Network namespace: Multiple containers in a pod can access the same IP and port range.
    • IPC namespace: Multiple containers in a pod are able to communicate using SystemV IPC or POSIX message queues.
    • UTS namespace: Multiple containers in a pod share a single hostname.

      • Volumes

    • 😛 individual containers in OD can access Volumes defined at the pod level.

    The lifecycle of a pod is managed through the Replication Controller; It is defined through a template, then assigned to a node to run, and the pod ends after the pod contains the container runs.

    Kubernetes designed a unique network configuration for pods, including: assigning an IP address to each pod, using the pod name as the hostname for communication during the glory period, etc.

    Six CNI

    common plugins

    CNI (Container Network Interface) container network interface, is a set of Linux container network configuration standards and libraries, users need to develop their own container network plug-ins according to these standards and libraries. CNI only focuses on container network connection and resource release when containers are destroyed, and provides a set of frameworks, so CNI can support a large number of different network modes and is easy to implement.

    The following is a diagram showing six commonly used CNI plugins:

    class=”rich_pages wxw-img” src=”https://mmbiz.qpic.cn/mmbiz_jpg/7lxf8j18x2IhgQuaQA5FWFtrIvmurYy1vXKe44WTW7dcX22Ok26odcicjicfXFDw5YArjniasah9mpZhEaHz1cpYg/640?wx_fmt=jpeg”>

    Layer 7 load balancing

    to

    improve load balancing has to mention the communication between servers first.

    IDC (Internet Data Center), also known as data center, computer room, used to place servers. The IDC network is a bridge for communication between servers.

    There are a lot of network devices drawn in the picture above, what are they used for?

    Routers, switches, and MGW/NAT are all network devices, which are divided into different roles according to performance and internal and external networks.

    • intranet access switch: also known as TOR (top of rack), is a server access network device. Each intranet access switch connects 40-48 servers and uses a CIDR block with a mask of /24 as the intranet segment of the server.
    • Intranet core switch: responsible for traffic forwarding and cross-IDC traffic forwarding of each intranet access switch in the IDC.
    • MGW/NAT: MGW is used for load balancing, and NAT is used for address translation when private network devices access the Internet.
    • Extranet core router: Unifies the Internet platform through static interconnection operators or BGP to Meituan.

    Let’s talk about layer load

    balancing first:

      Layer 2 load

    • balancing: Layer 2 load balancing based on MAC addresses.
    • Layer 3 load balancing: Server load balancing based on IP addresses.
    • Layer 4 load balancing: Load balancing based on IP + ports.
    • Layer 7 load balancing: Load balancing based on application layer information such as URLs.

    Here is a diagram to talk about the difference between layer 4 and layer 7 load balancing:

    The above four layers of service discovery mainly talk about the K8s native kube-proxy method. K8s’s exposure of the service is mainly through the NodePort method, by binding to a certain port of the minion host, and then forwarding and load balancing the pod’s requests, but this method has the following defects:

    • there may be many services. If each node host port is bound, the host needs to open the peripheral port for service calls, which is chaotic.
    • You can’t apply many company-required firewall rules.

    The ideal way is to bind a fixed port through an external load balancer, such as 80, and then forward to the subsequent service IP according to the domain name or service name, Nginx solves this requirement very well, but the problem is if there is a service joined, how to modify the configuration of Nginx, and load these configurations? The solution Kubernetes offers is Ingress. This is a 7-tier based scenario.

    Eight isolation dimensions

    class=”rich_pages wxw-img” src=”https://mmbiz.qpic.cn/mmbiz_jpg/7lxf8j18x2IhgQuaQA5FWFtrIvmurYy1gOTXOsnwAPKLxKYqRWUp6oLEDORDGYeUhcgzOhyyljpJJQLjco0Adg/640?wx_fmt=jpeg”>

    On the K8s cluster scheduling side, you need to make corresponding scheduling policies for the isolation from coarse-grained to fine-grained from top to bottom.

    Nine

    network model principles

    The K8s network model should conform to 4 basic principles, 3 network requirements principles, 1 architecture principle, and 1 IP principle.

    Each pod has a separate IP address, and it is assumed that all pods are in a flat, flat network space that can be connected directly, regardless of whether they are running on the same node or not, they can be accessed through the pod’s IP.

    The IP of the pod in K8s is the least granular IP. All containers in the same pod share a network stack, a model called the IP-per-Pod model.

      The

    • IP address and port seen inside

      • the

    • pod actually assigned by docker0

      • are the same as the outside

    • Different containers in the same pod share the network and can access each other’s ports through localhost, similar to different processes in the same VM.

    From the

    perspective of port allocation, domain name resolution, service discovery, load balancing, and application configuration, the IP-per-Pod model can be regarded as an independent VM or physical machine.

    • All containers can communicate with other containers without NAT.
    • All nodes can heartache with all containers in different NAT modes and vice versa.
    • The address of the container is the same as the address that others see.

    To conform to the following schema:

    From the above architecture, the IP concept is derived from the outside of the cluster to the inside of the cluster

    class=”rich_pages wxw-img” src=”https://mmbiz.qpic.cn/mmbiz_jpg/7lxf8j18x2IhgQuaQA5FWFtrIvmurYy1qQEOJKibiahZmBsTUw00OdE41jN7JhX74oMwDpDMoFIEPxG7kdoQlf0g/640?wx_fmt=jpeg”> 

    

     
    

    

    

    

    

    

    

    

    Buy Me A Coffee