Patent ID: 12192051

DETAILED DESCRIPTION

In the following detailed description of the invention, numerous details, examples, and embodiments of the invention are set forth and described. However, it will be clear and apparent to one skilled in the art that the invention is not limited to the embodiments set forth and that the invention may be practiced without some of the specific details and examples discussed.

Some embodiments of the invention provide a system for implementing one or more logical routers using a Kubernetes cluster having multiple nodes, each of which executes a set of pods. The nodes of the Kubernetes cluster, in some embodiments, collectively implement an edge device that processes data messages at a boundary between a logical network and external network. To implement the edge device, the pods in the cluster include a first pod that performs a first set of data message processing operations for multiple logical routers and at least one separate pod for each of the logical routers. Each of these separate pods performs a respective second set of data message processing operations for the corresponding logical router. In some cases, multiple Kubernetes clusters may be used to implement multiple such edge devices.

The multiple nodes of the Kubernetes cluster, in some embodiments, are grouped into various node pools, with each node pool including nodes all having the same configuration (i.e., each node pool includes nodes of a respective type). In some embodiments, changing the configuration of one node in a node pool causes the configuration of all of the other nodes in the pool to change. While having the same configuration, each node in a node pool executes a different set of pods, with varying amounts of resources allocated to each pod. In some embodiments, when a particular pod executing on a node requires more resources than are available on that node, an autoscaling operation is performed in order to provide additional resources for the particular pod or to provide additional instances of the particular pod to distribute the load that the particular pod is responsible for, as will be described further below.

In some embodiments, each of the pods executed by the nodes in the Kubernetes cluster is a virtual machine (VM) and each of the VMs execute on a same host computer. Alternatively, in some embodiments, the VMs are distributed across multiple host computers. For example, in some embodiments, different pools of VMs execute on different host computers, with each pool including VMs having the same configuration. In other embodiments, each node in the cluster is instead a physical machine.

The multiple nodes, in some embodiments, include (i) a master node and (ii) multiple worker nodes on which the pods execute. In some embodiments, the master node is responsible for scheduling the pods on the worker nodes by selecting which pods are to execute on which nodes. For example, in some embodiments, the first pod is scheduled to execute on a first one of the nodes and the separate pods for the individual logical routers are distributed across the multiple nodes (i.e., scheduled across the multiple nodes).

In some embodiments, certain pods are scheduled to certain nodes and cannot be rescheduled to other nodes. The first pod, in some embodiments, is affinitized to the first node (i.e., pinned to the first node so that it is not moved between nodes), while the separate pods for the logical routers are moved between the multiple nodes based on resource usage (or other factors). In some such embodiments, one or more of the separate pods for the multiple logical routers execute on the first node along with the first pod. Other pods that cannot be rescheduled can include, for example, pods having local storages and pods that are not managed by a controller.

FIG.1illustrates an overview of edge architecture100in which a Kubernetes cluster implements an edge device, in some embodiments. As shown, the edge architecture100includes a master node110that is part of the control plane105, and a cluster of worker nodes120,122, and124that each include a respective set of pods. For instance, worker node120includes an L4 pod130and two logical router pods132and134; worker node122includes logical router pods140,142, and144; and worker node124includes logical router pods150,152, and154. The cluster of worker nodes120-124, in some embodiments, collectively implement an edge device that implements multiple logical routers to handle data traffic between a logical network and an external network.

The master node110, in some embodiments, controls and manages the worker nodes120-124of the cluster. In different embodiments, a cluster can include one master node or multiple master nodes, depending on the size of the cluster deployment. When multiple master nodes are included for a large cluster, these master nodes provide high-availability solutions for the cluster.

Each master node110, in some embodiments, includes various components, such as an application programming interface (API) server used for communications to elements external to the cluster, a set of controllers to run the cluster, a state database for the cluster (e.g., Etcd), and a scheduler for scheduling pods across the worker nodes and for scheduling functionalities for worker nodes in the cluster. In some embodiments, the master node110executes on a first server, while the worker nodes managed by the master node110execute on one or more other servers.

In some embodiments, the L4 pod130executes a set of run-to-completion threads for processing data messages sent between the logical network and the external network and a set of control threads for handling control plane operations. Each respective run-to-completion thread executed by the L4 pod130, in some embodiments, is assigned to a respective core of a first set of cores of a computing device on which the L4 pod130executes, while the set of control threads are scheduled between a second set of cores of the computing device.

Half of the available cores, in some embodiments, are allocated to these run-to-completion data message processing threads by default, with each such thread pinned to one core. All other processes and threads, in some such embodiments, are scheduled between the remaining available cores. In some embodiments, the edge appliance supports a dynamic split of CPUs between the two categories of threads, based on the actual CPU usage of both categories, and makes adjustments if needed.

In some embodiments, due to the threads executed by the L4 pod130being assigned to the cores of the computing device, the L4 pod130is affinitized to the worker node120. Pods that are affinitized to a particular node, in some embodiments, cannot be rescheduled (i.e., moved) to a different node in the cluster. In some embodiments, other pods, such as pods that have local storage on the node and pods that are not managed by the controller, also cannot be rescheduled to other nodes. Unlike the L4 pod130, the LR pods132-154(i.e., where L5-7 services are implemented) can be scheduled on any available nodes in the node pool, in some embodiments.

Apart from the L4 pod130, each pod executing on the worker nodes120-124performs a set of data message processing operations for its respective logical router. These data message processing operations, in some embodiments, include L5-L7 service operations such as L7 firewall services, transport layer security (TLS) services, L7 load balancing services, uniform resource locator (URL) filtering, and domain name service (DNS) forwarding. In some embodiments, each service operation configured for at least one logical router is assigned to a separate pod (i.e., rather than the logical router being a single pod that collectively performs each of the service operations).

Some embodiments use exactly one pod per logical router, e.g., each logical router has a dedicated pod, which may require multiple service daemons running in each pod (i.e., each service daemon performing a service configured for the logical router). Different daemons can be put into separate containers or the same one, according to some embodiments. Alternatively, some embodiments use one pod per network function or service of a logical router (e.g., each service daemon running in the same pod). This arrangement works well for, e.g., an Autonomous Edge that includes a single logical router with many services. A logical router may be defined to include a distributed router and one or more centralized routers. In some embodiments, the separate pods for such a logical router includes a pod for the distributed router and additional pods for each of the centralized routers.

The L4 pod130, in some embodiments, performs its own respective set of data message processing operations for all of the logical routers. The set of data processing operations performed by the L4 pod130for multiple logical routers, in some embodiments, includes layer 2-layer 4 (L2-L4) operations, such as L2/L3 lookups, tunnel termination/encapsulation, firewall processing, packet updating, and byte counters. This can include logical network processing, such as logical switching and logical routing operations.

Instead of having a single L4 pod on one worker node, some embodiments instantiate an L4 pod on each worker node.FIG.2illustrates an overview of such an edge architecture200in which a Kubernetes cluster, that includes one L4 pod per worker node, implements an edge device, in some embodiments. The Kubernetes cluster in the edge architecture200includes a master node210that runs on the control plane205, and a set of worker nodes220,222, and224.

Unlike in the edge architecture100, the L4 pod in the edge architecture200is implemented as a daemonset on all worker nodes. For example, each worker node220-224in the edge architecture200executes a respective L4 pod230,240, and250. Additionally, the worker node220executes logical router pods232and234, the worker node222executes logical router pods242and244, and the worker node224executes logical router pods252and254. In some embodiments, the edge architecture200uses a high-availability implementation (e.g., all of the L4 pods can be active with ECMP routing supported or one of the L4 pods is designated as active and the other L4 pods are designated as standby).

With this discussion of the architecture of a Kubernetes-based edge device introduced, the basic operation of an edge device (also referred to as an edge node) will now be described. A typical edge node implemented either as a bare metal device or VM has fixed CPU and memory. In some embodiments, hugepage memory that includes half of the total available memory is reserved by default for data message processing user space drives, configuration databases, connection tables, etc. Each edge node, in some embodiments, can run multiple components belonging to a variety of component classes.

The components classes, in some embodiments, include fastpath, control path, services, and other. The fastpath class includes poll mode driver (PMD) threads that are pinned to specific processing cores and run the service pipeline in a run-to-completion model. In some embodiments, the services that are implemented this way include switching, routing, network address translation (NAT), firewall, IPsec (internet protocol security), etc. The control path class, in some embodiments, includes datapath slowpath (e.g., address replacement protocol (ARP) learning, MAC learning, bi-directional forwarding detection (BFD) finite state machine (FSM), firewall purging, etc.), dynamic routing daemon (e.g., free-range routing (FRR)) and its related components (e.g., RCPM, etc.), internet key exchange (iked) (e.g., IPsec control plane), edge-agent, ops-agent, nsx-proxy, nestdb, exporter, aggsvc, nsd, etc. The services class of some embodiments includes services such as L7 firewall, TLS proxy, L7 LB, URL filtering, DNS forwarder, etc. Lastly, the remaining class includes KNI kthread, HCX, etc., in some embodiments.

In some embodiments, the max packet rate that can be achieved with stateful L4 services, such as firewall and NAT, is 1M PPS per core. To handle 100 Gbps throughout with 64 B bytes, some embodiments require the edge node to handle 148M PPS. For pure routing, this would require approximately 24 cores (i.e., assuming linear scalability), and for stateful firewall, this would require 148 cores.

Data message processing at the edge includes multiple steps in some embodiments, such as L2/L3 lookups, tunnel termination/encapsulation, firewall processing, updating the data message and byte counters, as mentioned above. In many cases, the data message processing includes various combinations of these steps, with each step having the potential to be repeated several times. As such, every data message processed translates to a large number of memory accesses as various data structures (e.g., routing tables, configurations, etc.) are consulted to determine forwarding decisions. To alleviate potential issues caused by the large number of memory accesses (e.g., exceeding the capacity of CPU L1/L2/L3 caches, low CPU cache hit rates, low performance, etc.), the edge datapath in some embodiments implements a flow cache.

In some embodiments, the processing of a received data message by the datapath message processing thread is reduced to a match-action table lookup, as well as data message modification based on the actions recorded. Anything that can be accelerated by the pNIC/vNIC, in some embodiments, will be offloaded, such as stateless checksum, Large Segmentation Offload (LSO), Large Receive Offload (LRO), Receive Side Scaling (RSS), VLAN insertion/stripping, flow director, and UDP Segmentation Offload (USO).

Kernel NIC Interfaces (KNI) allow DPDK applications (e.g., the edge datapath) to pass data messages between user space applications and a kernel network stack. KNIs are faster than TUN/TAP interfaces by eliminating system calls and copy to/from user operations (i.e., they are zero-copy). These KNIs were originally designed for control path communications but have been repurposed to carry datapath messages (e.g., L2 Ethernet frames). For these datapath messages, multi-queue support, VLAN and checksum offload, and other optimizations can be added. However, performance (e.g., HTTP throughput) using KNIs is still limited in some cases. In addition, the use of KNIs involves various efforts to keep an out-of-tree kernel module up with regularly changing API/ABI. To support various hardware offload operations (e.g., VLAN, checksum, TLS offload, etc.), extra metadata tagged at the PNIC is carried with the data messages.

In order to scale out software networking appliances, some embodiments support a multi-node cluster. Rather than an edge device being a fixed appliance with fixed capabilities, the functions are broken down into micro-services. This multi-node cluster, in some embodiments, allows services to be scaled out across nodes. In some embodiments, active/active mode is supported for stateful services to improve the service scale-out performance. For instance, a cluster of Kubernetes edge nodes can be formed to offer such redundancy and scaling and can also use nodes provisioned for a conventional cluster. In some embodiments, this requires the networking fabric to support equal-cost multipath (ECMP) routing.

FIG.3illustrates an architectural overview of a control plane300of some embodiments. The control plane300includes API(s)305, management plane310, central control plane315, Kubernetes control plane320, and a local control plane edge agent330executing on an L4 pod340. The API(s)305, in some embodiments, enable an administrator to communicate with the management plane310to configure the edge (as well as other logical network aspects). As shown, the L4 pod340includes DPDK PMD threads342(i.e., fastpath data message processing threads), slowpath threads and processes344, and a database (e.g., nestDB)346. The DPDK PMD threads342are connected to a physical OVS bridge350that connects to a NIC354, as well as to an inter-pod OVS bridge352. In other embodiments, the DPDK PMD threads342connect directly to the NIC354(i.e., without an OVS bridge or any other virtual switch in between).

The management plane310, in some embodiments, receives and accepts requests (e.g., resource and object requests from an administrator) via the API(s)305. As shown, the management plane310connects to both the Kubernetes control plane320and the L4 pod340. The management plane310is responsible for managing pod lifecycles directly over the Kubernetes control plane320.

The central control plane315also connects to the L4 pod340over the Kubernetes control plane320. In some embodiments, the central control plane315is responsible for configuring logical networks and forwarding rules over the local control plane edge agent330. In some embodiments, the central control plane315is configured to perform other configurations over the local control plane in addition to logical network and forwarding rule configurations. For example, in some embodiments, configurations for the L4 pod340are pushed down from the central control plane315to the nestDB346for use by the local control plane agent330.

The Kubernetes control plane320includes a cluster state database325(e.g., etcd). While not shown, the Kubernetes control plane320in some embodiments includes an API server. This API server, in some embodiments, is a frontend to a Kubernetes cluster for all communications external to the cluster. In some embodiments, when a logical router is added to or deleted from the cluster, a corresponding logical router pod is also created or deleted. Additionally, when such logical routers are added or deleted, inter-pod bridges, such as the OVS bridge352of the L4 pod340, are created or removed.

Kubernetes support Cluster Autoscaler, Vertical Pod Autoscaler (VPA), and Horizontal Pod Autoscaler (HPA). In some embodiments, the Cluster Autoscaler allows one edge node to scale seamlessly to more nodes when demands for more CPU come and go, according to some embodiments. The VPA, in some embodiments, is a set of one or more vertical autoscaler pods that run as control plane processes. In some embodiments, the VPA provides recommendations for values and limits on resource requests and determines whether to modify the amount of resources available for another pod. These recommendations, in some embodiments, are based on size constraints (i.e., minimum and maximum size constraints) specified for a node pool. In some embodiments, a separate HPA exists on the control plane for each logical router, with each HPA performing autoscaling operations to increase or decrease the number of pods for its respective logical router based on a variety of metrics. These metrics, in some embodiments, include actual resource usage, custom metrics reported by other Kubernetes objects in the cluster, and external metrics from applications and services external to the cluster.

Additionally, logical router pods could also benefit from VPA and (if none of the services configured for the logical router are stateful services) HPA. When stateful services are enabled, the use of HPA to increase the number of pods for a logical router may be difficult because many stateful services require traffic from both directions to arrive on the same pod.

FIG.4illustrates a process400for using cluster autoscaling to increase the number of nodes in a node pool, in some embodiments. The process400is performed, in some embodiments, by a cluster autoscaler that runs in the master node (i.e., as part of the control plane for the edge device). As mentioned above, Kubernetes supports cluster autoscaler, VPA, and HPA. In some embodiments, the cluster autoscaler of the process400works in conjunction with a VPA and/or an HPA during its autoscaling operations.

The process400starts by monitoring (at410) resource usage on each node in the node pool. Rather than monitor resource usage by monitoring resource utilization, the cluster autoscaler monitors resource usage according to resource requests by pods executing on the nodes. In the edge architecture100, for instance, a cluster autoscaler (not shown) running on the master node110would monitor resource usage by the pods130-134,140-144, and150-154on each of the worker nodes120-124, respectively. The resources for which the cluster autoscaler monitors the resource requests for include at least CPU and memory in some embodiments.

The process determines (at420) that a particular node in the node pool has insufficient resources for its respective set of pods to adequately perform their respective sets of data message processing operations. The cluster autoscaler makes this determination, in some embodiments, based on determining that one or more pods have reached a specified limit of resource requests. Any number of the pods operating on a node may be affected by the insufficient amount of resources (e.g., all of the pods, a single pod, or a subset of the pods). For instance, a particular logical router pod may experience resource shortages due to a large influx of data traffic.

The process identifies (at430) one or more pods executing on the particular node that can be rescheduled to other nodes in the node pool. As described above, pods that are affinitized to the node, pods that have local storage on the node, and pods that are not managed by the controller, in some embodiments, cannot be rescheduled to other nodes. For instance, the L4 pod130that executes on the worker node120is affinitized to that worker node and would not generally be available to be rescheduled to a different node.

The process determines (at440) whether any nodes in the node pool have sufficient available resources to support one or more of the pods from the particular node. In some embodiments, for example, one or more nodes in the node pool may have available resources that have not been allocated to any pods yet, or may have underutilized resources that can be reallocated to new or rescheduled pods. When the process determines (at440) that there are nodes in the node pool that have sufficient resources to support one or more of the pods from the particular node, the process reschedules (at450) the identified pods to the nodes determined to have sufficient available resources. The process400then ends.

Otherwise, when the process determines (at440) that there are not any nodes in the node pool with sufficient available resources to support one or more of the pods from the particular node, the process determines (at460) a number of additional nodes to instantiate in the node pool based on resource requirements of the identified pods as well as size constraints (i.e., a maximum number of nodes) specified for the node pool. In some embodiments, when configuring the node pool and the cluster autoscaler, a user (e.g., network administrator) specifies a minimum size for the node pool and a maximum size for the node pool, and the cluster autoscaler automatically adjusts the number of nodes in the node pool as needed within the constraints of the specified minimum and maximum.

The process instantiates (at470) the determined number of additional nodes in the node pool. As mentioned above, each node in node pool has the same configuration, and, as such, the additional nodes instantiated in the node pool will also have the same configuration as each other existing node in the pool.FIG.5illustrates an updated overview500of the edge architecture described above forFIG.1. As shown, the updated architecture500includes an additional worker node526.

The process then reschedules (at480) the identified pods to the newly instantiated nodes. In the updated edge architecture500, for example, the logical router pod154has been removed from the worker node124and rescheduled to the newly instantiated worker node526. In some embodiments, after the pod or pods have been rescheduled, a VPA performs an additional autoscaling operation to increase the amount of resources allocated to the remaining pods on the worker node124and/or to increase the amount of resources allocated to the rescheduled pod154on the new node526. Following480, the process400ends.

FIG.6illustrates a process600for using cluster autoscaling to decrease the number of nodes in a node pool, in some embodiments. Like the process400, the process600is performed, in some embodiments, by a cluster autoscaler that runs in the master node (i.e., as part of the control plane). The process600starts by monitoring (at610) resource usage on each node in the node pool. As described above, the cluster autoscaler monitors resource usage by monitoring the number of resource requests made by pods on each of the nodes in the node pool.

The process determines (at620) that a particular node's resources are being underutilized by the pods executing on the particular node. In some embodiments, this determination is based on a lack of resource requests by the pods executing on the particular node. For instance, when data traffic processing is distributed across multiple logical router pods that are distributed across multiple nodes in a pool, each logical router pod may have a small amount of data traffic to process when there are a lot of other logical router pods. As a result, some nodes may have underutilized resources, and consolidating the pods to fewer nodes would result in more efficient use of the nodes' resources.

The process determines (at630) whether all of the pods executing on the particular node can be rescheduled. Pods that are affinitized to a particular node, pods that have local storage on a particular node, and pods that are not managed by the controller, in some embodiments, are pods that cannot be rescheduled to other nodes. For instance, in the edge architecture100, the L4 pod130is affinitized to the worker node120and cannot be rescheduled. As a result, the worker node120cannot be deleted without deleting the L4 pod130. When the process600determines (at630) that one or more pods executing on the particular node cannot be rescheduled, the process600ends.

Alternatively, when the process determines (at630) that all pods executing on the particular node can be rescheduled, the process determines (at640) whether there are any other nodes in the node pool that have sufficient available resources to adequately support the data messaging processing operations of the pods from the particular node. For instance, in some embodiments, the process600may determine at620that more than one particular node's resources are being underutilized, and thus the pods that execute on each of the nodes can be consolidated on fewer nodes. When there are no other nodes in the pool that have sufficient available resources, the process600ends.

Alternatively, when the process determines (at640) that there are available nodes in the pool that have sufficient available resources, the process transitions to650to reschedule the pods executing on the particular node to the other nodes determined to have sufficient available resources to support the data message processing operations of the particular node's pods.

FIG.7, for example, illustrates an updated overview700of the edge architecture described above forFIG.1. The logical router pods150-154previously executed on the worker node124in the edge architecture100and have been rescheduled across the other nodes. As shown, updated worker node720now executes L4 pod130, and logical router pods132,134,150, and152, while updated worker node722now executes logical router pods140,142,144, and154.

Returning to the process600, the process then deletes (at660) the particular node from the node pool. For instance, the worker node724in the edge architecture700no longer executes any pods and is illustrated with a dashed outline to indicate its deletion from the cluster. Following the deletion of the particular node at660, the process600ends.

In Kubernetes, networking functions are often delegated to network plugins referred to as container network interfaces (CNIs). These CNIs often at least implement a standard CNI API to provide basic networking to containers. Typically, multiple networking interfaces are used for a network function virtualization (NFV) or software-defined networking (SDN) virtual appliance, such as the edge appliance, for separation of control between management and data planes (e.g., multi-home pod, etc.) By default, a pod has only one network interface apart from a loopback interface. Some networking use cases may require multiple interfaces connected to each pod, or interfaces of different types. Multus-CNI is one example of such a CNI that allows multiple network interfaces in one pod. Multus-CNI is a meta CNI plugin that uses other CNI plugins for network attachment definitions. Other than the management NIC (often referred to as eth0), multus-CNI is commonly used to create high performance interfaces for pods performing networking functions to support more complex networking usages.

FIG.8illustrates a simplified example of a multi-homed pod that uses multus-CNI, according to some embodiments. As shown, this figure includes a pod820that has an interface822to a control plane810, an interface824to a first external network830, and an interface826to a second external network832. The pod820, in some embodiments, can be an L4 pod or a logical router pod. In some embodiments, multiple pods each having more than one interface to external networks may execute on the same worker node.

The Kubernetes single root I/O virtualization (SR-IOV) network device plug-in, in some embodiments, extends the capabilities of Kubernetes to address high-performance network I/O by first discovering, and then advertising, SR-IOV network virtual functions (VFs) on a Kubernetes host.FIG.9illustrates an embedded switch on the NIC being used to accelerate inter-pod networking, in some embodiments. This approach allows close-to-line rate processing by the NIC, according to some embodiments.

This figure shows a worker node905and an SR-IOV NIC940. The worker node905executes a kubelet910, a multus CNI920, an SR-IOV CNI925, an L4 pod930, and a logical router pod935. Additionally, the kubelet910includes a CNI915, while each of the pods930-935include respective network interfaces, and interfaces to the control plane. The SR-IOV NIC940includes an embedded switch945on which a set of VFs run.

The SR-IOV CNI925allows a single physical PCI resource to be partitioned into virtual PCI functions for allocation to application and network function pods, in some embodiments. As illustrated by the dashed lines between the network interfaces of the pods930-935and the VFs of the embedded switch945, the SR-IOV CNI925enables the pods to be directly attached to VFs by using a standard driver in the kernel of a container's host.

The SR-IOV CNI925, in some embodiments, works with both VFs in kernel drivers as well as in DPDK drivers. In some embodiments, the SR-IOV CNI925can be used to configure VFs in the pods930-935when the VFs are attached to kernel drivers, whereas a virtual network function (VNF) application is responsible for configuring the VFs when the VFs are attached using DPDK drivers, according to some embodiments.

In some embodiments, a split datapath is implemented for edge appliances that allows L2-L4 services to be offloaded to a Smart NIC (i.e., when one is available) as a new fastpath (e.g., offloaded by the L4 pod or with the smart NIC replacing the L4 pod), while also enabling more elastic and scalable L7 services. By offloading L2-L4 services to a Smart NIC, some embodiments achieve better performance and save on CPU costs. In some embodiments, a new memif (shared memory packet interface) device is implemented in open vSwitch (OVS). This new memif device allows for leveraging of a high-performance virtual interface (VIF) for inter-pod networking, while replacing the kernel network interface (KNI).

FIG.10illustrates the use of memif on a Kubernetes edge appliance for inter-pod networking between pods on the same worker node, in some embodiments. This figure shows a worker node1005and an Open vSwitch (OVS)1040. The worker node1005executes a kubelet1010, a multus CNI1020, an OVS VNI1025, an L4 pod with a DPDK driver1030, and a logical router pod1035. The kubelet1010also includes a CNI1015, while each of the pods1030-1035include respective network interfaces and control plane interfaces. The OVS1040includes a pair of memifs1045.

Each pod1030-1035executing on the worker node1005is connected to OVS1040via a respective memif1045, as illustrated. The memif devices1045allow containers running in the pods1030-1035to be connected together at the speed of the memory bus, according to some embodiments. By default, each pod in a Kubernetes edge appliance is connected to an OVS via one memif interface in some embodiments.

In some embodiments, the memifs1045use libmemif from vector packet processing (VPP) and can be leveraged (e.g., with Ligato) to enable rapid development of high-performance cloud-native VNFs. A new network type, netdev-memif, is implemented on top of OVS, in some embodiments, to support the memif device. In some embodiments, performance with a memif device can achieve higher than 10M PPS.

FIG.11illustrates the use of memif on a Kubernetes edge appliance for inter-pod networking between pods on different worker nodes, in some embodiments. This figure shows worker nodes1105aand1105b, and an OVS1140for facilitating inter-pod networking for pods running on each of the worker nodes1105aand1105b. While the switch is illustrated as an OVS, which provides high performance for DPDK mode, some embodiments use other switches that offer similar functionality, such as a Linux bridge.

Each of the worker nodes1105aand1105bincludes a respective kubelet1110aand1110beach having a respective CNI1115aand1115b, a respective multus CNI1120aand1120b, and a respective OVS CNI1125aand1125b. Additionally, the worker node1105aexecutes an L4 pod1130with DPDK, and the worker node1105bexecutes a logical router pod1135. The L4 pod1130executing on the worker node1105aand the logical router pod1135executing on the worker node1105bconnects to a respective memif1145of the OVS1140, as shown.

In some embodiments, resource plugins other than OVS CNI are utilized for tunnel interfaces for worker nodes, and forwarding rules can be programmed accordingly. For example, a forwarding rule for inter-pod forwarding between the L4 pod1130and the logical router pod1135on the worker nodes1105aand1105bmay specify a tunnel port for forwarding to a destination MAC address. Conversely, a forwarding rule for the L4 pod1030and the logical router pod1035on the worker node1005may specify a port for forwarding to a destination MAC address. These forwarding rules are pushed down from a network controller, in some embodiments.

A destination MAC address is assigned to an interface of the logical router pod1135that's attached to the OVS1140based on assignments provided by a network manager (e.g., NSX-T manager), in some embodiments. Such network interfaces, in some embodiments, are not visible to the pod network and are only used for inner-pod networking controlled by the network manager.

While the pure software approach with memif may not be as performant as the SR-IOV based approach in some cases, in an environment where device passthrough is not possible (e.g., such as some cloud environments), it provides a viable solution. However, in some embodiments, the socket application needs to be rewritten to call into the APIs provided by the device (a common tradeoff faced by such high-performance networking software solutions). When applied to an autonomous edge, it can be made to work with any Kubernetes environment with improved availability and scalability.

Smart NICs, in some embodiments, can be utilized for offloading and accelerating a range of networking datapath functions from the host CPU. These Smart NICs also offer more programmable network processing features and intelligence compared to a traditional NIC, according to some embodiments. Some common datapath functions supported by Smart NIC include multiple match-action processing, tunnel termination and origination, etc. The match-action table works very similarly with flow cache and can be offloaded with relatively small efforts, in some embodiments.

In some embodiments, it is relatively easy to achieve inter-pod networking with VF approach as the NIC already supports switching between VFs on the same embedded switch. For the memif approach, in some embodiments, OVS can be offloaded to a NIC ASIC's (e.g., CX6) match-action table for some relatively simple pipelines. In some such embodiments, the last action is to send to a destination memif on the same switch. Since the hardware does not understand memif, CPU needs to be involved to continue the remaining work to copy packets to the destination memif, in some embodiments. This work, in some embodiments, can be further offloaded to the NIC if it supports virtio datapath acceleration (vDPA). For example, vDPA offloads virtio datapath to the NIC and offers line-speed networking performance.

In some embodiments, virtio is implemented on hosts as an alternative to both KNI and memifFIG.12, for example, illustrates a virtio implementation on a host1200of some embodiments. As shown, the host1200includes hardware (HW) blocks1205, a host kernel space1210, and a host user space1215. The host user space1215includes an OVS DPDK1220and a virtual machine monitor (VMM)1230. In some embodiments, the VMM1230is a generic, open source machine emulator such as QEMU.

The OVS DPDK1220enables usage of a DPDK-optimized virtual host (vhost) path in OVS. For example, the OVS switchd1222runs on the OVS DPDK1220and includes an OVS forwarding plane1224. The OVS switchd1222also includes a DPDK port1226athat connects to a PNIC1240of the HW blocks1205, and a DPDK port1226bto which a vhost-user interface1228is attached. The vhost-user interface1228, in some embodiments, is a module that includes additional APIs inside the DPDK library.

The VMM1230executes a guest machine1232that includes a user space1234aand a kernel space1234b. A virtio-PMD interface1236executes within the user space1234aof the guest machine1232, as shown. The virtio-PMD interface1236is a poll mode driver that consumes dedicated cores and performs polling without any interruptions, according to some embodiments.

Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as computer-readable medium). When these instructions are executed by one or more processing unit(s) (e.g., one or more processors, cores of processors, or other processing units), they cause the processing unit(s) to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs, flash drives, RAM chips, hard drives, EPROMs, etc. The computer-readable media does not include carrier waves and electronic signals passing wirelessly or over wired connections.

In this specification, the term “software” is meant to include firmware residing in read-only memory or applications stored in magnetic storage, which can be read into memory for processing by a processor. Also, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while remaining distinct software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that together implement a software invention described here is within the scope of the invention. In some embodiments, the software programs, when installed to operate on one or more electronic systems, define one or more specific machine implementations that execute and perform the operations of the software programs.

FIG.13conceptually illustrates a computer system1300with which some embodiments of the invention are implemented. The computer system1300can be used to implement any of the above-described hosts, controllers, gateway, and edge forwarding elements. As such, it can be used to execute any of the above described processes. This computer system1300includes various types of non-transitory machine-readable media and interfaces for various other types of machine-readable media. Computer system1300includes a bus1305, processing unit(s)1310, a system memory1325, a read-only memory1330, a permanent storage device1335, input devices1340, and output devices1345.

The bus1305collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the computer system1300. For instance, the bus1305communicatively connects the processing unit(s)1310with the read-only memory1330, the system memory1325, and the permanent storage device1335.

From these various memory units, the processing unit(s)1310retrieve instructions to execute and data to process in order to execute the processes of the invention. The processing unit(s)1310may be a single processor or a multi-core processor in different embodiments. The read-only-memory (ROM)1330stores static data and instructions that are needed by the processing unit(s)1310and other modules of the computer system1300. The permanent storage device1335, on the other hand, is a read-and-write memory device. This device1335is a non-volatile memory unit that stores instructions and data even when the computer system1300is off. Some embodiments of the invention use a mass-storage device (such as a magnetic or optical disk and its corresponding disk drive) as the permanent storage device1335.

Other embodiments use a removable storage device (such as a floppy disk, flash drive, etc.) as the permanent storage device. Like the permanent storage device1335, the system memory1325is a read-and-write memory device. However, unlike storage device1335, the system memory1325is a volatile read-and-write memory, such as random access memory. The system memory1325stores some of the instructions and data that the processor needs at runtime. In some embodiments, the invention's processes are stored in the system memory1325, the permanent storage device1335, and/or the read-only memory1330. From these various memory units, the processing unit(s)1310retrieve instructions to execute and data to process in order to execute the processes of some embodiments.

The bus1305also connects to the input and output devices1340and1345. The input devices1340enable the user to communicate information and select commands to the computer system1300. The input devices1340include alphanumeric keyboards and pointing devices (also called “cursor control devices”). The output devices1345display images generated by the computer system1300. The output devices1345include printers and display devices, such as cathode ray tubes (CRT) or liquid crystal displays (LCD). Some embodiments include devices such as touchscreens that function as both input and output devices1340and1345.

Finally, as shown inFIG.13, bus1305also couples computer system1300to a network1365through a network adapter (not shown). In this manner, the computer1300can be a part of a network of computers (such as a local area network (“LAN”), a wide area network (“WAN”), or an Intranet), or a network of networks (such as the Internet). Any or all components of computer system1300may be used in conjunction with the invention.

Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a machine-readable or computer-readable medium (alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROM), recordable compact discs (CD-R), rewritable compact discs (CD-RW), read-only digital versatile discs (e.g., DVD-ROM, dual-layer DVD-ROM), a variety of recordable/rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic and/or solid state hard drives, read-only and recordable Blu-Ray® discs, ultra-density optical discs, any other optical or magnetic media, and floppy disks. The computer-readable media may store a computer program that is executable by at least one processing unit and includes sets of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.

While the above discussion primarily refers to microprocessor or multi-core processors that execute software, some embodiments are performed by one or more integrated circuits, such as application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself.

As used in this specification, the terms “computer”, “server”, “processor”, and “memory” all refer to electronic or other technological devices. These terms exclude people or groups of people. For the purposes of the specification, the terms “display” or “displaying” mean displaying on an electronic device. As used in this specification, the terms “computer-readable medium,” “computer-readable media,” and “machine-readable medium” are entirely restricted to tangible, physical objects that store information in a form that is readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral or transitory signals.

While the invention has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the invention can be embodied in other specific forms without departing from the spirit of the invention. Thus, one of ordinary skill in the art would understand that the invention is not to be limited by the foregoing illustrative details, but rather is to be defined by the appended claims.