Network plugin for multiple network interfaces

A new host is detected being added to a network cluster, wherein each of a plurality of hosts are on the network cluster. Available interfaces on each of the plurality of hosts on the network cluster are detected responsive to detecting the new host being added. A classless inter-domain routing (CIDR) range is calculated for hosts and interfaces on the network cluster using the available interfaces. Pod routes with interface range and L3 host routes are set for each host.

BACKGROUND

Container networking, hereinafter interchangeably referred to as pod networking, is a form of virtualization where applications are handled as discrete items, where these items are referred to as containers or pods. These containers/pods are similar to virtual machines in some ways, where these pods are largely used in situations where they can be used to run large applications in a distributed manner.

SUMMARY

Aspects of the present disclosure relate to a method, system, and computer program product relating to a network plugin for multiple network interfaces with automated L3 configuration. For example, the method includes detecting a new host being added to a network cluster, wherein each of a plurality of hosts are on the network cluster. The method also includes detecting available interfaces on each of the plurality of hosts on the network cluster responsive to detecting the new host being added. The method also includes calculating, using the available interfaces, classless inter-domain routing (CIDR) range for hosts and interfaces on the network cluster. The method also includes setting pod routes with interface range and L3 host routes for each host. A system and computer program configured to execute the method described above are also described herein.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to container networking, while more particular aspects of the present disclosure relate to a network plugin for multiple network interfaces with automated L3 configuration. While the present disclosure is not necessarily limited to such applications, various aspects of the disclosure may be appreciated through a discussion of various examples using this context.

Container networking, hereinafter interchangeably referred to as pod networking (e.g., where “containers” are interchangeably referred to as “pods”), is an increasingly common technique. When pod networking is used, there are a few ways to get pods to communicate with each other, such as enabling communication between the pods via tunnelling (e.g., encapsulation). This is common, but often requires high overhead, perhaps cutting potential bandwidth in half (if not an even greater loss of bandwidth). Alternatively, MacVLAN and IPvlan are drivers that can be used to expose container packets to underlaying host networks using the pod internet protocol (IP) without encapsulation. However, using drivers such as MacVLAN and IPvlan requires that packets be later routed by L2 or L3 network layers. Packets on L2 network layers can dynamically be routed by address resolution protocol (ARP) traffic, but virtualized environments, or virtual private clouds (VPCs) typically do not support/allow L2 traffic. Similarly, within conventional systems L3 traffic is not dynamically routable, but rather it requires explicit neighbor entries to be known/static.

Another common technique in this realm is the use of multiple network interface cards (NICs). High-performing computing (HPC) applications and/or artificial intelligence (AI) applications use multiple NICs to increase performance by spreading the execution of high-demand jobs across a significant number of computing nodes on a massive scale. Many conventional cloud computing offerings support multiple NICs for virtual machine (VM) instances in their VPC using a network plugin like Multus. However, it can become cumbersome to handle cluster-wide IP address management (IPAM) for a single network. For example, conflicts of subnets between multiple NICs must be manually analyzed and solved in conventional systems. For another example, IPAM produces per-pod, per-interface, per-host L3 routes on each host route table, such that L3 routes must be updated on all worker nodes every time a new pod is created and/or deleted (e.g., given that L3 communication is not dynamic as discussed above). As such, when any given worker node joins and/or leaves a cluster, all L3 routes need to be analyzed/updated accordingly.

Therefore, as would be understood by one of ordinary skill in the art, conventional interface-creation plugins rely on fixed interface names, where these names are often different on different hosts for a single network, especially in VPC environments. Accordingly, pod annotations are often limited to pre-scheduled hosts, even when cluster-wide IPAM is utilized, hampering intra-cluster communication efforts.

Aspects of this disclosure improve or address technical shortcomings of conventional systems in enabling networking interface autodetection and dynamic configuration. For example, aspects of this disclosure relate to a network plugin for multiple network interfaces with automatic L3 configuration. One or more computing devices that include or otherwise utilize one or more processing units executing instructions stored on one or more memories may provide the functionality that addresses these problems, where said computing device(s) are herein referred to as a controller.

For example, the controller may enable networking interface autodetection by automatically detecting available interfaces on each host whenever a new host is added to a networking cluster. The controller may use network address as a key to identify the subnet, and therein set a subnet index for the network. The controller may then automatically determine the non-conflict classless inter-domain routing (CIDR) range of each host and of each interface once the plugin definition is defined. The controller may determine the CIDR range using user-defined factors, such as global subnet, master network addresses, nodeBlock, interfaceBlock, or the like. The controller may then automatically set pod routes that are within the determined interface range and set L3 host routes (next hop via dev) at each host. From here, the next time that the plugin is called at pod creation, the controller may automatically create directly-IP-exposing interfaces (e.g., IPVLAN) and assign available IPs for each interface from the computed CIDR range of corresponding host. At this point the controller may repeat these steps as necessary each time that a new host is added. Further, when a pod is deleted from the cluster, the controller may detect such deletion and recycle the IP address of the deleted pod. Similarly, the controller may be configured to detect when a host is removed from a cluster and recycle/update the host index to reflect this removal.

In this way, aspects of this disclosure may dynamically detect changes on hosts and interfaces and determine non-conflict IP address range (CIDR) for multiple hosts and multiple interfaces by inputting a single global subnet, list of network addresses, interface block size (bits) and host block size (bits), to therein integrate IPAM-computed CIDR data for dynamically configuring L3 routes on host table to route pod packets across different hosts without network address translations (NATs) and overlay networks. Therefore, as would be understood by one of ordinary skill in the art, aspects of this disclosure may improve functionality of container networking while reducing overhead.

For example,FIG.1depicts an example CNI data flowchart100whereby aspects of this disclosure may automatically and dynamically detect changes of hosts and interfaces and update network settings accordingly. As depicted inFIG.1, orchestrator storage can be used to keep data synchronized by defining new custom resources, where inFIG.1these new custom resources include host interface data112, CIDR computation data114, and IP assignment data116. Data flowchart100is in a depicted system with a CNI controller110(IPAM service), CNI daemon118, and a CNI binary120(e.g., where both the CNI daemon and CNI binary120are on a host VM108).

Flowchart100begins with the configuration being defined. For example, at multus-CR102, one or more multiple NIC CNI types may be defined (e.g., defined by a user). This includes definitions for the master NetAddress list, the identified subnet, nodeBlock, interfaceBlock, and the like. From here controller110observes multus-CR102as/until these configurations are all defined across the network.

Once these configuration variables are defined, CNI controller110gathers interface information from CNI daemon118. For example, first controller110queries CNI daemon about the interfaces, in response to which CNI daemon118determines a list of all available interfaces and sends this list back to CNI controller110. Controller110stores this list in one of the new custom resources, where inFIG.1this is host interface data112.

At this point CNI controller110computes CIDR. For example, CNI controller110may receive as an input the subnet, startlP, nodelndex, interfacelndex, nodeBlock, interfaceBlock, or the like. From these, controller110outputs a unique ordered CIDR for all pods, hosts, and interfaces, along with a CIDR for IPvlan. For example, controller110may receive the following inputs: subnet=172.16.0.0/12, startIP=172.22.0.1, nodeBlock=10, interfaceBlock=3, nodeIndex=1, interfaceIndex=1. Controller110may use these inputs to determine the following:

Following this, controller110configures all routes of the network. This includes sending route modification instructions (e.g., addroute, deleteroute) to CNI daemon118, which therein accordingly executes each of these operations as sent from CNI controller110.

Subsequently the annotated pod is created. For example, this includes attaching the annotation definitions (net-attach-def) to this pod at pod annotation104. Multus106observes this to identify when the creation of the annotated pod is successfully completed. Once completed, multus106causes CNI binary120to be called by the CNI framework. CNI binary120then sends a request to CNI controller110to trigger IP allocation (e.g., where this request includes MultiNicNetwork, and/or NodeName). For example, MultiNicNetwork may include a defined watcher that creates a resource called NetAttachDefWatcher (e.g., which itself watches for network annotation definitions, as the resource name suggests) as it delegates CNI plugin details via single root input/output virtualization (SR-IOV) Network custom resources (CRs) for SR-IOV CNI to enable cluster-wide network definition.

Once CNI controller110receives this request, controller110may allocate all IP addresses using all of the information that it has gathered, and then record these allocations in IP assignment data116. Once allocated, controller110sends back this compiled information (e.g., interfaceName, IP addresses from valid CIDRs, etc.), in response to which CNI binary120may, e.g., create IPvlan with these assigned IP addresses.

In this way, the system ofFIG.1can be defined with NetworkAttachmentDefinition for dynamic and automatic configuration of interfaces. However, in other examples, aspects of this disclosure relate to a stand-alone system. For example, a stand-alone solution can be achieved by defining a static configuration at relevant hosts, similar to other CNI plugin.

FIG.2provides another flowchart as to how different components interact across different steps. For example, CNI152allocates (IP addresses) to IPAM154. IPAM154then sends the new IP addresses to numerous resources/components (Daemonwatcher156, MultiNicNetwork158, IPPoolWatcher160, CIDRWatcher162), and updates IP addresses within IPPool166. From here numerous of these monitor other elements (e.g., to verify if/when things occur, and react accordingly). For example, DaemonWatcher156monitors DaemonPod170, updates CIDR176, removes data from Daemon164, and creates and deletes interface and host information from HostInterface172as necessary. Similarly, MultiNicNetwork158monitors NetworkAttachDefinition174, and then adds or removes data from Daemon164as necessary. Similarly, IPPoolWatcher monitors AttachedPod168, and then deletes IP addresses from IPPool166as necessary. As used herein, resources should be understood to perform the plain meaning of their names when these resources are not otherwise widely known in the industry or defined herein—for example, the resource IPPoolWatcher should be understood to be a resource that monitors/watches the a “pool” of IP addresses as described herein.

As described above, some of the functionality described herein may include or be part of a computing device that includes a processor configured to execute instructions stored on a memory to execute the techniques described herein. For example,FIG.3is a conceptual box diagram of such computing system200, where for purposes of exampleFIG.3is provided as if controller110ofFIG.1were a full system with discrete memory, processors, and interfaces as described herein. While controller110is depicted as a single entity (e.g., within a single housing) for the purposes of illustration, in other examples, controller110may include two or more discrete physical systems (e.g., within two or more discrete housings). Controller110may include interface210, processor220, and memory230. Controller110may include any number or amount of interface(s)210, processor(s)220, and/or memory(s)230.

Controller110may include components that enable controller110to communicate with (e.g., send data to and receive and utilize data transmitted by) devices that are external to controller110. For example, controller110may include interface210that is configured to enable controller110and components within controller110(e.g., such as processor220) to communicate with entities external to controller110. Interface210may include one or more network interface cards, such as Ethernet cards and/or any other types of interface devices that can send and receive information. Any suitable number of interfaces may be used to perform the described functions according to particular needs.

As discussed herein, controller110may be configured to automate the dynamic recognition and management of hosts and interfaces on a network. Controller110may utilize processor220in this way. Processor220may include, for example, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and/or equivalent discrete or integrated logic circuits. Two or more of processor220may be configured to work together as described herein accordingly.

Processor220may provide a network plugin for multiple network interfaces with automated L3 configuration according to instructions232stored on memory230of controller110. Memory230may include a computer-readable storage medium or computer-readable storage device. In some examples, memory230may include one or more of a short-term memory or a long-term memory. Memory230may include, for example, random access memories (RAM), dynamic random-access memories (DRAM), static random-access memories (SRAM), magnetic hard discs, optical discs, floppy discs, flash memories, forms of electrically programmable memories (EPROM), electrically erasable and programmable memories (EEPROM), or the like. In some examples, processor220may provide a network plugin as described herein according to instructions232of one or more applications (e.g., software applications) stored in memory230of controller110.

In addition to instructions232, in some examples gathered or predetermined data or techniques or the like as used by processor220to provide a network plugin for multiple network interfaces with automated L3 configuration as described herein may be stored within memory230. For example, memory230may include information described above that is gathered from environment100. Specifically, as depicted inFIG.2, memory230may include hot routes data234, IP data236, host index data238, and user defined configuration data240.

Further, memory230may include threshold and preference data242. Threshold and preference data242may include thresholds that define a manner in which controller110is to provide a network plugin for multiple network interfaces with automated L3 configuration.

Memory230may further include machine learning techniques244that controller110may use to improve a process of providing a network plugin for multiple network interfaces with automated L3 configuration as described herein over time. Machine learning techniques244can comprise algorithms or models that are generated by performing supervised, unsupervised, or semi-supervised training on a dataset, and subsequently applying the generated algorithm or model to manage network host and interface routing. Using these machine learning techniques244, controller110may improve an ability of generating providing a network plugin for multiple network interfaces with automated L3 configuration.