Link-layer level link aggregation autoconfiguration

A computing device identifies a plurality of network interface controllers (NICs) that are available for link aggregation. The computing device probes a network using a plurality of protocols to detect a topology of at least one network to which the plurality of NICs are connected. The computing device automatically establishes a link aggregation configuration including one or more of the plurality of NICs based on the determined network topology.

TECHNICAL FIELD

Embodiments of the present invention relate to link aggregation, and more specifically to automatically discovering network topology and performing link aggregation based on the network topology.

BACKGROUND

The bandwidth used by a compute node in a data center typically exceeds the capabilities of a single network interface controller (NIC). Accordingly, link aggregation technologies are used to bundle network bandwidth and provide redundancy. In conventional systems, when a system administrator wants to set up link-layer level link aggregation, the system administrator uses his knowledge of the network topology to manually configure the link aggregation. This is often problematic because the administrator may lack knowledge as to the network topology and/or may not know how to set up the link aggregation.

DETAILED DESCRIPTION

Described herein is a method and system for automatically determining a network topology and configuring a link-layer link aggregation based on the determined network topology. Link aggregation is the generation of a single logical channel from the bundling of multiple physical ports. Link aggregation is performed to provide redundancy for network interface controllers (NICs), to provide bandwidth that is greater than a single NIC can provide, to perform load balancing, and for other reasons. However, link aggregation typically will not be performed successfully without first knowing which NICs to aggregate and the network topology of the network to which the NICs are connected. Accordingly, the configuration of link-layer level link aggregation is typically manually performed by a network administrator with knowledge about the network topology and the computing device for which link aggregation is to be set up.

Embodiments simplify the process of setting up link-layer level link aggregation configurations, and in some instances completely eliminate manual input by an administrator. Moreover, embodiments enable link-layer level link aggregation to be set up even in instances in which an administrator is not familiar with the network topology or the computing device to be connected to the network. Embodiments also provide an application programming interface (API) that other applications such as orchestration systems (e.g., Openstack) may use to automatically establish link-layer level link aggregation configurations.

In one embodiment, processing logic identifies multiple network interface controllers (NICs) of a computing device that are available for link aggregation. The processing logic then probes a network using multiple different protocols to detect a topology of a network to which the NICs are connected. The processing logic then automatically establishes a link aggregation configuration including one or more of the NICs based on the determined network topology.

Some embodiments are described herein with reference to physical switches. However, it should be appreciated that the principles introduced and described with reference to physical switches also apply to virtual network switches. Accordingly, embodiments of the present invention are not limited to physical switches, and those embodiments describing physical switches may be modified for use with virtual network switches.

FIG. 1is a block diagram that illustrates an embodiment of a computing device100. The computing device100may be a rackmount server, a workstation, a desktop computer, a notebook computer, a tablet computer, a mobile phone, a compute node of a data center, etc. The computing device100includes hardware105, which may include one or more processors120, one or more devices124, memory128, multiple physical network interface controllers (NICs)129-133, and other hardware components. The memory128may include volatile memory devices (e.g., random access memory (RAM)), non-volatile memory devices (e.g., flash memory), and/or other types of memory devices. The hardware105may also be coupled to external storage (not shown) via a direct connection or a local network. The computing device100may be a single machine or multiple machines arranged in a cluster.

Each of the devices124may be a physical device that is internal or external to the computing device100. Examples of internal devices include a graphics card, hardware RAID controller, secondary storage (e.g., hard disk drive, magnetic or optical storage based disks, tapes or hard drives), universal serial bus (USB) devices, internal input/output (I/O) devices, etc. Examples of external devices include a keyboard, mouse, speaker, external hard drive, external I/O devices, etc.

NICs129-133provide a hardware interface between the computing device100and a network. NICs may support wired standards, wireless standards, or both. The NICs129-133may be Ethernet controllers, Wi-Fi controllers, Token Rings, Infiniband controllers, and so forth. The NICs129-133provide functionality to communicate over a network using specific physical layer (OSI layer 1) and data link layer (OSI layer 2) standards (e.g., Ethernet, Wi-Fi, Token Ring, etc.). Each NIC129-133includes a unique media access control (MAC) address, which may be stored in a read only memory of the NIC129-133.

The computing device100includes an operating system (OS)110. The computing device may additionally include a server (e.g., a web server), a database and/or database management system (DBMS), a hypervisor and/or virtual machines, or other functionality. The computing device100may be configured to manage many connections to other devices over a network. Accordingly, the computing device may consume large amounts of bandwidth, and may accordingly have multiple NICs129-133to handle the large bandwidth usage.

In order to optimally use the multiple NICs129-133, the computing device100may configure one or more link-layer level link aggregations from some or all of these NICs129-133. In one embodiment, computing device100includes a link aggregator140that automatically sets up link-layer level link aggregation configurations.

Link aggregator140probes the NICs129-133and the remote endpoints to which the NICs are attached to detect a network topology of one or more networks to which the NICs129-133are connected. Link aggregator140uses multiple different protocols to probe the NICs and the networks. For example, link aggregator140may use at least two of link aggregation control protocol (LACP), address resolution protocol (ARP), link line discovery protocol (LLDP), internet protocol version six neighbor discovery protocol (IPv6 NDP), or dynamic host configuration protocol (DHCP) to send out probes. Other types of probes may also be used.

In the illustrated example, link aggregator140probes the NICs129-133to determine speeds or bandwidth capabilities of each of the NICs. Ethernet NICs typically support speeds of 10 Megabits per second (Mbits/s), 100 Mbits/s or 1000 Mbits/s. Wi-Fi NICs typically support speeds of between 54 Mbits/s and 866 Mbits/s.

Link aggregator140may also probe the NICs129-133and the endpoints (e.g., switches141-144) to which the NICs are connected to determine which NICs are on the same network. In one embodiment, Link aggregator140broadcasts or multicasts a message out from one or more of the NICs129-133and then listens for that broadcast message from the other NICs. Such a broadcast message will be sent to each endpoint on a network that receives the broadcast. Accordingly, if a broadcast is sent out on a NIC, it can be determined that each other NIC at which that broadcast message is received is connected to the same network segment. For example, if link aggregator140broadcasts a message from NIC129, that message would be received at NIC130and NIC131, but would not be received at NIC132and NIC133. Accordingly, link aggregator140could determine that NICs129-131are connected to first network150. Similarly, link aggregator140could determine that NICs132-133are connected to second network152.

Link aggregator140may additionally probe the switches141-144to which the NICs129-133are connected to determine which NICs are connected to the same switches. For example, link aggregator140may determine that NIC129and130are connected to the same switch141, and that NIC132and NIC133are connected to the same switch144. Link aggregator140may additionally probe the switches to determine capabilities of the switches, such as to determine whether the switches support LACP. Link aggregator140may also probe the switches141-144to determine additional information, such as whether there are any virtual local area networks (VLANs) that are used by any of the switches.

Once link aggregator140has obtained enough information to determine the network topologies for the first network150and the second network152, link aggregator140may apply one or more link aggregation rules to aggregate the NICs129-133. For example, link aggregator140might generate a first link-layer level link aggregation using NICs129-131and may generate a second link-layer level link aggregation using NICs132-133in one embodiment. Alternatively, link aggregator140may generate the first link-layer level link aggregation using just NIC129and NIC130. NIC131may be omitted from the first link aggregation, for example, if NIC131is a 100 Mbit/s NIC and NICS129-130are 1000 Mbit/s NICS and/or if switch141supports LACP. The link aggregator140is described in greater detail below with reference toFIG. 2.

FIG. 2is a block diagram of a link aggregator200, in accordance with one embodiment of present invention. In one implementation, link aggregator200corresponds to link aggregator140ofFIG. 1. Link aggregator200is configured to probe a network to determine a network topology and create link-layer level link aggregation configurations based on the network topology.

In one embodiment, the link aggregator200includes a network prober257, a topology determiner215and an aggregation determiner220. The link aggregator200may also include one or more aggregation rules262that may be used to set up link aggregation configurations.

Network prober257sends out multiple different types of probes260to endpoints that NICs of a computing device are connected to and/or to the NICs themselves. Multiple different protocols may be used to send out multiple different types of probes260. Each of the different types of probes260may be used to gather a different type of information. Ultimately, the network prober257should send out enough different probes260to gather information about the network that can be used to determine a topology of the network. Such information may include information about nodes (e.g., switches, servers, computer systems, etc.) on the network, properties of those nodes, and so on. Some information is received in the form of probe responses252, which may be forwards of the original probes, responses generated by nodes based on the probes, and so on. Network prober257may also receive some messages from other nodes without first sending out probes. For example, for some protocols such as LLDP nodes send out periodic messages. A non-exhaustive list of protocols that may be used to send out probes includes link aggregation control protocol (LACP), address resolution protocol (ARP), link line discovery protocol (LLDP), internet protocol version six neighbor discovery protocol (IPv6 NDP), or dynamic host configuration protocol (DHCP).

ARP probe messages, DHCP probe messages, IPv6 NDP probe messages, or other types of probe messages may be used to determine which NICs are on the same network segment. Any of these protocols may be used to broadcast a probe message from a NIC. Network prober257may then listen in on the other NICs to determine if they have received the broadcast probe message. If a NIC receives the broadcast probe, then that NIC is on a network segment that is connected to a network segment of the NIC from which the probe was broadcast. If a NIC fails to receive the broadcast probe, then that NIC is on a network segment that is not connected with the network segment of the NIC from which the probe was broadcast. No link aggregation can be performed for NICs that are on unconnected network segments.

ARP probe messages, DHCP probe messages, and/or IPv6 NDP probe messages may additionally be used to determine resident links between the NICs and switches or other nodes. The resident links may indicate how to reach a specific node on the network. If there are multiple resident links connected to a node, then there may be multiple different ways to connect to that node.

ARP is a telecommunications request and reply protocol for resolution of network layer addresses into link layer addresses. ARP probes are packets encapsulated by the line protocol and are communicated within the boundaries of a single network. The types of information that can be determined from ARP nodes is limited, but all switches should be able to forward ARP probes. Additionally, switches that have IP addresses may respond to ARP probes.

In some network environments, a switch may be prepopulated with all possible flows and/or a network may be configured such that all switches are prepopulated with all possible flows. In such instances, some protocols used for network discovery may be deactivated on the switches for security purposes. For example, firewalls and switches may be configured so as to no longer pass through ARP packets. In such an environment, protocols other than ARP would be used to determine which NICs are on the same network and/or the resident links for those NICs.

DHCP is a standardized networking protocol used on internet protocol (IP) networks that dynamically configures IP address and other information. DHCP allows devices on a network to receive IP addresses from a DHCP server, eliminating or reducing a need for a network administrator to configure these settings manually. The DHCP protocol includes a DHCP discovery mechanism in which a DHCP probe is broadcast on a physical subnet to discover available DHCP servers. Such DHCP probes may be used to discover similar information that the ARP protocol discovers. However, DHCP probes will typically be permitted to pass through switches even when ARP probes are blocked.

IPv6 NDP is a protocol that includes a set of messages or probes that may be used to determine relationships between neighboring nodes. IPv6 NDP may be used to obtain the same type of information as ARP, but for nodes that use IPv6. Some example IPv6 NDP probes include a neighbor solicitation (NS) probe, a neighbor advertisement (NA) probe, a router solicitation (RS) probe and a router advertisement (RA) probe.

LLDP is a standard link layer protocol used by networking devices to advertise their identity, capabilities and neighbors on a local area network (LAN). LLDP messages or probes may be sent at a fixed interval in the form of an Ethernet frame. Each frame may include one or more destination media access control (MAC) address, a source MAC address, a port identifier (ID), a time to live value, and/or other information. LLDP probes or messages may be used to gather information such as a system name and description, a port name and description, a virtual local area network (VLAN) name, an IP management address, system capabilities (e.g., switching, routing, etc.), MAC address, link aggregation capabilities, and so forth. LLDP-MED (media endpoint discovery) is an extension to LLDP that may also be used. LLDP-MED may be used for auto-discovery of LAN policies (e.g., VLANs, layer 2 priority, differentiated services, etc.), device location discovery, and other information.

LLDP may be used to discover VLANs configured on a port or node. If a node uses a VLAN, then that node may insert information about the VLAN into an LLDP message which will be received by the NIC. If a link layer level link aggregation is to be set up for a NIC that is connected to a node that uses a VLAN, then a VLAN tag may be inserted into messages for link validation. In addition to VLAN configurations, LLDP may be used to determine, for example, link speeds of network cards, data center bridging capabilities, and so forth.

LACP is a link aggregation protocol that can be used to aggregate links that share the same switch. That is, two NICs that are both connected to the same switch that supports LACP may be configured into a link-layer level link aggregation that uses the LACP protocol. LACP aggregated links have several advantages over traditional non-LACP aggregated links. In particular, LACP aggregated links may utilize all aggregated NICs for both uplinks and downlinks. However, non-LACP aggregated links typically use all NICs for uplinks but only a single NIC for the downlink.

Topology determiner215determines a topology of the network based on the probe responses252and/or other messages that are received from nodes on the network. In the case that the computing device on which link aggregator200runs is connected to multiple networks, topology determiner215will determine the topology of each of the networks. For example, if the computing device is connected to a data network and a management network, then topology determiner may discover the network topology of both the data network and the management network. Examples of information that may be determined about the network topology include identification of switches that each of the NICs of the computing device are connected to, a determination of which NICs are connected to the same switches, a determination of link speed for the switches and/or NICs, capabilities of the switches (e.g., whether they support LACP), whether switches use VLANs, and so forth.

Aggregation determiner220establishes one or more link-layer level link aggregation configurations based on the determined network topology. A separate link aggregation may be set up for each separate network that the computing device is connected to. For example, a first link aggregation may be set up for a data network and a second link aggregation may be set up for a management network.

In one embodiment, aggregation determiner220applies one or more aggregation rules262to set up the link aggregation configurations. Aggregation rules may be used to assign weights or preferences to specific individual NICs or combinations of NICs. For example, a first link aggregation rule262may assign weights to NICs based on their link speeds, where higher speed NICs are assigned a higher weighting than lower speed NICs. A second aggregation rule may assign a higher weighting to NICs that are connected to the same switch than to NICs that are connected to different switches. This is because typically higher performance gains can be realized for aggregations of links that are to the same switch. A third aggregation rule may assign a higher weighting to NICs that are connected to switches that support LACP than to NICs that are connected to switches that do not support LACP. Other aggregation rules may also be used. The aggregation rules may be used in combination to determine an optimal link aggregation configuration for a given network topology.

Multiple different types of link aggregation configurations are possible. For example, a first type of link aggregation configuration may combine multiple links in parallel to increase throughput above what a single NIC can sustain. A second type of link aggregation configuration may combine links in such a way that one or more secondary links provide redundancy in case a primary link fails. A third type of link aggregation performs load balancing between NICs. Other types of link configurations are also possible. An aggregated link may have a single logical address that is shared by multiple NICs or a single physical address that is shared by the multiple NICs.

FIGS. 3-5are flow diagrams of various embodiments of methods related to network topology discovery and automatic link aggregation configuration. The methods are performed by processing logic that may include hardware (circuitry, dedicated logic, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both. In one implementation, the methods are performed by a computing device running a link aggregator, such as link aggregator200ofFIG. 2. The methods ofFIGS. 3-5enable link aggregation to be performed without any user interaction. Thus, the process of link aggregation is simplified since administrators may not know anything about the network topology in embodiments.

FIG. 3is a flow diagram illustrating one embodiment for a method300of establishing a link-layer level link aggregation configuration. At block305of method300, processing logic identifies one or more NICs that are available for link-layer level link aggregation. In one embodiment, a network administrator provides a list or grouping of available NICs. Alternatively, processing logic may use one or more probes to identify NICs that are available for link aggregation.

At block312, processing logic probes a network to which the determined NICs are connected to detect a topology of the network. Multiple different types of probes using multiple different protocols are used to determine the network topology. At block330, processing logic then automatically establishes a link-layer level link aggregation configuration based on the detected network topology. In one embodiment, processing logic applies one or more link aggregation rules to set up the link aggregation.

The network topology may not be static, and may thus change over time. For example, nodes may change IP addresses or MAC addresses, nodes may be turned on or turned off, nodes may be relocated on the network, network partitions may occur, and so forth. Processing logic may automatically adapt to such changes in the network topology, which may include changes in the hardware level and/or changes in the software level.

At block332, processing logic receives updated information on the network topology. Such updated information may be received responsive to sending out new probes. Additionally, or alternatively, updated network topology information may be received periodically without sending out probes. For example, LLDP messages may periodically be sent by nodes in a network. Some network topology information may additionally be attached to data that is received through the course of standard operations (e.g., while downloading or uploading data).

At block335, processing logic determines whether a change in the network topology has occurred. If such a change is detected, the method continues to block340. At block340, processing logic updates the link-layer level link aggregation based on the changed network topology. For example, four NICs may have originally been available for link aggregation, and a first and second NIC may have been aggregated. However, if the first and second NIC fail, then processing logic may set up a new link aggregation configuration using the third and fourth NICs. In some instances, the link aggregation configuration will not change even though the network topology has changed. If no change in the network topology is detected, the method returns to block332. This process may continue indefinitely.

FIG. 4is a flow diagram illustrating one embodiment for a method400of determining a network topology and establishing a link-layer level link aggregation configuration. At block405of method400, processing logic receives a link aggregation command. Processing logic may provide an API that higher level management software such as an orchestration system may use to request that a network connection be set up.

Responsive to such a request, processing logic may determine whether or not link aggregation is appropriate for a computing device, and may automatically set up a link-layer level link aggregation if appropriate. Processing logic may also receive a request to set up a link aggregation configuration from an administrator, who may or may not identify NICs to consider for the link aggregation.

At block420, processing logic determines whether a list of available NICs has been provided. If no such list is provided, the method continues to block412and processing logic probes the NICs of the computing device to determine which are available. Otherwise the method continues to block415.

At block415, processing logic determines a protocol to use for probing the network. The protocol may be determined based on the type of information that is yet to be obtained about the network and/or based on information currently known about the network (e.g., knowledge that ARP probes will not be forwarded by switches). At block420, processing logic probes the network using the determined protocol.

At block425, processing logic determines whether the network topology is fully characterized. If the network topology is not fully characterized, the method returns to block415, and another type of probe to send out is identified. If the network topology is fully characterized, the method continues to block440. At block440, processing logic automatically establishes a link-layer level link aggregation configuration based on the determined network topology. The established link aggregation configuration may be an optimal configuration for the determined network topology. Processing logic may then report, for example, to an orchestration system or administrator that a network connection has been established.

FIG. 5is a flow diagram illustrating one embodiment for a method500of establishing multiple link aggregation configurations for a computing device. At block505of method500, processing logic receives a link aggregation command (e.g., from an orchestration system). The link aggregation command may be received as a command to set up a network connection to a data network and to a management network, for example.

At block510, processing logic probes the computing device to identify available NICs. At block512, processing logic probes the computing device to determine networks to which the NICs are connected. For example, processing logic may determine that a first set of NICs are connected to a first network (e.g., a data network) and that a second set of NICs are connected to a second network (e.g., a management network).

At block515, processing logic probes the first network and the second network and determines both a topology of the first network and a topology of the second network. At block520, processing logic sets up a first link aggregation configuration for one or more of the NICs connected to the first network. At block525, processing logic sets up a second link aggregation configuration for one or more NICs connected to the second network. The method then ends.

The processing device602represents one or more general-purpose processors such as a microprocessor, central processing unit, or the like. The term “processing device” is used herein to refer to any combination of one or more integrated circuits and/or packages that include one or more processors (e.g., one or more processor cores). Therefore, the term processing device encompasses a single core CPU, a multi-core CPU and a massively multi-core system that includes many interconnected integrated circuits, each of which may include multiple processor cores. The processing device602may therefore include multiple processors. The processing device602may include a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, processor implementing other instruction sets, or processors implementing a combination of instruction sets. The processing device602may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like.

The computer system600may further include one or more network interface devices622(e.g., NICs). The computer system600also may include a video display unit610(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device612(e.g., a keyboard), a cursor control device614(e.g., a mouse), and a signal generation device620(e.g., a speaker).

The secondary memory616may include a machine-readable storage medium (or more specifically a computer-readable storage medium)624on which is stored one or more sets of instructions654embodying any one or more of the methodologies or functions described herein (e.g., link aggregator680). The instructions654may also reside, completely or at least partially, within the main memory604and/or within the processing device602during execution thereof by the computer system600; the main memory604and the processing device602also constituting machine-readable storage media.

The modules, components and other features described herein (for example in relation toFIG. 2) can be implemented as discrete hardware components or integrated in the functionality of hardware components such as ASICS, FPGAs, DSPs or similar devices. In addition, the modules can be implemented as firmware or functional circuitry within hardware devices. Further, the modules can be implemented in any combination of hardware devices and software components, or only in software.