Patent Description:
It is the object of the present invention to reduce communications latency in a distributed computing system.

In cloud-based datacenters or other types of large scale distributed computing systems, overlay protocols such as Virtual Extensible Local Area Network and virtual switching can involve complex packet manipulation actions. As such, processing complexity related to server-based networking data plane has increased dramatically to support such overlay protocols. With ever increasing network interface bandwidths, performing these complex packet manipulation actions in software imposes a heavy burden on processing resources at the servers to leave little or no processing resources to run user applications.

To address such challenges, certain hardware circuitry has been developed for offloading at least a portion of the data plane processing from server processors. For example, servers can incorporate a Field Programmable Gate Array ("FPGA") by coupling the FPGA to a Network Interface Card ("NIC") and a Central Processing Unit ("CPU"). During runtime, a software controller at the CPU can program the FPGA to perform flow action matching or other suitable data plane actions. For instance, the FPGA can be configured to implement an inbound processing path that includes an inbound packet buffer for holding received inbound packets, a parser configured to parse headers of the inbound packets, a lookup circuit configured to locate one or more matching actions in a flow match table based on at least a portion of the parsed headers, and an action circuit configured to perform the one or more matching actions on the inbound packets. The FPGA can also include an outbound processing path that includes similar components coupled to one another in a reverse direction than the inbound processing path.

In operation, the inbound processing path can receive an inbound packet from a computer network via, for example, a top-of-rack switch ("TOR"), store the received inbound packet in the inbound packet buffer, parse headers of the received inbound packet, locate one or more matching actions for the packet based on at least a portion of the headers, and perform the one or more matching actions on the inbound packet before forwarding the processed inbound packet to the NIC. The outbound processing path can receive an outbound packet from, for example, the NIC or the CPU, store the outbound packet in an outbound packet buffer, parse the received outbound packet, locate one or more matching actions for the outbound packet, and perform the one or more matching actions on the outbound packet before forwarding the processed outbound packet to the computer network, for example, via the same TOR.

The foregoing offloading implementation, however, have several drawbacks. For example, the FPGA in the foregoing offloading implementation directly forwards inbound/outbound packets to either the NIC or the TOR. Such direct forwarding does not allow remote direct memory access ("RDMA") among applications and/or virtual machines on a virtual network implemented on an underlay network in the distributed computing system. RDMA is a technique that allows a computer, a virtual machine, or an application to directly access memory locations of a remote computer via a computer network without involving either one's operating system. An RDMA connection can allow ultra-low latency (e.g., less than about <NUM>) communications between computers. RDMA can also have low processor utilization and high bandwidth on individual connections. RDMA can be implemented in various manners. In one example, RDMA can be implemented using hardware components such as hardware connection adapters ("HCAs") to process RDMA traffic using queue pairs. Such an implementation, however, involves installing and maintaining hardware components such, i.e., HCAs, in addition to NICs or other types of adapters needed to handle TCP/IP traffic in a computer network.

Typically, RDMA operations involve routing packets in hardware between pairs of network endpoints (e.g., HCAs) with routable addresses in an underlay network. As such, attempt to route RDMA packets using virtual network addresses would simply be inoperable. For example, when the FPGA described above receives a request-for-connection packet ("request packet") from a first virtual machine to a second virtual machine for an RDMA connection between the pair, the FPGA simply forwards the request packet to the TOR. The TOR, however, would not understand source/destination addresses associated with the request packet because the TOR does not have any entries in an associated routing table for the virtual network addresses, but instead network addresses in the underlay network. Thus, the TOR would deem the request packet as invalid and drop the request packet, causing the RDMA connection request to fail.

Embodiments of the disclosed technology can address the foregoing FPGA implementation drawback by allowing the FPGA to route RDMA connection packets inside the FPGA. For example, in one implementation, the inbound processing path of the FPGA can further include an output buffer between the action circuit and the NIC. The outbound processing path can further include a NIC buffer operatively coupled to the action circuit in the outbound processing path to receive input from the action circuit. A multiplexer can be configured to receive input from both the output buffer and the NIC buffer alternately, in a round-the-robin, or other suitable fashions. The multiplexer can also be configured to provide an output to the NIC.

During operation, when the FPGA receives a RDMA request packet from a first virtual machine on a first host to a second virtual machine on a second host, the outbound processing path can parse a header of the request packet, attempt to match the request packet to a flow in the flow table based on at least a portion of the parsed header. The action circuit can then raise an exception because the FPGA does not contain a flow that matches the request packet. The action circuit in the outbound processing path can then forward the request packet along with an exception flag to the NIC buffer instead of the TOR. In turn, the multiplexer can then retrieve the request packet with the exception flag from the NIC buffer and forward the request packet to a software component (e.g., a virtual switch) via the NIC. The software component can then generate a flow for the request packet based on, for example, certain RDMA connection policies in the distributed computing system, and transmit the request packet back to the outbound processing path along with information of the generated flow.

Upon receiving the request packet along with the information of the generated flow, the outbound processing path can then process the request packet according to the flow to, for instance, encapsulate the request packet with an underlay network address of a host at which the second virtual machine is hosted, and transmit the processed request packet to the TOR. The TOR can then forward the request packet to the host at which the second virtual machine is hosted according to the underlay network address, and thus enabling RDMA connection between the first and second virtual machines on a virtual network. Several embodiments of the disclosed technology can thus enable RDMA connections between pairs of virtual machines on virtual networks without requiring installation of additional NICs, HCAs, or other hardware components in the distributed computing system. As such, communications latency in the distributed computing system can be reduced using RDMA without incurring additional capital costs.

Certain embodiments of systems, devices, components, modules, routines, data structures, and processes for routing RDMA network traffic in datacenters or other suitable distributed computing systems are described below. In the following description, specific details of components are included to provide a thorough understanding of certain embodiments of the disclosed technology. A person skilled in the relevant art will also understand that the technology can have additional embodiments. The technology can also be practiced without several of the details of the embodiments described below with reference to <FIG>.

As used herein, the term "distributed computing system" generally refers to an interconnected computer system having multiple network nodes that interconnect a plurality of servers or hosts to one another and/or to external networks (e.g., the Internet). The term "network node" generally refers to a physical network device. Example network nodes include routers, switches, hubs, bridges, load balancers, security gateways, or firewalls. A "host" generally refers to a physical computing device configured to implement, for instance, one or more virtual machines, virtual switches, or other suitable virtualized components. For example, a host can include a server having a hypervisor configured to support one or more virtual machines, virtual switches or other suitable types of virtual components.

A computer network can be conceptually divided into an overlay network implemented over an underlay network. An "overlay network" generally refers to an abstracted network implemented over and operating on top of an underlay network. The underlay network can include multiple physical network nodes interconnected with one another. An overlay network can include one or more virtual networks. A "virtual network" generally refers to an abstraction of a portion of the underlay network in the overlay network. A virtual network can include one or more virtual end points referred to as "tenant sites" individually used by a user or "tenant" to access the virtual network and associated computing, storage, or other suitable resources. A tenant site can host one or more tenant end points ("TEPs"), for example, virtual machines. The virtual networks can interconnect multiple TEPs on different hosts. Virtual network nodes in the overlay network can be connected to one another by virtual links individually corresponding to one or more network routes along one or more physical network nodes in the underlay network.

Further used herein, a Match Action Table ("MAT") generally refers to a data structure having multiple entries in a table format. Each of the entries can include one or more conditions and one or more corresponding actions. The one or more conditions can be configured by a network controller (e.g., an Software Defined Network or "SDN" controller) for matching a set of header fields of a packet. The action can also be programmed by the network controller to apply an operation to the packet when the conditions match the set of header fields of the packet. The applied operation can modify at least a portion of the packet to forward the packet to an intended destination. Further used herein, a "flow" generally refers to a stream of packets received/transmitted via a single network connection between two end points (e.g., servers, virtual machines, or applications executed in the virtual machines). A flow can be identified by, for example, an IP address and a TCP port number. A flow can have one or more corresponding entries in the MAT. Each entry can have one or more conditions and actions. Example conditions and actions are shown in <FIG>.

As used herein, a "packet" generally refers to a formatted unit of data carried by a packet-switched network. A packet typically can include user data along with control data. The control data can provide information for delivering the user data. For example, the control data can include source and destination network addresses/ports, error checking codes, sequencing information, hop counts, priority information, security information, or other suitable information regarding the user data. Typically, the control data can be contained in headers and/or trailers of a packet. The headers and trailers can include one or more data field containing suitable information.

<FIG> is a schematic diagram illustrating a distributed computing system <NUM> implementing RDMA network traffic routing in accordance with embodiments of the disclosed technology. As shown in <FIG>, the distributed computing system <NUM> can include an underlay network <NUM> interconnecting a plurality of hosts <NUM>, a plurality of client devices <NUM> associated with corresponding users <NUM>, and a platform controller <NUM> operatively coupled to one another. Even though particular components of the distributed computing system <NUM> are shown in <FIG>, in other embodiments, the distributed computing system <NUM> can also include additional and/or different components or arrangements. For example, in certain embodiments, the distributed computing system <NUM> can also include network storage devices, additional hosts, and/or other suitable components (not shown) in other suitable configurations.

As shown in <FIG>, the underlay network <NUM> can include one or more network nodes <NUM> that interconnect the multiple hosts <NUM> and the users <NUM>. In certain embodiments, the hosts <NUM> can be organized into racks, action zones, groups, sets, or other suitable divisions. For example, in the illustrated embodiment, the hosts <NUM> are grouped into three host sets identified individually as first, second, and third host sets 107a-107c. Each of the host sets 107a-107c is operatively coupled to a corresponding network nodes 112a-112c, respectively, which are commonly referred to as "top-of-rack" network nodes or "TORs. " The TORs 112a-112c can then be operatively coupled to additional network nodes <NUM> to form a computer network in a hierarchical, flat, mesh, or other suitable types of topology. The underlay network can allow communications among hosts <NUM>, the platform controller <NUM>, and the users <NUM>. In other embodiments, the multiple host sets 107a-107c may share a single network node <NUM> or can have other suitable arrangements.

The hosts <NUM> can individually be configured to provide computing, storage, and/or other suitable cloud or other suitable types of computing services to the users <NUM>. For example, as described in more detail below with reference to <FIG>, one of the hosts <NUM> can initiate and maintain one or more virtual machines <NUM> (shown in <FIG>) upon requests from the users <NUM>. The users <NUM> can then utilize the provided virtual machines <NUM> to perform computation, communications, and/or other suitable tasks. In certain embodiments, one of the hosts <NUM> can provide virtual machines <NUM> for multiple users <NUM>. For example, the host 106a can host three virtual machines <NUM> individually corresponding to each of the users 101a-101c. In other embodiments, multiple hosts <NUM> can host virtual machines <NUM> for the users 101a-101c.

The client devices <NUM> can each include a computing device that facilitates the users <NUM> to access cloud services provided by the hosts <NUM> via the underlay network <NUM>. In the illustrated embodiment, the client devices <NUM> individually include a desktop computer. In other embodiments, the client devices <NUM> can also include laptop computers, tablet computers, smartphones, or other suitable computing devices. Though three users <NUM> are shown in <FIG> for illustration purposes, in other embodiments, the distributed computing system <NUM> can facilitate any suitable numbers of users <NUM> to access cloud or other suitable types of computing services provided by the hosts <NUM> in the distributed computing system <NUM>.

The platform controller <NUM> can be configured to manage operations of various components of the distributed computing system <NUM>. For example, the platform controller <NUM> can be configured to allocate virtual machines <NUM> (or other suitable resources) in the distributed computing system <NUM>, monitor operations of the allocated virtual machines <NUM>, or terminate any allocated virtual machines <NUM> once operations are complete. In the illustrated implementation, the platform controller <NUM> is shown as an independent hardware/software component of the distributed computing system <NUM>. In other embodiments, the platform controller <NUM> can also be a datacenter controller, a fabric controller, or other suitable types of controller or a component thereof implemented as a computing service on one or more of the hosts <NUM>.

<FIG> is a schematic diagram illustrating certain hardware/software components of the distributed computing system <NUM> in accordance with embodiments of the disclosed technology. In particular, <FIG> illustrates an overlay network <NUM>' that can be implemented on the underlay network <NUM> in <FIG>. Though particular configuration of the overlay network <NUM>' is shown in <FIG>, In other embodiments, the overlay network <NUM>' can also be configured in other suitable ways. In <FIG>, only certain components of the underlay network <NUM> of <FIG> are shown for clarity.

In <FIG> and in other Figures herein, individual software components, objects, classes, modules, and routines may be a computer program, procedure, or process written as source code in C, C++, C#, Java, and/or other suitable programming languages. A component may include, without limitation, one or more modules, objects, classes, routines, properties, processes, threads, executables, libraries, or other components. Components may be in source or binary form. Components may include aspects of source code before compilation (e.g., classes, properties, procedures, routines), compiled binary units (e.g., libraries, executables), or artifacts instantiated and used at runtime (e.g., objects, processes, threads).

Components within a system may take different forms within the system. As one example, a system comprising a first component, a second component and a third component can, without limitation, encompass a system that has the first component being a property in source code, the second component being a binary compiled library, and the third component being a thread created at runtime. The computer program, procedure, or process may be compiled into object, intermediate, or machine code and presented for execution by one or more processors of a personal computer, a network server, a laptop computer, a smartphone, and/or other suitable computing devices.

Equally, components may include hardware circuitry. A person of ordinary skill in the art would recognize that hardware may be considered fossilized software, and software may be considered liquefied hardware. As just one example, software instructions in a component may be burned to a Programmable Logic Array circuit, or may be designed as a hardware circuit with appropriate integrated circuits. Equally, hardware may be emulated by software. Various implementations of source, intermediate, and/or object code and associated data may be stored in a computer memory that includes read-only memory, random-access memory, magnetic disk storage media, optical storage media, flash memory devices, and/or other suitable computer readable storage media excluding propagated signals.

As shown in <FIG>, the first host 106a and the second host 106b can each include a processor <NUM>, a memory <NUM>, and network interface card <NUM>, and a packet processor <NUM> operatively coupled to one another. In other embodiments, the hosts <NUM> can also include input/output devices configured to accept input from and provide output to an operator and/or an automated software controller (not shown), or other suitable types of hardware components.

The processor <NUM> can include a microprocessor, caches, and/or other suitable logic devices. The memory <NUM> can include volatile and/or nonvolatile media (e.g., ROM; RAM, magnetic disk storage media; optical storage media; flash memory devices, and/or other suitable storage media) and/or other types of computer-readable storage media configured to store data received from, as well as instructions for, the processor <NUM> (e.g., instructions for performing the methods discussed below with reference to <FIG> and <FIG>). Though only one processor <NUM> and one memory <NUM> are shown in the individual hosts <NUM> for illustration in <FIG>, in other embodiments, the individual hosts <NUM> can include two, six, eight, or any other suitable number of processors <NUM> and/or memories <NUM>.

The first and second hosts 106a and 106b can individually contain instructions in the memory <NUM> executable by the processors <NUM> to cause the individual processors <NUM> to provide a hypervisor <NUM> (identified individually as first and second hypervisors 140a and 140b) and a virtual switch <NUM> (identified individually as first and second virtual switches 141a and 141b). Even though the hypervisor <NUM> and the virtual switch <NUM> are shown as separate components, in other embodiments, the virtual switch <NUM> can be a part of the hypervisor <NUM> (e.g., operating on top of an extensible switch of the hypervisors <NUM>), an operating system (not shown) executing on the hosts <NUM>, or a firmware component of the hosts <NUM>.

The hypervisors <NUM> can individually be configured to generate, monitor, terminate, and/or otherwise manage one or more virtual machines <NUM> organized into tenant sites <NUM>. For example, as shown in <FIG>, the first host 106a can provide a first hypervisor 140a that manages first and second tenant sites 142a and 142b, respectively. The second host 106b can provide a second hypervisor 140b that manages first and second tenant sites 142a' and 142b', respectively. The hypervisors <NUM> are individually shown in <FIG> as a software component. However, in other embodiments, the hypervisors <NUM> can be firmware and/or hardware components. The tenant sites <NUM> can each include multiple virtual machines <NUM> for a particular tenant (not shown). For example, the first host 106a and the second host 106b can both host the tenant site 142a and 142a' for a first tenant 101a (<FIG>). The first host 106a and the second host 106b can both host the tenant site 142b and 142b' for a second tenant 101b (<FIG>). Each virtual machine <NUM> can be executing a corresponding operating system, middleware, and/or applications.

Also shown in <FIG>, the distributed computing system <NUM> can include an overlay network <NUM>' having one or more virtual networks <NUM> that interconnect the tenant sites 142a and 142b across multiple hosts <NUM>. For example, a first virtual network 142a interconnects the first tenant sites 142a and 142a' at the first host 106a and the second host 106b. A second virtual network 146b interconnects the second tenant sites 142b and 142b' at the first host 106a and the second host 106b. Even though a single virtual network <NUM> is shown as corresponding to one tenant site <NUM>, in other embodiments, multiple virtual networks <NUM> (not shown) may be configured to correspond to a single tenant site <NUM>.

The virtual machines <NUM> can be configured to execute one or more applications <NUM> to provide suitable cloud or other suitable types of computing services to the users <NUM> (<FIG>). The virtual machines <NUM> on the virtual networks <NUM> can also communicate with one another via the underlay network <NUM> (<FIG>) even though the virtual machines <NUM> are located on different hosts <NUM>. Communications of each of the virtual networks <NUM> can be isolated from other virtual networks <NUM>. For example, different virtual networks <NUM> can have different domain names, virtual network addresses, and/or other suitable identifiers. In certain embodiments, communications can be allowed to cross from one virtual network <NUM> to another through a security gateway or otherwise in a controlled fashion. A virtual network address can correspond to one of the virtual machine <NUM> in a particular virtual network <NUM>. Thus, different virtual networks <NUM> can use one or more virtual network addresses that are the same. Example virtual network addresses can include IP addresses, MAC addresses, and/or other suitable addresses. To facilitate communications among the virtual machines <NUM>, the virtual switches <NUM> can be configured to switch or filter packets (not shown) directed to different virtual machines <NUM> via the network interface card <NUM> and facilitated by the packet processor <NUM>.

As shown in <FIG>, to facilitate communications with one another or with external devices, the individual hosts <NUM> can also include a network interface card ("NIC") <NUM> for interfacing with a computer network (e.g., the underlay network <NUM> of <FIG>). A NIC <NUM> can include a network adapter, a LAN adapter, a physical network interface, or other suitable hardware circuitry and/or firmware to enable communications between hosts <NUM> by transmitting/receiving data (e.g., as packets) via a network medium (e.g., fiber optic) according to Ethernet, Fibre Channel, Wi-Fi, or other suitable physical and/or data link layer standards. During operation, the NIC <NUM> can facilitate communications to/from suitable software components executing on the hosts <NUM>. Example software components can include the virtual switches <NUM>, the virtual machines <NUM>, applications <NUM> executing on the virtual machines <NUM>, the hypervisors <NUM>, or other suitable types of components.

In certain implementations, a packet processor <NUM> can be interconnected and/or integrated with the NIC <NUM> in order to facilitate network processing operations for enforcing communications security, performing network virtualization, translating network addresses, maintaining a communication flow state, or performing other suitable functions. In certain implementations, the packet processor <NUM> can include a Field-Programmable Gate Array ("FPGA") integrated with or independent from the NIC <NUM>. An FPGA can include an array of logic circuits and a hierarchy of reconfigurable interconnects that allow the logic circuits to be "wired together" like logic gates by a user after manufacturing. As such, a user can configure logic blocks in FPGAs to perform complex combinational functions, or merely simple logic operations to synthetize equivalent functionality executable in hardware at much faster speeds than in software. In the illustrated embodiment, the packet processor <NUM> has one network interface communicatively coupled to the NIC <NUM> and another coupled to a network switch (e.g., a Top-of-Rack or "TOR" switch) at the other. In other embodiments, the packet processor <NUM> can also include an Application Specific Integrated Circuit ("ASIC"), a microprocessor, or other suitable hardware circuitry. In any of the foregoing embodiments, the packet processor <NUM> can be programmed by the processor <NUM> (or suitable software components provided by the processor <NUM>) to route packets inside the packet processor <NUM> in order to enable RDMA network traffic between two virtual machines <NUM> on a single or multiple host <NUM>, as described in more detail below with reference to <FIG>.

In operation, the processor <NUM> and/or a user <NUM> (<FIG>) can configure logic circuits in the packet processor <NUM> to perform complex combinational functions or simple logic operations to synthetize equivalent functionality executable in hardware at much faster speeds than in software. For example, the packet processor <NUM> can be configured to process inbound/outbound packets for individual RDMA flows according to configured policies or rules contained in a flow table such as a MAT. The flow table can also contain data representing processing actions corresponding to each flow for enabling private virtual networks with customer supplied address spaces, scalable load balancers, security groups and Access Control Lists ("ACLs"), virtual routing tables, bandwidth metering, Quality of Service ("QoS"), etc..

As such, once the packet processor <NUM> identifies an inbound/outbound packet as belonging to a particular flow, the packet processor <NUM> can apply one or more corresponding policies in the flow table before forwarding the processed packet to the NIC <NUM> or TOR <NUM>. For example, as shown in <FIG>, the virtual machine <NUM>, and/or other suitable software components on the first host 106a can generate an outbound packet <NUM> destined to, for instance, another virtual machine <NUM>' at the second host 106b. Example outbound packet <NUM> can include RDMA connection request packet, RDMA connection reply packet, or RDMA data packet. The NIC <NUM> at the first host 106a can forward the generated packet <NUM> to the packet processor <NUM> for processing according to certain policies in a flow table. Once processed, the packet processor <NUM> can forward the outbound packet <NUM> to the first TOR 112a, which in turn forwards the packet <NUM> to the second TOR 112b via the overlay/underlay network <NUM> and <NUM>'.

The second TOR 112b can then forward the packet <NUM> to the packet processor <NUM> at the second host 106b to be processed according to other policies in another flow table at the second hosts 106b. If the packet processor <NUM> cannot identify a packet as belonging to any flow, the packet processor <NUM> can forward the packet <NUM> to the processor <NUM> via the NIC <NUM> for exception processing. In another example, when the first TOR 112a receives an inbound packet <NUM>', for instance, from the second host 106b via the second TOR 112b, the first TOR 112a can forward the packet <NUM>' to the packet processor <NUM> to be processed according to a policy associated with a flow of the packet <NUM>'. The packet processor <NUM> can then forward the processed packet <NUM>' to the NIC <NUM> to be forwarded to, for instance, the application <NUM> or the virtual machine <NUM>.

In certain implementations, the packet processor <NUM> is configured to always forward packets <NUM>/<NUM>' to either the NIC <NUM> or the TOR <NUM> following a direct forwarding scheme. Such a direct forwarding scheme, however, would not allow RDMA to be implemented. For example, according to the direct forwarding scheme, the packet processor <NUM> may directly forward a RDMA connection reply packet from the virtual machine <NUM>" to the TOR 112b. The RDMA connection reply packet, however, is identified by virtual network addresses corresponding to the first and second virtual machines <NUM>' and <NUM>". Upon receiving the RDMA connection reply packet, the TOR 112b would deem the packet to be invalid because the TOR 112b can only route packets identified by network addresses in the underlay network <NUM> (<FIG>). As such, the RDMA connection attempt would fail.

Several embodiments of the disclosed technology can address at least some aspects of the foregoing limitations by implementing network traffic routing inside the packet processor <NUM>. For example, a NIC buffer <NUM> (shown in <FIG>) can be implemented in the packet processor <NUM> to temporarily store the RDMA connection reply packet while consulting the virtual switch <NUM> (or other suitable software components) for flow information of a flow to which the RDMA connection reply packet belongs. Once the flow information is received, the packet processor <NUM> can re-route the RDMA connection reply packet back to the inbound processing path to be processed according to the received flow information. As such, the packet processor <NUM> can facilitate establishment of an RDMA connection between the first and second virtual machines <NUM>' and <NUM>", as described in more detail below with reference to <FIG>.

<FIG> are schematic diagrams illustrating a hardware packet processor <NUM> implemented at a host <NUM> in a distributed computing system <NUM> during processing of an outbound packet for establishing an RDMA connection in accordance with embodiments of the disclosed technology. As shown in <FIG>, in certain implementations, the packet processor <NUM> can include an inbound processing path 138a and an outbound processing path 138b in opposite processing directions. As shown in <FIG>, the inbound processing path 138a can include a set of processing circuits having an inbound packet buffer <NUM> (shown as "IN Packet Buffer" in <FIG>), a parser <NUM>, a lookup circuit <NUM>, an action circuit <NUM>, and an output buffer <NUM> interconnected with one another in sequence. The outbound processing path 138b can include another set of processing circuits having an outbound packet buffer <NUM>' (shown as "OUT Packet Buffer" in <FIG>), a parser <NUM>', a lookup circuit <NUM>', and an action circuit <NUM>' interconnected with one another in sequence and in the opposite processing direction.

In accordance with embodiments of the disclosed technology, the packet processor <NUM> can also include a NIC buffer <NUM> and an inbound multiplexer <NUM> in the inbound processing path 138a. As shown in <FIG>, the NIC buffer <NUM> and the output buffer <NUM> are arranged to provide an output to the inbound multiplexer <NUM>. In turn, the inbound multiplexer <NUM> can be configured to receive input from each of the NIC buffer <NUM> and the output buffer <NUM> in the inbound processing path 138a and provide an output to the NIC <NUM>. The action circuit <NUM>' in the outbound processing path 138b can include an input from the lookup circuit <NUM>' and a first output to the TOR <NUM> and a second output to the NIC buffer <NUM>.

As shown in <FIG>, the packet processor <NUM> can also include a memory <NUM> containing data representing a flow table having one or more policies or rules <NUM>. The rules <NUM> can be configured by, for example, the virtual switch <NUM> or other suitable software components provided by the processor <NUM> to provide certain actions when corresponding conditions are met. Example conditions and actions are described in more detail below with reference to <FIG>. Even though the flow table is shown being contained in the memory <NUM> in the packet processor <NUM>, in other embodiments, the flow table may be contained in a memory (not shown) outside of the packet processor <NUM>, in the memory <NUM> (<FIG>), or in other suitable storage locations.

<FIG> shows an operation of the packet processor <NUM> when receiving an outbound RDMA packet from the first virtual machine <NUM>' or a virtualized function (e.g., a virtualized NIC) associated with the first virtual machine <NUM>'. As used herein, the term "virtualized function" generally refers to a virtualized portion of a physical resource at the host <NUM>. In the example shown in <FIG>, the outbound RDMA packet is shown as a RDMA connection request packet <NUM>' (referred to as "request packet"). In other examples, the outbound RDMA packet can also be a RDMA connection reply packet or other suitable types of RDMA packet. In <FIG> and other figures herein, solid connecting arrows represent a used processing path while dashed connecting arrows represent un-used processing path during certain operations.

As shown in <FIG>, the first virtual machine <NUM>' or a virtualized function associated therewith can transmit the request packet <NUM>' to the packet processor <NUM> via the NIC <NUM>. Upon receiving the request packet <NUM>', the packet processor <NUM> can store the received request packet <NUM>' in the outbound packet buffer <NUM>'. The outbound parser <NUM>' can parse at least a portion of the header of the request packet <NUM>' and forward the parsed header to the lookup circuit <NUM>' in the outbound processing path 138b. The lookup circuit <NUM>' can then attempt to match the request packet <NUM>' to a flow in the flow table in the memory <NUM> based on the parsed header and identify an action for the request packet <NUM>' as contained in the flow table. However, when lookup circuity <NUM>' cannot match the request packet <NUM>' to any existing flow in the flow table because the request packet <NUM>' may be a first packet for a new RDMA flow. As such, the action circuit <NUM>' can then attach an exception flag to the request packet <NUM>' and forward the request packet <NUM>" with the exception flag to the NIC buffer <NUM> instead of the TOR <NUM>.

As shown in <FIG>, the multiplexer <NUM> in the inbound processing path 138a can then retrieve the request packet <NUM>" with the exception flag and forward the request packet <NUM>" to the virtual switch <NUM> (or other suitable software components) via the NIC <NUM> for flow information associated with the request packet <NUM>". In response, as shown in <FIG>, the virtual switch <NUM> (or other suitable software components) can then generates data representing flow information of a flow to which the request packet <NUM>" belongs. The flow information can contain one or more rules <NUM> for the flow.

The virtual switch <NUM> can then transmit the created rules <NUM> to the packet processor <NUM> to be stored in the memory <NUM>. In certain embodiments, the virtual switch <NUM> can forward the request packet <NUM>" along with the rules <NUM> to the packet processor <NUM>, which in turn processes the request packet <NUM>" according to the rules <NUM>. In other embodiments, the virtual switch <NUM> can process the request packet <NUM>" (e.g., by encapsulating the request packet <NUM>" with a network address of the second host 106b in the underlay network <NUM> of <FIG>) and transmit the processed request packet <NUM> along with the rules <NUM> to the packet processor <NUM>. In such embodiments, the virtual switch <NUM> can also include a special flag with the request packet <NUM> indicating to the packet processor <NUM> that processing of the request packet <NUM> is complete and the request packet <NUM> is to be forwarded directly to the TOR <NUM>. In turn, the packet processor <NUM> can either process the request packet <NUM> according to the rules <NUM> received from the virtual switch <NUM> or forward the request packet <NUM> directly to the TOR <NUM> via the outbound processing path 138b. Upon receiving the request packet <NUM>, the TOR <NUM> can then forward the request packet <NUM> to the second host 106b via the underlay network <NUM> (<FIG>).

<FIG> are schematic diagrams illustrating a hardware packet processor <NUM> implemented at a host <NUM> in a distributed computing system <NUM> during processing of an inbound packet for establishing an RDMA connection in accordance with embodiments of the disclosed technology. In the illustrated embodiment, the inbound packet is shown as a request packet <NUM>. In other embodiments, the inbound packet can also be other suitable types of RDMA packet.

As shown in <FIG>, upon receiving the request packet <NUM> from the first host 106a, the TOR <NUM> can forward the request packet <NUM> to the packet processor <NUM> to be stored in the inbound packet buffer <NUM>. The inbound parser <NUM> can parse at least a portion of the header of the request packet <NUM> and forward the parsed header to the lookup circuit <NUM> in the inbound processing path 138a. The lookup circuit <NUM> can then attempt to match the request packet <NUM> to a flow in the flow table in the memory <NUM> based on the parsed header and identify an action for the request packet <NUM> as contained in the flow table. However, when lookup circuity <NUM>' cannot match the request packet <NUM> to any existing flow in the flow table because the request packet <NUM> may be a first packet for a RDMA connection. In response, the action circuit <NUM> can attach an exception flag to the request packet <NUM> and forward the request packet <NUM>' with the exception flag to the output buffer <NUM> for further processing.

As shown in <FIG>, the multiplexer <NUM> in the inbound processing path 138a can then retrieve the request packet <NUM>' with the exception flag and forward the request packet <NUM>' to the virtual switch <NUM> (or other suitable software components) via the NIC <NUM> for flow information associated with the request packet <NUM>'. In response, as shown in <FIG>, the virtual switch <NUM> (or other suitable software components) can then generates data representing a flow to which the request packet <NUM>' belongs and one or more rules <NUM> for the flow. The virtual switch <NUM> can then forward the request packet <NUM>' along with the rules <NUM> to the outbound processing path 138b. In turn, the outbound processing path 138b can process the request packet <NUM>' according to the rules <NUM> to, for example, decapsulate the request packet <NUM>' and remove a portion of the header of the request packet <NUM>' containing the underlay network address of the second host 106b. The action circuit <NUM>' can then forward the processed request packet <NUM>" to the NIC buffer <NUM> instead of the TOR <NUM> according to the rules <NUM> received from the virtual switch <NUM>. Then, the multiplexer <NUM> can retrieve the processed request packet <NUM>" and forward the retrieved request packet <NUM>" to the second virtual machine <NUM>" or a virtual function associated with the second virtual machine <NUM>" for further processing. Upon receiving the request packet <NUM>", the second virtual machine <NUM>" can then generate a reply packet (not shown) to the requested RDMA connection with the first virtual machine <NUM>' and transmit the reply packet to the packet processor <NUM> via the NIC <NUM>. The reply packet can then be processed by the packet processor <NUM> in operations generally similar to those described above with reference to <FIG>.

Upon establishing the RDMA connection between the first and second virtual machines <NUM>' and <NUM>", data packets <NUM> can be transmitted, as shown in <FIG> and <FIG>. As shown in <FIG>, upon receiving a data packet <NUM> from, for example, the first virtual machine <NUM>' (<FIG>) at the first host 106a, the packet processor 138a at the second host 106b can store the data packet <NUM> in the inbound packet buffer <NUM>, parsing a header of the data packet <NUM> with the parser <NUM>, matching the data packet <NUM> with a flow with corresponding rules <NUM> with the lookup circuit <NUM>, and perform actions identified by the rules <NUM> with the action circuit <NUM>. The identified rules <NUM> can include those receive from the virtual switch <NUM> during connection establishment as described above with reference to <FIG>. Examples actions can include decapsulating the data packet <NUM> to expose virtual network addresses contained in the header of the data packet <NUM>, or other suitable operations. The action circuit <NUM> can then forward the processed data packet <NUM>' to the output buffer <NUM>. In turn, the multiplexer <NUM> can retrieve the processed data packet <NUM>' and forward the data packet <NUM>' to the second virtual machine <NUM>' or a virtual function associated therewith. Based on the received data packet <NUM>', the second virtual machine <NUM>" can, for example, write certain data in the data packet <NUM>' directly into a memory space allocated to the second virtual machine <NUM>" on the second host 106b. As shown in <FIG>, processing of outbound RDMA data packet <NUM> can be generally similar to processing the inbound data packet <NUM> in <FIG> except the outbound processing path 138b is utilized.

<FIG> is a schematic diagram illustrating example conditions and corresponding actions for a rule <NUM> (<FIG>) as an entry in a flow table in accordance with embodiments of the disclosed technology. In certain embodiments, as shown in <FIG>, the rule <NUM> can include actions upon matching packets in a MAT model. When creating an entry, a network controller (not shown) can be expressive while reducing fixed policy in a data plane.

As shown in <FIG>, the rule <NUM> can include a condition list containing multiple conditions <NUM>, and one or more corresponding actions <NUM>. Example conditions <NUM> can include source/destination MAC, source/destination IP, source/destination TCP port, source/destination User Datagram Protocol ("UDP") port, general routing encapsulation key, Virtual Extensible LAN identifier, virtual LAN ID, or other metadata regarding the payload of the packet. Conditions <NUM> can have a type (such as source IP address) and a list of matching values (each value may be a singleton, range, or prefix). For a condition to match a packet, any of the matching values can match as in an OR clause. For an rule <NUM> to match, all conditions <NUM> in the rule <NUM> match as in an AND clause.

The action <NUM> can also contain a type and a data structure specific to that type with data needed to perform the action. For example, an encapsulation rule <NUM> can takes as input data a source/destination IP address, source/destination MAC address, encapsulation format and key to use in encapsulating the packet. As shown in <FIG>, the example actions can include allow/circuit a packet according to, for example, ACLs, network name translation (L3/L4), encapsulation/decapsulation, quality of service operations (e.g., rate limit, mark differentiated services code point, metering, etc.), encryption/decryption, stateful tunneling, and routing (e.g., equal cost multiple path routing).

The rule <NUM> can be implemented via a callback interface, e.g., initialize, process packet, and de-initialize. If a rule type supports stateful instantiation, the virtual switch <NUM> (<FIG>) or other suitable types of process handler can create a pair of flows in the packet processor <NUM> (<FIG>). Flows can also be typed and have a similar callback interface to rules <NUM>. A stateful rule <NUM> can include a time to live for a flow, which is a time period that a created flows can remain in a flow table after a last packet matches unless expired explicitly by a TCP state machine. In addition to the example set of actions <NUM> in <FIG>, user-defined actions can also be added, allowing the network controllers to create own rule types using a language for header field manipulations.

<FIG> and <FIG> are flowcharts illustrating processes for RDMA network traffic routing in accordance with embodiments of the disclosed technology. Though the processes <NUM> and <NUM> are described below in light of the distributed computing system <NUM> of <FIG>, in other embodiments, the processes can also be performed in other computing systems with similar or different components.

As shown in <FIG>, the process <NUM> can include receiving an outbound packet at stage <NUM>. In certain embodiments, the outbound packet can be an RDMA connection request packet, RDMA connection reply packet, or other suitable types of packet received via a NIC <NUM> (<FIG>) from an application <NUM> (<FIG>), a virtual machine <NUM> (<FIG>), or other software components on a host <NUM> (<FIG>). The process <NUM> can then include matching the outbound packet with a flow in a flow table at stage <NUM>. In certain embodiments, matching the inbound packet can include parsing a header of the inbound packet, matching at least a portion of the header to an entry in a flow table, and identifying an action corresponding to the entry. In other embodiments, matching the inbound packet can also include forwarding the inbound packet to a software component for further processing when an entry in the flow table cannot be located as matching the inbound packet.

The process <NUM> can then include a decision stage <NUM> to determine whether the outbound packet is matched to at least one flow in the flow table. In response to determining that the outbound packet is matched to at least one flow in the flow table, the process <NUM> can include performing actions associated with the identified flow and forwarding the processed outbound packet to the TOR at stage <NUM>. The TOR <NUM> can then forward the outbound packet to a suitable destination in the distributed computing system <NUM> (<FIG>) via the overlay/underlay network <NUM>'/<NUM>. In response to determining that the outbound packet is not matched to at least one flow in the flow table, the process <NUM> can include forwarding the outbound packet to a NIC buffer <NUM> (<FIG>) with an exception flag at stage <NUM>. The process <NUM> can then include forwarding the inbound packet with the exception flag to a software component in the host and receiving flow information from the software component at stage <NUM>. The process <NUM> can further include performing outbound processing of the packet and forwarding the packet to the TOR at stage <NUM>.

<FIG> illustrates a process <NUM> for inbound RDMA network traffic routing using a request packet as an example in accordance with embodiments of the disclosed technology. The process <NUM> can include receiving a request packet from a TOR <NUM> (<FIG>) at stage <NUM>. The process <NUM> can then include matching the received request packet with a flow in a flow table at stage <NUM>. The matching operations can be generally similar those described above with reference to the process <NUM> in <FIG>.

The process <NUM> can then include a decision stage <NUM> to determine whether the request packet matches a flow in the flow table. In response to determining that the request packet matches a flow in the flow table, the process <NUM> can include performing actions associated with the identified flow and forwarding the request packet to the NIC <NUM>, by, for example, copying the request packet into a buffer of the NIC <NUM> at stage <NUM>. Otherwise, the process <NUM> can include forwarding the request packet to a NIC buffer <NUM> (<FIG>) with an exception flag at stage <NUM>. The process <NUM> can then include forwarding the request packet with the exception flag to a software component in the host and receiving flow information from the software component at stage <NUM>. The process <NUM> can further include performing outbound processing of the packet and forwarding the packet to the TOR at stage <NUM>.

<FIG> is a computing device <NUM> suitable for certain components of the distributed computing system <NUM> in <FIG>. For example, the computing device <NUM> can be suitable for the hosts <NUM>, the client devices <NUM>, or the platform controller <NUM> of <FIG>. In a very basic configuration <NUM>, the computing device <NUM> can include one or more processors <NUM> and a system memory <NUM>. A memory bus <NUM> can be used for communicating between processor <NUM> and system memory <NUM>.

Depending on the desired configuration, the processor <NUM> can be of any type including but not limited to a microprocessor (µP), a microcontroller (µC), a digital signal processor (DSP), or any combination thereof. The processor <NUM> can include one more levels of caching, such as a level-one cache <NUM> and a level-two cache <NUM>, a processor core <NUM>, and registers <NUM>. An example processor core <NUM> can include an arithmetic logic unit (ALU), a floating point unit (FPU), a digital signal processing core (DSP Core), or any combination thereof. An example memory controller <NUM> can also be used with processor <NUM>, or in some implementations memory controller <NUM> can be an internal part of processor <NUM>.

Depending on the desired configuration, the system memory <NUM> can be of any type including but not limited to volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, etc.) or any combination thereof. The system memory <NUM> can include an operating system <NUM>, one or more applications <NUM>, and program data <NUM>. As shown in Figure <NUM>, the operating system <NUM> can include a hypervisor <NUM> for managing one or more virtual machines <NUM>. This described basic configuration <NUM> is illustrated in <FIG> by those components within the inner dashed line.

The computing device <NUM> can have additional features or functionality, and additional interfaces to facilitate communications between basic configuration <NUM> and any other devices and interfaces. For example, a bus/interface controller <NUM> can be used to facilitate communications between the basic configuration <NUM> and one or more data storage devices <NUM> via a storage interface bus <NUM>. The data storage devices <NUM> can be removable storage devices <NUM>, non-removable storage devices <NUM>, or a combination thereof. Examples of removable storage and non-removable storage devices include magnetic disk devices such as flexible disk drives and hard-disk drives (HDD), optical disk drives such as compact disk (CD) drives or digital versatile disk (DVD) drives, solid state drives (SSD), and tape drives to name a few. Example computer storage media can include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules, or other data. The term "computer readable storage media" or "computer readable storage device" excludes propagated signals and communication media.

The system memory <NUM>, removable storage devices <NUM>, and non-removable storage devices <NUM> are examples of computer readable storage media. Computer readable storage media include, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other media which can be used to store the desired information and which can be accessed by computing device <NUM>. Any such computer readable storage media can be a part of computing device <NUM>. The term "computer readable storage medium" excludes propagated signals and communication media.

The computing device <NUM> can also include an interface bus <NUM> for facilitating communication from various interface devices (e.g., output devices <NUM>, peripheral interfaces <NUM>, and communication devices <NUM>) to the basic configuration <NUM> via bus/interface controller <NUM>. Example output devices <NUM> include a graphics processing unit <NUM> and an audio processing unit <NUM>, which can be configured to communicate to various external devices such as a display or speakers via one or more A/V ports <NUM>. Example peripheral interfaces <NUM> include a serial interface controller <NUM> or a parallel interface controller <NUM>, which can be configured to communicate with external devices such as input devices (e.g., keyboard, mouse, pen, voice input device, touch input device, etc.) or other peripheral devices (e.g., printer, scanner, etc.) via one or more I/O ports <NUM>. An example communication device <NUM> includes a network controller <NUM>, which can be arranged to facilitate communications with one or more other computing devices <NUM> over a network communication link via one or more communication ports <NUM>.

The network communication link can be one example of a communication media. Communication media can typically be embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and can include any information delivery media. A "modulated data signal" can be a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media can include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), microwave, infrared (IR) and other wireless media. The term computer readable media as used herein can include both storage media and communication media.

The computing device <NUM> can be implemented as a portion of a small-form factor portable (or mobile) electronic device such as a cell phone, a personal data assistant (PDA), a personal media player device, a wireless web-watch device, a personal headset device, an application specific device, or a hybrid device that include any of the above functions. The computing device <NUM> can also be implemented as a personal computer including both laptop computer and non-laptop computer configurations.

Claim 1:
A method (<NUM>) for routing Remote Direct Memory Access, RDMA, network traffic in a distributed computing system having a plurality of hosts interconnected by a computer network, the individual hosts having a main processor, a network interface card, NIC, and a hardware packet processor operatively coupled to one another, the method comprising:
receiving (<NUM>), from the computing network, a connection request packet at the packet processor of a host for establishing an RDMA connection with a virtual machine on the host and having access to the NIC via a corresponding virtual function;
determining (<NUM>) whether the received connection request packet matches a flow in a flow table in the packet processor; and
in response to determining that the received connection request packet does not match with any flow in the flow table,
forwarding (<NUM>), via the NIC, the connection request packet with an exception flag to a software component provided by the main processor;
receiving (<NUM>), via the NIC, a copy of the connection request packet and flow information of a flow corresponding to the connection request packet from the software component;
processing (<NUM>) the copy of the connection request packet according to the received flow information; and
forwarding the processed connection request packet back to the NIC to be delivered to the virtual function of the virtual machine, thereby allowing the virtual machine to establish the RDMA connection.