Graphics processing unit (GPU) as a programmable packet transfer mechanism

Discloses is an apparatus including a network interface controller (NIC), memory, and an accelerator. The accelerator can include a direct memory access (DMA) controller configured to receive data packets from the NIC and to provide the data packets to the memory. The accelerator can also include processing circuitry to generate processed data packets by implementing packet processing functions on the data packets received from the NIC, and to provide the processed data packets to at least one processing core. Other methods, apparatuses, articles and systems are also described.

TECHNICAL FIELD

Embodiments relate generally to information processing and communications and, more particularly, to circuits that communicate data from a host computer system over a computer network.

BACKGROUND

Currently, network cards receive data packets and pass them on to a computing device for processing. As the need for speed in processing network data increases, efforts have been generally directed to improving packet processing speed.

DETAILED DESCRIPTION

Network interface performance has been increasing in recent years. However, packet processing has not kept pace with network interface performance gains.

Various parameters and conditions can affect the performance of packet processing. For example, central processing unit (CPU) speed and utilization, interrupt overhead, bus bandwidth, memory latency, and I/O latency can affect the performance of packet processing applications. Some examples of CPU overhead that affects packet processing can include overhead involved with memory allocation and deallocation for each packet, copying data between kernel and user-space, expensive cache misses, per-packet system calls, and the transformation of the parallelized processing of packets by the queues of multi-queue network interface controllers (NICs) to a serialized one. The latter condition can occur when all packets converge to one single point, thus creating a bottleneck.

The architecture of Graphical Processing Units (GPU) can enable highly parallel processing in GPUs, and this capability can be used to improve packet processing performance to address the above concerns, as well as other concerns. Programmable GPUs can act as co-processors by receiving code, data, and commands from the host CPU. Recent efforts in GPU development have been directed to increasing parallelism. As such, GPUs today can have hundreds or thousands cores. Together, the thousands of cores in a GPU can provide greater throughput than the single CPU core for many types of highly parallel tasks. GPU usefulness can be enhanced by launching multiple threads per GPU core to provide high thread-level parallelism. By using high thread-level parallelism, a GPU can reduce memory access latency by executing a considerable number of threads. Similarly, the processing power of the GPU can be used for computing-intensive operations that run on software routers and thus overcome the bottleneck posed by the CPU. Accordingly, embodiments described herein take advantage of the high thread-level parallelism capability of GPUs to perform packet processing functions to reduce or eliminate at least one source of system bottleneck and slowdown.

Embodiments that can use a GPU in combination with a NIC to perform packet processing are described herein. The NIC can also be referred to as a host-fabric interface (HFI). Using the GPU and NIC in combination can make it appear to a host that a single integrated Ethernet device can provide various kinds of programmable offloads and network virtualization. As disclosed herein, this unique combination of components and techniques combines the GPU and CPU operation to result in improved performance for data center operators and other networking customers. In addition, embodiments can allow for more flexible and programmable packet processing functions.

In some prior systems, data packets enter a CPU directly from the NIC. In contrast, in embodiments disclosed herein, the data packets can be processed prior to entering the CPU by combining the NIC and GPU as one unit. Thus, the data packets can be pre-processed on the GPU after NIC processing, before entering the CPU, and vice versa when packets are transmitted. Stated another way, the CPU and the NIC can be daisy-chained so as to appear transparently to the system software as a single complex device. The MC can transfer (e.g. via a direct memory access (DMA) controller) data packets directly to and from the GPU and the GPU can apply additional packet processing functions before making the final transfer to and from the CPU.

An embodiment of the combined NIC and GPU can be represented via a newly defined virtual I/O model that combines both hardware functions of the NIC and the GPU. The combined GPU and NIC functions as a single software layer can be viewed as a “virtual NIC.” The meta-data presented by the virtual NIC may represent parameters such as the combined offload capability of the GPU and NIC, and the offloads performed or to be performed on a packet by the GPU and NIC, as well as other parameters. This allows software applications using the combined GPU/NIC to view and/or control the offloads performed.

Using GPUs for packet processing can provide improved performance for specific offloads. For example, using GPUs for packet processing can show increased offload performance for stateless packet processing. As disclosed herein, computational time for some algorithms can be improved when various data packets are processed on a GPU. The increase in speed can be limited by factors such as, but not limited to, the bandwidth between a CPU and the GPU, and availability of computation resources.

The bandwidth issues can be addressed using a hardware block supporting queue management (HQM) or equivalent interface improvements and pipelining of input stages as disclosed herein. The usage of HQM or equivalent interface improvements and pipelining can help to minimize chatter between virtual devices and processing cores due to the virtual device providing additional processing of packets before they are transferred to the processing cores of the CPU. Offloading workload to the GPU can make more computation resources available for other software functions on cores that may or may not be limited by computational resource availability.

FIG. 1illustrates a schematic of an architecture100in accordance with some embodiments. As shown inFIG. 1, the architecture100can include a packet processing application102, a software device driver104, at least one processing core106, a NIC108, and an accelerator110. In some embodiments, the accelerator is a GPU. However, embodiments are not limited thereto and the accelerator110can be implemented by using, for example, a field programmable gate array (FPGA).

During operation some data packet processing functions can be done by the GPU110as described later herein. Consequently, the at least one core106(e.g., at least a first core, and possibly a second core or more cores) of a CPU that runs packet processing applications can be freed from running these functions. As a result, the cores of the CPU can be utilized to process other data packets.

In a traditional computing device, the GPU is used for graphics processing and video stream transcoding. As disclosed herein, the GPU110can be repurposed to cooperate with the NIC108such that they appear as a single device (apparatus200,FIG. 2) to software. In other words, software that utilizes the data packets processed by the apparatus200combination will in most embodiments be unaware that the data packets were not processed by the processing core106. Consistent with embodiments disclosed herein, data packets can be transferred from the NIC108to the GPU110and back to the NIC108without entering the cores106. In addition, the workload can be offloaded from partial cores as well. For example, some functions can be offloaded from a core106to move functions previously performed in software on the core106to a combined device, such as the apparatus200(FIG. 2).

FIG. 2illustrates an apparatus200in accordance with some embodiments. The apparatus200can include the NIC108to communicate with one or more hardware components. The apparatus200can further include an accelerator110(e.g., a GPU, FPGA, etc.). The apparatus200can also be referred to in some embodiments as a virtual NIC, as the apparatus200combines accelerator110and NIC108functions under a single software layer. The accelerator110can include memory202and a DMA controller204configured to receive data packets from the NIC108and to move the data packets to the memory202. GPUs have their own on-board device memory, which can be as large as 6 GB or more. GPUs can access the device memory directly, so data can be copied via DMA over the PCIe bus. The PCIe bus is also used for CPU-GPU communication, such as launching GPU kernels and synchronizing states as the computation progresses.

The accelerator110can include processing circuitry206that generates processed data packets by implementing packet processing functions on the data packets received from the NIC108. The processing circuitry206can also provide the processed data packets to at least one processing core106(FIG. 1) of a CPU. It will be understood that any or all of the functions and operations performed by processing circuitry206can be executed with hardware, software, firmware, or any combination thereof. In some embodiments, processing circuitry206can comprise one or more processing cores. The processing circuitry206can perform the various operations described herein with regard to the data packet processing pipeline (FIG. 3) by implementing instructions208(e.g., “software”) stored within the memory202. As such, the memory202can include machine-readable media.

FIG. 3illustrates an example data packet processing pipeline300in accordance with some embodiments. As shown inFIG. 3, a plurality of CPU cores302can run an application that can use a virtual NIC interface304to send and receive data packets using the apparatus200.

The GPU110can include a variety of data packet processing, receiving, and transmitting functions. For example, the GPU110can include a data packet classifier306that can maintain a set of rules that classify which packet flow a packet belongs to based on the contents of the packet headers. Actions taken for each class (e.g., packet scheduling in an output queue, routing decisions, etc.) can further be performed in the GPU110or in other systems. The GPU110can include an inline decryption function308. The GPU110can also include ingress packet traffic management310functions, which delays some or all datagrams to bring them into compliance with a desired traffic profile to enhance or guarantee performance, latency, and usable bandwidth for some kinds of packets. The GPU110can additionally include packet distribution functions312that can be used to process received data packets before sending the processed data packets to the plurality of CPU cores302. The packet distribution functions312can also include another DMA controller (not shown inFIG. 3) interfacing to/from the virtual NIC interface304to assist with distribution of processed data packets to the plurality of CPU cores302. In other embodiments, a mesh connection can be provided.

The GPU110can perform other functions328for ingress QoS, such as metering functions, marking functions, policing, traffic class separation, segmentation and reassembly (SAR).

The GPU110can also include egress packet traffic management314, and egress QoS functions316. Egress QoS functions can include scheduling and shaping. Scheduling functions can include first come first serve (FCFS), round robin, string priority, earliest deadline first schemes, and generalized processor sharing based on weight factors associated with packet flows. Shaping functions can decrease the burstiness of traffic by, for example, delaying the flow of packets designated as less important than those of prioritized traffic streams. The GPU110can include inline encryption318functionality for handling data packets received from the plurality of CPU cores302. The GPU110can also include a packet transmitter320, and a packet receiver322. While some of the available interconnections and communication paths have been shown inFIG. 3, it will be appreciated that other functionalities can be provided and these functionalities can communicate in various ways that are not shown inFIG. 3for clarity purposes only.

The GPU110can include a transport agent324to transport packets to and from the NIC108(or HFI), to hand over packets for other packet processing within the GPU, and to pass packets to and from the virtual NIC interface304. In other words, the transport agent324can be considered an interworking function to track the state of packet processing and to handle NIC108interfaces. In some embodiments, the DMA controller204functions can be incorporated or viewed as a part of the transport agent324functions. The transport agent324can provide additional processing to packets by, for example, adding or stripping headers or addresses from data packets, among other operations, whether for providing the packets to the cores302or for receiving packets from the cores302for transmission to the network. For simplicity, the transport agent324has been shown in two locations inFIG. 3although as described above the transport agent324can operate in various aspects of the pipeline300. Functions of the transport agent324are also shown inFIGS. 4A and 4B.

In addition, the various functions and the sequence of the functions can be programmed and modified depending on application needs. For example, if certain data packets do not need to be encrypted or do not arrive encrypted, inline decryption function308and inline encryption318can be bypassed or otherwise omitted. In addition, the GPU110can interface with NIC108packet receiving and transmitting functions. Furthermore, an exception handling path can utilize CPU intervention with regard to NIC108operations.

FIGS. 4A and 4Billustrate examples of data packet ingress and egress in accordance with some embodiments. In addition,FIGS. 4A and 4Billustrate examples of potential communications mechanisms between various stages defined via at least interrupts and user level interrupts. For example, as shown inFIGS. 4A and 4B, operations410and452(e.g., transmit (TX) and receive (RX) packet indications) can be implemented as interrupts, polling, or user level interrupt mechanisms. Similarly, operations426and454(e.g., virtual NIC TX and RX packet indications) can be implemented using interrupt, polling, user level interrupt, or callback mechanisms.

As an example,FIGS. 4A and 4Billustrate how a transport agent324can interface with memory202and the various functional blocks shown inFIG. 3. The packet engines can implement various stages of packet processing functions. In operation402, a data packet can arrive from a network400at NIC108. In operation404, the NIC can provide the data packet to memory202(or to a DMA controller204, which can move the data packet to memory202as described earlier herein). In operation406, the NIC108can provide a data packet descriptor to the memory202. The memory202can then provide the data packet to data packet classifier306in operation408. Additionally or alternatively, the NIC108can generate an interrupt (or use any other mechanism such as polling, or user-level interrupt mechanisms) in operation410to signal to the transport agent324that a data packet has been received, and the transport agent324can provide the data packet in operation412to the data packet classifier306. In operation414, the data packet classifier306can provide the classified packet to the transport agent324, which can in turn add classification headers or other information to the classified packet based on the information provided by the data packet classifier.

In operation416, the transport agent324can provide the classified packet (e.g., the packet generated at operation414) to the inline decryption function308. Alternatively, in operation418, the memory202can provide an encrypted data packet to the inline decryption function308. In operation420, the inline decryption function308provides the decrypted packet to the memory202. Additionally or alternately, in operation422, the inline decryption function308provides the decrypted data packet to the transport agent324.

In operation424, the transport agent324performs load balancing to determine which core/s302should receive packets. The transport agent324can split packets (on a per-packet basis, a per-flow basis or other basis) among multiple paths (e.g., to different cores) to achieve load balancing. The transport agent324can use algorithms such as round-robin scheduling or schemes based on detection of utilization levels for various core/s302to perform load balancing.

In operation426, the transport agent324uses interrupts, polling, user level interrupts or a callback mechanism over the virtual NIC interface304to provide the processed data packet to a core302. Alternatively, memory202can provide the packet data to the virtual NIC interface304.

In operation430, the CPU core302can perform any packet processing, such as by providing data packets to applications running on the core302. In some embodiments, the CPU core302can perform transmissions of data packets. Accordingly, the core302can indicate the availability of data packets for transmission in operation432through mechanisms that can include interrupts, polling, user-level interrupts, callbacks, etc. In some embodiments, in operation434, the core302can provide the data packets to the transport agent324and the transport agent324can provide the data packet to an inline encryption318engine for encryption in operation436. In other embodiments, in operation432, decrypted packets can be provided to the memory202and the memory202can provide the decrypted packet for encryption in operation438. In operation442, encrypted data packets are provided to the transport agent324. In operation444, the transport agent324can indicate that the packets are ready for transmission, to the memory202and in operation446, the transport agent324can request that the NIC108transmit the corresponding data packet. In operations450,452, and454, the NIC108can transmit the packet to the network400and provide confirmation to the transport agent324, which in turn provides confirmation to the core302.

The various transmission pathways shown inFIGS. 4A and 4Bcan be altered depending on the application. For example, as indicated above, certain data packets may not need to be encrypted and thus, the transmission pathway for those data packets can omit a pathway that would route them to encryption circuitry. In addition, after processing by the application cores302, the data packets may need to be encrypted before transmission to the network400. Thus, the transmission pathway can be altered to include a pathway that would route the data packets to encryption circuitry. Stated another way, the transmission pathways can be static or dynamic depending on the application.

EXAMPLE METHODS

Various methods can be implemented in accordance with various embodiments to perform functions of usage models described above, as well as other usage models.FIG. 5is a flow diagram of an example method500in accordance with some embodiments. An apparatus200, as described here or elements thereof (e.g., GPU110, NIC108, etc.), can perform operations of the example method500. Accordingly, the example method500will be described with reference to components ofFIGS. 1-4. For example, processing circuitry206can perform one or more operations of example method500.

The example method500begins with operation502with the processing circuitry206receiving data packets. As described herein the data packets can be received from the NIC108.

The example method500continues with operation504with processing circuitry206processes the data packets. For example, the processing circuitry206can perform any of the functions in the data packet processing pipeline shown inFIG. 3. Consistent with embodiments disclosed herein, the data packets are processed such that the software operating on the CPU cores302are not aware that the data packets have been processed by the GPU110beforehand. In other words, the CPU cores302see the NIC108and GPU110as a single integrated device (e.g., the apparatus200(FIG. 2)) instead of two separate components.

The example method500continues with operation506with processing circuitry206transmitting the processed data packets to the CPU cores302. Once the GPU110has processed the data packets, the processed data packets can then be sent to the CPU cores302for use by various applications.

The example method500of receiving, processing and transmitting data can also be carried out in reverse. For example, the GPU110can receive data packets from the CPU cores302, process the data packets, and transmit the processed data packets to the NIC108for transmission to the network400.

In embodiments, the example method500can include additional operations such as, for example, detecting traffic load characteristics corresponding to the plurality of packet streams and directing one or more packet streams to the one or more of the CPU cores302based on traffic load characteristics and load balancing performed by the GPU110.

Examples, as described herein, may include, or may operate on, logic or a number of circuits, components, modules, or mechanisms. Modules are tangible entities (e.g., hardware) capable of performing specified operations and may be configured or arranged in a certain manner. In an example, circuits may be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner as a module. In an example, at least a part of one or more computer systems (e.g., a standalone, client or server computer system) or one or more processors of the GPU110may be configured by firmware or software (e.g., instructions208(FIG. 2), an application portion, or an application) as a module that operates to perform specified operations. In an example, the software may reside on at least one machine-readable medium. In an example, the software, when executed by the underlying hardware of the module (e.g., the GPU110), can include instructions208(FIG. 1) to cause the hardware to perform the specified operations.

For example, instructions208can cause GPU110to detect characteristics of a plurality of packet streams received at a NIC108. In various embodiments, the instructions208can optionally cause the hardware to detect traffic load characteristics corresponding to the plurality of packet streams and to direct one or more packet streams to the one or more CPU cores302based on traffic load characteristics.

While a machine-readable medium may include a single medium, the term “machine-readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers).

The term “machine-readable medium” may include any medium that is capable of storing, encoding, or carrying instructions208for execution by a machine (e.g., the apparatus200, GPU110, or any other processor or circuitry) and that cause the machine to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. In other words, the processing circuitry206(FIG. 2) can include instructions and can therefore be termed a machine-readable medium in the context of various embodiments. Other non-limiting machine-readable medium examples may include solid-state memories, and optical and magnetic media. Specific examples of machine-readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

The instructions208may further be transmitted or received over a communications network using a transmission medium utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), TCP, user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks ((e.g., channel access methods including Code Division Multiple Access (CDMA), Time-division multiple access (TDMA), Frequency-division multiple access (FDMA), and Orthogonal Frequency Division Multiple Access (OFDMA) and cellular networks such as Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), CDMA 2000 1×* standards and Long Term Evolution (LTE)), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802 family of standards including IEEE 802.11 standards (WiFi), IEEE 802.16 standards (WiMax®) and others), peer-to-peer (P2P) networks, or other protocols now known or later developed.

The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by hardware processing circuitry, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

Additional Notes & Examples

Example 1 includes subject matter (such as a device, processor, computer device, electrical apparatus, etc.) including a network interface controller (NIC); memory; and an accelerator comprising: a direct memory access (DMA) controller configured to receive data packets from the NIC and to move the data packets to the memory; and processing circuitry configured to generate processed data packets by implementing packet processing functions on the data packets received from the NIC, and provide the processed data packets to at least one processing core of a central processing unit (CPU).

In Example 2, the subject matter of Example 1 can optionally include wherein the accelerator comprises a graphics processing unit (GPU).

In Example 3, the subject matter of Example 2 can optionally include a transport agent to transport packets between the NIC and the accelerator.

In Example 4, the subject matter of Example 3 can optionally include wherein the packet processing functions include packet classifier functions.

In Example 5, the subject matter of Example 4 can optionally include wherein the transport agent is further configured to provide packets to the packet classifier functions.

In Example 6, the subject matter of Example 4 can optionally include wherein the transport agent is further configured to track a state of packet processing.

In Example 7, the subject matter of Example 2 can optionally include wherein the accelerator comprises packet transmit and packet transmit functions.

In Example 8, the subject matter of any one of Examples 1-7 can optionally include wherein the NIC and the accelerator comprise a software layer that forms a virtual NIC interface to provide the processed data packets to the at least one processing core.

In Example 9, the subject matter of Example 8 can optionally include wherein the virtual NIC presents offload capability information for the accelerator and the NIC to the at least one processing core.

In Example 10, the subject matter of Example 9 can optionally include a second DMA controller for providing the processed data packets to the virtual NIC interface.

In Example 11, the subject matter of any of Examples 1-10 can optionally include wherein the accelerator comprises a field programmable array (FPGA).

In Example 12, the subject matter of any of Examples 1-11 can optionally include wherein the processed data packets are provided to the at least one processing core using a mesh connection.

In Example 13, the subject matter of any of Examples 1-12 can optionally include wherein the processing circuitry is configured to perform load balancing operations.

In Example 14, the subject matter of any of Examples 1-13 can optionally include circuitry for receiving data packets from the at least one processing core for transmission using the NIC.

In Example 15, the subject matter of any of Examples 1-14 can optionally include wherein the packet processing functions include decryption functions for data packets provided to the at least one processing core, and encryption functions for data packets received from the at least one processing core.

In Example 16, the subject matter of Example 15 can optionally include wherein the packet processing functions includes scheduling functions for scheduling transmission of data packets received from the at least one processing core.

Example 17 includes subject matter (such as a data center system, networking system, or other system) including a network interface controller (NIC) configured to transmit and receive data from a network; a central processing unit (CPU) including a first core and a second core; an accelerator in electrical communication with the CPU and the NIC, the accelerator including a direct memory access (DMA) controller, processing circuitry and a memory storing instructions that, when executed by the accelerator, cause the accelerator to perform operations comprising: generating processed data packets by implementing packet processing functions on the data packets received from the NIC, and providing the processed data packets to at least one of the first core or the second core.

In Example 18, the subject matter of Example 17 can optionally include wherein the packet processing functions include at least one of packet encryption, packet decryption, and packet classification.

In Example 19, the subject matter of any of Examples 17-18 can optionally include wherein the accelerator is further configured to perform load balancing operations to balance processing loads between at least the first core and the second core.

Example 20 includes subject matter such as a machine-readable medium including instructions that, when executed on a machine (such as a control device, electrical apparatus, graphics processing unit (GPU), processor, NIC card, or other Ethernet device, etc.) cause the machine to perform operations comprising: providing data packets from a network interface controller (NIC) at a direct memory access (DMA) controller of the GPU; processing the data packets using packet processing functions, the packet processing functions including at least one of a decryption function and a packet classification function, to generate processed data packets; and providing the processed data packets to at least one processing core of a central processing unit (CPU).

In Example 21, the subject matter of Example 20 can optionally include wherein the instructions further include providing the processed data packets to two or more processing cores of the CPU according to a load balancing operation.

In Example 22, the subject matter of any of Examples 20-21 can optionally include receiving data packets from the at least one processing core; and providing the data packets to the NIC according to a scheduling function.

Example 23 includes a graphics processing unit (GPU) having means for transmitting and receiving data packets to and from a network; means for transmitting processed data packets to a plurality of cores of a central processing unit (CPU); and means for generating the processed data packets for transmission to the plurality of cores.

In Example 24, the subject matter of Example 23 can optionally include wherein the means for transmitting the processed data packets to the plurality of cores includes a mesh connection.

In Example 21, the subject matter of Example 19 can optionally include means for performing load balancing between the plurality of cores.