Method of handling network traffic through optimization of receive side scaling

An information handling system includes a plurality of processors that each includes a cache memory, and a receive side scaling (RSS) indirection table with a plurality of pointers that each points to one of the processors. A network data packet received by the information handling system determines a pointer to a first processor. In response to determining the pointer, information associated with the network data packet is transferred to the cache memory of the first processor, The information handling system also includes a process scheduler that moves a process associated with the network data packet from a second processor to the first processor, and an RSS module that directs the process scheduler to move the process and associates the first pointer with the processor in response to directing the process scheduler.

FIELD OF THE DISCLOSURE

The present disclosure generally relates to information handling systems, and more particularly relates to handling network traffic in an information handling system through optimization of receive side scaling.

BACKGROUND

DETAILED DESCRIPTION OF THE DRAWINGS

The following description in combination with the Figures is provided to assist in understanding the teachings disclosed herein. The description is focused on specific implementations and embodiments of the teachings, and is provided to assist in describing the teachings. This focus should not be interpreted as a limitation on the scope or applicability of the teachings. Other teachings can be used in this application, and the teachings can be used in other applications and with different types of architectures, such as a client-server architecture, a distributed computing architecture, or a middleware server architecture and associated resources.

In a particular embodiment, information handling system100includes a network interface110, a main memory120, a group of processors130each including one or more central processing unit (CPU) cores and a cache memory, and a group of user processes140. Network interface110represents an interface device between information handling system100and an external network (not illustrated), and operates to provide receive side scaling for network traffic received by the information handling system. In a particular embodiment, when network traffic150is received by network interface110, the information152included in the network traffic is sent to the cache of the processor130that is associated with the network flow, and an interrupt154is provided to the CPU core associated with the network traffic. When the CPU core receives interrupt154, the CPU core retrieves the data156and provides the data158to the user process140associated with the network traffic. In another embodiment, when network traffic160is received by network interface110, the information162included in the network traffic is sent to a receive buffer122of main memory120, and an interrupt164is provided to the CPU core associated with the network traffic. The CPU core retrieves the information166to the cache memory of the processor130that is associated with the network flow, retrieves the data168from the cache memory, and provides the data170to the user process140associated with the network traffic.

Network interface110can be implemented as a network interface card (NIC) of information handling system100or as a network capability that resides on a system board of the information handling system. In implementing receive side scaling (RSS), network interface110can provide interrupts154and164as hardware interrupts, as software interrupts, as virtual interrupts in a virtual machine environment, or as a combination thereof. In a particular embodiment, the RSS capability of network interface110is limited as to the number of available RSS channels, and by extension, to the number of processors130that can be used to handle network traffic. In particular, the number of RSS channels can be less than the number of processors130in information handling system100. In one embodiment, the processors130that are associated with the RSS channels are predetermined by a firmware component of network interface110when information handling system100is powered on. In another embodiment network interface110operates to determine if a particular processor130is idle or lightly loaded, and modifies the associations between the RSS channels and the processors to provide the task of handling network traffic to idle or lightly loaded processors.

In yet another embodiment, network interface110operates in conjunction with a CPU scheduler (not illustrated) to optimize the loading of network traffic tasks by ensuring that the network processing is performed by a processor130that is also handling the user process140associated with the network traffic. As such, network interface110can provide a prompt to the CPU scheduler to move a particular task associated with a network traffic flow to a processor130that is associated with an RSS channel, or the CPU scheduler can modify the associations between the RSS channels and the processors to map the network data directly to the user process140associated with the network traffic flow. In a particular embodiment, network interface110operates with the CPU scheduler to optimize the loading of network traffic tasks in response to changes in the flow rate of network traffic into the network interface.

In another embodiment, the selection of a particular RSS channel is based upon the application of a hashing function to incoming data packets. Here, network interface110can choose from among several hashing methods, or can select different fields, or tuples, of the data packets on which the hashing methods operate, in order to more effectively optimize the processing network traffic flows. In another embodiment, the CPU scheduler modifies the associations between the RSS channels and processors130based upon prompts received from network traffic intensive applications, or can track outgoing network traffic from the processors and modifies the associations based upon the outgoing network traffic. In another embodiment, user process140can provide prompts to the CPU scheduler or to network interface110to modify the associations.

FIG. 2illustrates an information handling system200that is similar to information handling system100, including a network interface220and a host system230. Network interface220includes a hash type module222, a hash function module224, an RSS indirection table226, a message signaled interrupt (MSI-X) table228, and an RSS module229. Host system230includes a host memory232, a CPU scheduler234, and CPUs0-n(labeled242,244,246, and248, respectively. Host memory232includes a receive buffer233, and each CPU242-248includes a respective cache memory243,245,247, and249. Network interface220is connected to a network210to receive network data traffic. As illustrated, MSI-X table228is shown as a part of network interface220, but this need not always be so. For example, MSI-X table228can be implemented as part of a chipset component of host system230, or elsewhere in information handling system200, as needed or desired.

In operation, information handling system200performs functions similar to information handling system100, as described above. In particular, when a data packet250is received260, the data packet is analyzed by hash type module222to determine262one or more fields252, or tuples of the data packet that are to be analyzed by hash function module224, and the fields are forwarded264to the hash function module for determination266of a hash value254of the received data packet. Hash value254is provided268as a pointer into indirection table226. For example, as illustrated, indirection table226includes four RSS channels such that when hash value254is in a first range of values, a pointer0is selected that points to CPU0(242), when hash value254is in a second range of values, a pointer1is selected that points to CPU1(244), when hash value254is in a third range of values, a pointer2is selected that points to CPU2(246), and when hash value254is in a fourth range of values, a pointer3is selected that points to CPU3(248). Indirection table226can include more or fewer RSS channels, as needed or desired. When a particular pointer is selected from indirection table226, such as pointer2in the illustrated example, the information256that is extracted from data packet250is directed270by the indirection table pointer to be transferred272to the cache247of the selected CPU246.

When the pointer is selected from indirection table226, the pointer also serves to select278an entry from MSI-X table228. In a particular embodiment, there is a one-to-one correspondence between the number of RSS channels implemented on network interface220, that is, the number of pointers in indirection table226, and the number of interrupts implemented in MSI-X table, but this is not necessarily so. For example, pointer0corresponds to interrupt0that serves to direct an interrupt to CPU0(242), pointer1corresponds to interrupt1that serves to direct an interrupt to CPU1(244), pointer2corresponds to interrupt2that serves to direct an interrupt to CPU2(246), and pointer3corresponds to interrupt3that serves to direct an interrupt to CPU3(248). When a particular interrupt is selected from MSI-X table228, such as interrupt2in the illustrated example, an interrupt280is generated to the selected CPU246. In another embodiment, when the pointer is selected from indirection table226, the information256that is extracted from data packet250is directed270by the indirection table pointer to be transferred274to the receive buffer233of host memory232. Then, when the interrupt280is generated to the selected CPU246, the selected CPU operates to read information256from RS buffer233to cache247.

In one embodiment, the CPUs242-248that are associated with the RSS channels are predetermined by a firmware component of network interface220when information handling system200is powered on. In another embodiment illustrated inFIG. 3, CPU scheduler234operates to determine if a particular CPU242,244,246, or248is idle or lightly loaded, illustrated here as CPU n (248). Then CPU scheduler234directs304RSS module229to modify the associations between the RSS channels302in indirection table226and the interrupts306in MSI-X table228and the CPUs to provide the task of handling network traffic to idle or lightly loaded processors. In this way, when the hash value is in a range that selects, for example hash2, the information356in an associated data packet is directed308to cache249, and the interrupt210is directed to CPU248. In yet another embodiment, RSS module229operates in conjunction with CPU scheduler234to optimize the loading of network traffic tasks by ensuring that the network processing is performed by a CPU242,244,246, or248that is also handling a user process associated with the network traffic. As such, RSS module229can provide a prompt to CPU scheduler234to move a particular task associated with a network traffic flow to a processor242,244,246, or248that is associated with an RSS channel, or the CPU scheduler can modify the associations between the RSS channels and the CPUs to map the network data directly to the user process associated with the network traffic flow. In a particular embodiment, RSS module229operates with CPU scheduler234to optimize the loading of network traffic tasks in response to changes in the flow rate of network traffic into the network interface.

In another embodiment, RSS module229operates to select a different hash type such that hash type module222selects different fields, or tuples, of the data packets on which hash function module224operates. In yet another embodiment, RSS module229operates to select a different hash function such that hash function module224performs a different hash function on the fields provided by hash type module222, in order to more effectively optimize the processing network traffic flows.

In a particular embodiment, hash type module222can operate as a more generalized tuple selector, to select, for example a source or destination IP address field, a TCP port field, or the like. Here further, hash function module224can operate to provide a hash value254for a particular value of the selected tuple. In this way, a one-to-one mapping between a network flow and a CPU can be established. Here, for example, hash function module224can be implemented in whole or in part by a tertiary content addressable memory (TCAM) of an appropriate size.

FIG. 4illustrates method of handling network traffic in an information handling system through optimization of receive side scaling. The method begins in block402where scheduling priorities for user processes and CPU utilization are retrieved. For example, RSS module229can determine the network traffic needs for a particular user process, and can determine the CPU242-248that is being scheduled by CPU scheduler234to handle the user process. Further, RSS module229can determine that one or more CPU242-248is idle, or is currently lightly loaded, and is therefore a candidate for rescheduling to handle the user process and to receive the network traffic flows associated with the user process. The interrupt capabilities for an MSI-X table are retrieved in block404. For example, RSS module220can determine the status of the interrupts in MSI-X table228. The indirection capabilities for an indirection table are retrieved in block406. For example, RSS module220can determine the status of the pointers in indirection table226. The hash type and hash function capabilities of the information handling system are retrieved in block408. For example, RSS module229can determine the supported hash types from hash type module222, and the supported hash functions from hash function module224. An optimal RSS configuration is determined in block410. In response to the determination of the optimal RSS configuration, the scheduling priorities for user processes and CPU utilization are set in block412, the interrupt capabilities for the MSI-X table are set in block414, the indirection capabilities for the indirection table are set in block416, the hash type and hash function is set in block418, and the method returns to block402where the scheduling priorities for the user processes and the CPU utilization are retrieved.

In a particular embodiment, in determining the optimal RSS configuration in block410, a greedy algorithm is implemented which starts with the process and associated networks flows with the highest aggregate frame rate or data rate, and assigns the indirection table entries mapped to by the current hash function for said flows to the CPU or CPUs in use by that process. The algorithm repeats this on the next process with highest aggregate frame rate or data making assignments in the indirection table entries, assuming those entries had not been previously assigned within the algorithm. In another embodiment, in determining the optimal RSS configuration in block410, a hash selection algorithm is implemented which picks between hash functions (or assignment of hash function parameters) that provide maximize a utility function. The utility function is calculated as the weighted sum of correct number of flows that map in the indirection table to CPUs which currently host a process. The weighting may be based on frame rate or data rate for the given flow. The number of hash functions (or parameter settings) may be excessive so any running of the algorithm may only evaluate a set number of has functions or parameters. In yet another embodiment, in determining the optimal RSS configuration in block410, both of the above algorithms can be performed concurrently.

FIG. 5is a block diagram illustrating an embodiment of an information handling system500, including a processor510, a chipset520, a memory530, a graphics interface540, an input/output (I/O) interface550, a disk controller560, a network interface570, and a disk emulator580. In a particular embodiment, information handling system500is used to carry out one or more of the methods described herein. In another embodiment, one or more of the systems described herein are implemented in the form of information handling system500.

Chipset520is connected to and supports processor510, allowing the processor to execute machine-executable code. In a particular embodiment (not illustrated), information handling system500includes one or more additional processors, and chipset520supports the multiple processors, allowing for simultaneous processing by each of the processors and permitting the exchange of information among the processors and the other elements of the information handling system. Chipset520can be connected to processor510via a unique channel, or via a bus that shares information among the processor, the chipset, and other elements of information handling system500.

Memory530is connected to chipset520. Memory530and chipset520can be connected via a unique channel, or via a bus that shares information among the chipset, the memory, and other elements of information handling system500. In another embodiment (not illustrated), processor510is connected to memory530via a unique channel. In another embodiment (not illustrated), information handling system500includes separate memory dedicated to each of the one or more additional processors. A non-limiting example of memory530includes static random access memory (SRAM), dynamic random access memory (DRAM), non-volatile random access memory (NVRAM), read only memory (ROM), flash memory, another type of memory, or any combination thereof.

Graphics interface540is connected to chipset520. Graphics interface540and chipset520can be connected via a unique channel, or via a bus that shares information among the chipset, the graphics interface, and other elements of information handling system500. Graphics interface540is connected to a video display542. Other graphics interfaces (not illustrated) can also be used in addition to graphics interface540as needed or desired. Video display542includes one or more types of video displays, such as a flat panel display, another type of display device, or any combination thereof.

I/O interface550is connected to chipset520. I/O interface550and chipset520can be connected via a unique channel, or via a bus that shares information among the chipset, the I/O interface, and other elements of information handling system500. Other I/O interfaces (not illustrated) can also be used in addition to I/O interface550as needed or desired. I/O interface550is connected via an I/O interface552to one or more add-on resources554. Add-on resource554is connected to a storage system590, and can also include another data storage system, a graphics interface, a network interface card (NIC), a sound/video processing card, another suitable add-on resource or any combination thereof I/O interface550is also connected via I/O interface552to one or more platform fuses556and to a security resource558. Platform fuses556function to set or modify the functionality of information handling system500in hardware. Security resource558provides a secure cryptographic functionality and includes secure storage of cryptographic keys. A non-limiting example of security resource558includes a Unified Security Hub (USH), a Trusted Platform Module (TPM), a General Purpose Encryption (GPE) engine, another security resource, or a combination thereof.

Disk controller560is connected to chipset520. Disk controller560and chipset520can be connected via a unique channel, or via a bus that shares information among the chipset, the disk controller, and other elements of information handling system500. Other disk controllers (not illustrated) can also be used in addition to disk controller560as needed or desired. Disk controller560includes a disk interface562. Disk controller560is connected to one or more disk drives via disk interface562. Such disk drives include a hard disk drive (HDD)564, and an optical disk drive (ODD)566, and can include one or more disk drive as needed or desired. ODD566can include a Read/Write Compact Disk (R/W-CD), a Read/Write Digital Video Disk (R/W-DVD), a Read/Write mini Digital Video Disk (R/W mini-DVD, another type of optical disk drive, or any combination thereof. Additionally, disk controller560is connected to disk emulator580. Disk emulator580permits a solid-state drive584to be coupled to information handling system500via an external interface582. External interface582can include industry standard busses such as USB or IEEE 1394 (Firewire) or proprietary busses, or any combination thereof. Alternatively, solid-state drive584can be disposed within information handling system500.

Network interface device570is connected to I/O interface550. Network interface570and I/O interface550can be coupled via a unique channel, or via a bus that shares information among the I/O interface, the network interface, and other elements of information handling system500. Other network interfaces (not illustrated) can also be used in addition to network interface570as needed or desired. Network interface570can be a network interface card (NIC) disposed within information handling system500, on a main circuit board such as a baseboard, a motherboard, or any combination thereof, integrated onto another component such as chipset520, in another suitable location, or any combination thereof. Network interface570includes a network channel572that provide interfaces between information handling system500and other devices (not illustrated) that are external to information handling system500. Network interface570can also include additional network channels (not illustrated).

Information handling system500includes one or more application programs532, and Basic Input/Output System and Firmware (BIOS/FW) code534. BIOS/FW code534functions to initialize information handling system500on power up, to launch an operating system, and to manage input and output interactions between the operating system and the other elements of information handling system500. In a particular embodiment, application programs532and BIOS/FW code534reside in memory530, and include machine-executable code that is executed by processor510to perform various functions of information handling system500. In another embodiment (not illustrated), application programs and BIOS/FW code reside in another storage medium of information handling system500. For example, application programs and BIOS/FW code can reside in HDD564, in a ROM (not illustrated) associated with information handling system500, in an option-ROM (not illustrated) associated with various devices of information handling system500, in storage system590, in a storage system (not illustrated) associated with network channel572, in another storage medium of information handling system500, or a combination thereof. Application programs532and BIOS/FW code534can each be implemented as single programs, or as separate programs carrying out the various features as described herein.

In the embodiments described herein, an information handling system includes any instrumentality or aggregate of instrumentalities operable to compute, classify, process, transmit, receive, retrieve, originate, switch, store, display, manifest, detect, record, reproduce, handle, or use any form of information, intelligence, or data for business, scientific, control, entertainment, or other purposes. For example, an information handling system can be a personal computer, a consumer electronic device, a network server or storage device, a switch router, wireless router, or other network communication device, a network connected device (cellular telephone, tablet device, etc.), or any other suitable device, and can vary in size, shape, performance, price, and functionality. The information handling system can include memory (volatile (e.g. random-access memory, etc.), nonvolatile (read-only memory, flash memory etc.) or any combination thereof), one or more processing resources, such as a central processing unit (CPU), a graphics processing unit (GPU), hardware or software control logic, or any combination thereof. Additional components of the information handling system can include one or more storage devices, one or more communications ports for communicating with external devices, as well as, various input and output (I/O) devices, such as a keyboard, a mouse, a video/graphic display, or any combination thereof. The information handling system can also include one or more buses operable to transmit communications between the various hardware components. Portions of an information handling system may themselves be considered information handling systems.

When referred to as a “device,” a “module,” or the like, the embodiments described herein can be configured as hardware. For example, a portion of an information handling system device may be hardware such as, for example, an integrated circuit (such as an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), a structured ASIC, or a device embedded on a larger chip), a card (such as a Peripheral Component Interface (PCI) card, a PCI-express card, a Personal Computer Memory Card International Association (PCMCIA) card, or other such expansion card), or a system (such as a motherboard, a system-on-a-chip (SoC), or a stand-alone device). The device or module can include software, including firmware embedded at a device, such as a Pentium class or PowerPC™ brand processor, or other such device, or software capable of operating a relevant environment of the information handling system. The device or module can also include a combination of the foregoing examples of hardware or software. Note that an information handling system can include an integrated circuit or a board-level product having portions thereof that can also be any combination of hardware and software.