Patent Description:
Traditionally, in each cloud server that implements a hypervisor, a certain percentage of the cloud server's CPU (central processing unit) cores will be reserved for hypervisor use. While this reservation ensures that the hypervisor has sufficient compute resources to carry out its functions, it also reduces the number of CPU cores available for use by, e.g., customer VMs. At scale, this can result in a meaningful reduction in the overall customer-facing compute capacity of the cloud platform. <NPL>, relates to Azure Accelerated Networking (AccelNet), a solution for offloading host networking to hardware, using custom Azure SmartNICs based on FPGAs. Modern cloud architectures rely on each server running its own networking stack to implement policies such as tunneling for virtual networks, security, and load balancing. These networking stacks are becoming increasingly complex as features are added and as network speeds increase. Running these stacks on CPU cores takes away processing power from VMs, increasing the cost of running cloud services, and adding latency and variability to network performance. The goals of AccelNet are programmability comparable to software, and performance and efficiency comparable to hardware. It is shown that FPGAs are the best current platform for offloading the networking stack as ASICs do not provide sufficient programmability, and embedded CPU cores do not provide scalable performance, especially on single network flows.

The object of the invention is to optimize the utilization of computing resources for executing virtual machines on a server.

A physical server with an offload system including a SoC (system-on-chip) and a FPGA (field programmable gate array) is disclosed. The offload system is on a card. The SoC is configured to offload one or more first hypervisor functions from a CPU complex of the server that are suited for execution in software, and the FPGA is configured to offload one or more second hypervisor functions from the CPU complex that are suited for execution in hardware.

In the following description, for purposes of explanation, numerous examples and details are set forth to provide an understanding of various embodiments. It will be evident, however, to one skilled in the art that certain embodiments can be practiced without some of these details or can be practiced with modifications or equivalents thereof.

Embodiments of the present disclosure are directed to a physical server design that employs an offload card comprising a SoC (system-on-chip) and a FPGA (field-programmable gate array). The SoC and FPGA can run hypervisor functions traditionally executed by the server's CPU complex, thereby offloading the processing burden for those functions from the CPU complex. For example, the SoC of the offload card can run hypervisor functions that require or benefit from the flexibility of a general purpose processor (e.g., networking and storage control plane functions), while the FPGA of the offload card can run hypervisor functions that are suited for implementation/acceleration in hardware (e.g., networking and storage data plane functions).

With this general architecture, it is possible to move most, if not all, hypervisor processing from the server's CPU complex to the offload card, which advantageously allows the CPU complex to focus on running tenant (e.g., customer) VM workloads. In cases where the hypervisor is completely vacated from the CPU complex, tenant code can potentially run in a "bare metal" manner on the CPU complex (i.e., without any intervening hypervisor virtualization layer).

Further, because the execution of hypervisor code/logic on the offload card is physically isolated from the execution of tenant code on the CPU complex, this solution protects the hypervisor from side-channel attacks that may attempt to use the tenant code as an attack vector.

Yet further, by employing an FPGA for accelerating certain hypervisor functions that are amenable to hardware implementation, the offload card can improve the server's efficiency while at the same time maintaining architectural flexibility. For example, if needed, the FPGA can be re-programmed from accelerating one type/class of functions (e.g., networking) to accelerating another type/class of functions (e.g., storage). This is not possible with a hard logic-based accelerator such as an ASIC (application-specific integrated circuit).

The foregoing and other aspects of the present disclosure are described in further detail in the sections that follow.

<FIG> is a simplified block diagram illustrating the high-level topology of a physical server <NUM> according to certain embodiments of the present disclosure. In one set of embodiments, physical server <NUM> may be a cloud server that is deployed as part of the infrastructure of a cloud platform. In these embodiments, physical server <NUM> may be mounted in a server rack within a data center operated by the cloud platform provider. In other embodiments, physical server <NUM> may be deployed in other contexts and/or via other form factors, such as in an on-premises enterprise IT environment in the form of, e.g., a standalone server.

As noted in the Background section, cloud servers often implement a hypervisor for virtualization, which allows the cloud platform to offer services such as IaaS (Infrastructure-as-a-Service). However, due to using a portion of their platform resources, including CPU cores, for hypervisor (also known as "host") use, conventional cloud servers cannot expose all of their CPU capacity to VMs, thereby reducing the efficiency of the platform.

To address this and other issues, physical server <NUM> includes a novel offload card <NUM> comprising a SoC <NUM> and a FPGA <NUM>. In the embodiment shown, offload card <NUM> is implemented as a PCIe (Peripheral Component Interface Express)-based expansion card and thus interfaces with the mainboard of physical server <NUM> via a standard PCIe x16 <NUM> edge connector interface <NUM>. In other embodiments, offload card <NUM> may be implemented using any other type of peripheral interface.

As shown, SoC <NUM> has its own RAM (random access memory) <NUM> and flash memory <NUM> and is communicatively coupled with FPGA <NUM> via at least two interfaces that are internal to offload card <NUM>: a PCIe interface <NUM> and an Ethernet interface <NUM>. In addition, SoC <NUM> is communicatively coupled with a baseboard management controller (BMC) <NUM> of physical server <NUM> through I2C interface <NUM> and a number of other channels (e.g., USB and COM).

FPGA <NUM> also has its own RAM <NUM> and flash memory <NUM> and is communicatively coupled with a CPU complex <NUM> of physical server <NUM> through PCIe edge connector interface <NUM>. This CPU complex comprises the main CPU cores and associated RAM modules of physical server <NUM>. In addition, FPGA <NUM> includes two external Ethernet interfaces, one of which connects to an external network <NUM> (via, e.g., a TOR (top-of-rack) switch or some other network device) and the other of which connects to a NIC (network interface card/controller) <NUM> within physical server <NUM>.

Generally speaking, the topology shown in <FIG> enables some or all of the hypervisor functions traditionally run on CPU complex <NUM> of physical server <NUM> to instead be run on, and thus offloaded to, SoC <NUM> and FPGA <NUM> of offload card <NUM>. For example, hypervisor functions that benefit from the flexibility of a general purpose processor (or are simply too complex/dynamic to implement in hardware) can be run on SoC <NUM>, which incorporates one or more general purpose processing cores. Examples of such functions include SDN (software-defined networking) control plane functions, which require complex routing computations and need to be updated relatively frequently to support new protocols and features.

On the other hand, hypervisor functions that are suited to hardware acceleration can be implemented via logic blocks on FPGA <NUM>. Examples of such functions include SDN data plane functions, which involve forwarding network data traffic according to control plane decisions, and storage data plane functions such as data replication, de-duplication, and so on.

With this solution, a number of advantages are achieved over conventional server designs. First, by relieving CPU complex <NUM> of certain host processing duties, the amount of platform resources, including CPU cores in CPU complex <NUM>, used by the hypervisor can be decreased, which in turn increases the platform capacity available to VMs (also known as "guests"). This is particularly beneficial in public cloud platforms where every incremental increase in server efficiency can have a significant impact at scale. In some embodiments, the hypervisor may be entirely vacated from CPU complex <NUM> and moved to offload card <NUM>, in which case CPU complex <NUM> can run a minimal hypervisor that deals with issues that can only be run on the CPU complex itself, such as accessing certain registers, or no hypervisor at all and the remainder of the compute capacity of CPU complex <NUM> can be dedicated to guest workloads.

Second, by implementing both SoC <NUM> (which handles non-hardware accelerated functions) and FPGA <NUM> (which handles hardware accelerated functions) on offload card <NUM> and tightly coupling these two, it is easier for the hypervisor code running on SoC <NUM> to interact with the logic implemented in FPGA <NUM> and vice versa. It is possible to have alternative implementations that solely include a hardware accelerator on offload card <NUM>, but these implementations require data flows for properly coordinating the activities of the hardware accelerator with the server's main CPUs. Additionally, these alternative implementations may not support "bare metal" platforms and may not offload as much of the work.

Third, because the host code running on offload card <NUM> is physically isolated from guest code running on CPU complex <NUM>, it is more difficult for malicious entities to perpetrate an attack on the hypervisor via the VMs. This particularly relevant in light of the recent discoveries of certain side-channel vulnerabilities in modern CPU architectures. Although these known vulnerabilities can be patched, other similar vulnerabilities may be found in the future.

Fourth, by using a FPGA rather than an ASIC for hardware acceleration, offload card <NUM> can be easily re-purposed for different use cases or the same use case improved by re-programming the FPGA, and the FPGA logic can be updated if needed. This is advantageous in large-scale deployments where it may not be desirable to pull and replace a large number cards that are already in the field.

It should be appreciated that the specific topology shown for physical server <NUM> in <FIG> is illustrative and various modifications are possible. For example, although SoC <NUM> and FPGA <NUM> are shown as being implemented on an expansion card (i.e., offload card <NUM>) that interfaces with the physical server's mainboard via a peripheral (e.g., PCIe) interface, in some embodiments an alternative offload architecture may be used.

As another example, although NIC <NUM> is depicted as being a standalone component, in some embodiments the functionality of NIC <NUM> may be incorporated into one or more other components shown in <FIG>, such as in FPGA <NUM>. One of ordinary skill in the art will recognize other variations, modifications, and alternatives.

<FIG> is a schematic diagram <NUM> that presents additional details regarding the architecture of offload card <NUM> of <FIG> according to certain embodiments. Various aspects of this architecture are discussed in turn below.

SoC <NUM> can be implemented using any one of a number of existing system-on-chip designs that include one or more general purpose processing cores, interfaces for memory, storage, and peripherals, and a NIC. In a particular embodiment, SoC <NUM> may incorporate general purpose processing cores based on the ARM microprocessor architecture.

As shown, SoC <NUM> is communicatively coupled with FPGA <NUM> via three separate interfaces, which are discussed in section <NUM> below. In addition, SoC is connected to (<NUM>) one or more DRAM (dynamic RAM) modules <NUM> corresponding to RAM <NUM> of <FIG> via a memory interface <NUM>, (<NUM>) an eMMC (embedded multimedia card) device <NUM> corresponding to flash memory <NUM> of <FIG> via a storage interface <NUM>, (<NUM>) a BIOS flash memory component <NUM> via SPI (Serial Peripheral Interface) interfaces <NUM> and an intervening security chip <NUM>, and (<NUM>) a number of I2C (Inter Integrated Device) devices such as EEPROM <NUM>, hotswap controller <NUM>, and temperature sensor <NUM> via an I2C bus <NUM> (which is also connects to FPGA <NUM> and PCIe edge connector interface <NUM>).

Regarding (<NUM>), SoC <NUM> can use DRAM module(s) <NUM> as its working memory for running program code, including hypervisor code offloaded from CPU complex <NUM> of physical server <NUM>. The specific number and capacity of DRAM module(s) <NUM> and the specification of memory interface <NUM> can vary depending on the implementation. In a particular embodiment, DRAM module(s) <NUM> can comprise <NUM> GB (gigabytes) of DDR4 DRAM organized as a single <NUM> (megabit) x <NUM> bit + ECC (error correction code) memory bank and memory interface <NUM> can be configured as a single DDR4-<NUM> memory channel.

Regarding (<NUM>), SoC <NUM> can use eMMC device <NUM> as a non-transitory storage medium for storing and booting program code to be executed on the SoC, including hypervisor code offloaded from CPU complex <NUM>, as well as storing FPGA configuration images to be applied to FPGA <NUM>.

Regarding (<NUM>), BIOS flash memory component <NUM> can hold the system firmware for SoC <NUM> and security chip <NUM> can, among other things, ensure that this system firmware is not purposefully or inadvertently modified or corrupted by an attacker.

Regarding (<NUM>), I2C devices <NUM>, <NUM>, and <NUM> can provide various pieces of management information regarding offload card <NUM> to BMC <NUM>. These pieces of information can include information such as operating temperature data, manufacturing information, and power consumption data.

In addition to the above, SoC <NUM> includes USB (Universal Serial Bus), COM, and JTAG (Joint Test Action Group) interfaces <NUM>, <NUM>, and <NUM> to external headers <NUM>, <NUM>, and <NUM> respectively, which can be used to connect SoC <NUM> with BMC <NUM> or external devices for debugging or management. There is also a power throttle signal <NUM> that can be sent by BMC <NUM> to SoC <NUM> through PCIe edge connector interface <NUM>.

As mentioned previously, SoC <NUM> is communicatively coupled with FPGA <NUM> via three internal, chip-to-chip interfaces in <FIG>: a PCIe interface <NUM>, an Ethernet interface <NUM>, and a JTAG interface <NUM>. In various embodiments, PCIe interface <NUM> provides both control and data transfer/exchange capabilities. For control capabilities, SoC <NUM> can use PCIe interface <NUM> (or alternatively a JTAG interface) to manage and update FPGA <NUM>. For example, SoC <NUM> can validate FPGA configuration images transferred from RAM <NUM> to FPGA <NUM> and can update the image on the FPGA or in the FPGA's flash memory <NUM> using this interface. For data capabilities, PCIe interface <NUM> can enable program code running on SoC <NUM> to send data to, and receive data from, FPGA <NUM>. This is useful for, e.g., hypervisor code that is already written to exchange data over PCIe, because such code can be ported for execution on SoC <NUM> (or implementation on FPGA <NUM>) with relatively few changes. In a particular embodiment, PCIe interface <NUM> can have <NUM> PCI <NUM> lanes (i.e., correspond to a PCI <NUM>8x interface). In other embodiments any other number of PCI lanes, such as <NUM>, <NUM>, <NUM>, etc., may be supported.

Ethernet interface <NUM> allows SoC <NUM> and FPGA <NUM> to exchange data in the form of network packets. This is useful for, e.g., hypervisor code that is already written to exchange data via network packets, because such code can be ported for execution on SoC <NUM> (or implementation on FPGA <NUM>) with relatively few changes. For example, consider a scenario where network flow-based forwarding is implemented in hardware on FPGA <NUM> and a network control plane for determining routes for network flows is implemented in software on SoC <NUM>. In this case, flow table exceptions and rules can be communicated between FPGA <NUM> and SoC <NUM> in the form of network packets. In a particular embodiment, Ethernet interface <NUM> can support <NUM> (gigabit) Ethernet.

JTAG interface <NUM> provides a way for SoC <NUM> to communicate with FPGA <NUM> for low-level testing (e.g., debugging) and programming purposes. In some embodiments, a JTAG multiplexer can be inserted in the JTAG path between SoC <NUM> and FPGA <NUM> that allows an external programmer device connected via external header <NUM> to drive interface <NUM>. In these embodiments, a "present" signal from the external programmer device will switch the signal path of JTAG interface <NUM> from SoC <NUM> to the device. This is useful for initial offload card bring-up when loading initial bit streams, and for FPGA application development when the SoC to FPGA JTAG path is not ready. <FIG> depicts an example diagram <NUM> of this architecture with a JTAG multiplexer <NUM> according to certain embodiments.

FPGA <NUM> can be implemented using any one of number of existing FPGA chips. In a particular embodiment, FPGA <NUM> can be implemented using an existing FPGA chip that supports a certain minimum number of programmable logic elements (e.g., <NUM> elements) and a certain minimum transceiver/FPGA fabric speed grade (e.g., grade <NUM>). As shown in <FIG>, FPGA <NUM> is communicatively coupled with I2C bus <NUM> and with SoC <NUM> via interfaces <NUM>-<NUM> discussed above. In addition, FPGA <NUM> is connected to (<NUM>) PCIe edge connector interface <NUM> via internal PCIe interface <NUM>, (<NUM>) one or more DRAM module(s) <NUM> corresponding to RAM <NUM> of <FIG> via a memory interface <NUM>, (<NUM>) a QSPI (Quad Serial Peripheral Interface) flash memory module <NUM> corresponding to flash memory <NUM> of <FIG> via a storage interface <NUM>, and (<NUM>) two network transceiver modules <NUM> and <NUM> via Ethernet interfaces <NUM> and <NUM> respectively.

Regarding (<NUM>), internal PCIe interface <NUM> enables FPGA <NUM> to communicate with CPU complex <NUM> and other PCIe devices installed in physical server <NUM> (including, e.g., NIC <NUM>). In a particular embodiment, PCIe interface <NUM> may be a PCIe <NUM> x16 interface.

Regarding (<NUM>), FPGA <NUM> can use DRAM module(s) <NUM> as its working memory when executing logic programmed into the device, including hypervisor logic offloaded from CPU complex <NUM>. The specific number and capacity of DRAM module(s) <NUM> and the specification of memory interface <NUM> can vary depending on the implementation. In a particular embodiment, DRAM module(s) <NUM> can comprise <NUM> GB (gigabytes) of DDR4 DRAM organized as two 4GB banks of <NUM> x <NUM> bit + ECC and memory interface <NUM> can be configured as dual DDR4-<NUM> memory channels.

Regarding (<NUM>), QSPI flash memory module <NUM> can hold one or more FPGA configuration images that FPGA <NUM> can load upon power-up into order to configure itself to perform its designated functions. In certain embodiments, QSPI flash memory module <NUM> can hold at least three separate images, which is described in section <NUM> below. In addition to configuration from flash memory, FPGA <NUM> can also support configuration via an external JTAG programmer device, JTAG commands sent by SoC <NUM> over JTAG interface <NUM>, CvP (Configuration via Protocol) over PCIe, and partial reconfiguration over PCIe.

Regarding (<NUM>), network transceiver module <NUM> enables FPGA <NUM> to receive incoming network traffic from and transmit outgoing network traffic to external network <NUM>. Further, network transceiver module <NUM> enables FPGA <NUM> to exchange network traffic with NIC <NUM>. This is useful in scenarios where FPGA <NUM> implements network plane functions because FPGA <NUM> can receive outgoing network packets from NIC <NUM> via module <NUM>, process/transform them appropriately, and send them out to external network <NUM> via module <NUM>. Conversely, FPGA <NUM> can receive incoming network packets from external network <NUM> via module <NUM>, process/transform them appropriately, and send them to NIC <NUM> via module <NUM> (at which point they can be communicated to the correct destination VM). An example network data flow that leverages FPGA <NUM> for network data plane acceleration in this manner is discussed in section <NUM> below. In a particular embodiment, network transceiver modules <NUM> and <NUM> can be QSFP28 optical modules and Ethernet interfaces <NUM> and <NUM> can support <NUM> Ethernet.

In one set of embodiments, QSPI flash memory module <NUM> can store a minimum of three separate configuration images for FPGA <NUM>: a golden image, a failsafe image, and a user application image. The golden image is factory tested at the time of initial manufacturing and comprises the normal intended functionality for FPGA <NUM>. The failsafe image is programmed at the factory and is never overwritten after manufacturing. In various embodiments, this failsafe image contains a minimum set of functions required by offload card <NUM> at power-up and the network interfaces of FPGA <NUM> are forced into a bypass mode where all traffic is passed directly between the interfaces without any intermediate processing by the FPGA. Finally, the user application image is an image that has been defined by a user/customer.

At the time offload card <NUM> is powered-on, by default the golden image will be loaded from QSPI flash memory module <NUM> and applied to FPGA <NUM> for configuring its structures. If there any errors with this power-on process (or if problems are found during server runtime), the card can be rebooted to load the failsafe image instead of the golden image.

With the foregoing offload card architecture in mind, <FIG> depicts a flowchart <NUM> of an example network processing workflow that may be implemented by physical server <NUM> according to certain embodiments. Flowchart <NUM> assumes that FPGA <NUM> of offload card <NUM> is configured to maintain a flow table comprising network flows determined by a network control plane running on SoC <NUM> and to forward data packets in accordance with the flow table.

Starting with block <NUM>, NIC <NUM> of physical server <NUM> can present an SR-IOV (single root IO virtualization) interface to a VM running on server <NUM>. This SR-IOV interface (referred to as a virtual function) enables the VM to directly communicate with NIC <NUM>, without involving the hypervisor.

At block <NUM>, the VM can create a data payload for a network packet to be transmitted to a remote destination and can notify NIC <NUM> of this. In response, NIC <NUM> can read the data payload from the guest memory space of the VM (block <NUM>), assemble the data payload into one or more network packets with headers identifying, among other things, the IP address of the VM and the IP address of the intended destination (block <NUM>), and output the network packet out of its egress port connected to network transceiver module <NUM> of FPGA <NUM> (block <NUM>).

At blocks <NUM> and <NUM>, FPGA <NUM> can receive the network packet and apply its network data plane logic to perform a lookup of the network packet's <NUM>-tuple (source IP address, source port, destination IP address, destination port, protocol) into a flow table. If a matching entry is found in the table (block <NUM>), FPGA <NUM> can identify the next-hop destination for the network packet in the entry (block <NUM>), update the header of the packet (block <NUM>), and send the packet out of network transceiver module <NUM> to external network <NUM> (block <NUM>), thereby ending the workflow.

On the other hand, if a matching entry is not found in the table at block <NUM> (indicating that this is the first packet in a flow), FPGA <NUM> can send the network packet to SoC <NUM> over internal Ethernet interface <NUM> (block <NUM>). A network control plane component running on SoC <NUM> can then calculate a next-hop destination for the packet and add a new entry for the packet's network flow to the FPGA's flow table via interface <NUM> (block <NUM>). Using this new entry, FPGA <NUM> can execute blocks <NUM> and <NUM> and the workflow can end.

The above description illustrates various embodiments of the present disclosure along with examples of how aspects of these embodiments may be implemented. The above examples and embodiments should not be deemed to be the only embodiments, and are presented to illustrate the flexibility and advantages of the present disclosure as defined by the following claims.

Claim 1:
A server (<NUM>) comprising:
a central processing unit, CPU, complex (<NUM>); and
an offload card (<NUM>) including:
a system-on-chip, SoC, (<NUM>); and
a field programmable gate array, FPGA, (<NUM>),
wherein the CPU complex is configured to execute one or more virtual machines, VMs,
wherein the SoC includes one or more general purpose processing cores and is configured to execute, in software, one or more first functions of a hypervisor associated with the one or more VMs, and
wherein the FPGA is configured to execute, in hardware, one or more second functions of the hypervisor associated with the one or more VMs,
wherein the SoC and the FPGA are communicatively coupled with each other via a Peripheral Component Interconnect Express, PCle, interface that is internal to the offload card and via an Ethernet interface that is internal to the offload card.