Patent ID: 12248708

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

This disclosure describes a novel system that enables multiple host devices to access the virtual functions of SR-IOV capable PCIe devices, including and not limited to solid state drives, redundant array of independent disk (RAID) devices, field programmable gate array (FPGA) devices, network interface cards, and graphics processing units (GPUs), that are installed external to the host devices. In some implementations, the SR-IOV capable PCIe device is an NVMe device that has multiple namespaces, in which each namespace represents an amount of storage space of the SR-IOV capable NVMe device. The single-root input/output virtualization enables the namespace to be accessed by one or more physical functions and a plurality of virtual functions supported by the NVMe device. A controller of the NVMe device attaches one or more namespaces to each of one or more of the virtual functions. A PCIe switch is provided to communicate with multiple host devices and assign one or more virtual functions to each host device, and enable the host devices to access the namespaces using the assigned virtual functions.

In some examples, the SR-IOV capable NVMe device supports sharing of one or more namespaces by multiple virtual functions. The NVMe controller sets one or more namespaces to a “shared” state and attaches the one or more shared namespaces to multiple virtual functions. The PCIe switch assigns the virtual functions to host devices and enables the host devices to access the one or more shared namespaces using the assigned virtual functions. A management central processor unit (CPU) configures the PCIe switch and the NVMe controller to enable the host devices to access the shared namespaces using the virtual functions.

In some implementations, the SR-IOV capable NVMe device includes a controller memory buffer (CMB) that has one or more partitions. The single-root input/output virtualization enables the controller memory buffer to be accessed by one or more physical functions and a plurality of virtual functions supported by the NVMe device. The NVMe controller attaches one or more partitions of the controller memory buffer to each of one or more virtual functions supported by the NVMe device. The PCIe switch assigns one or more virtual functions to each host device, and enables the host devices to access the controller memory buffer using the assigned virtual functions.

In some examples, the SR-IOV capable NVMe device supports sharing of one or more partitions of the controller memory buffer by multiple virtual functions. The NVMe controller sets one or more partitions of the controller memory buffer to a “shared” state and attaches one or more shared partitions of the controller memory buffer to multiple virtual functions. The PCIe switch assigns the virtual functions to host devices and enables the host devices to access the one or more shared partitions of the controller memory buffer using the assigned virtual functions. The management central processor unit configures the PCIe switch and the NVMe controller to enable the host devices to access the one or more shared partitions of the controller memory buffer using the virtual functions. Data can be transferred between the host devices and the one or more shared partitions of the controller memory buffer using direct memory access (DMA) transfers.

In some implementations, the SR-IOV capable PCIe device is a graphics card that includes one or more GPU cores and has graphics memory. The single-root input/output virtualization enables the GPU cores and the graphics memory to be accessed by one or more physical functions and a plurality of virtual functions supported by the graphics card. A controller of the graphics card attaches one or more GPU cores and/or one or more partitions of the graphics memory to each of the one or more virtual functions. The PCIe switch assigns one or more virtual functions to each host device, and enables the host devices to send instructions to the GPU cores and access the graphics memory using the assigned virtual functions.

In some examples, the SR-IOV capable graphics card supports sharing of one or more partitions of the graphics memory by multiple virtual functions. The graphics controller sets one or more partitions of the graphics memory to a “shared” state and attaches one or more shared partitions of the graphics memory to multiple virtual functions. The PCIe switch assigns the virtual functions to the host devices and enables the host devices to access the one or more shared partitions of the graphics memory using the assigned virtual functions. The management central processor unit configures the PCIe switch and the graphics controller to enable the host devices to access the one or more shared partitions of the graphics memory using the virtual functions. Data can be transferred between the host devices and the one or more shared partitions of the graphics memory using direct memory access (DMA) transfers.

In some implementations, the SR-IOV capable PCIe device is a RAID controller card that includes a RAID controller and a cache memory. The RAID controller controls access to a redundant array of independent disks, referred to as RAID storage devices. The single-root input/output virtualization enables the RAID storage devices and the cache memory to be accessed by one or more physical functions and a plurality of virtual functions supported by the RAID controller card. The RAID controller attaches one or more partitions of the cache memory to each of one or more virtual functions provided by the RAID controller card. The PCIe switch assigns one or more virtual functions to each host device, and enables the host devices to access the cache memory using the assigned virtual functions.

In some examples, the SR-IOV capable RAID controller card supports sharing of one or more partitions of the cache memory by multiple virtual functions. The RAID controller sets one or more partitions of the cache memory to a “shared” state and attaches one or more shared partitions of the cache memory to multiple virtual functions. The PCIe switch assigns the virtual functions to host devices and enables the host devices to access the one or more shared partitions of the cache memory using the assigned virtual functions. The management central processor unit configures the PCIe switch and the RAID controller to enable the host devices to access the one or more shared partitions of the cache memory using the virtual functions. Data can be transferred between the host devices and the one or more shared partitions of the cache memory using direct memory access (DMA) transfers.

In some implementations, the SR-IOV capable PCIe device is an FPGA card that includes an FPGA device and embedded memory. The single-root input/output virtualization enables the FPGA device and the embedded memory to be accessed by one or more physical functions and a plurality of virtual functions supported by the FPGA card. An FPGA controller attaches one or more partitions of the embedded memory to each of one or more virtual functions provided by the FPGA card. The PCIe switch assigns one or more virtual functions to each host device, and enables the host devices to access the embedded memory using the assigned virtual functions.

In some examples, the SR-IOV capable FPGA card supports sharing of one or more partitions of the embedded memory by multiple virtual functions. The FPGA controller sets one or more partitions of the embedded memory to a “shared” state and attaches one or more shared partitions of the embedded memory to multiple virtual functions. The PCIe switch assigns the virtual functions to host devices and enables the host devices to access the one or more shared partitions of the FPGA embedded memory using the assigned virtual functions. The management central processor unit configures the PCIe switch and the FPGA controller to enable the host devices to access the one or more shared partitions of the FPGA embedded memory using the virtual functions. Data can be transferred between the host devices and the one or more shared partitions of the FPGA embedded memory using direct memory access (DMA) transfers.

In some implementations, the SR-IOV capable PCIe device is a network interface card (NIC) that includes NIC interfaces and a buffer memory. The single-root input/output virtualization enables the NIC interfaces and the buffer memory to be accessed by one or more physical functions and a plurality of virtual functions supported by the network interface card. A network interface card controller attaches one or more partitions of the buffer memory to each of one or more virtual functions provided by the network interface card. The PCIe switch assigns one or more virtual functions to each host device, and enables the host devices to access the NIC buffer memory using the assigned virtual functions.

In some examples, the SR-IOV capable network interface card supports sharing of one or more partitions of the buffer memory by multiple virtual functions. The network interface card controller sets one or more partitions of the buffer memory to a “shared” state and attaches one or more shared partitions of the buffer memory to multiple virtual functions. The PCIe switch assigns the virtual functions to host devices and enables the host devices to access the one or more shared partitions of the NIC buffer memory using the assigned virtual functions. The management central processor unit configures the PCIe switch and the network interface card controller to enable the host devices to access the one or more shared partitions of the NIC buffer memory using the virtual functions. Data can be transferred between the host devices and the one or more shared partitions of the NIC buffer memory using direct memory access (DMA) transfers.

FIG.28is a block diagram of an example of a host device2800that includes a central processing unit2802, a memory device2804, an SR-IOV capable PCIe device2806, a hypervisor2808, and several virtual machines. The host device2800can be, for example, a personal computer, a workstation computer, or a server computer. The host device2800can include a housing or chassis, and a motherboard is installed inside the housing. The central processing unit2802and the memory device2804are mounted on the motherboard. The motherboard can have PCIe slots, and the SR-IOV capable PCIe device2806can be inserted into one of the PCIe slots. The SR-IOV capable PCIe device2806supports a physical function and multiple virtual functions, such as virtual function 1 (2810a), virtual function 2 (2810b), and virtual function 3 (2810c). The host device executes virtualization software, e.g., the hypervisor, and executes multiple virtual machines, such as virtual machine 1 (2812a), virtual machine 2 (2812b), and virtual machine 3 (2812c). For example, the single-root input/output virtualization allows the virtual machine 1 (2812a) to use the virtual function 1 (2810a), the virtual machine 2 (2812b) to use the virtual function 2 (2810b), and the virtual machine 3 (2812c) to use the virtual function 3 (2810c). In this example, the resources of the SR-IOV capable PCIe device2806can be used by the host device2800. However, it is difficult for a second host device to use the resources of the SR-IOV capable PCIe device2804installed in the PCIe slot of the first host device2800.

In the past, if a company has multiple host devices, such as multiple workstation computers, each workstation computer can install SR-IOV capable PCIe devices so that the virtual machines executing in the workstation computer can access the resources provided by the SR-IOV capable PCIe devices installed within the workstation computer. Sometimes this may not provide the most efficient use of the SR-IOV capable PCIe devices. For example, the virtual machines on a first workstation computer can have low workloads so that some of the SR-IOV capable PCIe devices installed in the first workstation computer are idle or not fully utilized, whereas the virtual machines on a second workstation computer can have high workloads that need more resources than the SR-IOV capable PCIe devices installed in the second workstation computer can provide.

The following describes a solution to the above problem by providing a novel external PCIe switch box system that includes SR-IOV capable PCIe devices, connecting the SR-IOV capable PCIe devices to the host devices through PCIe links (e.g., a PCIe switched fabric), and configuring the SR-IOV capable PCIe devices in novel ways to support sharing of computing resources.

FIG.1is a diagram of an example of a PCIe switch box system100that enables multiple host devices, e.g.,102,104,106, to access the virtual functions of SR-IOV resources124, such as SR-IOV capable PCIe devices, e.g.,108a,108b,108c,108d, collectively referenced as108, which can include and are not limited to one or more of solid state drives, redundant array of independent disk (RAID) devices, field programmable gate array (FPGA) devices, network interface cards, graphics processing units (GPUs), or any combination of the above. Each SR-IOV capable PCIe device is connected to the PCIe interface and complies with the SR-IOV specification.

Each host device can be, e.g., a workstation computer, a server computer, a personal computer, an industrial control computer, or any other computing device that communicates with the PCIe switch box system100through a PCIe link. Each host device can support multiple virtual machines, and each virtual machine can access the virtual functions of the SR-IOV capable PCIe devices108.

In some implementations, each SR-IOV capable PCIe device108is an NVMe device that has a namespace identifier110and multiple namespaces (e.g.,112a,112b,112c, collectively referenced as112), in which each namespace112represents an amount of storage space of the SR-IOV capable NVMe device108. The single-root input/output virtualization enables the namespace112to be accessed by one or more physical functions and a plurality of virtual functions. A controller of the NVMe device108attaches one or more namespaces to each of one or more NVMe virtual functions supported by the NVMe device108. A PCIe switch116is provided to communicate with the host devices102,104,106and assign one or more NVMe virtual functions to each host device, and enable the host devices to access the namespaces using the assigned NVMe virtual functions. The PCIe switch116is configured to assign different virtual functions associated with a shared namespace to different host devices and enable the different host devices to access the shared namespace using the assigned virtual functions.

In some implementations, the PCIe switch box system100includes a housing (or enclosure), in which a motherboard is disposed in the housing. The PCIe switch116is mounted on the motherboard. The motherboard includes PCIe interfaces, and the PCIe switch116communicates with the SR-IOV capable PCIe devices through the PCIe interfaces. For example, the PCIe interfaces can include PCIe slots, and the SR-IOV PCIe devices can be configured as PCIe peripheral cards that are inserted into the PCIe slots. In this document, the PCIe switch116is also referred to as a management PCIe switch because it manages the assignment of virtual functions to host devices.

In some implementations, the PCIe switch116can be model PEX88096 PCIe Gen4 Switch, available from Broadcom, San Jose, California. For example, the SR-IOV capable PCIe device108can be an SR-IOV capable NVMe device. The PCIe switch116assigns the virtual functions of the SR-IOV capable NVMe devices to different host ports, so that different hosts can access (e.g., read from and/or write to) the namespace from the same NVMe device. For example, both host A126and host B130can access the namespace from the same NVMe device. The registers of the PCIe switch116can be set to allow the downstream port NVMe virtual function of the PCIe switch116to be assigned to any upstream host port of the PCIe switch116.

By comparison, in a conventional system, the physical and virtual functions of an SR-IOV capable NVMe device can be accessed by a single host. In the conventional system, one of host A126or host B130can see and access the physical and virtual functions of the NVMe device. In the conventional system, host A126and host B130cannot see or access the namespace that belongs to the same NVMe device.

One or more memory devices120store management software that when executed by a management CPU118causes the management CPU118to configure the PCIe switch116to enable the host devices to access the namespaces using the virtual functions. The PCIe switch box system100includes a root complex device122that connects the CPU118and the memory devices120to the PCIe switch116.

Each host device includes a central processing unit that communicates with the PCIe switch116through a local PCIe switch. For example, the host device102includes a host CPU126and a local PCIe switch128, the host device104includes a host CPU130and a local PCIe switch132, and the host device106includes a host CPU134and a local PCIe switch136. For example, each of the local PCIe switch128,132can be model PEX88032 switch card, available from Broadcom. The PEX88032 switch card can operate in fanout mode. The host A126uses the local PCIe switch128to access (e.g., read/write) the NVMe namespaces in the PCIe switch box system100. In this document, the PCIe switch116is sometimes referred to as the “switch box PCIe switch,” and the PCIe switch128or132is sometimes referred to as the “host PCIe switch.”

For example, the host device106includes a virtual machine manager138that manages multiple instances of virtual machines (e.g.,114a,114b,114c, collectively referenced as114). The host CPUs126,130, and134can access the SR-IOV resources124at the PCIe switch box system100. When virtual machines114are executed at the host device106, each virtual machine114can access the SR-IOV resources124at the PCIe switch box system100.

A management computer140is provided to enable an administrator to remotely configure the PCIe switch box system100. For example, through the management computer140, the administrator of the PCIe switch box system100can set the privileges, access levels, and quotas for each host device. The management computer140can review requests from the host devices, and determine whether to grant to deny the requests. For example, if a host device requests an amount of solid state storage that exceeds the quota for the host device, the management computer140can either partially grant the request by allocating an amount of solid state storage that equals the quota to the host device, or increasing the quota for the host device. If the host device requests access to a resource that is beyond its access level, the management computer140can deny the request.

The PCIe switch box system100allows the host devices to be set up in an efficient manner. For example, the host device102may need a large amount of solid state storage for a few days per month to process a large amount of transaction data, and needs a smaller amount of solid state storage for the remaining days of the month. In this case, it is not economical for the host device to be installed with the large amount of solid state storage since it is only used for a small percentage of time. The host device102can be installed with the smaller amount of solid state storage that is needed most of the time, and the host device102can request additional solid state storage from the PCIe switch box system100when needed. The PCIe switch box system100can include a large number of solid state storage devices that are shared among the multiple host devices (e.g.,102,104,106), such that the solid stage storage is more fully utilized.

For example, the host device104may need to access several powerful graphics processing units for a few hours a week in order to train a large scale artificial intelligence neural network. After the neural network has been trained, the host device104may only need a smaller number of graphics processing units to perform the other day-to-day graphical processing tasks. In this example, the host device104can request access to additional graphics processing units from the PCIe switch box system100when needed. The PCIe switch box system100can include a large number of expensive and powerful graphics processing units that are shared among the multiple host devices (e.g.,102,104,106) such that the graphics processing units are more fully utilized.

For example, the host device106can provide software as a service and execute several instances of virtual machines114to support many remote users. The number of virtual machines114can vary depending on the number of remote users and the software applications. Each virtual machine114can request access to resources such as redundant array of independent disk (RAID) devices, and the amount of resources can vary. In this example, the host device106itself does not need to include a large number of RAID devices. Rather, the virtual machines114can request additional resources, such RAID devices, from the PCIe switch box system100when needed. This allows the host device106to be set up at a lower cost while still able to support a large number of instances of virtual machines to service a large number of remote users.

The PCIe switch box system100allows companies to be more flexible in deploying their computing resources. For example, the PCIe switch box system100can be located in a server room, and the host devices102,104, and106can be located in various offices remote from the server room. The large number of solid state storage devices, graphics processing units, and RAID devices can generate a large amount of heat and require special cooling facilities that can be noisy. By locating the computing resources, such as solid state storage devices, graphics processing units, and RAID devices away from the host devices and managing them centrally at the server room, the company can manage the computing resources more efficiently.

The PCIe switch box system100can have excess capacity and provide redundancy to allow the host devices102,104,106to operate continuously with a low down time in case some of the sources fail. For example, when one of the SR-IOV capable devices108fail, the PCIe switch box system100can quickly switch to another SR-IOV capable device108and continue to service the hosts102,104,106.

The PCIe switch box system100enables the company to more conveniently upgrade their systems. For example, in a conventional system in which the solid state storage devices are installed locally at each host device, when the solid state storage devices need to be upgraded to provided more storage capacity, the host device needs to be shut down, and the housing of the host device needs to be opened up to allow the storage device to be upgraded. This results in downtime and inconvenience for the user. When the PCIe switch box system100is used, the host devices can request as much additional storage capacity as needed. The PCIe switch box system100can be designed such that the SR-IOV capable devices are hot pluggable, such that individual SR-IOV capable devices can be installed or removed without shutting down the PCIe switch box system100. The administrator can upgrade the storage devices at the PCIe switch box system100without interrupting the operations of the host devices102,104,106.

In some implementations, the management computer140provides an application programming interface (API) (referred to as the “SR-IOV configuration API”) that allows host devices to configure the parameters of the SR-IOV capable PCIe devices. The parameters that are configurable can be different for different types of devices. As an example, for an NVMe storage device, the SR-IOV configuration API can be used to set the namespace configurations, the number of partitions in the storage device, the size of each partition, or the namespace identifier of the partition. The SR-IOV configuration API can assign a particular namespace identifier to a particular virtual function, set a namespace identifier to a “shared” state to allow the namespace identifier to be shared with another host device, or set a namespace identifier to a “private” state so that the namespace identifier is not shared with another host device. For example, the SR-IOV configuration API can send instructions to the controller of the PCIe device to perform the configuration actions mentioned above. For example, the host device can, through the SR-IOV configuration API, cause the PCIe device controller to set the namespace to the “private” state during certain periods of time, and set the namespace to the “shared” state during other periods of time. This way, the host device or a virtual machine executing at the host device can have exclusive use of the namespace during some periods of time, and share the namespace with other host devices or virtual machines during other periods of time.

For example, the host device (e.g.,102,104, or106) can send requests to the SR-IOV configuration API for configuring the parameters of an SR-IOV capable NVMe device, and the management computer140can determine whether to grant the requests. If the request is granted, the virtual function of the NVMe device is assigned to the host device. A PCI device tree stores information about the PCI devices accessible to the host device. For example, the PCI device tree can also store information about the physical and virtual functions that are accessible to the host device. When a new virtual function is assigned to the host device, the new virtual function is added to the PCI device tree. When a virtual function is removed from the host device, the virtual function is also removed from the PCI device tree.

For example, in a conventional SR-IOV system, if a set of virtual functions is assigned to a host device, when the host device is turned off and on again, the SR-IOV drivers are not loaded automatically because the basic input/output system (BIOS) cannot see the virtual functions, and the host device no longer have access to the set of virtual functions. The host device need to follow a procedure to reload or reset the set of virtual functions.

By comparison, the PCIe switch box system100manages and stores the SR-IOV configuration parameters, so the SR-IOV functions are still available to the host device after the host device reboots. For example, suppose a set of physical and virtual functions are assigned to a host device, the virtual functions have particular namespace configurations, a storage device assigned to the host device has a particular number of partitions, each partition has a particular size, each partition has a particular namespace identifier, a particular namespace identifier is set to the “shared” state, etc., these configurations will still be available to the host device after the host device reboots. For example, the PCIe switch box system100can store information about the PCI device tree showing which physical and virtual functions can be accessed by the host device and provides the PCI device tree to the host device after the host device reboots.

In some implementations, the PCIe switch box system100enables hot-plug capability so that a hardware PCIe peripheral card (e.g., NVMe storage device card, or GPU card) can be plugged into a PCIe slot in the PCIe switch box system100without turning off the PCIe switch box system100. When the peripheral card is plugged into the PCIe slot, the peripheral device is assigned to the PCIe space. If the peripheral card is removed from the PCIe slot, the peripheral device is un-assigned from the PCIe space.

In some implementations, when the administrator assigns a virtual function to a host device, if the host device has the hot-plug function, the host device will see the virtual function. If the host device does not have the hot-plug function, the host device can reboot and then the host device will see the virtual function.

The PCIe switch box system100performs a centralized management of the PCIe resources that can be virtualized and assigned to the host devices. The host devices can access management functions of the PCIe switch box system100through the SR-IOV configuration API. The specific management functions that can be accessed by a particular host device depends on the access level of the host device. For example, if the PCIe switch box system100includes SR-IOV capable GPU devices and the GPU functions are virtualized, it is possible to configure the host device access level such that the host device can see the GPU device on the PCI device tree during certain time periods, and the GPU device does not appear on the PCI device tree at other time periods.

For example, if the PCIe switch box system100includes SR-IOV capable NVMe storage devices and the NVMe storage functions are virtualized, it is possible to configure the host device access level such that the host device can see the NVMe device on the PCI device tree during certain time periods, and the NVMe device does not appear on the PCI device tree at other time periods.

In some implementations, the PCIe switch box system100can aggregate the physical and virtual functions of the SR-IOV capable PCIe devices so that a host device can access all, or a subset, of the physical and virtual functions of the PCIe devices. For example, if the PCIe switch box system100has 16 NVMe devices installed, and each NVMe device supports 4 virtual functions, then the PCIe switch box system100can support 64 virtual functions. These 64 virtual functions can be pooled together so that a single host can see all 64 virtual functions, or a subset of the 64 virtual functions depending on the access level of the host device. The PCIe switch box system100can provide virtualized NVMe solid state drives, virtualized GPU devices, virtualized RAID devices, and/or virtualized network interface cards.

Referring toFIG.2, the host devices can access the physical and virtual functions of an SR-IOV capable NVMe device located in the PCIe switch box system100. The figure shows an example in which the SR-IOV capable PCIe device is an NVMe solid state drive154. The NVMe solid state drive154provides an NVMe physical function150, a first NVMe virtual function152a, a second NVMe virtual function152b, and an M-th NVMe virtual function152c, and so forth. The NVMe solid state drive154includes an NVMe drive controller156. The PCIe switch116assigns a first downstream NVMe virtual function152ato an upstream host A126, which allows the upstream host A126to access the first virtual function152a. The host device102includes a first virtual function152a′ shown in dashed lines, indicating that the host A126can access the first virtual function152aas if the first virtual function152ais provided locally at the host device102. The PCIe switch116assigns a first namespace identifier110ato the first NVMe virtual function152aand allows the first NVMe virtual function152ato access a first namespace A112a.

The PCIe switch116assigns a second downstream NVMe virtual function152bto an upstream host B130, which allows the upstream host B130to access the second virtual function152b. The host device104includes a second virtual function152b′ shown in dashed lines, indicating that the host B130can access the second virtual function152bas if the second virtual function152bis provided locally at the host device104. The NVMe drive controller156assigns the first namespace identifier110ato the second NVMe virtual function152band allows the second NVMe virtual function152bto access the first namespace A112a. This way, both the host A126and the host B130can access the same namespace A112a.

The host device106executes three virtual machines114a,114b,114c. The PCIe switch116assigns a third downstream NVMe virtual function152cto the upstream virtual machine114a, which allows the upstream virtual machine114ato access the third NVMe virtual function152c. The host device106includes a third NVMe virtual function152c′ shown in dashed lines, indicating that the virtual machine114acan access the third virtual function152cas if the third virtual function152cis provided locally at the host device106. In a similar manner, the PCIe switch116assigns fourth and fifth downstream NVMe virtual functions152dand152eto the upstream virtual machines114band114c, which allows the upstream virtual machines114band114cto access the fourth and fifth NVMe virtual functions152dand152e, respectively. The host device106includes fourth and fifth NVMe virtual functions152d′ and152e′ shown in dashed lines, indicating that the virtual machines114band114ccan access the fourth and fifth virtual functions152dand152eas if the fourth and fifth virtual functions152dand152eare provided locally at the host device106. The NVMe drive controller156assigns the second namespace identifier110bto the third, fourth, and fifth NVMe virtual functions152c,152d,152eand allows the virtual machines114a,114b,114cto access the second namespace B112bthrough the virtual functions152c,152d, and152e.

The above is merely an example, the NVMe drive controller156can assign the namespaces to the virtual functions differently. For example, the NVMe drive controller156can assign the first namespace identifier110ato the third NVMe virtual function152cand allow the virtual machine114ato access, through the third NVMe virtual function152c, the namespace A112a.

The management computer140can determine how the PCIe switch116assigns the NVMe virtual functions to the host devices and virtual machines, and how the namespace identifiers are assigned to the NVMe virtual functions. For example, some namespaces can correspond to storage devices having higher throughput and greater security, and are reserved to host devices having higher privileges. Some host devices can belong to the same work group and can share access to the same files stored in a common namespace, so the NVMe drive controller156can attach the same namespace identifier to the virtual functions assigned to those host devices. If two different host devices do not share access to the same files, then the NVMe drive controller156assigns different namespaces to the virtual functions assigned to host devices and ensures that the files of each host device cannot be accessed by the other host device. A first virtual function assigned to a first host is hooked to a first synthetic PCIe tree that can be seen by the first host and allows the first host to access a first namespace attached to the first virtual function. A second virtual function assigned to a second host is hooked to a second synthetic PCIe tree that can be seen by the second host and allows the second host to access a second namespace attached to the second virtual function. The second virtual function is not hooked to the first synthetic PCIe tree, so the first host cannot identify the second virtual function and cannot access the second namespace. Likewise, the first virtual function is not hooked to the second synthetic PCIe tree, so the second host cannot identify the first virtual function and cannot access the first namespace.

FIG.3is a diagram of an example in which workstation computers160and162access one or more of SR-IOV capable devices, such as a GPU164, an NVMe device190, and another SR-IOV capable device192through the PCIe switch box system100. In some implementations, the PCIe switch box system100includes a communication interface166that allows the PCIe switch box system100to communicate with the management computer140. For example, the management computer140can, through the communication interface166, issue instructions to the management software executing in the PCIe switch box system100. The PCIe switch box system100includes a first PCIe redriver168athat functions as an interface between the PCIe switch116and the PCIe switch128of the first workstation computer160. A second PCIe redriver168bis provided as an interface between the PCIe switch116and the PCIe switch132of the second workstation computer162. For example, the first and second PCIe redrivers168a,168b(also referred to as repeater integrated circuit) can condition the signals transmitted between the switch box PCIe switch116and the host PCIe switches128,132, respectively, such as boosting some frequency portions of the signals to counteract the frequency-dependent attenuations caused by the interconnections. The redrivers can condition transmitted signals through the physical layer and reduce jitter in the signals. Use of the redrivers can improve the quality of the signals transmitted between the switch box PCIe switch116and the host PCIe switches128,132.

Referring toFIG.4, in some implementations, the GPU device164(FIG.3) provides a GPU physical function182, a first GPU virtual function170a, and a second GPU virtual function170b. In this example, the switch box PCIe switch116assigns the first GPU virtual function170ato the first workstation computer160, and assigns the second GPU virtual function170bto the second workstation computer162. The first GPU virtual function170a′ and the second GPU virtual function170b′ shown in dashed lines in the first workstation computer160and the second workstation computer162indicate that the CPU186of the first workstation160can access the first GPU virtual function170aas if it is provided locally, and the CPU188of the second workstation162can access the second GPU virtual function170bas if it is provided locally.

In some implementations, the NVMe device190(FIG.3) provides an NVMe physical function184, a first NVMe virtual function176a, and a second NVMe virtual function176b. The switch box PCIe switch116assigns the first NVMe virtual function176ato the first workstation computer160, and assigns the second NVMe virtual function176bto the second workstation computer162. The first NVMe virtual function170a′ and the second NVMe virtual function176b′ shown in dashed lines in the first workstation computer160and the second workstation computer162indicate that the CPU186of the first workstation160can access the first NVMe virtual function176aas if it is provided locally, and the CPU188of the second workstation162can access the second NVMe virtual function176bas if it is provided locally.

A PCIe device controller157assigns a second namespace identifier178to the first NVMe virtual function176a. This allows the CPU186of the first workstation computer160to access the namespace B180associated with the first NVMe virtual function176a. The PCIe device controller157assigns a third namespace identifier194to the second NVMe virtual function176b. This allows the CPU188of the second workstation computer162to access the namespace C196associated with the second NVMe virtual function176b.

In the example ofFIG.4, the workstation computers160and162can use the GPU virtual functions170and access the namespaces of the NVMe device190. The PCIe switch box system100can be configured to perform other assignments of the virtual functions to the workstation computers, and assignments of the namespace identifiers to the virtual functions, depending on the requirements of the host devices and the available SR-IOV capable resources. For example, the namespace B180and the namespace C196can be the same namespace, and the namespace ID2 and the namespace ID3 can be the same identifier. The namespace B180/namespace C196is shared by the first and second workstation computers160,162allowing the first GPU virtual function170a′ executing on the first workstation computer160and the second GPU virtual function170b′ executing on the second workstation computer162to access the shared namespace. The first GPU virtual function170a′ can write data directly to the shared namespace, and the second GPU virtual function170b′ can read the data directly from the shared namespace. SeeFIG.6for additional information regarding access to a shared namespace by two host devices.

FIG.5is a diagram showing the signal paths between host devices (e.g.,200,202, and204), the management computer140, and an SR-IOV capable NVMe device240in the PCIe switch box system100. In this example, the host device200can be a personal computer executing MacOS, Linux, or Windows operating system. The host device202can be a server computer that executes Docker software. The host device204can include a virtual machine manager206and execute multiple virtual machines208. The SR-IOV capable resource can be an NVMe solid state storage device that provides a physical function210, a first virtual function212, a second virtual function214, and a k-th virtual function216.

In some implementations, the CPU118executes one or more management drivers (e.g., PCIe switch management drivers920inFIG.9) stored in the memory120to configure the PCIe switch116to function as a switch manager that manages the assignments of virtual functions to the host devices. For example, the PCIe switch116assigns the first virtual function212to the first host device200, assigns the second virtual function214to the second host device202, and assigns the k-th virtual function216to the virtual machines208. The host device200accesses (e.g., read/write) the first virtual function212through a first PCIe data path218. For example, the PCIe data path218can comply with PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path218extends from the PCIe switch220of the host device200to the PCIe redriver222of the PCIe switch box system100, from the PCIe redriver222to the PCIe switch116, and from the PCIe switch116to the first virtual function212. The host device202accesses (e.g., read/write) the second virtual function214through a second PCIe data path224, which can comply with, e.g., PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path224extends from the PCIe switch226of the host device202through the PCIe redriver228of the PCIe switch box system100and the PCIe switch116to the second virtual function214. The virtual machines208of the host device204access (e.g., read/write) the k-th virtual function216through a third PCIe data path230, which can comply with, e.g., PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path230extends from the PCIe switch232of the host device204through the PCIe redriver234of the PCIe switch box system100and the PCIe switch116to the k-th virtual function216.

The management computer140communicates with the communication interface166of the PCIe switch box system100through a secure communication channel236, such as a secure Ethernet link. The management computer140can provide a user interface238that allows the administrator to conveniently determine the capabilities of the PCIe switch box system100, such as what SR-IOV capable devices are available, which physical and virtual functions are available, what namespace identifiers are available, and what namespaces are available. Through the user interface238, the administrator can assign particular physical functions or virtual functions to particular host devices.

FIG.6is a diagram showing the signal paths between host devices250and252, and namespaces that can be accessed by the host devices250,252. In this example, through the user interface262at the management computer140, the administrator can configure namespace A256as a shared namespace (step264), attach namespace A256to the first virtual function212(step266), attach namespace A256to the second virtual function214(step266), issue the instruction “VF_DevFunc(0,1)@Host A” (step270), and issue the instruction “VF_DevFunc(0,2)@Host B” (step272). In step270, a resource mapping between NVMe virtual function to host device A is recorded, and in step272, a resource mapping between NVMe virtual function to host device B is recorded.

As a result of the configuration instructions issued by the management computer140, the first namespace identifier254is assigned to the first NVMe virtual function212and the second NVMe virtual function214, which allows the first NVMe virtual function212and the second NVMe virtual function214to access the namespace256. The host device250accesses the namespace256through the PCIe data path258, and the host device252accesses the namespace256through the PCIe data path260. For example, the PCIe data paths258,260can comply with PCIe 4.0, 5.0, 6.0, and/or 7.0 specification.

Referring toFIG.7, an SR-IOV capable NVMe device700can have a controller memory buffer (CMB)280that can store queues and data for direct memory access (DMA). The queues and data for direct memory access can be stored in the host memory if the controller memory buffer280is not used. In this example, the controller memory buffer280includes partitions A, B, C, and D. In some examples, the NVMe controller memory buffer280is configured such that some partitions are configured to be shared by two or more virtual functions. For example, the NVMe controller memory buffer280can be configured to have partitions C and D shared by the first virtual function212and the second virtual function214. The first virtual function212is assigned to a host A250, and the second virtual function214is assigned to a host B252. Because the partitions C and D of the controller memory buffer are shared by the first and second virtual functions212,214, the partitions C and D of the controller memory buffer can be accessed by both the host A250and the host B252. This enables communication between host A250and host B252over the PCIe fabric through shared access to the partitions C and D of the controller memory buffer. For example, the host A250can write data to the partition(s) C and/or D of the controller memory buffer, and the host B252can read the data from the partition(s) C and/or D of the controller memory buffer. Similarly, the host B252can write data to the partition(s) C and/or D of the controller memory buffer, and the host B252can read the data from the partition(s) C and/or D of the controller memory buffer.

One of the features of the PCIe switch box system100is that the SR-IOV capable PCIe devices are configured such that the controller memory buffer of each SR-IOV capable PCIe device is exposed on the PCIe bus. This allows the controller memory buffer to be accessed (e.g., read/write) by other devices connected to the PCIe bus. The SR-IOV capable PCIe devices can include, e.g., redundant array of independent disk (RAID) devices, field programmable gate array (FPGA) devices, network interface cards, and graphics processing units. The controller memory buffer can be implemented using the memory devices on board the SR-IOV capable PCIe devices. When a first host device transmits data to a second host device, the transmission of data can be accomplished using the PCIe fabric without additional external peripheral interfaces.

In some implementations, the PCIe switch box system100is configured such that when the namespace of an NVMe device is set to a “shared” state, the PCIe switch box system100allows different host devices to use different virtual functions to access the same NVMe namespace. This design has the advantage that, because different host devices can access the same NVMe namespace, the transfer of data between different host devices can be made much faster. Another advantage is that because it is not necessary to separately install network interface cards for the purpose of transferring data between the host devices, the hardware and software costs associated with the network interface cards can be reduced or eliminated.

In some implementations, the PCIe switch box system100is configured such that when the controller memory buffer of an NVMe device is set to a “shared” state, the PCIe switch box system100allows different host devices to use different virtual functions to access the same NVMe controller memory buffer. This design has the advantage that, because different host devices can access the same NVMe controller memory buffer, the transfer of data between different host devices can be made much faster. For example, the operating system on each of the first and second host devices can manage access to the shared controller memory buffer to avoid conflicts.

In some implementations, the CPU118configures the PCIe switch116to enable the host devices to transfer data using a shared namespace or a shared controller memory buffer through the PCIe fabric by using the processes shown inFIGS.24to26.

FIG.24is a diagram of an example of a process2400for configuring the PCIe switch116to assign an NVMe SR-IOV virtual function of an SR-IOV capable PCIe device to a host port, which can be a port of a particular host device, e.g.,102,104,106inFIG.1. At step2402, a host port synthetic PCIe tree is initialized. At step2404, the host device sends a PCIe configuration transaction layer packet (TLP) to inquire information about the PCIe devices that are available. The PCIe configuration transaction layer packet is redirected by the switch box PCIe switch116to the management CPU118. At step2406, the management CPU118modifies the PCIe configuration transaction layer packet in a way such that the packet received by the PCIe device is similar to the packet that the PCIe device would receive if the PCIe device were installed in the host device. Thus the PCIe device behaves in the same manner as if it were installed in the host device. At step2408, the management CPU118loads the NVMe drivers for the NVMe physical functions (PF) to enable the management software in the PCIe switch box system100to perform setup of the NVMe drive, such as generating namespaces, attaching a namespace to an NVMe virtual function. At step2410, the NVMe SR-IOV function is enabled.

At step2412, the management CPU118sends an NVMe admin command to the NVMe drive controller156(FIG.2) to generate an NVMe namespace. At step2414, the management CPU118sends an NVMe admin command to the NVMe drive controller156to attach an NVMe namespace to one of NVMe SR-IOV virtual functions (VF). At step2416, the management CPU118sends an NVMe admin command to the NVMe drive controller156to set a virtual queue (VQ) and a virtual interrupt (VI) for the NVMe virtual function (VF). The virtual queue resource (VQ resource) is a type of controller resource that manages one submission queue (SQ) and one completion queue (CQ). The virtual interrupt resource (VI resource) is a type of controller resource that manages one interrupt vector. The NVM subsystem includes primary controller(s) and secondary controller(s), in which the secondary controller(s) depend on the primary controller(s) for dynamically assigned resources. At step2418, the management CPU118assigns an NVMe virtual function (VF) to the host port (e.g., insert a synthetic device to the synthetic PCIe tree). At step2420, the management CPU118sets up a PCI identity (ID) trap for data transfer from the NVMe device to the host port. The PCI identity trap is set up at a downstream port to provide identity (ID) routing information for upstream routes (TO device to the host device). For example, this can occur when the IO device initiates a DMA data transfer. The address routing will be transformed to ID routing, since the address value is in the host address space.

At step2422, the management CPU118sets up the fabric path (across different chips) for sending data from the PCIe device to the host port, and from the host port to the PCIe device. For example, this provides routing information when the destination is not in the source switch. Thus, the fabric path can be used in cross-switch or cross-domain environments, e.g., switch cascade. This supports up to 256 domains and up to 256 busses per domain.

At step2424, when the host device writes configuration data, the PCI identifier (ID) translations for G2H (management CPU to host) and H2G (host to management CPU) are set up. For example, this translates the requester ID (RID) between host (local) domain and mCPU (global) domain. The TLP travels between the host domain and the mCPU domain, so the requester ID needs to be translated to a proper value. This provides local-to-global and global-to-local RID translation.

At step2426, when the host device writes to the base address registers (BARs), an address trap for translating the address from the host device to the management CPU118domain is set up. For example, this translates addresses between the host device and the PCIe device. The setup at a host port (BAR access) is as follows: The host address space will be translated to mCPU address space within a specific range. The setup at a downstream (PCIe device) port is as follows: The first device address will be translated to another device address for peer-to-peer transfer.

FIG.25is a diagram of an example of a process2500for configuring the PCIe switch116to assign an NVMe SR-IOV virtual function with a shared NVMe namespace to the host port. Steps2502to2510are similar to the steps2402to2410, respectively, ofFIG.24. At step2512, the management CPU118sends an NVMe admin command to the NVMe drive controller156(FIG.2) to generate an NVMe shared namespace. The NVMe device is designed such that a namespace can be set to a “private” state or a “shared” state. If the namespace is set to the “private” state, the namespace can only be attached to a single virtual function and be accessed by that single virtual function. When the single virtual function is assigned to a particular host device, the private namespace can only be accessed by the particular host device through the virtual function. If the namespace is set to the “shared” state, the namespace can be attached to multiple virtual functions and be accessed by those multiple virtual functions. When the virtual functions are assigned to host devices, the shared namespace can be accessed by the corresponding host devices through the virtual functions. At step2514, the management CPU118sends an NVMe admin command to the NVMe drive controller156to attach an NVMe shared namespace to one of NVMe SR-IOV virtual functions (VF). Steps2516to2526are similar to the steps2416to2426, respectively.

FIG.26is a diagram of an example of a process2600for configuring the PCIe switch116to assign an NVMe SR-IOV virtual function with a shared namespace and a shared NVMe controller memory buffer to a host port. Steps2602to2616are similar to steps2502to2516, respectively, ofFIG.25. At step2618, the management CPU118sends an NVMe admin command to the NVMe drive controller156to set an NVMe controller memory buffer (CMB) to a “shared” state, and attach the shared controller memory buffer to one of the NVMe virtual functions (VF). In some examples, the controller memory buffer can have multiple partitions. One or more of the partitions can be set to the “shared” state, while other partitions are set to the “private” state. The NVMe device is designed such that the controller memory buffer, or a partition in the controller memory buffer, can be set to a “private” state or a “shared” state. If a partition in the controller memory buffer is set to the “private” state, the CMB partition can only be attached to a single virtual function and be accessed by that single virtual function. When the single virtual function is assigned to a particular host device, the private CMB partition can only be accessed by the particular host device through the virtual function. If the CMB partition is set to the “shared” state, the CMB partition can be attached to multiple virtual functions and be accessed by those multiple virtual functions. When the virtual functions are assigned to host devices, the shared CMB partition can be accessed by the corresponding host devices through the virtual functions. Steps2620to2628are similar to steps2518to2526, respectively, ofFIG.25.

Similar principles can be applied to enable the host devices to access a shared GPU controller memory buffer.FIG.27is a diagram of an example of a process2700for configuring the PCIe switch116to assign a GPU SR-IOV virtual function with shared GPU controller memory buffer to a host port. Steps2702to2706are similar to steps2602to2606, respectively, ofFIG.26. At step2708, the management CPU118loads the GPU driver for each GPU physical function (PF) to enable the management software in the PCIe switch box system100to perform setup of the GPU. At step2710, the GPU SR-IOV function is enabled. At step2712, the management software configures each of the attached GPU through vendor specific instructions.

At step2714, the management CPU118sends a GPU vendor specific command to the GPU controller to set a GPU controller memory buffer (CMB) to a “shared” state, and attach the shared controller memory buffer to one of the GPU virtual functions (VF). In some examples, the controller memory buffer can have multiple partitions. One or more of the partitions can be set to the “shared” state, while other partitions are set to the “private” state. The GPU is designed such that the controller memory buffer, or a partition in the controller memory buffer, can be set to a “private” state or a “shared” state. If a partition in the controller memory buffer is set to the “private” state, the CMB partition can only be attached to a single virtual function and be accessed by that single virtual function. When the single virtual function is assigned to a particular host device, the private CMB partition can only be accessed by the particular host device through the virtual function. If the CMB partition is set to the “shared” state, the CMB partition can be attached to multiple virtual functions and be accessed by those multiple virtual functions. When the virtual functions are assigned to host devices, the shared CMB partition can be accessed by the corresponding host devices through the virtual functions.

At step2716, the management CPU118assigns a GPU virtual function (VF) to the host port (e.g., insert a synthetic device to the synthetic PCIe tree). At step2718, the management CPU118sets up a PCI identity (ID) trap for data transfer from the GPU to the host port. The PCI identity trap is set up at a downstream port to provide identity (ID) routing information for upstream routes (IO device to the host device). For example, this can occur when the IO device initiates a DMA data transfer. The address routing will be transformed to ID routing, since the address value is in the host address space. Steps2720to2724are similar to steps2624to2628, respectively, ofFIG.6.

Thus, the PCIe switch box system100allows the host devices to transfer data using a shared namespace or a shared controller memory buffer of an NVMe device or a GPU through the PCIe fabric. This significantly increases the speed of data transfer between host devices.

Referring toFIG.8, the switch box100allows direct memory access (DMA) data transfers from a first host device to a second host device to be performed faster as compared to a conventional SR-IOV system that uses remote DMA (RDMA) through an Ethernet interface card. In some implementations, an NVMe device800includes a DMA engine802that can read data from the NVMe controller memory buffer280and write the data to a main memory804of a host device A806, or read data from the main memory804of the host device A806and write the data to the NVMe controller memory buffer280. Similarly, the DMA engine802can read data from the NVMe controller memory buffer280and write the data to a main memory808of a host device B810, or read data from the main memory808of the host device B810and write the data to the NVMe controller memory buffer280. By using the process2600ofFIG.26, one or more partitions of the controller memory buffer280can be shared by host device A806and host device B810. The DMA engine802of the NVMe device800can then be used to transfer data from the main memory804of the host device A806to the main memory808of the host device B810through the shared controller memory buffer280. Similarly, the DMA engine802of the NVMe device800can be used to transfer data from the main memory808of the host device B810to the main memory804of the host device A806through the shared controller memory buffer280.

Referring toFIG.9, the PCIe switch box system100includes hardware and software components. The software components can be divided into a kernel space902and a user space904. The kernel space902can include, e.g., system drivers906and device drivers908. The system drivers906can include, e.g., PCIeHP drivers910and SR-IOV drivers912. The device drivers908can include, e.g., NVMe drivers914and PCIe switch management divers920(e.g., PEX88096 management drivers). For example, the NVMe drivers914can provide a SysFS interface916and an IOCTL interface918. The applications in the user space904can issue operation commands to the NVMe drivers912using Sysfs and IOCTL function calls.

The user space904can include management utility applications950that include management daemons922, e.g., an NVMe administrator daemon924, an SR-IOV daemon926, a system daemon928, and a PCIe switch daemon930. The NVMe administrator daemon924can manage and store information, e.g., an NVMe information page932and an NVMe configuration page934. The NVMe information page932can include, e.g., model name, serial number, controller information, and namespace information. The NVMe configuration page934can include, e.g., namespace number, namespace size, and namespace share/private state information.

The SR-IOV daemon926can manage and store information, e.g., an information page936and a configuration page938. The information page936can include, e.g., virtual function number, virtual function and namespace identifier relation information. The configuration page938can store, e.g., virtual function number, and virtual function and namespace identifier relation information. The system daemon928can manage and store information, e.g., an information page940and a configuration page942. The information page940can include, e.g., MAC address, RTC, internet protocol (IP) address, firmware information. The configuration page942can store, e.g., RTC, IP, firmware update information. The PCIe switch daemon930can manage and store information, e.g., an information page944and a configuration page946. The information page944can store, e.g., firmware information, driver parameters, and log files. The configuration page946can store, e.g., firmware update information, driver parameters, and chip reset information.

A web graphical user interface948can be provided to allow the user to easily configure various functions and parameters of the PCIe switch box system100.

Referring toFIG.10, in some implementations, the PCIe switch box system100includes software components that include a management socket1000that can support multiple functions, e.g., web server, JAR, JSON.

Referring toFIG.11, in some implementations, a host server or workstation1100(which can be similar to, e.g.,102,104, or106ofFIG.1) can include hardware components1102and software components1104. The hardware components1102include a PCIe switch1106that allows the host server1100to access the remote NVMe virtual functions. In this example, the PCIe switch1106is capable of accessing four remote PCIe slots1108a,1108b,1108c, and1108d. The first remote PCIe slot1108ais empty. The second, third, and fourth remote PCIe slots1108b,1108c,1108dare assigned to remote NVMe virtual functions DevFunc(m, a), DevFunc(m, b), and DevFunc(m, c), respectively.

The software components1104include a kernel space1110and a user space1112. The kernel space1110can include device drivers1114, such as NVMe drivers1116and VFIO drivers1122. The NVMe drivers can provide a SysFS interface1118and an IOCTL interface1120. The applications in the user space1112can issue operation commands to the NVMe drivers1116using Sysfs and IOCTL function calls.

The user space1112can include, e.g., Docker software1124, database software1126, administration software1128, virtual machine hypervisors1130, virtual machines1132, virtual machine NVMe drivers1134, and storage performance development kits1136.

For example, the management computer can issue instructions, e.g., configuration namespace, attach namespace to virtual function, assign DevFunc(0,0) to the host server1100, assign virtual function DevFunc(m,a) to host A, assign virtual function DevFunc(m,b) to the host server1100, and assign virtual function DevFunc(m,c) to the host server1100.

FIG.12shows the signal paths between the PCIe switch box system100and a host device1200, which can be similar to, e.g.,102,104,106ofFIG.1. The signal paths include a NVMe virtual function direct memory access (DMA) path1202, which extends from the main memory1204of the host device1200to the root complex1214of the host device1200, from the root complex1214to the PCIe switch1206, from the PCIe switch1206to the PCIe redriver1208, from the PCIe redriver1208to the PCIe switch1210, and from the PCIe switch1210to the virtual function 11212of the NVMe device1232. An NVMe virtual function 1 TLP configuration path and an NVMe virtual function 1 memory-mapped I/O (MMIO) path1216extend from the CPU1218at the host device1200to the root complex1214of the host device1200, from the root complex1214to the PCIe switch1206, from the PCIe switch1206to the PCIe redriver1208, from the PCIe redriver1208to the PCIe switch1210, and from the PCIe switch1210to the virtual function 11212of the NVMe device1232.

A TLP configuration path/memory-mapped I/O (MMIO) interrupt path1220extends from the PCIe switch1210to the root complex1224of the PCIe switch box system100, and from the root complex1224to the management CPU1222. In the PCIe switch box system100, a native NVMe driver path1226extends from the operating system1228to the physical function1230of the NVMe device1232. In the host device1200, a native NVMe driver path1226extends from the operating system1238to the remote virtual function 11240of the synthetic NVMe device1242. In the PCIe switch box system100, a basic input/output system (BIOS) sizing BAR path1234extends from the BIOS1236of the PCIe switch box system100to the physical function1230of the NVMe device1232. In the host device1200, a basic input/output system (BIOS) sizing BAR path1234extends from the BIOS1246of the host device1200to the remote virtual function 11240of the synthetic NVMe device1242. A synthetic PCIe path1244extends from the PCIe switch1206to the remote virtual function1240. The PCIe switch1206generates a synthetic PCIe tree.

FIG.13shows various steps of exchange of information between the PCIe switch box system100, a host device1300(which can be similar to, e.g.,102,104,106ofFIG.1), and the management computer140for enabling virtualization of NVMe device functions. An administrator or user of the host device1300first specifies the NVMe requirements1302, such as the quality of service (QoS) and volume size. At step 1, the administrator or the user of the host device1300sends1304a request to the administrator of the PCIe switch box system100. At step 2, the administrator acknowledges1306the receipt of the request from the host device100. At step 3, the administrator, by using the PCIe management interface, configures1308the NVMe device according to the request from the host device1300. At step 4, the PCIe management interface1312sends1314an acknowledgement that the NVMe configuration instructions have been received. At step 5, the PCIe management interface1312sends1316the instructions for configuring and/or assigning an NVMe device to the PCIe switch box system100. At step 6, the PCIe switch box system100sends1318an acknowledgement that the NVMe configuration and/or assignment instructions have been received. At step 7, the NVMe virtual function is hooked1320under the synthetic tree. These steps enable the host CPU1322to access the remote NVMe virtual function1310through the PCIe switch1324.

The figure also shows the native NVMe driver path1326and the BIOS sizing BAR path1328.

Referring toFIG.14, the PCIe switch box system100includes a baseboard management controller1400, which can be a system-on-chip that manages the operation of various components of the PCIe switch box system100, including monitoring the temperatures of various chips and the fan speeds. The baseboard management controller1400allows the remote user to know the parameters of the enclosure of the PCIe switch box system100. For example, the baseboard management controller1400can be implemented using model AST2500 from ASPEED Technology, Inc., Hsinchu City, Taiwan. The baseboard management controller1400can store the management daemons1402including, e.g., the NVMe administration daemon1404, the SR-IOV daemon1406, the system daemon1408, and the PCIe switch daemon1410.

Referring toFIG.15, some of the management daemons1402can be accessed through an external management CPU operation system. In this example, the NVMe administration daemon1404, the SR-IOV daemon1406, and the PCIe switch daemon1410can be accessed through the external management CPU operation system.

FIG.16shows the hardware architecture of the PCIe switch box system100and the host device1600. The PCIe switch box system100includes a PCIe switch116, and the host device1600includes a host PCIe switch1602. For example, the switch box PCIe switch116can be the PEX88096 chip, but other PCIe switch integrated circuits can also be used. For example, the host PCIe switch1602can be the PEX88032 chip, but other PCIe integrated circuits can also be used. The host PCIe switch1602can operate as a fan-out mode PCIe switch card.

At the host side1600, the host PCIe switch1602includes an upstream port1604and a downstream port1606. The upstream port1604communicates with the host side PCI root port1608. The downstream port1606communicates with an upstream port1610of the switch box PCIe switch116. At the PCIe switch box system100, the switch box PCIe switch116includes the upstream port1610and a downstream port1612. The upstream port1610communicates with the downstream port1606of the host PCIe switch1602. The downstream port1612of the switch box PCIe switch116communicates with the PCIe device1614, including the NVMe physical function1616and the NVMe virtual functions1618.

FIG.17is a flow diagram of an example of a process1700that includes configuration steps performed in the switch box PCIe switch116to assign an NVMe SR-IOV virtual function to a host port. Steps1702to1706are similar to the steps2402to2406ofFIG.24. Steps1708to1716are similar to the steps2418to2426ofFIG.24.

FIG.18is a flow diagram of an example of a process1800for implementing a boot up sequence when using the PCIe switch box system100that includes SR-IOV capable NVMe devices. The process1800includes steps1802to1824that are performed at the PCIe switch box system100, and steps1826to1834that are performed at the host server or workstation. At step1802, the PCIe switch box system100is powered on. At step1804, the baseboard management controller (BMC) system on chip is boot up. For example, the baseboard management controller can be the AST2500 chip. At step1806, the external management CPU (emCPU) board is boot up. For example, the external management CPU can be the CPU118ofFIG.1. At step1808, the basic input/output system (BIOS) of the PCIe switch box system100scans for the PCIe devices installed in the PCIe switch box system100. At step1810, the BIOS finds the NVMe physical function(s) and reserves the PCIe bus number(s) and the memory mapped IO. In step1810, the BIOS configures two PCie device resources: the bus number and the BAR space. After these PCIe device resources are configured, the management CPU can read from or write to the PCIe device registers. After BIOS scans and identifies an NVMe physical function in step1808, in step1810the BIOS configures the bus number and the BAR space of the NVMe physical function, and enables the management CPU to read from or write to the registers of the NVMe physical function. The NVMe specification defines the relevant registers that need to be configured in step1810.

At step1812, the system boots into the management operating system. At step1814, the management operating system loads the NVMe driver(s) for the physical functions. At step1816, the operating system enables the NVMe SR-IOV functions of the SR-IOV capable NVMe devices. At step1818, the operating system loads the management driver of the switch box PCIe switch116. At step1820, the operating system executes the switch box PCIe switch116management daemons.

In some implementations, the PCIe switch box system100has already been configured in which certain namespaces are attached to certain virtual functions, and certain virtual functions are assigned to certain nodes of a synthetic PCIe tree. When the PCIe switch box system100is powered down, these configurations are stored in a non-volatile storage device. At step1822, the previously stored configuration data for the attachment of namespaces are loaded, and the NVMe namespaces are attached to the NVMe virtual functions according to the configuration data. At step1824, the previously stored configuration data for the assignment of the NVMe virtual functions are loaded, and the NVMe virtual functions are assigned to the nodes of the synthetic PCIe tree according to the configuration data.

At step1826, the host server is powered on or rebooted. At step1828, the BIOS of the host server scans for available PCIe devices. At step1830, the BIOS finds the NVMe virtual function(s) and reserves the bus number and memory mapped JO (MMIO) ranges. After the BIOS scans and identifies the NVMe virtual function in step1828, in step1830the BIOS configures the bus number and the BAR space of the NVMe virtual function to enable the host device side CPU to read from or write to the NVMe virtual function registers. The NVMe specification defines the relevant registers that need to be configured in step1830. Note that at step1828, after scanning for available PCIe devices, the host server does not identify the physical function of the SR-IOV capable NVMe device in the PCIe switch box system100. Rather, the host server identifies the NVMe virtual function on the synthetic PCIe tree generated by the PCIe switch box system100.

At step1832, the host server boots into the operating system. At step1834, the operating system loads the NVMe driver for the virtual function or loads the virtual function IO driver for the NVMe virtual function. There are two ways for the host server operating system to access the NVMe virtual function namespace. For example, when the file system of the host server reads from or writes to the namespace attached to the NVMe virtual function, the file system can use the NVMe driver (which has been developed according to the NVMe specification) to communicate with the NVMe device controller. The NVMe driver can reside in the kernel layer of the host server operating system. As another example, the application programs in the user space can use the virtual function IO drivers (for the NVMe virtual function) residing in the kernel layer of the host server operating system to read from or write to the namespace attached to the NVMe virtual function. For example, the virtual machines can “pass-through” the hypervisor to directly access the virtual function by using the virtual function IO drivers (for the NVMe virtual function) to directly read from or write to the namespace attached to the NVMe virtual function. Note that the PCIe switch box system100allows the virtual machines or file systems of multiple host servers to read from or write to the namespace attached to the same NVMe drive, or the controller memory buffer associated with the same NVMe drive. Some host servers can load the kernel layer NVMe driver, and some host servers can load the kernel layer virtual function IO driver for the NVMe virtual function.

FIG.19is an example of a flow diagram of a process1900for assigning/inserting an NVMe virtual function to a synthetic PCIe tree when the host operating system does not support NVMe hot plug. In this case, when a new NVMe device is hot plugged into the PCIe switch box system100, the host server or workstation will not be able to detect the new NVMe device. The NVMe device is not hooked to the downstream port of a synthetic PCIe tree of any host port, so the host server does not detect any change in the PCIe devices. In order for the host server to detect the newly inserted NVMe device, the NVMe device needs to be hooked to the downstream port of the synthetic PCIe tree, and a TLP needs to be sent to the host port to notify the host server about the changes in the PCIe devices, Without the above steps, some host servers may not allocate appropriate PCIe resources to the newly inserted NVMe device. The host server has to reboot in order for the BIOS to allocate appropriate PCIe resources and be able to use the newly added NVMe device. When the NVMe virtual function is hooked to the synthetic PCIe tree, the host server needs to reboot in order to allocate PCIe resources for the NVMe virtual function, such as the bus number of the NVMe virtual function and the MMIO resources. The process1900includes steps1902and1904that are performed at the PCIe switch box system100, and steps1906to1914that are performed at the host server or workstation. At step1902, the switch box PCIe switch116assigns or inserts the NVMe virtual function to the synthetic PCIe tree. At step1904, the switch box PCIe switch116issues a hotplug MSI TLP to the synthetic downstream port (DSP). The TLP packet notifies the host server that a PCIe device hotplug event has occurred at a certain downstream port of the synthetic PCIe tree.

At step1906, the host server is powered on or rebooted. At step1908, the host server BIOS scans for available PCIe devices. At step1910, the host server BIOS finds the NVMe virtual functions, and reserves the PCIe bus number and the memory mapped IO. At step1912, the host server boots into the operating system. At step1914, the host server operating system loads the NVMe driver for the virtual function, or loads the virtual function IO driver for the NVMe virtual function.

FIG.20is a flow diagram of an example of a process2000for un-assigning/removing an NVMe virtual function from a synthetic PCIe tree when the host operating system does not support NVMe hot plug. In this case, when an NVMe device is unassigned from the host server or removed from the PCIe switch box system100, the host server will not detect that the NVMe device has been unassigned or removed. The process2000is performed to enable the host server to accurately determine the available PCIe devices. The process2000includes steps2002and2004that are performed at the PCIe switch box system100, and steps2006to2012that are performed at the host server or workstation. At step2002, the NVMe virtual function is unassigned or removed from the synthetic PCIe tree. At step2004, the switch box PCIe switch116issues a hotplug MSI TLP to the synthetic downstream port (DSP).

At step2006, the host server is powered on or rebooted. At step2008, the host server BIOS scans for available PCIe devices. At step2010, the host server BIOS finds the IO enabler endpoint, and reserves the PCIe bus number and the memory mapped IO. When the NVMe virtual function is removed from the synthetic PCIe tree, the vacant position is filled in using an IO enabler endpoint PCIe device. The function of the IO enabler endpoint is to reserve appropriate PCIe resources (e.g., the bus number and BAR space) so that when the host server BIOS allocates resources for the synthetic PCIe tree generated by the PCIe switch box system100, the host server BIOS can allocate the bus number and the BAR space to the IO enabler endpoint. When an NVMe virtual function is added to the synthetic PCIe tree, the IO enabler endpoint can be removed from the synthetic PCIe tree and be replaced by the NVMe virtual function. This way, the host server operating system does not need to reboot in order for the BIOS to configure the PCI resource of the NVMe virtual function. At step2012, the host server boots into the operating system.

FIG.21is a flow diagram of an example of a process2100for assigning/inserting an NVMe virtual function to a synthetic PCIe tree when the host operating system supports NVMe hot plug. When a new NVMe device is hot plugged into the PCIe switch box system100, the host server or workstation, the NVMe device is initially not hooked to the downstream port of a synthetic PCIe tree of any host port, so the host server does not detect any change in the PCIe devices. In order for the host server to detect the newly inserted NVMe device, the NVMe device needs to be hooked to the downstream port of the synthetic PCIe tree, and a TLP needs to be sent to the host port to notify the host server about the changes in the PCIe devices. The process2100includes steps2102and2104that are performed at the PCIe switch box system100, and steps2106and2108that are performed at the host server or workstation. Steps2102and2104are similar to steps1902and1904ofFIG.19. At step2106, the host server operating system calls a pre-registered PCIe hotplug interrupt service routine (ISR) and tries to add a new PCIe device (the NVMe virtual function). In this example, the host server operating system uses a previously registered PCIe hotplug interrupt service routine to allocate system resources to any PCIe device (e.g., NVMe virtual function) newly added to the synthetic PCIe tree. Step2108is similar to step1914ofFIG.19.

FIG.22is a flow diagram of an example of a process2200for un-assigning/removing an NVMe virtual function from a synthetic PCIe tree when the host operating system supports NVMe hot plug. In this case, when an NVMe device is unassigned from the host server or removed from the PCIe switch box system100, the host server can detect the unassignment or removal of the NVMe device without rebooting. The process2200includes steps2202and2204that are performed at the PCIe switch box system100, and steps2206and2208that are performed at the host server or workstation. Steps2202and2204are similar to steps2002and2004ofFIG.20. At step2206, the host server operating system calls a pre-register PCIe hotplug ISR and tries to remove the offlined PCIe device (the NVMe virtual function). At step2208, the host server operating system unloads the NVMe driver for the virtual function, or unloads the virtual function IO driver for the NVMe virtual function.

Referring toFIG.23, in some implementations, two or more PCIe switch box systems100,2300can provide redundancy so that if one PCIe switch box system fails, the host devices160,162can still access the SR-IOV functions provided by the other PCIe switch box system. In this example, each host device includes two PCIe switches for interfacing with the two PCIe switch box systems. For example, the host device160includes a first PCIe switch128for interfacing with the switch box100, and a second PCIe switch2302for interfacing with the switch box2300. The host device162includes a first PCIe switch132for interfacing with the switch box100, and a second PCIe switch2304for interfacing with the switch box2300.

Initially, the PCIe switch box system100is the primary system, and the host devices160and162accesses the virtualized PCIe device physical and virtual functions provided by the PCIe switch box system100. The management computer140monitors the health status of the PCIe switch box system100and2300. If the PCIe switch box system100fails, the management computer140notifies the host devices160and162to change to using the PCIe switch box system2300.

In the examples ofFIGS.1and3, the SR-IOV capable PCIe devices can include graphics processing units (GPUs).FIG.4shows an example in which the first and second workstation computers160,162can access GPU virtual functions170using the PCIe switch box system100. The following describes additional examples of PCIe switch box systems that allow multiple hosts to share resources of the GPUs.

Referring toFIG.29, a PCIe switch box system2900allows multiple host devices, such as a first host device (host A)2902and a second host device (host B)2904, to access virtual functions of an SR-IOV capable graphics processing unit (GPU) (also referred to as a graphics card)2906installed at the PCIe switch box system2900. Although the figure shows a single GPU, two or more GPUs can be installed in the PCIe switch box system2900. The single-root input/output virtualization enables the GPU resources to be accessed by one or more physical functions2908and a plurality of virtual functions2910supported by the one or more GPUs. The physical functions2908can be accessed by, e.g., a PCI SR-IOV driver and a GPU driver. The system architecture shown inFIG.29provides several technical advantages compared to a conventional SR-IOV GPU setup.

In the conventional SR-IOV system, a first GPU is installed at the first host device, and virtual machines at the first host device can access the virtual functions provided by the first GPU. In some examples, the second host device cannot access the resources provided by the first GPU that is installed at the first host device unless a first network interface card is installed at the first host device, and the first host device is configured to share the resources of the first GPU with other devices through the first network interface card. A second GPU is installed at the second host device, and virtual machines at the second host device can access the virtual functions provided by the second GPU. In some examples, the first host device cannot access the resources provide by the second GPU installed at the second host device unless a second network interface card is installed at the second host device, and the second host device is configured to share the resources of the second GPU with other devices through the second network interface card. This setup results in inefficient allocation of the GPU resources because different hosts can have peak GPU requirements at different times. Furthermore, if a GPU installed at the host device needs to be upgraded or repaired, the host device may have to be turned off, and the chassis of the host device may have to be opened in order to access the GPU. This results in down time of the host device.

By comparison, one or more GPUs can be installed at the PCIe switch box system2900, and the GPU resources can be shared by the host devices2902,2904. When multiple GPUs are installed at the PCIe switch box system2900, the GPU resources (e.g., physical functions2908, virtual functions2910, graphics processor cores, and/or graphics memory) from multiple GPUs form a GPU resource pool that can be accessed by the host devices2902,2904. The amount of GPU resources (e.g., the number of graphics processor cores and/or the amount of graphics memory) allocated to each host device2902or2904can be adjusted dynamically depending on the requirements of the host devices, resulting in efficient use of the GPU resources.

Referring toFIG.31A, in some implementations, the GPU2906includes multiple processor cores3100, a graphics memory (or GPU memory)3102that can be divided into multiple memory spaces (or partitions), a graphics controller3104, the physical functions2908, and the virtual functions2910. The term “processor core” refers to a computation unit in the GPU. Different GPU manufacturers have different ways of partitioning the computation units of the GPU. For example, some models of GPUs can each have a smaller number (e.g., 20 or less) of processor cores in which each processor core can perform complicated data processing, whereas some models of GPUs can each have a larger number of processor cores (e.g., 1000 or more) in which each processor core can perform simple arithmetic calculations. Some processor cores can be dedicated to particular types of tasks or calculations. A processor core can be, e.g., a CUDA (Compute Unified Device Architecture) core, a tensor core, or a ray tracing core. Different models of GPUs can have different types of processor cores. For example, the GPU memory3102can be high speed dynamic random-access memory (DRAM), such as graphics double data rate synchronous dynamic random-access memory (GDDR SDRAM). For example, the graphics memory3102can comply with GDDR4 SDRAM, GDDR5 SDRAM, GDDR6 SDRAM standards, and/or other graphics memory standards. A portion of the graphics memory3102can be used as the graphics controller memory buffer. The graphics controller memory buffer is exposed to the PCIe bus and can be accessed (e.g., read data from and/or write data to) by the management CPU118, the CPUs of the host devices (e.g., CPU2802of the host device2800inFIG.28), and/or other PCIe devices connected to the PCIe bus, Other than the graphics controller memory buffer, the majority of the graphics memory3102is used exclusively by the GPU processor cores3100and the graphics controller3104.

Examples of GPUs include NVIDIA GeForce RTX series GPUs, NVIDIA Quadro® RTX series GPUs, NVIDIA A100 GPUs, NVIDIA H100 GPUs, AMD Radeon™ series GPUs, and Intel® Arc™ series GPUs, but can also be other types of GPUs. The technology for graphics processing units advance rapidly. The PCIe switch box system2900can include multiple types of GPUs from multiple manufacturers, including current and future models of GPUs. When newer models of GPUs are available, the operator of the PCIe switch box system2900can replace the older GPUs with newer GPUs, upload new graphics drivers for the new GPUs to the system2900, reconfigure the system2900to allocate the GPU resources of the new GPUs to new virtual functions, and assign the new virtual functions to the host devices.

Each virtual function2910can be associated with one or more processor cores3100, and/or one or more memory spaces (or partitions)3102. In some examples, each virtual function can be associated with a virtual GPU (vGPU), each virtual GPU can be allocated a certain amount of computing capability (e.g., a certain number of CUDA cores) and a certain amount of memory.FIG.31Ashows an example of the resources that can be provided by a GPU, it is understood that other types of resources are also possible. In some examples, the GPU can be designed to provide additional resources, such as various audio/video codecs, ray-tracing accelerators, artificial intelligence computation accelerators, and/or physical simulation accelerators. In some examples, a physical graphics processor core can be associated with multiple virtual functions on a time-share basis, in which different virtual functions access the physical graphics processor core at different time slots.

Referring back toFIG.29, in some implementations, the PCIe switch box2900includes a PCIe switch2940that communicates with the host devices2902,2904and assigns one or more GPU virtual functions2910to each host device, and enables the host devices to access the GPU resources using the assigned GPU virtual functions2910. The GPU resources can include, e.g., graphics processor cores and/or graphics memory. In some implementations, the PCIe switch2940can be model PEX88096 PCIe Gen4 Switch, available from Broadcom, San Jose, California. For example, the GPU2906can be an SR-IOV capable GPU. The PCIe switch2940assigns the virtual functions of the SR-IOV capable GPUs to different host ports, so that different hosts can access the GPU resources. For example, both host A2902and host B2904can access the same graphics memory from the same GPU. The registers of the PCIe switch2940can be set to allow the downstream port GPU virtual function of the PCIe switch2940to be assigned to any upstream host port of the PCIe switch2940.

One or more memory devices120store management software that when executed by a management CPU118causes the management CPU118to configure the PCIe switch2940to enable the host devices to access the GPU resources using the GPU virtual functions. The PCIe switch box system2900includes a root complex device122that connects the CPU118and the memory devices120to the PCIe switch2940.

In some implementations, the host device can be a personal computer or workstation executing MacOS, Linux, or Windows operating system. For example, the host device can be a server computer that executes Docker software. For example, the host device can include a virtual machine manager and execute multiple virtual machines. Each of the first host device2902and the second host device2904executes a GPU driver2936associated with the GPU2906. The GPU driver2936enables the host devices to use the application programming interfaces (APIs) provided by the GPU2906. The graphics driver take graphics rendering instructions from the host device operating system and translate them into instructions that the GPU2906can understand and execute. The GPU driver2936also informs the host device operating system the capabilities and configurations of the GPU2906(e.g., DirectX and OpenGL feature levels). In some examples, the graphics driver can optimize the instructions that the application programs (e.g., games) executing on the host device are sending to be more efficient. The first host device2902includes a PCIe switch2932a, and the second host device2904includes a PCIe switch2932b. The PCIe switch box2900includes a first PCIe redriver (e.g., redrive board)2934aand a second PCIe redriver (e.g., redrive board)2934b. The PCIe switch2932ais electrically connected to the PCIe redriver2934athrough a PCIe extension cable, and the PCIe switch2932bis electrically connected to the PCIe redriver2934bthrough a PCIe extension cable.

In some implementations, the CPU118executes one or more management drivers (e.g., PCIe switch management drivers920inFIG.9) stored in the memory120to configure the PCIe switch2940to function as a switch manager that manages the assignments of GPU virtual functions to the host devices. For example, the PCIe switch2940assigns a first GPU virtual function2910ato the first host device2902, and assigns a second GPU virtual function2910bto the second host device2904. The host device2902accesses the first GPU virtual function2910athrough a first PCIe data path2924. Accessing a GPU virtual function can mean, e.g., sending data processing instructions to the graphics processor cores, reading data from the graphics memory, or writing data to the graphics memory. For example, the PCIe data path2924can comply with PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path2924extends from the PCIe switch2932aof the host device2902to the PCIe redriver2934aof the PCIe switch box system100, from the PCIe redriver2934ato the PCIe switch2940, and from the PCIe switch2940to the first GPU virtual function2910a. The host device2904accesses the second GPU virtual function2910bthrough a second PCIe data path2926, which can comply with, e.g., PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path2926extends from the PCIe switch2932bof the host device2904to the PCIe redriver2934bof the PCIe switch box system2900, from the PCIe redriver2934bto the PCIe switch2940, and from the PCIe switch2940to the second GPU virtual function2910b.

A management computer2914communicates with a communication interface2930of the PCIe switch box system2900through a secure communication channel2942, such as a secure Ethernet link. The management computer2914can provide a user interface2912that allows the administrator to conveniently determine the capabilities of the PCIe switch box system2900, such as what SR-IOV capable devices are available, which physical and virtual functions are available, what graphics processor cores are available, and what graphics memory spaces (or partitions) are available. Through the user interface2912, the administrator can assign particular physical functions or virtual functions to particular host devices.

Through the user interface2912at the management computer2914, the administrator can allocate GPU resources to GPU virtual function 1 (step2916), allocate GPU resources to GPU virtual function 2 (step2918), issue the instruction “VF_DevFunc(0,1)@Host A” (step2920), and issue the instruction “VF_DevFunc(0,2)@Host B” (step2922). In step2920, a resource mapping between GPU virtual function 1 to host device A2902is recorded, and in step2922, a resource mapping between GPU virtual function 2 to host device B2904is recorded.

As a result of the configuration instructions issued by the management computer2914, certain GPU resources are allocated to the first GPU virtual function2910aand the second GPU virtual function2910b. The host device2902accesses the first GPU virtual function2910athrough the PCIe data path2924, and the host device2904accesses the second GPU virtual function2910bthrough the PCIe data path2926. For example, the PCIe data paths2924,2926can comply with PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The first host device2902remotely accesses the first GPU virtual function2910aas if accessing a local GPU virtual function2944a. The second host device2904remotely accesses the second GPU virtual function2910bas if accessing a local GPU virtual function2944b.

Referring toFIG.30, in some implementations, a PCIe switch box system3000allows multiple host devices (e.g.,2902,2904) to access virtual functions of an SR-IOV capable network interface card (NIC)3002installed at the PCIe switch box system3000. Although the figure shows a single network interface card, two or more network interface cards can be installed in the PCIe switch box system3000. The single-root input/output virtualization enables the NIC resources to be accessed by one or more physical functions3018and a plurality of virtual functions3006supported by the one or more NICs3002. The physical functions3018of the NIC3002can be accessed by, e.g., a PCI SR-IOV driver and a NIC driver. The system architecture shown inFIG.30provides several technical advantages, similar to the example ofFIG.29, compared to a conventional SR-IOV NIC setup.

One or more NICs3002can be installed at the PCIe switch box system3000, and the NIC resources can be shared by the host devices2902,2904. When multiple NICs are installed at the PCIe switch box system3000, the NIC resources (e.g., physical functions, virtual functions, network ports (e.g., 2.5 Gb Ethernet ports, 10 Gb Ethernet ports, 25 Gb SFP optical communication ports), and the resources connected to the network ports (e.g., network attached storage devices, network attached scanners, network attached printers, network attached3D printers)) from multiple NICs form a NIC resource pool that can be accessed by the host devices2902,2904. The amount of NIC resources (e.g., the network bandwidth, network attached storage, network attached scanners, network attached printers, and network attached3D printers) allocated to each host device2902or2904can be adjusted dynamically depending on the requirements of the host devices, resulting in efficient use of the NIC resources.

Referring toFIG.31B, in some implementations, the NIC3002includes multiple network ports3110, a buffer memory3112, a network interface controller3114, the physical functions3018, and the virtual functions3006. In some examples, the network ports3110can include several types of network ports, such as 2.5 Gb Ethernet ports, 10 Gb Ethernet ports, 25 Gb SFP optical ports. The buffer memory3112can have one or more partitions. In some implementations, the network interface controller3114can set one or more partitions of the buffer memory3112to a “shared” state and attaches one or more shared partitions of the buffer memory3112to multiple virtual functions. The PCIe switch2940assigns the virtual functions to host devices2902,2904and enables the host devices2902,2904to access the one or more shared partitions of the NIC buffer memory3112using the assigned virtual functions. The management central processor unit118configures the PCIe switch2940and the network interface controller3114to enable the host devices2902,2904to access the one or more shared partitions of the NIC buffer memory3112using the virtual functions. Data can be transferred between the host devices2902,2904and the one or more shared partitions of the MC buffer memory3112using direct memory access (DMA) transfers.

Each virtual function3006can be associated with one or more network ports3110, and the host device that is assigned the virtual function3006can have its own MAC address and IP address. Different host devices that use different virtual functions to access the same network port can have different MAC addresses and different IP addresses. In some examples, each virtual function can be associated with a virtual NIC (vNIC), each virtual NIC can be allocated a certain number of network ports or a certain amount of network bandwidth, and/or one or more partitions of the buffer memory3112.FIG.31Bshows an example of the resources that can be provided by a MC, it is understood that other types of resources are also possible. In some examples, the NIC can be designed to provide additional resources, such as virtual private network functions. In some examples, a physical network port can be associated with multiple virtual functions on a time-share basis, in which different virtual functions access the physical network port at different time slots.

Referring back toFIG.30, in some implementations, the PCIe switch box3000includes a PCIe switch2940that communicates with the host devices2902,2904and assigns one or more NIC virtual functions to each host device, and enables the host devices to access the NIC resources using the assigned NIC virtual functions3006. The NIC resources can include, e.g., network ports having various communication capabilities (e.g., various network communication speeds). In some implementations, the PCIe switch2940can be model PEX88096 PCIe Gen4 Switch, available from Broadcom, San Jose, California. For example, the NIC3002can be an SR-IOV capable NIC. The PCIe switch2940assigns the virtual functions of the SR-IOV capable NICs3002to different host ports, so that different hosts can access the NIC resources through the NIC virtual functions. For example, both host A2902and host B2904can access the network port through different virtual functions of the NIC3002. The registers of the PCIe switch2940can be set to allow the downstream port NIC virtual function of the PCIe switch2940to be assigned to any upstream host port of the PCIe switch2940.

In some implementations, the host device can be a personal computer or workstation executing MacOS, Linux, or Windows operating system. For example, the host device can be a server computer that executes Docker software. For example, the host device can include a virtual machine manager and execute multiple virtual machines. Each of the first host device2902and the second host device2904executes a NIC driver3004associated with the NIC3002. The NIC driver3004enables the host devices to use the application programming interfaces (APIs) provided by the MC3002. The first host device2902includes a PCIe switch2932a, and the second host device2904includes a PCIe switch2932b. The PCIe switch box2900includes a first PCIe redriver (e.g., redrive board)2934aand a second PCIe redriver (e.g., redrive board)2934b. The PCIe switch2932ais electrically connected to the PCIe redriver2934athrough a PCIe extension cable, and the PCIe switch2932bis electrically connected to the PCIe redriver2934bthrough a PCIe extension cable.

In some implementations, the CPU118executes one or more management drivers (e.g., PCIe switch management drivers920inFIG.9) stored in the memory120to configure the PCIe switch2940to function as a switch manager that manages the assignments of NIC virtual functions to the host devices. For example, the PCIe switch2940assigns a first virtual function3006aof the MC3002to the first host device2902, and assigns a second virtual function3006bof the NIC3002to the second host device2904. The host device2902accesses the first NIC virtual function3006athrough a first PCIe data path2924. Accessing an NIC virtual function can mean, e.g., accessing a network port, obtaining an associated MAC address and IP address, and sending and/or receiving data packets and control signals through the network port. For example, the PCIe data path2924can comply with PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path2924extends from the PCIe switch2932aof the host device2902to the PCIe redriver2934aof the PCIe switch box system100, from the PCIe redriver2934ato the PCIe switch2940, and from the PCIe switch2940to the first NIC virtual function3006a. The host device2904accesses the second NIC virtual function3006bthrough a second PCIe data path2926, which can comply with, e.g., PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The PCIe data path2926extends from the PCIe switch2932bof the host device2904to the PCIe redriver2934bof the PCIe switch box system2900, from the PCIe redriver2934bto the PCIe switch2940, and from the PCIe switch2940to the second NIC virtual function3006b.

A management computer2914communicates with a communication interface2930of the PCIe switch box system3000through a secure communication channel2942, such as a secure Ethernet link. The management computer2914can provide a user interface2912that allows the administrator to conveniently determine the capabilities of the PCIe switch box system3000, such as what SR-IOV capable devices are available, which physical and virtual functions are available, and what network interface functions are available. Through the user interface2912, the administrator can assign particular physical functions or virtual functions to particular host devices.

Through the user interface2912at the management computer2914, the administrator can allocate NIC resources to NIC virtual function 1 (step3008), allocate NIC resources to NIC virtual function 2 (step3010), issue the instruction “VF_DevFunc(0,1)@Host A” (step3012), and issue the instruction “VF_DevFunc(0,2)@Host B” (step3014). In step3012, a resource mapping between NIC virtual function 1 to host device A2902is recorded, and in step3014, a resource mapping between NIC virtual function 2 to host device B2904is recorded.

As a result of the configuration instructions issued by the management computer2914, certain NIC resources are allocated to the first NIC virtual function3006aand the second NIC virtual function3006b. The host device2902accesses the first NIC virtual function3006athrough the PCIe data path2924, and the host device2904accesses the second NIC virtual function3006bthrough the PCIe data path2926. For example, the PCIe data paths2924,2926can comply with PCIe 4.0, 5.0, 6.0, and/or 7.0 specification. The first host device2902remotely accesses the first NIC virtual function3006aas if accessing a local NIC virtual function3016a. The second host device2904remotely accesses the second NIC virtual function3006bas if accessing a local NIC virtual function3016b.

FIG.32shows the hardware architecture of the PCIe switch box system2900and the host device2902. The PCIe switch box system2900includes a PCIe switch2940and the host device2902includes a host PCIe switch2932a. For example, the switch box PCIe switch2940can be the PEX88096 chip, but other PCIe switch integrated circuits can also be used. For example, the host PCIe switch2932acan be the PEX88032 chip, but other PCIe integrated circuits can also be used. The host PCIe switch2940can operate as a fan-out mode PCIe switch card.

At the host side2902, the host PCIe switch2932aincludes an upstream port1604and a downstream port1606. The upstream port1604communicates with the host side PCI root port1608. The downstream port1606communicates with an upstream port1610of the switch box PCIe switch2940. At the PCIe switch box system2900, the switch box PCIe switch2940includes the upstream port1610and a downstream port1612. The upstream port1610communicates with the downstream port1606of the host PCIe switch2932a. The downstream port1612of the switch box PCIe switch2940communicates with the GPU2906, including the GPU physical function2908and the GPU virtual functions2910.

FIG.33shows the hardware architecture of the PCIe switch box system3000and the host device2902. The components of the host device2902are the same as or similar to the example ofFIG.32. The PCIe switch box system3000includes a PCIe switch2940, similar to the example ofFIG.32. At the PCIe switch box system3000, the switch box PCIe switch2940includes the upstream port1610and a downstream port1612. The upstream port1610communicates with the downstream port1606of the host PCIe switch2932a. The downstream port1612of the switch box PCIe switch2940communicates with the NIC3002, including the MC physical function3018and the NIC virtual functions3006.

FIG.34is a diagram of an example of a process3400for configuring the PCIe switch2940of the PCIe switch box system2900to assign a GPU SR-IOV virtual function of an SR-IOV capable GPU PCIe device to a host port, which can be a port of a particular host device, e.g.,2902,2904inFIG.29. The majority of the process3400is performed in the PCIe switch box system2900. At step3402, a host port synthetic PCIe tree is initialized. At step3404, the host device sends a PCIe configuration transaction layer packet (TLP) to inquire information about the PCIe devices that are available. The PCIe configuration transaction layer packet is redirected by the switch box PCIe switch2940to the management CPU118. At step3406, the management CPU118modifies the PCIe configuration transaction layer packet in a way such that the packet received by the PCIe device is similar to the packet that the PCIe device would receive if the PCIe device were installed in the host device. Thus, the PCIe device behaves in the same manner as if it were installed in the host device. At step3408, the management CPU118loads the GPU drivers for the GPU physical functions (PF) (a GPU driver for each corresponding physical function) to enable the management software in the PCIe switch box system2900to perform setup of the GPU, such as assigning computing and memory resources to a GPU virtual function. At step3410, each of attached GPU is configured through vender specific instructions, including assigning customized GPU resources (e.g., processor cores, memory) to each candidate GPU virtual function (VF). For example, the GPU can be configured by default settings if this step is skipped. At step3412, the GPU SR-IOV function is enabled.

At step3414, the management CPU118assigns a GPU virtual function (VF) to the host port (e.g., insert a synthetic device to the synthetic PCIe tree). At step3416, the management CPU118sets up a PCI identity (ID) trap for data transfer from the GPU device to the host port. The PCI identity trap is set up at a downstream port to provide identity (ID) routing information for upstream routes (10device to the host device). For example, this can occur when the IO device initiates a DMA data transfer. The address routing will be transformed to ID routing, since the address value is in the host address space.

At step3418, the management CPU118sets up the fabric path (across different chips) for sending data from the PCIe device to the host port, and from the host port to the PCIe device. For example, this provides routing information when the destination is not in the source switch. Thus, the fabric path can be used in cross-switch or cross-domain environments, e.g., switch cascade. This supports up to 256 domains and up to 256 busses per domain.

At step3420, when the host device writes configuration data, the PCI identifier (ID) translations for G2H (management CPU to host) and H2G (host to management CPU) are set up. For example, this translates the requester ID (RID) between host (local) domain and mCPU (global) domain. The TLP travels between the host domain and the mCPU domain, so the requester ID needs to be translated to a proper value. This provides local-to-global and global-to-local RID translation.

At step3422, when the host device writes to the base address registers (BARs), an address trap for translating the address from the host device to the management CPU118domain is set up. For example, this translates addresses between the host device and the PCIe device. The setup at a host port (BAR access) is as follows: The host address space will be translated to mCPU address space within a specific range. The setup at a downstream (PCIe device) port is as follows: The first device address will be translated to another device address for peer-to-peer transfer.

FIG.35is a diagram of an example of a process3500for configuring the PCIe switch2940of the PCIe switch box system3000to assign an NIC SR-IOV virtual function of an SR-IOV capable NIC PCIe device to a host port, which can be a port of a particular host device, e.g.,2902,2904inFIG.30. The majority of the process3500is performed in the PCIe switch box system3000. At step3502, a host port synthetic PCIe tree is initialized. At step3504, the host device sends a PCIe configuration transaction layer packet (TLP) to inquire information about the PCIe devices that are available. The PCIe configuration transaction layer packet is redirected by the switch box PCIe switch2940to the management CPU118. At step3506, the management CPU118modifies the PCIe configuration transaction layer packet in a way such that the packet received by the PCIe device is similar to the packet that the PCIe device would receive if the PCIe device were installed in the host device. Thus, the PCIe device behaves in the same manner as if it were installed in the host device. At step3508, the management CPU118loads the NIC drivers for the NIC physical functions (PF) (an NIC driver for each corresponding physical function) to enable the management software in the PCIe switch box system3000to perform setup of the NIC, such as assigning quality of service (QoS) parameter and MAC address. At step3510, each of attached NIC is configured through vender specific instructions, including assigning customized NIC resources (e.g., quality of service (QoS) parameter, MAC address) to each candidate NIC virtual function (VF). For example, the NIC can be configured by default settings if this step is skipped. At step3512, the NIC SR-IOV function is enabled.

At step3514, the management CPU118assigns an NIC virtual function (VF) to the host port (e.g., insert a synthetic device to the synthetic PCIe tree). At step3516, the management CPU118sets up a PCI identity (ID) trap for data transfer from the NIC to the host port. The PCI identity trap is set up at a downstream port to provide identity (ID) routing information for upstream routes (IO device to the host device). For example, this can occur when the IO device initiates a DMA data transfer. The address routing will be transformed to ID routing, since the address value is in the host address space.

At step3518, the management CPU118sets up the fabric path (across different chips) for sending data from the PCIe device to the host port, and from the host port to the PCIe device. For example, this provides routing information when the destination is not in the source switch. Thus, the fabric path can be used in cross-switch or cross-domain environments, e.g., switch cascade. This supports up to 256 domains and up to 256 busses per domain.

At step3520, when the host device writes configuration data, the PCI identifier (ID) translations for G2H (management CPU to host) and H2G (host to management CPU) are set up. For example, this translates the requester ID (RID) between host (local) domain and mCPU (global) domain. The TLP travels between the host domain and the mCPU domain, so the requester ID needs to be translated to a proper value. This provides local-to-global and global-to-local RID translation.

At step3522, when the host device writes to the base address registers (BARs), an address trap for translating the address from the host device to the management CPU118domain is set up. For example, this translates addresses between the host device and the PCIe device. The setup at a host port (BAR access) is as follows: The host address space will be translated to mCPU address space within a specific range. The setup at a downstream (PCIe device) port is as follows: The first device address will be translated to another device address for peer-to-peer transfer.

FIG.36is a flow diagram of an example of a process3600for implementing a boot up sequence when using the PCIe switch box system2900that includes SR-IOV capable graphics processing unit (GPU) devices. The process3600includes steps3602to3622that are performed at the PCIe switch box system2900, and steps3624to3632that are performed at the host server or workstation. At step3602, the PCIe switch box system2900is powered on. At step3604, the baseboard management controller (BMC) system on chip is boot up. For example, the baseboard management controller can be the AST2500 chip. At step3606, the external management CPU (emCPU) board is boot up. For example, the external management CPU can be the CPU118ofFIG.29. At step3608, the basic input/output system (BIOS) of the PCIe switch box system2900scans for the PCIe devices installed in the PCIe switch box system2900. At step3610, the BIOS finds the GPU physical function(s) and reserves the PCIe bus number(s) and the memory mapped IO. In step3610, the BIOS configures two PCIe device resources: the bus number and the BAR space. After these PCIe device resources are configured, the management CPU can read from or write to the PCIe device registers. After BIOS scans and identifies a GPU physical function in step3608, in step3610the BIOS configures the bus number and the BAR space of the GPU physical function, and enables the management CPU to read from or write to the registers of the GPU physical function. The GPU specification defines the relevant registers that need to be configured in step3610.

At step3612, the system boots into the management operating system. At step3614, the management operating system loads the GPU driver(s) for the physical functions. At step3616, the management CPU118sets the GPU resources (e.g., processor cores and memory) for the candidate GPU virtual functions. At step3618, the operating system enables the GPU SR-IOV functions of the SR-IOV capable GPU devices. At step3620, the operating system loads the management driver of the switch box PCIe switch2940. At step3622, the operating system executes the switch box PCIe switch2940management daemons.

At step3624, the host server is powered on or rebooted. At step3626, the BIOS of the host server scans for available PCIe devices. At step3628, the BIOS finds the GPU virtual function(s) and reserves the bus number and memory mapped IO (MMIO) ranges. After the BIOS scans and identifies the GPU virtual function in step3626, in step3628the BIOS configures the bus number and the BAR space of the GPU virtual function to enable the host device side CPU to read from or write to the GPU virtual function registers. The GPU specification defines the relevant registers that need to be configured in step3628. Note that at step3626, after scanning for available PCIe devices, the host server does not identify the physical function of the SR-IOV capable GPU device in the PCIe switch box system2900. Rather, the host server identifies the GPU virtual function on the synthetic PCIe tree generated by the PCIe switch box system2900.

At step3630, the host server boots into the operating system. At step3632, the operating system loads the GPU driver for the virtual function or loads the virtual function IO driver for the GPU virtual function. There are two ways for the host server operating system to access the GPU virtual function. For example, when the file system of the host server reads from or writes to the GPU memory attached to the GPU virtual function, the file system can use the GPU driver (which has been developed according to the GPU specification) to communicate with the GPU device controller. The GPU driver can reside in the kernel layer of the host server operating system. As another example, the application programs in the user space can use the virtual function IO drivers (for the GPU virtual function) residing in the kernel layer of the host server operating system to read from or write to the GPU memory attached to the GPU virtual function. For example, the virtual machines can “pass-through” the hypervisor to directly access the virtual function by using the virtual function IO drivers (for the GPU virtual function) to directly read from or write to the GPU memory attached to the GPU virtual function. Note that the PCIe switch box system2900allows the virtual machines or file systems of multiple host servers to read from or write to the GPU memory of the same GPU, or the controller memory buffer associated with the same GPU. Some host servers can load the kernel layer GPU driver, and some host servers can load the kernel layer virtual function IO driver for the GPU virtual function.

FIG.37is a flow diagram of an example of a process3700for implementing a boot up sequence when using the PCIe switch box system3000that includes SR-IOV capable network interface cards (NICs). The process3600includes steps3702to3722that are performed at the PCIe switch box system3000, and steps3724to3732that are performed at the host server or workstation. At step3702, the PCIe switch box system3000is powered on. At step3704, the baseboard management controller (BMC) system on chip is boot up. For example, the baseboard management controller can be the AST2500 chip. At step3706, the external management CPU (emCPU) board is boot up. For example, the external management CPU can be the CPU118ofFIG.30. At step3708, the basic input/output system (BIOS) of the PCIe switch box system3000scans for the PCIe devices installed in the PCIe switch box system3000. At step3710, the BIOS finds the GPU physical function(s) and reserves the PCIe bus number(s) and the memory mapped IO. In step3710, the BIOS configures two PCIe device resources: the bus number and the BAR space. After these PCIe device resources are configured, the management CPU can read from or write to the PCIe device registers. After BIOS scans and identifies an NIC physical function in step3708, in step3710the BIOS configures the bus number and the BAR space of the MC physical function, and enables the management CPU to read from or write to the registers of the NIC physical function. The MC specification defines the relevant registers that need to be configured in step3710.

At step3712, the system boots into the management operating system. At step3714, the management operating system loads the NIC driver(s) for the physical functions. At step3716, the management CPU118sets the NIC resources (e.g., quality of service (QoS) parameter, MAC address) for the candidate NIC virtual functions. At step3718, the operating system enables the NIC SR-IOV functions of the SR-IOV capable NIC devices. At step3720, the operating system loads the management driver of the switch box PCIe switch2940. At step3722, the operating system executes the switch box PCIe switch2940management daemons.

At step3724, the host server is powered on or rebooted. At step3726, the BIOS of the host server scans for available PCIe devices. At step3728, the BIOS finds the NIC virtual function(s) and reserves the bus number and memory mapped IO (MMIO) ranges. After the BIOS scans and identifies the MC virtual function in step3726, in step3728the BIOS configures the bus number and the BAR space of the NIC virtual function to enable the host device side CPU to read from or write to the NIC virtual function registers. The NEC specification defines the relevant registers that need to be configured in step3728. Note that at step3726, after scanning for available PCIe devices, the host server does not identify the physical function of the SR-IOV capable NIC device in the PCIe switch box system3000. Rather, the host server identifies the NIC virtual function on the synthetic PCIe tree generated by the PCIe switch box system3000.

At step3730, the host server boots into the operating system. At step3732, the operating system loads the MC driver for the virtual function or loads the virtual function IO driver for the MC virtual function. There are two ways for the host server operating system to access the NIC virtual function. For example, when the file system of the host server reads from or writes to the MAC address attached to the NIC virtual function, the file system can use the NIC driver (which has been developed according to the NIC specification) to communicate with the NIC device controller. The NIC driver can reside in the kernel layer of the host server operating system. As another example, the application programs in the user space can use the virtual function IO drivers (for the NIC virtual function) residing in the kernel layer of the host server operating system to read from or write to the MAC address attached to the NIC virtual function. For example, the virtual machines can “pass-through” the hypervisor to directly access the virtual function by using the virtual function IO drivers (for the NIC virtual function) to directly read from or write to the MAC address attached to the GPU virtual function. Note that the PCIe switch box system3000allows the virtual machines or file systems of multiple host servers to read from or write to the GPU memory of the same GPU, or the controller memory buffer associated with the same GPU. Some host servers can load the kernel layer NIC driver, and some host servers can load the kernel layer virtual function IO driver for the NIC virtual function.

FIG.38is an example of a flow diagram of a process3800for assigning/inserting a graphics processing unit (GPU) virtual function to a synthetic PCIe tree when the host operating system does not support GPU hot plug. In this case, when a new GPU device is hot plugged into the PCIe switch box system2900, the host server or workstation will not be able to detect the new GPU device. The GPU device is not hooked to the downstream port of a synthetic PCIe tree of any host port, so the host server does not detect any change in the PCIe devices. In order for the host server to detect the newly inserted GPU device, the GPU device needs to be hooked to the downstream port of the synthetic PCIe tree, and a TLP needs to be sent to the host port to notify the host server about the changes in the PCIe devices. Without the above steps, some host servers may not allocate appropriate PCIe resources to the newly inserted GPU device. The host server has to reboot in order for the BIOS to allocate appropriate PCIe resources and be able to use the newly added GPU device. When the GPU virtual function is hooked to the synthetic PCIe tree, the host server needs to reboot in order to allocate PCIe resources for the GPU virtual function, such as the bus number of the GPU virtual function and the MMIO resources. The process3800includes steps3802and3804that are performed at the PCIe switch box system2900, and steps3806to3814that are performed at the host server or workstation. At step3802, the switch box PCIe switch2940assigns or inserts the GPU virtual function to the synthetic PCIe tree. At step3804, the switch box PCIe switch2940issues a hotplug MSI TLP to the synthetic downstream port (DSP). The MP packet notifies the host server that a PCIe device hotplug event has occurred at a certain downstream port of the synthetic PCIe tree.

At step3806, the host server is powered on or rebooted. At step3808, the host server BIOS scans for available PCIe devices. At step3810, the host server BIOS finds the GPU virtual functions, and reserves the PCIe bus number and the memory mapped M. At step3812, the host server boots into the operating system. At step3814, the host server operating system loads the GPU driver for the virtual function, or loads the virtual function IO driver for the GPU virtual function.

FIG.39is an example of a flow diagram of a process3900for assigning/inserting a network interface card (MC) virtual function to a synthetic PCIe tree when the host operating system does not support NIC hot plug. In this case, when a new NIC is hot plugged into the PCIe switch box system3000, the host server or workstation will not be able to detect the new NIC. The MC is not hooked to the downstream port of a synthetic PCIe tree of any host port, so the host server does not detect any change in the PCIe devices. In order for the host server to detect the newly inserted NIC, the NIC needs to be hooked to the downstream port of the synthetic PCIe tree, and a TLP needs to be sent to the host port to notify the host server about the changes in the PCIe devices. Without the above steps, some host servers may not allocate appropriate PCIe resources to the newly inserted NIC. The host server has to reboot in order for the BIOS to allocate appropriate PCIe resources and be able to use the newly added NIC. When the NIC virtual function is hooked to the synthetic PCIe tree, the host server needs to reboot in order to allocate PCIe resources for the NIC virtual function, such as the bus number of the MC virtual function and the MMIO resources. The process3900includes steps3902and3904that are performed at the PCIe switch box system3000, and steps3906to3914that are performed at the host server or workstation. At step3902, the switch box PCIe switch2940assigns or inserts the MC virtual function to the synthetic PCIe tree. At step3904, the switch box PCIe switch2940issues a hotplug MSI TLP to the synthetic downstream port (DSP). The TLP packet notifies the host server that a PCIe device hotplug event has occurred at a certain downstream port of the synthetic PCIe tree.

At step3906, the host server is powered on or rebooted. At step3908, the host server BIOS scans for available PCIe devices. At step3910, the host server BIOS finds the NIC virtual functions, and reserves the PCIe bus number and the memory mapped IO. At step3912, the host server boots into the operating system. At step3914, the host server operating system loads the NIC driver for the virtual function, or loads the virtual function IO driver for the MC virtual function.

FIG.40is a flow diagram of an example of a process4000for un-assigning/removing a graphics processing unit (GPU) virtual function from a synthetic PCIe tree when the host operating system does not support GPU hot plug. In this case, when a GPU device is unassigned from the host server or removed from the PCIe switch box system2900, the host server will not detect that the GPU device has been unassigned or removed. The process4000is performed to enable the host server to accurately determine the available PCIe devices. The process4000includes steps4002and4004that are performed at the PCIe switch box system2900, and steps4006to4012that are performed at the host server or workstation. At step4002, the GPU virtual function is unassigned or removed from the synthetic PCIe tree. At step4004, the switch box PCIe switch2940issues a hotplug MSI TLP to the synthetic downstream port (DSP).

At step4006, the host server is powered on or rebooted. At step4008, the host server BIOS scans for available PCIe devices. At step4010, the host server BIOS finds the IO enabler endpoint, and reserves the PCIe bus number and the memory mapped IO. When the GPU virtual function is removed from the synthetic PCIe tree, the vacant position is filled in using an IO enabler endpoint PCIe device. The function of the IO enabler endpoint is to reserve appropriate PCIe resources (e.g., the bus number and BAR space) so that when the host server BIOS allocates resources for the synthetic PCIe tree generated by the PCIe switch box system2900, the host server BIOS can allocate the bus number and the BAR space to the IO enabler endpoint. When a GPU virtual function is added to the synthetic PCIe tree, the IO enabler endpoint can be removed from the synthetic Pete tree and be replaced by the GPU virtual function. This way, the host server operating system does not need to reboot in order for the BIOS to configure the PCI resource of the GPU virtual function. At step4012, the host server boots into the operating system.

FIG.41is a flow diagram of an example of a process4100for un-assigning/removing a network interface card (NIC) virtual function from a synthetic PCIe tree when the host operating system does not support NIC hot plug. In this case, when an NIC is unassigned from the host server or removed from the PCIe switch box system3000, the host server will not detect that the MC has been unassigned or removed. The process4100is performed to enable the host server to accurately determine the available PCIe devices. The process4100includes steps4102and4104that are performed at the PCIe switch box system3000, and steps4106to4112that are performed at the host server or workstation. At step4102, the MC virtual function is unassigned or removed from the synthetic PCIe tree. At step4104, the switch box PCIe switch2940issues a hotplug MSI TLP to the synthetic downstream port (DSP).

At step4106, the host server is powered on or rebooted. At step4108, the host server BIOS scans for available PCIe devices. At step4110, the host server BIOS finds the IO enabler endpoint, and reserves the PCIe bus number and the memory mapped IO. When the NIC virtual function is removed from the synthetic PCIe tree, the vacant position is filled in using an IO enabler endpoint PCIe device. The function of the IO enabler endpoint is to reserve appropriate PCIe resources (e.g., the bus number and BAR space) so that when the host server BIOS allocates resources for the synthetic PCIe tree generated by the PCIe switch box system3000, the host server BIOS can allocate the bus number and the BAR space to the IO enabler endpoint. When an NIC virtual function is added to the synthetic PCIe tree, the IO enabler endpoint can be removed from the synthetic PCIe tree and be replaced by the NIC virtual function. This way, the host server operating system does not need to reboot in order for the BIOS to configure the PCI resource of the NIC virtual function. At step4112, the host server boots into the operating system.

FIG.42is a flow diagram of an example of a process4200for assigning/inserting a GPU virtual function to a synthetic PCIe tree when the host operating system supports GPU hot plug. When a new GPU device is hot plugged into the PCIe switch box system2900, the host server or workstation, the GPU device is initially not hooked to the downstream port of a synthetic PCIe tree of any host port, so the host server does not detect any change in the PCIe devices. In order for the host server to detect the newly inserted GPU device, the GPU device needs to be hooked to the downstream port of the synthetic PCIe tree, and a TLP needs to be sent to the host port to notify the host server about, the changes in the PCIe devices. The process4200includes steps4202and4204that are performed at the PCIe switch box system2900, and steps4206and4208that are performed at the host server or workstation. Steps4202and4204are similar to steps3802and3804ofFIG.38. At step4206, the host server operating system calls a pre-registered PCIe hotplug interrupt service routine (ISR) and tries to add a new PCIe device (the GPU virtual function). In this example, the host server operating system uses a previously registered PCIe hotplug interrupt service routine to allocate system resources to any PCIe device (e.g., GPU virtual function) newly added to the synthetic PCIe tree. Step4208is similar to step3814ofFIG.38.

FIG.43is a flow diagram of an example of a process4300for assigning/inserting an NIC virtual function to a synthetic PCIe tree when the host operating system supports NIC hot plug. When a new network interface card is hot plugged into the PCIe switch box system2900, the host server or workstation, the NIC is initially not hooked to the downstream port of a synthetic PCIe tree of any host port, so the host server does not detect any change in the PCIe devices. In order for the host server to detect the newly inserted NIC, the NIC needs to be hooked to the downstream port of the synthetic PCIe tree, and a TIP needs to be sent to the host port to notify the host server about the changes in the PCIe devices. The process4300includes steps4302and4304that are performed at the PCIe switch box system3000, and steps4306and4308that are performed at the host server or workstation. Steps4302and4304are similar to steps3802and3804ofFIG.38. At step4306, the host server operating system calls a pre-registered PCIe hotplug interrupt service routine (ISR) and tries to add a new PCIe device (the MC virtual function). In this example, the host server operating system uses a previously registered PCIe hotplug interrupt service routine to allocate system resources to any PCIe device (e.g., NIC virtual function) newly added to the synthetic PCIe tree. Step4308is similar to step3814ofFIG.38.

FIG.44is a flow diagram of an example of a process4400for un-assigning/removing a GPU virtual function from a synthetic PCIe tree when the host operating system supports GPU hot plug. In this case, when a GPU card is unassigned from the host server or removed from the PCIe switch box system2900, the host server can detect the unassignment or removal of the GPU card without rebooting. The process4400includes steps4402and4404that are performed at the PCIe switch box system2900, and steps4406and4408that are performed at the host server or workstation. Steps4402and4404are similar to steps4002and4004ofFIG.40. At step4406, the host server operating system calls a pre-register PCIe hotplug ISR and tries to remove the offlined PCIe device (the GPU virtual function). At step4408, the host server operating system unloads the GPU driver for the virtual function, or unloads the virtual function IO driver for the GPU virtual function.

FIG.45is a flow diagram of an example of a process4500for un-assigning/removing a network interface card virtual function from a synthetic PCIe tree when the host operating system supports NIC hot plug. In this case, when an NIC is unassigned from the host server or removed from the PCIe switch box system3000, the host server can detect the unassignment or removal of the NIC without rebooting. The process4500includes steps4502and4504that are performed at the PCIe switch box system3000, and steps4506and4508that are performed at the host server or workstation. Steps4502and4504are similar to steps4102and4104ofFIG.41. At step4506, the host server operating system calls a pre-register PCIe hotplug ISR and tries to remove the offlined PCIe device (the NIC virtual function). At step4508, the host server operating system unloads the NIC driver for the virtual function, or unloads the virtual function IO driver for the MC virtual function.

FIG.46is a diagram of an example of a rack system4600that can provide an NVMe/GPU/NIC SR-IOV pooling solution for cloud servers. The rack system4600includes a plurality of compute servers4602and a PCIe switch box4604. The compute server4602can be any of several types of data processing servers, such as cloud storage server, communications network server, cloud data processing server. The computer servers can provide a variety of services, such as one or more of cloud computing services, social network data processing services, gaming services, artificial intelligence computation services, weather and climate simulation services, healthcare data processing services, financial data processing services, logistics data processing services, autonomous vehicle AI engine training services, omniverse data processing services, and metaverse data processing services.

The PCIe switch box4604can be configured and operate in a manner similar to the PCIe switch box100(FIGS.1to10),2900(FIG.29), or3000(FIG.30). The PCIe switch box4604includes various SR-IOV capable resources, such as one or more SR-IOV capable NVMe solid state drives4606, one or more SR-IOV capable network interface cards4608, and one or more SR-IOV GPUs4610. The SR-IOV capable NVMe solid state drives4606, the SR-IOV capable network interface cards4608, and the SR-IOV GPUs4610can be configured and operate in a manner similar to those described above. Each compute server4602communicates with the PCIe switch box4604through a dedicated PCIe link, which can comply with, e.g., PCIe 4.0, 5.0, 6.0, or 7.0 specification.

The workload of each compute server4602can vary depending on user demand. When the workload of a compute server4602increases, the compute server4602can request additional NVMe/GPU/NIC resources to be allocated to the virtual function assigned to the compute server4602. When the workload of a compute server4602decreases, the compute server4602can request some NVMe/GPU/NIC resources to be released from the virtual function assigned to the compute server4602.

The rack system4600has several technical advantages over conventional rack systems that uses Ethernet cables to connect the compute servers to storage servers. For example, the data bandwidth over PCIe links can be greater than the data bandwidth over Ethernet links. By using high speed PCIe links to connect the compute servers4602to the PCIe switch box4604, each compute server4602can access high speed NVMe storage, high speed graphics data processing, and high speed network connection resources as if those resources were installed locally at the compute server4602.

FIG.47is a diagram showing an example of GPU pooling architecture. In some implementations, a system4700includes host devices4702,4704,4706, and4708. The system4700includes PCIe switch boxes4710and4712. Each PCIe switch box4710,4712includes two PCIe switches. The PCIe switch box4710includes a first PCIe switch4714and a second PCIe switch4716. The first PCIe switch4714is electrically coupled to four SR-IOV capable GPUs4718. The second PCIe switch4716is electrically coupled to four SR-IOV capable GPUs4720. The PCIe switch box4712includes a first PCIe switch4722and a second PCIe switch4724. The first PCIe switch4722is electrically coupled to four SR-IOV capable GPUs4726. The second PCIe switch4724is electrically coupled to four SR-IOV capable GPUs4728.

The host device4702communicates with the first PCIe switch4714and the second PCIe switch4716of the PCIe switch box4710using a first host bus adapter4730. The host device4702communicates with the first PCIe switch4722and the second PCIe switch4724of the PCIe switch box4712using and a second host bus adapter4732. The host bus adapters4730,4732communicate with the PCIe switches4714,4716,4722,4724through signal paths4734, which can be, e.g., PCIe Gen4×8 links, such as mini SAS HD cables. This design allows the host device4702to be able to access every GPU4718,4720,4726, and4728supported by the PCIe switch boxes4710and4712.

Each of the host devices4704,4706, and4708communicates with the PCIe switches4714,4716of the PCIe switch box4710and the PCIe switches4722,4724of the PCIe switch box4712in a manner similar to the host device4702. Thus, each of the host devices4704,4706, and4708is also able to access every GPU4718,4720,4726, and4728supported by the PCIe switch boxes4710and4712. This design provides flexibility in allocating the pool of GPU resources. In some examples, all of the resources provided by the pool of GPUs can be allocated to the virtual machines assigned to a single host device. In some examples, the resources provided by the pool of GPUs can be divided in 4 ways and allocated to virtual machines that are assigned to the four host devices4702,4704,4706, and4708according to the needs to the host devices.

FIG.48is a diagram showing an example of NVMe pooling architecture. In some implementations, a system4800includes a first PCIe switch4802and a second PCIe switch4804. In this example, each PCIe switch supports 144 PCIe lanes. The first PCIe switch4802communicates with IO upstream PCIe ×8 host ports4806and24downstream SR-IOV capable PCIe ×2 NVMe solid state drives4808. Each host device connected to a corresponding host port4806can be assigned a virtual function provided by any of the SR-IOV capable NVMe SSDs4808. The second PCIe switch4804communicates with IO upstream PCIe ×8 host ports4810and24downstream SR-IOV capable PCIe ×2 NVMe solid state drives4812. Each host device connected to a corresponding host port4810can be assigned a virtual function provided by any of the SR-IOV capable NVMe SSDs4812.

In some implementations, the PCIe switch4802communicates with the PCIe switch4804through a PCIe fabric link4814. Each host device connected to a corresponding host port4806can be assigned a virtual function provided by any of the SR-IOV capable NVMe SSDs4812. The signal path from the host device connected to the host port4806to the NVMe SSD4812passes through the PCIe switch4802, the PCIe fabric link4814, and the PCIe switch4804. Similarly, each host device connected to a corresponding host port4810can be assigned a virtual function provided by any of the SR-IOV capable NVMe SSDs4808. The signal path from the host device connected to the host port4810to the NVMe SSD4808passes through the PCIe switch4804, the PCIe fabric link4814, and the PCIe switch4802.

The design shown inFIG.48allows each of the 20 host devices connected to the PCIe×8 host ports4806,4810to be able to access the virtual functions provided by any of the 48 SR-IOV capable PCIe×2 NVMe SSDs4808,4812. In some examples, the resources provided by the entire pool of NVMe SSDs4808,4812can be allocated to virtual functions that are assigned to a single host device connected to one of the host ports4806,4810. In some examples, the resources provided by the pool of NVMe SSDs can be divided in 20 ways and allocated to virtual machines that are assigned to the 20 host devices connected to the host ports4806,4810according to the needs to the host devices.

FIG.49is a diagram showing another example of NVMe pooling architecture. In some implementations, a system4900includes a first PCIe switch4802, a second PCIe switch4804, a group of 24 SR-IOV capable PCIe ×2 NVMe solid state drives4808, and a group of 24 SR-IOV capable PCIe ×2 NVMe solid state drives4812, all housed within a JBOD (just a bunch of disks) enclosure4816. In this example, the 48 NVMe SSDs, each using 2 PCIe lanes, are configured as JBOD and appear as a single large volume disk drive. Any portion of the JBOD drive can be allocated to a virtual function and be assigned to one of the host devices4902,4904. This provides an efficient use of the NVMe resources.

For example, the PCIe devices and PCIe links described in this document can comply with the PCI Express® 3.0, 4.0, 5.0, 6.0, and/or 7.0 Base Specifications, which are incorporated by reference. For example, the SR-IOV specification, such as the “Single Root I/O Virtualization and Sharing Specification” (including several revisions), which are incorporated by reference, can be accessed through the PCI-SIG web site “https://pcisig.com/specifications/iov/single_root”. In some implementations, the PCIe switch box system100can include two or more PCIe switch controllers to allow more host devices to access more SR-IOV capable PCIe devices. The PCIe links between the PCIe switch box system100and the host devices (e.g.,102,104,106ofFIG.1) can have lengths of, e.g., a few feet, tens of feet, or hundreds of feet. For example, the PCIe switch box system100and the host servers can be rackmount devices, the PCIe switch box system100and the host servers can be mounted on a same rack, in which the PCIe switch box system100host PCIe resources (e.g., GPU, NVMe storage) for the host servers in the rack.

In some implementations, the switch box system can work with devices that comply with standards other than the PCI Express and NVM Express standards, e.g., new standards that enhance or replace the PCIe and NVMe standards. For example, the switch box system can comply with, and work with SR-IOV devices and other components that comply with Compute Express Link (CXL). The CXL™ 3.0 Specification is available from the CXL™ Consortium (https://www.computeexpresslink.org) and is herein incorporated by reference.

In some implementations, the software modules and drivers for implementing the PCIe switch box system and the management computer can be provided on computer-readable media (e.g., RAM, ROM, SDRAM, hard disk, optical disk, and flash memory). The term “computer-readable medium” refers to a medium that participates in providing instructions to a processor for execution, including without limitation, non-volatile media (e.g., optical or magnetic disks), and volatile media (e.g., memory) and transmission media. Transmission media includes, without limitation, coaxial cables, copper wire, fiber optics and free space.

The features described above can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., C, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, a browser-based web application, or other unit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructions include, e.g., general purpose microprocessors, special purpose microprocessors, digital signal processors, single-core or multi-core processors, of any kind of computer. Generally, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memories for storing instructions and data. Generally, a computer will also include, or be operatively coupled to communicate with, one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM, DVD-ROM, and Blu-ray BD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits).

Particular embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or in sequential order, to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous.

Although some examples have been discussed above, other implementations and applications are also within the scope of the following claims. For example, the hosts (e.g.,160,162) can be different from what is described above.

In some implementations, the NVMe solid state drives can be replaced with other types of solid state drives. The non-volatile memory used in the solid state drives can be based on various types of technology, including e.g., single-level cell flash memory, triple-level cell flash memory, and/or multi-level cell flash memory.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments.