Patent Description:
For example, a peripheral proxy subsystem is placed between multiple hosts, each having a root controller, and single root I/O virtualization (SR-IOV) peripheral devices that are to be shared. The peripheral proxy subsystem provides a root controller for coupling to the endpoint of the SR-IOV peripheral device or devices and multiple endpoints for coupling to the root controllers of the hosts. The peripheral proxy subsystem maps the virtual functions of an SR-IOV peripheral device to the multiple endpoints as desired to allow the virtual functions to be allocated to the hosts. The physical function of the SR-IOV peripheral device is managed by the peripheral proxy device to enable the desired number of virtual functions. The peripheral proxy subsystem can also map individual physical functions of a multi-function peripheral device to the multiple endpoints in the peripheral proxy subsystem as desired to allow the individual functions to be allocated to the hosts. The virtual functions of the SR-IOV peripheral device or individual physical function of a multi-function peripheral device are then presented to the appropriate host as a physical function or a virtual function.

This allows an SR-IOV peripheral device or a multi-function peripheral device to function as a multiple root I/O virtualization (MR-IOV) peripheral device so that a multi-root domain can be developed without the need for hard to find MR-IOV peripheral devices and multi-root aware (MRA) switches.

Referring now to <FIG>, a computer system <NUM> is illustrated that is one example of a configuration with a host <NUM><NUM> and a host <NUM><NUM> connected to a PCIe switch <NUM>, which in turn has a PCIe NVM Express™ (NVMe™) memory <NUM> and a PCIe <NUM> Ethernet network controller <NUM> connected to the PCIe switch <NUM>. This configuration ideally allows host <NUM><NUM> and host <NUM><NUM> to access the NVMe memory <NUM> for shared memory storage and utilize the <NUM> Ethernet network controller <NUM> to access external systems at high speed. In practice, however, such a configuration is not as simple as it appears in <FIG>. Problems develop because of the ability of PCIe devices to be shared by multiple root controllers. A host, such as host <NUM><NUM> and host <NUM><NUM>, has a PCIe root controller to act as the top of the PCIe tree or network. PCIe provides two methods of sharing peripheral devices. The first is referred to as single root I/O virtualization (SR-IOV). The second is referred to as multiple root I/O virtualization (MR-IOV) and devices are multiple root aware (MRA). SR-IOV allows multiple virtual machines and the host operating system or hypervisor contained on a physical processor chip that operate through a single PCIe root controller to access the shared peripheral device. The shared peripheral device presents a physical function and multiple virtual function register sets at the PCIe interface. Each of the virtual machines or cores is assigned a virtual function or multiple virtual functions, while the physical function is assigned to the operating system or hypervisor. However, SR-IOV is not useful when two hosts, each containing a PCIe root controller, must share the same PCIe peripheral device. For multiple hosts with multiple root controllers to share a PCIe peripheral device, an MR-IOV peripheral device may be used, provided any intervening PCIe switches are also compliant with MR-IOV. While there are numerous SR-IOV enabled peripheral devices, there are very few MR-IOV enabled peripheral devices and very few MR-IOV switches. This is because of the added complexity that is required in each of the device peripheral devices and the switches. To be MR-IOV compliant, an MRA PCIe switch must be aware of the address spaces used by the hosts to properly route transactions. MR-IOV compliant peripheral devices may have more complicated interfaces to provide a basic function and to present multiple virtual endpoints, each virtual endpoint having both a physical function and virtual functions. These additional requirements have limited the availability of MR-IOV compliant switches and peripheral devices.

<FIG> is an example of a computer system <NUM> which contains host <NUM><NUM> and host <NUM><NUM> connected to an MRA PCIe switch <NUM>, which is MR-IOV compliant. An SR-IOV peripheral device <NUM> is connected to the MRA PCIe switch <NUM>. This SR-IOV peripheral device <NUM> can only be utilized by one of the two hosts <NUM>, <NUM>. An MR-IOV peripheral device <NUM> is connected to the MRA PCIe switch <NUM> and can be utilized by both hosts <NUM>, <NUM>. The SR-IOV peripheral device <NUM> presents a physical function (PF) zero (PF0) <NUM> and two virtual functions (VFs), VF1 <NUM> and VF2 <NUM> at the endpoint (EP) PCIe interface EP0 <NUM>. In contrast, the MR-IOV peripheral device <NUM> presents a basic function (BF) <NUM> used to manage the multiple root features of the MR-IOV peripheral device <NUM> and two virtual endpoints (VEP), VEP2 <NUM> and VEP3 <NUM> at EP1 <NUM>. VEP2 <NUM> presents PF0 <NUM>, while VEP3 <NUM> presents PF1 <NUM> and VF1 <NUM>. In the illustrated embodiment example host <NUM><NUM> has control of the SR-IOV peripheral device <NUM> and has illustrated module <NUM> to interact with PF0 <NUM>, module <NUM> to interact with VF1 <NUM> and module <NUM> to interact with VF2 <NUM>. Host <NUM><NUM> also has control of VEP2 <NUM> and has a module <NUM> to interact with PF0 <NUM>. Host <NUM><NUM> has control of VEP3 <NUM> and has modules <NUM> to interact with PF <NUM><NUM> and <NUM> to interact with VF1 <NUM>. The MRA PCIe switch <NUM> includes a multi-root PCI manager (MR-PCIM) <NUM>, which is responsible for discovering and configuring the virtual hierarchies within the multi-root topology, in the indicated example, just the MR-IOV peripheral device <NUM>. The MRA PCIe switch <NUM> manages the multi-root aspects, so that the host <NUM><NUM> and host <NUM><NUM> just see SR-IOV, multi-function or single function peripheral devices.

<FIG> illustrates a computer system <NUM> which provides for multiple hosts with individual root controllers to share an SR-IOV peripheral device. Host <NUM><NUM> and host <NUM><NUM> are connected to a peripheral proxy subsystem <NUM>. The peripheral proxy subsystem <NUM> is connected to an SR-IOV peripheral device <NUM>. The peripheral proxy subsystem <NUM> in <FIG> provides an PCIe endpoint interface EP2 <NUM> to cooperate with a root controller (RC) interface RC1 <NUM> of host <NUM><NUM>. The peripheral proxy subsystem <NUM> also provides an PCIe endpoint interface EP3 <NUM> to cooperate with the root controller PCIe interface RC2 <NUM> of host <NUM><NUM>. The peripheral proxy subsystem <NUM> provides a root controller PCIe interface RC3 <NUM> which cooperates with an PCIe endpoint interface EP0 <NUM> on the SR-IOV peripheral device <NUM>, which includes PF0 <NUM>, VF1 <NUM> and VF2 <NUM>. These are controlled by RC3 <NUM> in the peripheral proxy subsystem <NUM>. The peripheral proxy subsystem <NUM> has modules <NUM> for interacting with the PF0 <NUM>, <NUM> for interacting with VF1 <NUM> and <NUM> for interacting with VF2 <NUM>. The peripheral proxy subsystem <NUM> presents a first cloned instance of PF0 <NUM> designated PFO' <NUM> at EP2 <NUM> coupled to RC1 <NUM>. In the example of <FIG>, PFO' <NUM> is VF1 <NUM> of the peripheral device <NUM> presented by the peripheral proxy subsystem <NUM> as a PCIe endpoint physical function. Similarly, EP3 <NUM> presents a second cloned instance of PF0 <NUM> designated PF0" <NUM>, which is VF2 <NUM> of the peripheral device <NUM> in the example of <FIG>. In this way, the virtual functions of the peripheral device <NUM> can be arbitrarily divided among the cloned instances, PFO' <NUM> and PF0" <NUM>, and each of the cloned instances of the physical functions present an independent subset of the virtual functions.

The peripheral proxy subsystem <NUM> includes various manager modules. A policy manager <NUM> stores the configuration allocation information for the particular physical and virtual functions of the attached peripheral devices to the various attached hosts. A configuration manager <NUM> includes a physical function manager <NUM> which manages configuration of PF0 <NUM> of the peripheral device <NUM>. An endpoint manager <NUM> is contained in the configuration manager <NUM> and performs the remainder of the endpoint management functions to allow the hosts to identify the shared peripheral device and to be interrupted by the shared peripheral device. A map and forward module <NUM> manages the memory mapping contained in the peripheral proxy subsystem <NUM> and the forwarding routes for data.

<FIG> illustrates an example of a more complex multi-root PCIe system <NUM>. A host <NUM><NUM> and a host <NUM><NUM> are connected to an MRA PCIe switch <NUM>. An MR-IOV peripheral device <NUM> is connected to the MRA PCIe switch <NUM> and presents two virtual endpoints VEP2 <NUM> and VEP3 <NUM>. This combination creates a multi-root domain <NUM>. A non-MRA PCIe switch <NUM> is connected to the MRA PCIe switch <NUM> and has connected to it an SR-IOV peripheral device <NUM> and a single function peripheral device <NUM>. The MR-IOV peripheral device <NUM> is shared between the host <NUM><NUM> and host <NUM><NUM>, whereas the SR-IOV peripheral device <NUM> and the single function peripheral device <NUM> are assigned to either host <NUM><NUM> or host <NUM><NUM> as they are not multi-root aware.

<FIG> is an example of a more complex system <NUM> including a peripheral proxy subsystem. Host <NUM><NUM> is connected to a first PCIe switch <NUM>, the PCIe switch <NUM> not being multi-root aware. Host <NUM><NUM> is connected to a second PCIe switch <NUM>. PCIe switch <NUM> is connected to an PCIe endpoint interface EP2 <NUM> of a peripheral proxy subsystem <NUM>, while the PCIe switch <NUM> is connected to an PCIe endpoint interface EP3 <NUM>. Root controller interface RC3 <NUM> of the peripheral proxy subsystem <NUM> is connected to a PCIe switch <NUM>. A first SR-IOV peripheral device <NUM> is connected to the PCIe switch <NUM>. A second SR-IOV peripheral device <NUM> is also connected to the PCIe switch <NUM>. This collection of devices together forms a multi-root domain <NUM> in the system <NUM>. Contrasting it with the multi-root domain <NUM> of <FIG>, it is noted that none of the host, switch, or peripheral devices in the multi-root domain <NUM> are multi-root aware devices. However, peripheral proxy subsystem <NUM> allows more commonly available SR-IOV peripheral devices and normal PCIe switches are used, and multi-root operation is still provided. A single function peripheral device <NUM> is connected to PCIe switch <NUM>, while a single function peripheral device <NUM> is connected to PCIe switch <NUM> and a single function peripheral device <NUM> is connected to PCIe switch <NUM>. In this configuration the virtual functions of the SR-IOV peripheral devices <NUM>, <NUM> can be shared between the hosts <NUM> and <NUM>. The single function peripheral device <NUM> can be utilized by either host <NUM><NUM> or host <NUM><NUM>. Single function peripheral device <NUM> is dedicated to host <NUM><NUM>, while single function peripheral device <NUM> is dedicated to host <NUM><NUM>. With the use of the peripheral proxy subsystem <NUM>, a multi-root domain <NUM> is developed using the more commonly available SR-IOV devices and non-multi-root aware switches and without the use of multi-root aware devices.

Referring now to <FIG>, one example of initialization of the peripheral proxy subsystem <NUM> of <FIG> is illustrated. In operation <NUM>, hardware link negotiation between RC3 <NUM> and EP0 <NUM> is performed. In operation <NUM>, a PCIe software stack that is present in the peripheral proxy subsystem <NUM> enumerates PF0 <NUM>. In operation <NUM>, a PCIe driver in the physical function manager <NUM> is bound to PF0 <NUM> so that the SR-IOV peripheral device <NUM> can be configured. In operation <NUM>, the policy manager <NUM> determines the number of virtual functions in SR-IOV peripheral device <NUM> to enable. In operation <NUM>, the physical function manager <NUM> writes this number of virtual functions into the peripheral device registers of PF0 <NUM>. In operation <NUM>, the PCI software stack enumerates the virtual functions of EP0 <NUM>. In operation <NUM>, a PCIe endpoint function driver in the endpoint manager <NUM> is bound to EP2 <NUM> and EP3 <NUM> to allow initialization of the endpoint instances connected to the host <NUM><NUM> and host <NUM><NUM> with the appropriate virtual function configuration space data. In operation <NUM>, the policy manager <NUM> determines the endpoint instances that are connected to hosts and the virtual functions that each endpoint instance should be initialized with. In some examples, a physical function also provides a function beyond just management, and in those cases the physical function may also be mapped to an endpoint instance. In yet other examples, as described below, a peripheral device is directly assigned to an endpoint instance. In operation <NUM>, the endpoint manager <NUM> initializes the endpoint instances with the virtual function configuration space data, and/or the relevant portions of the physical function configuration space used to provide the functions of a physical function. For example, if VF1 <NUM> is to be mapped to EP2 <NUM>, the values in the VF1 <NUM> configuration space data are copied to EP2 <NUM> to present as the configuration space data for the appropriate cloned function, such as PFO' <NUM> or a virtual function. Mapping of basic physical function data is described below. In operation <NUM>, the map and forward module <NUM> configures various translation lookaside buffers (TLBs) and transaction forwarders in the peripheral proxy subsystem <NUM> to perform translation of memory addresses in memory transactions and to forward those memory transactions from the virtual functions in the SR-IOV peripheral device <NUM> to either EP2 <NUM> or EP3 <NUM>. In operation <NUM>, hardware link negotiation is performed between RC1 <NUM> and EP2 <NUM> and between RC2 <NUM> and EP3 <NUM>. In operation <NUM>, normal software execution begins in RC1 <NUM> and RC2 <NUM>. With this, the peripheral proxy subsystem <NUM> is initialized and ready for operation.

A routing mechanism in the peripheral proxy subsystem <NUM> provides a way for transactions to be forwarded from virtual functions VF1 <NUM> and VF2 <NUM> in SR-IOV peripheral device <NUM> to host1 <NUM> and host2 <NUM>, respectively, in one direction and from host1 <NUM> and host2 <NUM> to VF1 <NUM> and VF2 <NUM>, respectively, in SR-IOV peripheral device <NUM> in the other direction. The peripheral proxy subsystem provides peripheral virtualization unit (PVU) (<NUM>), virtual ID map logic <NUM>, and outbound address translation units <NUM> and <NUM> in the routing mechanism for transactions to be forwarded from virtual function VF1 <NUM> and VF2 <NUM> in SR-IOV peripheral device <NUM> to host1 <NUM> and host2 <NUM>, respectively. The peripheral proxy subsystem provides base address registers, inbound and outbound address translation units (described below) in the routing mechanism for transactions to be forwarded from host1 <NUM> and host2 <NUM> to VF1322 and VF2 <NUM>, respectively, in the SR-IOV peripheral device <NUM>.

<FIG> illustrates the register configuration performed by the endpoint manager <NUM>. The registers contained in the cloned instances PFO' <NUM> and PF0" <NUM> of the peripheral proxy subsystem and the corresponding PF0 <NUM> of the device to be shared are illustrated. The endpoint manager <NUM> initializes the configuration space of the endpoint instances EP2 <NUM> and EP3 <NUM>. The endpoint manager <NUM> initializes device identification capabilities in PFO' <NUM> and PF0" <NUM> with values from the standard configuration space headers and SR-IOV capability in the extended configuration space of EP0 <NUM>. The endpoint manager <NUM> need not initialize any other capabilities present in PFO' <NUM> and PF0" <NUM> that depend on the capabilities of the PF0 <NUM> in the EP0 <NUM> except for MSI capabilities, as that operation is performed in step <NUM> if necessary. As to the MSI capabilities, certain fields, such as the multiple message capable field, which indicates the number of desired interrupt vectors, are initialized based on the respective physical or virtual function of the SR-IOV peripheral device <NUM> being mapped. If the physical function being presented to the root controller is mapped from a virtual function, then the VF configuration space data will have been copied to the PF configuration space being presented in step <NUM>.

<FIG> illustrates PCI memory transfers from the SR-IOV peripheral device <NUM> to host <NUM><NUM> and host <NUM><NUM>. When VF1 <NUM> and VF2 <NUM> are enumerated, internal addresses are reserved or assigned. The PCIe driver of host <NUM><NUM> includes a memory buffer MEM(A) <NUM> and the PCIe driver of host <NUM><NUM> includes a memory buffer MEM(B) <NUM>. The PCIe drivers in host <NUM><NUM> and host <NUM><NUM> initialize the buffer addresses of their memory buffers MEM(A) <NUM> and MEM(B) <NUM> in the respective buffer address register <NUM> of VF1 <NUM> and buffer address register <NUM> of VF2 <NUM>. Thus, VF1 <NUM> includes access to memory buffer MEM(A) <NUM> while VF2 <NUM> includes access to memory buffer MEM(B) <NUM>. With the host addresses now known, entries in TLB <NUM> and TLB <NUM> can be programmed to translate from the host addresses to the reserved internal addresses and the destination endpoint. Similarly, outbound address translation units <NUM> and <NUM> are programmed to translate from the internal addresses back into host addresses. The above steps are initialization steps and do not have to be performed for each transaction. VF1 <NUM> and VF2 <NUM> each have a requester ID developed during initialization operations. The requester ID (ReqID) is used in the transactions to identify the particular PF or VF. When a PF or VF provides a memory transaction, the memory transaction includes a requester ID and the host memory address. For example, a memory transaction from VF1 <NUM> will include a requester ID of 01x01 and address in the buffer memory MEM(A) <NUM>. The peripheral proxy subsystem <NUM> includes virtual ID map logic <NUM>. The virtual ID map logic <NUM> converts a requester ID in a transaction from a VF into a virtual ID (virtID) and forwards the transaction to the PVU <NUM>. The PVU <NUM> includes the TLBs <NUM> and <NUM>, one per virtID or VF. The entries in the TLBs <NUM> and <NUM> convert the memory addresses in a transaction to an internal address of the peripheral proxy subsystem <NUM> and provide the transaction to a designated endpoint. The selection of a TLB based on the virtID and the provision of the converted transaction to a designated endpoint act together to route the transaction to the proper endpoint. TLB <NUM> contains the translation lookaside buffer entries for virtID0, which corresponds to VF1 <NUM>, while TLB <NUM> contains the translation lookaside buffer entries for virtID1, which corresponds to VF2 <NUM>. When the PVU <NUM> receives a transaction from the virtual ID map logic <NUM>, the virtID is used to select the appropriate TLB <NUM> or <NUM>. The TLB <NUM>, <NUM> translates the host address in the transaction to a peripheral proxy subsystem <NUM> internal outbound address for the relevant endpoint EP2 <NUM> or EP3 <NUM>. The transaction with the now converted address is provided to a respective outbound address translation unit (ATU) <NUM> in PFO' <NUM> or outbound ATU <NUM> in PF0" <NUM>. The outbound ATU <NUM> in PFO' <NUM> or outbound ATU <NUM> in PF0" <NUM> translates the internal outbound address back into the appropriate host address for provision of the transaction to the particular host and provides an appropriate ReqID for the particular function based on the internal address. After this final transaction address translation, the transaction is provided from EP2 <NUM> to RC1 <NUM> for host <NUM><NUM> or from EP3 <NUM> to RC2 <NUM> for host <NUM><NUM>.

To summarize, for a memory operation from the peripheral device <NUM> to host <NUM><NUM> or host <NUM><NUM>, the peripheral device <NUM> provides a transaction with a ReqID of the appropriate function, such as PF0 <NUM>, VF1 <NUM> or VF2 <NUM>, and a host memory address. The ReqID is converted to a virtID in the virtual ID map logic <NUM>. A TLB in the PVU <NUM> translates the host memory address into an internal outbound memory address. An outbound ATU translates the internal outbound memory address back to the host memory address and the transaction is provided to the host. In this way, each of the cloned physical functions PFO' <NUM> and PF0" <NUM> appears, to its respective host, to be a function of a separate and independent SR-IOV peripheral device assigned only to the respective host.

<FIG> illustrates memory transfer operations from a host to the peripheral device. At initialization, each virtual function maps its base address registers (BARs) to internal memory addresses using an inbound ATU. VF1 <NUM> has BARs <NUM> which map to peripheral device <NUM> internal memory locations Access(A) <NUM> using inbound ATU <NUM>. VF2 <NUM> has BARs <NUM> which map to internal memory locations Access(B) <NUM> using the inbound ATU <NUM>. Next, during root controller enumeration of the virtual functions, memory is allocated from the outbound address space and initialized in the virtual function. RC3 <NUM> allocates memory block <NUM> for the VF1 BARs <NUM> in the outbound address space <NUM>. RC3 <NUM> allocates memory block <NUM> for the VF2 BARs <NUM> in the outbound address space <NUM>. RC3 <NUM> sets the memory block <NUM> to access the BARs <NUM> and memory block <NUM> to access the BARs <NUM>. Next, EP2 <NUM> uses inbound ATU <NUM> and EP3 <NUM> uses inbound ATU <NUM> to map their base address registers <NUM>, <NUM> to outbound address blocks <NUM>, <NUM>. Following this, the host <NUM><NUM> and host <NUM> allocate MEM(C) <NUM> and MEM(D) <NUM>, respectively, the host outgoing memory buffers, and initialize those addresses in the PFO' <NUM> BARs <NUM> and PF0" <NUM> BARs <NUM>. This completes initialization of the peripheral proxy subsystem <NUM> and the peripheral device <NUM> to receive a memory write transaction.

To perform a write operation, host <NUM><NUM> uses RC1 <NUM> and provides a memory transaction to EP2 <NUM> using the addresses mapped in MEM(C) <NUM>. EP2 <NUM> obtains the transaction and translates the host <NUM><NUM> address into internal addresses in the memory block <NUM> allocated for VF1 <NUM> and provides the transaction to RC <NUM>. RC3 <NUM> provides the transaction to EP0 <NUM>. The inbound ATU <NUM> translates the addresses according to BARs <NUM> and provides the transaction to the designated memory location Access(A) <NUM>. Similarly, host <NUM><NUM> uses RC2 <NUM> and provides a memory transaction to EP3 <NUM> using the addresses mapped in MEM(D) <NUM>. EP3 <NUM> obtains the transaction and translates the host <NUM><NUM> address into internal addresses in the memory block <NUM> allocated for VF2 <NUM> and provides the transaction to RC3 <NUM>. RC3 <NUM> provides the transaction to EP0 <NUM>. The inbound ATU <NUM> translates the addresses according to BARs <NUM> and provides the transaction to the designated memory locations Access(B) <NUM>. The mapping of the internal addresses to the BARs of the virtual functions of the peripheral device and the mapping of the inbound ATUs to those internal addresses acts to route a host to peripheral device transaction to the proper physical or virtual function in the peripheral device.

PCIe uses message signaled interrupts (MSIs). With MSI, interrupt messages are provided to host memory locations. <FIG> illustrates one example of MSI operations. Host <NUM><NUM> has a memory block MSI(P) <NUM> reserved for MSI use and host <NUM><NUM> has a memory block MSI(Q) <NUM> reserved for MSI use. EP2 <NUM> includes an outbound MSI memory block (X) <NUM>, while EP3 <NUM> includes an outbound MSI memory block (Y) <NUM>. The addresses of the outbound MSI memory blocks (X) <NUM> and (Y) <NUM> are programmed into the MSI registers of VF1 <NUM> and VF2 <NUM>, respectively. The host <NUM><NUM> programs the address of memory block MSI(P) <NUM> into the MSI registers of PFO' <NUM>, while the host <NUM><NUM> programs the address of memory block MSI(Q) <NUM> into the MSI registers of PF0" <NUM>. This completes the initialization operations. When VF1 <NUM> or VF2 <NUM> desires to send an MSI transaction, the virtual function provides a transaction with its requester ID and the address reserved in the outbound MSI memory block (X) <NUM> or (Y) <NUM>, respectively. The virtual ID map logic <NUM> maps the requester ID to a virtual ID and provides the transaction to the PVU <NUM>. The PVU <NUM> translates the MSI address to an internal outbound address, if necessary and routes the MSI transaction. The outbound ATU <NUM>, <NUM> translates the internal outbound MSI address to the host memory block MSI(P) <NUM> or MSI(Q) <NUM> and EP2 <NUM> or EP3 <NUM> provides the transaction to the host <NUM><NUM> or host <NUM><NUM>.

<FIG> illustrates the flexibility of the mapping of physical functions and virtual functions to the endpoints of the peripheral proxy subsystem <NUM> and thus to the hosts. In a first example, the peripheral proxy subsystem <NUM> manages PF0 <NUM>, VF1 <NUM> is mapped to PFO' <NUM>, VF2 <NUM> is mapped to PF0" and VF3 <NUM> is mapped to VF1 <NUM> of PCIe endpoint <NUM>. This provides MR-IOV operation of the SR-IOV peripheral device <NUM>. In a second example, PF0 <NUM> is managed by the peripheral proxy subsystem <NUM>, with VF1 <NUM> mapped as PFO', VF2 <NUM> mapped as PF1 <NUM> and VF3 <NUM> mapped as PF2 (not shown). This maps all virtual functions to host <NUM><NUM> as physical functions, so that peripheral device <NUM> appears to be a multifunction PCIe endpoint. In a third example, PF0 <NUM> is managed by the peripheral proxy subsystem <NUM>, VF1 <NUM> is mapped to PFO' <NUM>, VF2 <NUM> is mapped to VF1 <NUM> and VF3 <NUM> is mapped to VF2 <NUM>. This allows different virtual machines in host <NUM><NUM> to utilize the three virtual functions as normal in SR-IOV operation. In a fourth example, the SR-IOV peripheral device <NUM> is directly assigned to EP2 <NUM>, so that PF0 <NUM> maps to PFO' <NUM>, VF1 <NUM> maps to VF1 <NUM>, VF2 <NUM> maps to VF2 <NUM> and VF3 <NUM> maps to VF3 (not shown) of PCIe endpoint <NUM>. This flexibility in mapping is accomplished by the routing performed by the PVU <NUM> and the TLBs for peripheral device memory transfer and MSI operations and the internal memory space allocations and BAR values for host memory transfer operations in conjunction with the copying of PF and VF configuration space data to present the desired functions. These are just four examples of the flexibility of mapping physical functions and virtual functions using the peripheral proxy subsystem <NUM> and other mappings can readily be performed.

<FIG> is a hardware block diagram of a peripheral proxy subsystem <NUM>, such as that of <FIG>. Processors <NUM>, such as ARM R5F cores, provide the basic processing function in the peripheral proxy subsystem <NUM>. SRAM <NUM> is provided as operating memory and to act as the memory blocks used by the various devices internal to the peripheral proxy subsystem <NUM>. Various I/O ports, such as SPI <NUM>, UART <NUM> and I2C <NUM> are present to allow external communication with and management by other devices in a system. Alternatively, management of the peripheral proxy subsystem <NUM> could be performed through extended capabilities of a PF presented at one of the endpoints. A nonvolatile (NV) RAM interface <NUM> is connected to a non-transitory NVRAM <NUM> which contains the firmware modules used to provide the functions of the peripheral proxy subsystem <NUM>. For example, the policy manager <NUM>, the configuration manager <NUM> with its physical function manager <NUM> and endpoint manager <NUM>, and the map and forward module <NUM> are contained in NVRAM <NUM> to be executed by the processors <NUM>. An operating system and/or hypervisor <NUM> are stored in the NVRAM <NUM>, as is the PCIe software stack <NUM> mentioned previously.

A first PCIe endpoint <NUM> acts as an endpoint such as EP2 <NUM> and includes the outbound ATU <NUM>, outbound MSI memory <NUM>, the MSI capability register <NUM>, inbound ATU <NUM> and the base address registers <NUM>. A second PCIe endpoint <NUM> is provided to act as a second endpoint, such as EP3 <NUM>, and includes outbound ATU <NUM>, outbound MSI memory <NUM>, MSI capability registers <NUM>, inbound ATU <NUM> and the base address registers <NUM>.

A PCIe root controller <NUM> is present to act as the root controller for SR-IOV peripheral devices to be shared between hosts connected to PCIe endpoints <NUM>, <NUM>. An example of the PCIe root controller <NUM> is RC3 <NUM>. The PCIe root controller <NUM> includes outbound address space <NUM>, which includes the various virtual function base address register spaces. The virtual ID map logic <NUM> is provided, as is the PVU <NUM> with the various virtual ID TLBs <NUM> and <NUM> and others as needed. More endpoints and root controllers can be present on the peripheral proxy subsystem <NUM> than the two endpoints and one root controller illustrated in the various Figures. Each root controller and endpoint will operate as described above.

Multiple SR-IOV peripheral devices can be connected to the root controller of the peripheral proxy subsystem <NUM> and each is mapped and managed as described above.

<FIG> is a block diagram of an exemplary SoC <NUM> that can form the hosts <NUM> and <NUM><NUM>, <NUM>. A series of more powerful microprocessors <NUM>, such as ARM® A72 or A53 cores, form the primary general-purpose processing block of the SoC <NUM>, while a digital signal processor (DSP) <NUM> provides specialized computing capabilities. Simpler microprocessors <NUM>, such as ARM R5F cores, provide general control capability in the SoC <NUM>. A high-speed interconnect <NUM> connects the microprocessors <NUM>, DSP <NUM> and microprocessors <NUM> to various other components in the SoC <NUM>. For example, a shared memory controller <NUM>, which includes onboard memory or RAM <NUM>, is connected to the high-speed interconnect <NUM> to act as the onboard RAM for the SoC <NUM>. A DDR memory controller system <NUM> is connected to the high-speed interconnect <NUM> and acts as an external memory interface to external DRAM memory. A video acceleration module <NUM> and a radar processing accelerator (PAC) module <NUM> are similarly connected to the high-speed interconnect <NUM>. A vision processing accelerator module <NUM> is connected to the high-speed interconnect <NUM>, as is a depth and motion PAC module <NUM>. A graphics acceleration module <NUM> is connected to the high-speed interconnect <NUM>. A display subsystem <NUM> is connected to the high-speed interconnect <NUM> and includes conversion logic <NUM> and output logic <NUM> to allow operation with and connection to various video monitors if appropriate. A system services block <NUM>, which includes items such as DMA controllers, memory management units, general-purpose I/O's, mailboxes and the like, is provided for normal SoC <NUM> operation. A serial connectivity module <NUM> is connected to the high-speed interconnect <NUM> and includes modules as normal in an SoC. A vehicle connectivity module <NUM> provides interconnects for external communication interfaces, such as PCIe block <NUM>, USB block <NUM> and an Ethernet switch <NUM>. A capture/MIPI module <NUM> includes a four-lane CSI-<NUM> compliant transmit block <NUM> and a four-lane CSI-<NUM> receive module and hub. Further details on the CSI-<NUM> receive module and hub are provided below.

An MCU island <NUM> is provided as a secondary subsystem and handles operation of the integrated SoC <NUM> when the other components are powered down to save energy. A processor <NUM>, such as one or more ARM R5F cores, operates as a master and is coupled to the high-speed interconnect <NUM> through an isolation interface <NUM>. An MCU general purpose I/O (GPIO) block <NUM> operates as a slave. MCU RAM <NUM> is provided to act as local memory for the MCU ARM processor <NUM>. A CAN bus block <NUM>, an additional external communication interface, is connected to allow operation with a conventional CAN bus environment in a vehicle. An Ethernet MAC (media access control) block <NUM> is provided for further connectivity in the vehicle. Nonvolatile memory (NVM) is connected to the MCU ARM processor <NUM> through an external NVRAM interface <NUM>.

This is an example host configuration and many other host configurations can be utilized.

While the use of virtID map logic to translate ReqIDs to virtIDs and TLBs to translate host memory addresses to internal memory addresses are described in the examples above, in other examples ReqIDs are used to route transactions from the peripheral device to the hosts and the virtID map logic is not needed. In yet other examples, an internal memory space is not used and the TLBs are not needed.

In the example described above, a TLB is provided for each VF in the peripheral device <NUM>. In another example, only a single TLB is provided, but that TLB is extended to include the ReqID or virtID as lookup values and to provide the desired endpoint as an output value.

Claim 1:
A peripheral proxy subsystem (<NUM>) comprising:
a subsystem Peripheral Component Interconnect Express, PCIe, root controller (<NUM>) for coupling to a peripheral device Peripheral Component Interconnect Express, PCIe, endpoint (<NUM>);
a first subsystem PCIe endpoint (<NUM>) for coupling to a first PCIe root controller (<NUM>) of a first host (<NUM>), the first subsystem PCIe endpoint (<NUM>) is configured to present a first physical function (<NUM>) to the first PCIe root controller (<NUM>), the first physical function (<NUM>) being mapped to a first peripheral device virtual function (<NUM>);
a second subsystem PCIe endpoint (<NUM>) for coupling to a second PCIe root controller (<NUM>) of a second host (<NUM>), the second subsystem PCIe endpoint (<NUM>) is configured to present a second physical function (<NUM>) to the second PCIe root controller (<NUM>), the second physical function (<NUM>) being mapped to a second peripheral device virtual function (<NUM>);
a routing mechanism for routing PCIe memory transactions that are directed to a virtual function (<NUM>, <NUM>) configured to be selected from among the first peripheral device virtual function (<NUM>) and the second peripheral device virtual function (<NUM>) between the subsystem PCIe root controller (<NUM>) and a corresponding endpoint (<NUM>) from among the first subsystem PCIe endpoint (<NUM>) and the second subsystem PCIe endpoint (<NUM>) based on whether the virtual function is mapped to the first physical function (<NUM>) or the second physical function (<NUM>), and
at least a first module (<NUM>) and a second module (<NUM>), the first module (<NUM>) configured to interact with the first peripheral device virtual function (<NUM>) and the second module (<NUM>) configured to interact with the second peripheral device virtual function (<NUM>), the peripheral proxy subsystem (<NUM>) configured to present a first cloned instance of the first physical function (<NUM>) designated as instance PFO' (<NUM>) at first subsystem PCIe endpoint (<NUM>) coupled to first PCIe root controller (<NUM>), the peripheral proxy subsystem (<NUM>) configured to present a second cloned instance of the second physical function (<NUM>) designated as instance PFO' (<NUM>) at second subsystem PCIe endpoint (<NUM>) coupled to second PCIe root controller (<NUM>).