Patent Description:
A platform for a conventional graphics processing system includes a central processing unit (CPU), a graphics processing unit (GPU), one or more system memories (such as a dynamic random access memory, DRAM), and a bus to support communication between these entities. In some cases, the platform is implemented as a system-on-a-chip (SoC). The CPU initiates graphics processing by issuing draw calls to the GPU. In response to receiving a draw call, the GPU renders images for display using a pipeline formed of a sequence of programmable shaders and fixed-function hardware blocks. The system memory in the conventional graphics processing system is partitioned into a first portion that is visible to a host operating system (OS) executing on the graphics processing system and a second portion that is dedicated to the GPU, e.g., to provide a frame buffer. The second portion, which is sometimes referred to as a carveout or a GPU carveout, is not visible to the host OS. A GPU virtual manager (VM), which is managed by a graphics device driver, translates the virtual addresses in memory access requests to physical addresses in the system memory such as physical addresses in the GPU carveout region of the system memory. In some cases, the GPU VM performs the address translation using a corresponding translation lookaside buffer (TLB) that caches frequently requested address translations from a page table.

The following documents are acknowledged: <CIT>, Multiple Input-Output Memory Management Units with Fine Grained Device Scopes for Virtual Machines; <CIT>, Apparatus and Method for Memory Management in a Graphics Processing Environment; <CIT>, Method and Apparatus for Establishing System-on-Chip (SOC) Security through Memory Management Unit (MMU) Virtualization; <NPL>; <CIT>, Chained Hybrid IOMMU.

In accordance with one aspect, an apparatus includes a networked input/output memory management unit (IOMMU) comprising a plurality of IOMMUs. The networked IOMMU is configured to receive a memory access request that includes a domain physical address generated by a first address translation layer and selectively translate the domain physical address into a physical address in a system memory using one of the plurality of IOMMUs that is selected based on a type of a device that generated the memory access request.

In some embodiments, the device is a graphics processing unit (GPU) or one of a plurality of peripheral devices, and wherein the plurality of IOMMUs include a primary IOMMU configured to receive the memory access request from the first address translation layer and a secondary IOMMU connected to the primary IOMMU and disposed proximate to circuitry associated with the device. In some aspects, the primary IOMMU performs address translation of the domain physical address in response to the memory access request being received from the GPU, while the secondary IOMMU performs the address translation in response to the memory access request being received from the peripheral device.

Further, the primary IOMMU can perform the address translation of the domain physical address received from the GPU by performing a page table walk using a first translation lookaside buffer (TLB) and a first set of page tables associated with the primary IOMMU. In some aspects, the primary IOMMU provides the memory access request to the secondary IOMMU in response to the request being received from the peripheral device, and the secondary IOMMU performs the address translation of the virtual address in the memory access request received from the primary IOMMU by performing the page table walk using a second TLB and a second set of page tables associated with the secondary IOMMU.

In some embodiments, a location of the second IOMMU relative to the circuitry associated with the peripheral device is determined based on a latency requirement of the peripheral device. In such embodiments, the apparatus further can include a plurality of secondary IOMMUs connected to the primary IOMMU, wherein the plurality of secondary IOMMUs are deployed proximate circuitry associated with a plurality of peripheral devices, each of the plurality of secondary IOMMUs configured to perform address translations of domain physical addresses in memory access requests received from the circuitry associated with a corresponding one of the plurality of peripheral devices. The plurality of secondary IOMMUs may be integrated into the circuitry associated with the plurality of peripheral devices.

The apparatus further can include a command queue configured to receive memory access requests from the first address translation layer and selectively provide the memory access requests to the primary IOMMU or the secondary IOMMU based on the type of the device that generated the memory access request.

In accordance with another aspect, a method includes receiving, at a networked input/output memory management unit (IOMMU) comprising a plurality of lOMMUs, a memory access request that includes a domain physical address generated by a first address translation layer, selecting one of the plurality of IOMMUs based on a type of a device that generated the memory access request, and selectively translating the domain physical address into a physical address in a system memory using the selected one of the plurality of IOMMUs. In some embodiments,the device is a graphics processing unit (GPU) or one of a plurality of peripheral devices and receiving the memory access request comprises receiving the memory access from the first address translation layer request at a primary IOMMU in the plurality of IOMMUs. In such instances, selecting the one of the plurality of IOMMUs can include selecting the primary IOMMU in response to the device being the GPU, and selecting a secondary IOMMU in response to the device being the one of the plurality of peripheral devices, wherein the secondary IOMMU is connected to the primary IOMMU and disposed proximate to circuitry associated with the one of the plurality of peripheral devices. The method further can comprise performing address translation of the domain physical address at the primary IOMMU in response to the memory access request being received from the GPU, wherein performing the address translation of the domain physical address at the primary IOMMU comprises performing a page table walk using a first translation lookaside buffer (TLB) and a first set of page tables associated with the primary IOMMU. In some embodiments, the method further includes performing the address translation at the secondary IOMMU in response to the memory access request being received from the peripheral device. The method further can include providing the memory access request from the primary IOMMU to the secondary IOMMU in response to the request being received from the peripheral device, wherein performing the address translation of the virtual address at the secondary IOMMU comprises performing the page table walk using a second TLB and a second set of page tables associated with the secondary IOMMU.

In some embodiments, the method also includes receiving memory access requests at a command queue from the first address translation layer and selectively providing the memory access requests from the command queue to the primary IOMMU or the secondary IOMMU based on the type of the device that generated the memory access request.

In accordance with yet another aspect, a networked input/output memory management unit (IOMMU) configured to be connected to a graphics processing unit (GPU), at least one peripheral device, and a memory, includes a command queue configured to receive memory access requests from a first address translation layer, wherein the memory access request includes a domain physical address generated by the first address translation layer, a primary IOMMU configured to translate the domain physical address to a physical address in the memory in response to the memory access request being received from the GPU, and at least one secondary IOMMU configured to translate the domain physical address to the physical address in the memory in response to the memory access request being received from the at least one peripheral device. In some embodiments, the primary IOMMU performs the address translation of the domain physical address received from the GPU by performing a page table walk using a first translation lookaside buffer (TLB) and a first set of page tables associated with the primary IOMMU and the at least one secondary IOMMU performs the address translation of the virtual address in the memory access request by performing the page table walk using a second TLB and a second set of page tables associated with the secondary IOMMU. At least one location of the at least one second IOMMU relative to the at least one peripheral device may be determined based on a latency requirement of the at least one peripheral device.

The present disclosure is better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings.

Changes in the security infrastructure and requirements driven by vendors of operating systems such as Microsoft Windows® are expected to impact the memory access performance of the processing system. For example, the size of the GPU carveout may be reduced to increase the amount of memory available for dynamic allocation to the GPU from the OS-controlled portion of the memory. For another example, virtualization-based security (VBS) provides memory protection against kernel mode malware by creating a secure partition in the system memory that is accessed using a first address translation layer managed by a device driver, e.g., using page tables and translation lookaside buffers (TLBs) to cache frequently requested address translations from the page tables. The page table and TLBs are associated with a GPU virtual manager (VM). A second address translation layer used to access the secure partition is controlled by a hypervisor or secure OS. The first address translation layer is contiguous and high performance. The second address translation layer handles physical memory management challenges such as memory fragmentation and access security. Consequently, the second address translation layer typically determines overall address translation performance. The second address translation layer is implemented in a system-wide input/output memory management unit (IOMMU) that supports address translation and system memory access protection on direct memory access (DMA) transfers from devices including the GPU and one or more peripheral devices.

In response to receiving a memory access request from the GPU, the first address translation layer translates a device-generated address in the memory access request to a domain physical address. The second address translation layer implemented in the IOMMU translates the domain physical address into a system physical address in the system memory. For example, the IOMMU assigns a domain context and a distinct set of page tables to each device in the processing system. When a device attempts to read or write system memory, the IOMMU intercepts the access and determines the domain context to which the device has been assigned. Additional permissions like read, write, execute, and the like are encoded into entries in the page tables and TLBs that are used to perform the second layer translation. The IOMMU therefore uses the TLB entries associated with the domain or the page tables associated with the device to determine whether the access is to be permitted and the location in system memory that is to be accessed. For example, in response to determining that the memory access request from the device is permitted, the IOMMU generates a physical address in the system memory from a domain physical address generated by the first address translation layer.

Funneling all memory access requests from peripheral devices and the GPU through the IOMMU leads to several problems. For example, the IOMMU provides service to real-time-dependent device client blocks such as video decoders, video encoders, and display framebuffer scanout circuitry, which have strict latency requirements. Performing page tablewalks for memory access requests from multiple entities at a single IOMMU introduces processing delays that increase latency. Moreover, a single IOMMU cannot be positioned near all of the peripheral devices and the GPU, so round trip times between some of the entities and the IOMMU further increase the processing latency at the IOMMU. Consequently, a central IOMMU cannot service all memory requests close to the single IOMMU and within hard access deadlines, e.g., with low latency. A system of distinct and disparate IOMMUs could be deployed proximate the different devices or the GPU. However, providing programming support for device-specific IOMMUs requires different programming models in system software, complicating the host OS and other system software architectures that use the IOMMU as a software-targeted system device.

<FIG> disclose embodiments of a processing system that includes a graphics processing unit (GPU) and a networked input/output memory management unit (IOMMU) that satisfies the memory access latency requirements of peripheral devices using an architected programming model that views the networked IOMMU structure as a single device. The networked IOMMU receives memory access requests including a domain physical address generated from a device-generated address by a first address translation layer, e.g., a GPU VM and associated TLBs, that is managed by a device driver. The networked IOMMU selectively translates the domain physical address into a physical address in system memory using one of a plurality of IOMMUs that form the networked IOMMU. In some embodiments, the networked IOMMU include a first IOMMU that receives memory access requests from the GPU and peripheral devices via circuitry in the processing system such as display circuitry or camera circuitry. The networked IOMMU also includes one or more secondary IOMMUs that are connected to the primary IOMMU, which interfaces with the operating system (OS) or hypervisor (HV) software. The networked IOMMU implements the primary and secondary IOMMUs in a master-slave network, a star network, or other type of network such that the primary IOMMU acts as the front end for the networked IOMMU. Each of the secondary IOMMUs are disposed proximate to (or integrated within) corresponding circuitry associated with the one or more peripheral devices. For example, secondary IOMMUs for a display and a camera are disposed proximate display circuitry and camera circuitry. The location of the secondary IOMMUs are determined, at least in part, by latency requirements of the one or more peripheral devices.

In response to receiving a memory access request including a domain physical address from a first translation layer, the primary IOMMU selectively performs an address translation of the domain physical address or bypasses the address translation based on the type of device that provided the memory access request. In some embodiments, the primary IOMMU performs address translations of domain physical addresses associated with memory access requests from the GPU by performing a page tablewalk using a first set of page tables and a first translation lookaside buffer (TLB) associated with the primary IOMMU. The primary IOMMU bypasses the address translations of domain physical addresses in memory access requests received from peripheral devices. Instead, the primary IOMMU provides the memory access requests to a secondary IOMMU associated with the peripheral device that provided the memory access request. The secondary IOMMU performs address translations of domain physical addresses by performing page tablewalks using a second set of page tables and a second TLB associated with the second IOMMU. Some embodiments of the primary IOMMU include (or are associated with) a command queue that receives commands associated with the primary and secondary IOMMUs. The command queue allows system software to initiate page tablewalks and device rescans, which are processed in the primary IOMMU or selectively forwarded to one of the secondary IOMMUs, as discussed above. The command queue also supports rescan and synchronization of system software with the peripheral devices to ensure that software doesn't modify table data that is currently in flight.

<FIG> is a block diagram of a processing system <NUM> according to some embodiments. The processing system <NUM> includes or has access to a system memory <NUM> or other storage component that is implemented using a non-transitory computer readable medium such as a dynamic random access memory (DRAM). However, some embodiments of the memory <NUM> are implemented using other types of memory including static random access memory (SRAM), nonvolatile RAM, and the like. The processing system <NUM> also includes a bus <NUM> to support communication between entities implemented in the processing system <NUM>, such as the memory <NUM>. Some embodiments of the processing system <NUM> include other buses, bridges, switches, routers, and the like, which are not shown in <FIG> in the interest of clarity.

The processing system <NUM> includes a graphics processing unit (GPU) <NUM> that renders images for presentation on a display <NUM>. For example, the GPU <NUM> renders objects to produce values of pixels that are provided to the display <NUM>, which uses the pixel values to display an image that represents the rendered objects. Some embodiments of the GPU <NUM> include multiple processing elements (not shown in <FIG> in the interest of clarity) that execute instructions concurrently or in parallel. The processing elements are referred to as compute units, processor cores, or using other terms. Some embodiments of the GPU <NUM> are used for general purpose computing. In the illustrated embodiment, the GPU <NUM> communicates with the memory <NUM> over the bus <NUM>. However, some embodiments of the GPU <NUM> communicate with the memory <NUM> over a direct connection or via other buses, bridges, switches, routers, and the like. The GPU <NUM> executes instructions stored in the memory <NUM> and the GPU <NUM> stores information in the memory <NUM> such as the results of the executed instructions. For example, the memory <NUM> stores a copy <NUM> of instructions that represent a program code that is to be executed by the GPU <NUM>.

Some embodiments of the GPU <NUM> perform virtual-to-physical address translations using a GPU VM <NUM> and one or more corresponding TLBs <NUM> (only one TLB <NUM> is shown in <FIG> for clarity). For example, in some cases, the GPU VM <NUM> and the TLB <NUM> are implemented as part of a first address translation layer that generates a domain physical address from a virtual address that is included in a memory access request received at the GPU <NUM> or generated by the GPU <NUM>. Although the GPU VM <NUM> and the TLB <NUM> are depicted as integral parts of the GPU <NUM> in <FIG>, some embodiments of the GPU VM <NUM> or the TLB <NUM> are implemented external to the GPU <NUM>.

The processing system <NUM> also includes a central processing unit (CPU) <NUM> that implements multiple processing elements <NUM>, <NUM>, <NUM>, which are collectively referred to herein as "the processing elements <NUM>-<NUM>. " The processing elements <NUM>-<NUM> execute instructions concurrently or in parallel. The CPU <NUM> is connected to the bus <NUM> and communicates with the GPU <NUM> and the memory <NUM> via the bus <NUM>. The CPU <NUM> executes instructions such as program code <NUM> stored in the memory <NUM> and the CPU <NUM> stores information in the memory <NUM> such as the results of the executed instructions. The CPU <NUM> is also able to initiate graphics processing by issuing draw calls to the GPU <NUM>.

An input/output (I/O) engine <NUM> handles input or output operations associated with the display <NUM>, as well as other elements of the processing system <NUM> such as keyboards, mice, printers, external disks, and the like. In the illustrated embodiment, the I/O engine <NUM> also handles input and output operations associated with a camera <NUM>. The I/O engine <NUM> is coupled to the bus <NUM> so that the I/O engine <NUM> is able to communicate with the memory <NUM>, the GPU <NUM>, or the CPU <NUM>. In the illustrated embodiment, the I/O engine <NUM> reads information stored on an external storage component <NUM>, which is implemented using a non-transitory computer readable medium such as a compact disk (CD), a digital video disc (DVD), and the like. The I/O engine <NUM> also writes information to the external storage component <NUM>, such as the results of processing by the GPU <NUM> or the CPU <NUM>.

The processing system <NUM> includes a networked I/O memory management unit (IOMMU) <NUM> that includes a set of IOMMUs for processing memory access requests from devices such as the GPU <NUM> and peripheral devices including the display <NUM>, the camera <NUM>, and the external storage component <NUM>. The memory access requests include a device-generated address such as a virtual address that is used to indicate a location in the system memory <NUM>. Some embodiments of the networked IOMMU <NUM> receive memory access requests that include a domain physical address generated by a first address translation layer that is managed by a driver such as a graphics driver implemented by the GPU <NUM>. For example, the first address translation layer can include the GPU VM <NUM> and the TLB <NUM>. The networked IOMMU <NUM> selectively translates the domain physical address into a physical address in the system memory <NUM> using one of the set of IOMMUs that is selected based on a type of a device that generated the memory access request. The types include a first type for the GPU <NUM> and a second type for peripheral devices such as the display <NUM>, the camera <NUM>, and the external storage component <NUM>.

In the illustrated embodiment, the networked IOMMU <NUM> includes a primary IOMMU <NUM> that receives the memory access requests from the first address translation layer and secondary IOMMUs <NUM>, <NUM> connected to the primary IOMMU <NUM> and disposed proximate to circuitry (not shown in <FIG> in the interest of clarity) associated with the peripheral devices such as the display <NUM>, the camera <NUM>, and the external storage component <NUM>. The primary IOMMU <NUM> and the secondary IOMMUs <NUM>, <NUM> are deployed as a master-slave network, a star network, or other type of network that allows the memory access requests to be received initially by the primary IOMMU <NUM> and then selectively distributed, if necessary, to the secondary IOMMUs <NUM>, <NUM>. Some embodiments of the primary IOMMU <NUM> are responsible for passing through TLB shoot-downs or other software commands to the secondary IOMMUs <NUM>, <NUM>.

The networked IOMMU <NUM> performs address translations using address translations that are stored in page tables <NUM>. Each process that is executing on a device in the processing system <NUM> has a corresponding page table. The page table <NUM> for a process translates the device-generated (e.g., virtual) addresses that are being used by the process to physical addresses in the system memory <NUM>. The primary IOMMU <NUM> and the secondary IOMMUs <NUM>, <NUM> independently perform tablewalks of the page tables <NUM> to determine translations of addresses in the memory access requests. Translations that are frequently used by the networked IOMMU <NUM> are stored in TLBs <NUM>, which are used to cache frequently requested address translations. Separate TLBs <NUM> are associated with the primary IOMMU <NUM> and the secondary IOMMUs <NUM>, <NUM>. Entries including frequently used address translations are written from the page tables <NUM> into the TLBs <NUM> for the primary IOMMU <NUM> and the secondary IOMMUs <NUM>, <NUM>. The primary IOMMU <NUM> and the secondary IOMMUs <NUM>, <NUM> are therefore independently able to access the address translations from the TLB <NUM> without the overhead of searching for the translation in the page table <NUM>. Entries are evicted from the TLBs <NUM> to make room for new entries according to a TLB replacement policy. The TLB <NUM> is depicted as an integrated part of the networked IOMMU <NUM> in <FIG>. However, in other embodiments, the TLB <NUM> is implemented in a separate entity accessible by the networked IOMMU <NUM>. In some embodiments, the TLB <NUM> and the TLB <NUM> are implemented in a single architecture, although this is not required in all embodiments.

<FIG> is a block diagram of a portion <NUM> of a processing system that implements a conventional GPU carveout in memory. The portion <NUM> includes a GPU <NUM> and a system memory <NUM>. In the illustrated embodiment, the system memory <NUM> is partitioned into a host partition <NUM> and a GPU partition <NUM>, which is also referred to as a GPU carveout. For example, the GPU partition <NUM> includes up to <NUM> GB of dedicated frame buffer if the system memory <NUM> includes a total of <NUM> GB or more. The GPU partition <NUM> is reserved for exclusive access by the GPU <NUM>. The GPU partition <NUM> is not visible to the OS executing on the processing system. Consequently, the GPU partition <NUM> may be underutilized when the processing system is running applications that are non-graphics centric or otherwise do not consume significant resources of the GPU <NUM>. As discussed herein, the GPU partition <NUM> cannot be allocated to other applications because the GPU partition <NUM> is not visible to the OS.

<FIG> is a block diagram of a portion <NUM> of a processing system that implements a GPU carveout in conjunction with dynamic allocation of memory to a GPU according to some embodiments. The portion <NUM> includes a GPU <NUM> and a system memory <NUM>. In the illustrated embodiment, the system memory <NUM> is partitioned into a host partition <NUM> and a GPU partition <NUM>. The size of the GPU partition <NUM> is limited to a predetermined (relatively small) size. In some embodiments, the GPU partition <NUM> includes <NUM>% or less of system memory <NUM>, e.g., <NUM> MB for an <NUM> GB system memory <NUM>. The processing system compensates for the small size of the GPU partition <NUM> by dynamically allocating portions <NUM>, <NUM>, <NUM> (referred to herein as "the portions <NUM>-<NUM>") of the host partition <NUM>.

The portion <NUM> of the processing system includes a networked IOMMU <NUM> to translate device-generated addresses in memory access requests to physical addresses in the GPU partition <NUM> or the portions <NUM>-<NUM>. For example, a GPU VM and associated TLB can translate a virtual memory address in a memory access request to a domain physical address and provide the memory access request including the domain physical address to the networked IOMMU <NUM>. In some embodiments, page tables are defined in response to allocation of the portion <NUM>-<NUM> to processes executing on the GPU <NUM>. For example, virtual addresses used by a process executing on the GPU <NUM> are mapped to physical addresses in the portion <NUM> that is allocated to the process. The mapping is stored in entries of the page table associated with the process. The networked IOMMU <NUM> includes a set of IOMMUs and the networked IOMMU <NUM> selectively translates the domain physical address into a physical address in the system memory <NUM> using one of the set of IOMMUs that is selected based on a type of a device that generated the memory access request. For example, a primary IOMMU in the set of IOMMUs translates the domain physical address into the physical address in the system memory <NUM> in response to receiving a memory access request from the GPU <NUM>. For another example, the primary IOMMU bypasses the translation and provides the memory access request to a secondary IOMMU for translation in response to receiving a memory access request from a peripheral device such as a display or camera.

<FIG> is a block diagram illustrating translation <NUM> of device-generated addresses in a memory access request according to some embodiments. The translation <NUM> is performed in some embodiments of the processing system <NUM> shown in <FIG> and the portion <NUM> of the processing system shown in <FIG>. In the illustrated embodiment, a device <NUM> issues a memory access request such as a request to write information to a system memory <NUM> or a request to read information from the system memory <NUM>. The device <NUM> represents some embodiments of the GPU <NUM> shown in <FIG> or peripheral devices such as the display <NUM>, the camera <NUM>, or the external storage medium <NUM> shown in <FIG>.

The memory access request includes a device-generated address such as a virtual address used by an application executing on or associated with the device <NUM>. In the illustrated embodiment, a virtualization based security (VBS) provides memory protection (e.g., against kernel mode malware) using a two-level translation process that includes a first level translation <NUM> managed by an OS or device driver <NUM> and a second layer translation <NUM> managed by a hypervisor <NUM>. The first level translation <NUM> translates a device-generated address such as a virtual address in the memory access request to a domain physical address such as a GPU physical address. In some embodiments, the first level translation <NUM> is performed by a GPU VM and associated TLB, as discussed herein. The domain physical address is passed to the second level translation <NUM>, which translates the domain physical address into a physical address that indicates a location within the system memory <NUM>. As discussed herein, the second level translation <NUM> also verifies that the device <NUM> is authorized to access the region of the system memory <NUM> indicated by the physical address, e.g., using permission information that is encoded into entries in associated page tables and translation lookaside buffers (TLBs) that are used to perform the second layer translation <NUM>.

<FIG> is a block diagram of a portion <NUM> of a processing system that implements a networked IOMMU <NUM> according to some embodiments. The portion <NUM> represents a portion of some embodiments of the processing system <NUM> shown in <FIG> and the networked IOMMU <NUM> represents some embodiments of the networked IOMMU <NUM> shown in <FIG> and the networked IOMMU <NUM> shown in <FIG>.

The networked IOMMU <NUM> receives memory access requests via a unified software interface <NUM> to a primary IOMMU <NUM>. In the illustrated embodiment, the memory access requests are provided by software such as an IOMMU driver <NUM> that is implemented in the processing system. The IOMMU driver <NUM> receives the memory access requests from a first address translation layer, e.g., an address translation layer that includes a GPU VM and associated TLB (not shown in <FIG> in the interest of clarity). The first address translation layer translates a device-generated address such as a virtual memory address to a domain physical address and provides the memory access request including the domain physical address to the networked IOMMU <NUM>. The device-generated addresses (e.g., virtual addresses) are used by the software to indicate locations within a system memory <NUM>.

The primary IOMMU <NUM> and the unified software interface <NUM> support an architected programming model that is targeted as a single device by system software (such as the IOMMU driver <NUM>). Thus, the programming model does not require dedicated control mechanisms and software to operate disparate IOMMU hardware units for real time and conventional direct memory access (DMA) processing. However, some device client blocks that require IOMMU services to satisfy worst case latency requirements, e.g., for video decoder in code, display frame buffer scan out, and the like. A single primary IOMMU <NUM> is not always able to satisfy the latency requirements.

At least in part to address the worst case latency requirements of device client blocks, the networked IOMMU <NUM> includes one or more secondary IOMMU <NUM>, <NUM> that are deployed proximate corresponding device client blocks for peripheral devices. In the illustrated embodiment, the peripheral device circuitry includes display circuitry <NUM> that supports communication with a display such as the display <NUM> shown in <FIG> and camera circuitry <NUM> that supports communication with a camera such as the camera <NUM> shown in <FIG>. The secondary IOMMUs <NUM>, <NUM> are integrated in the corresponding circuitry <NUM>, <NUM>. However, in some embodiments, the secondary IOMMUs <NUM>, <NUM> are deployed proximate the corresponding circuitry <NUM>, <NUM> at a (physical or logical) distance that is determined based on the latency requirements of the corresponding circuitry <NUM>, <NUM>. Lower latency requirements imply that the secondary IOMMUs <NUM>, <NUM> are deployed (physically or logically) closer to the corresponding circuitry <NUM>, <NUM>. The primary IOMMU <NUM> and the secondary IOMMUs <NUM>, <NUM> are implemented in a master-slave relationship, a star network, or other configuration that allows the primary IOMMU <NUM> to act as a single device to receive memory access requests via the unified software interface <NUM>.

In operation, the primary IOMMU <NUM> performs address translations for memory access requests from devices of the first type (e.g., requests from a GPU) and bypasses performing address translations for memory access requests from devices of a second type (e.g., requests from peripheral devices). The primary IOMMU <NUM> forwards memory access requests from devices of the second type to corresponding secondary IOMMUs <NUM>, <NUM>. For example, the primary IOMMU <NUM> forwards memory access requests associated with a display to the secondary IOMMU <NUM> and memory access requests associated with a camera to the secondary IOMMU <NUM>. Thus, the IOMMU driver <NUM> issues a single command to access the system memory <NUM> (e.g., a single memory access request) via the interface <NUM>. The single command is then selectively handled by either the primary IOMMU <NUM> or one of the specialized secondary IOMMUs <NUM>, <NUM>. The IOMMU driver <NUM> therefore implements the access policy without being required to address the dedicated IOMMU <NUM>, <NUM>, <NUM> separately or independently.

In response to receiving a memory access request from a device of the appropriate type, the primary IOMMU <NUM> or the secondary IOMMUs <NUM>, <NUM> access entries in corresponding TLBs <NUM>, <NUM>, <NUM> (collectively referred to herein as "the TLBs <NUM>-<NUM>") to attempt to locate a translation of the address included in the memory access request. The entries in the TLBs <NUM>-<NUM> encode information indicating whether the requesting device is permitted to access the system memory <NUM>. If the address hits in the corresponding TLB <NUM>-<NUM> and the device has the appropriate permissions, the memory access request is forwarded to the system memory <NUM>. If the address misses in the corresponding TLB <NUM>-<NUM>, the memory access request is forwarded to a corresponding page table <NUM>, <NUM>, <NUM>, which returns the appropriate translation of the address. Entries in the TLBs <NUM>-<NUM> are updated based on the replacement policy implemented by the TLBs <NUM>-<NUM>. If the device does not have the appropriate permissions, the memory access request is denied.

Some embodiments of the networked IOMMU <NUM> include a command queue <NUM> that receives memory access requests from the IOMMU driver <NUM> and stores the access requests before they are issued to the primary IOMMU <NUM>. The command queue <NUM> allows system software to initiate page table and device rescans that are forwarded to the primary IOMMU <NUM> or the secondary IOMMUs <NUM>, <NUM>, which are therefore able to cache relevant data in the corresponding TLBs <NUM>-<NUM>. Some embodiments of the command queue <NUM> also allow rescans and synchronization of system software with hardware units to ensure that the software does not modify table data that is in flight.

<FIG> is a block diagram of a usage model <NUM> of a processing system that implements a system-on-a-chip (SOC) device translation block <NUM> according to some embodiments. The usage model <NUM> is implemented in some embodiments of the processing system <NUM> shown in <FIG>, the portion <NUM> of the processing system shown in <FIG>, and the portion <NUM> of the processing system shown in <FIG>.

The SOC device translation block <NUM> includes a primary IOMMU <NUM> that receives memory access requests from devices including a graphics pipeline (GFX) <NUM> in a GPU and peripheral devices such as a display <NUM>, a camera <NUM>, and the like. In some embodiments, the memory access requests are received from a first address translation layer, e.g., an address translation layer that is implemented using a GPU VM and TLB, and the memory access requests include a domain physical address generated by the first address translation layer. The memory access requests are used to access system memory such as DRAM <NUM>. As discussed herein, the primary IOMMU <NUM> selectively performs address translations on the addresses included in the memory access requests based on the type of the device that issued the request. Memory access requests from device types that are not translated by the primary IOMMU <NUM> are forwarded to a distributed remote IOMMU network <NUM> that includes one or more IOMMUs associated with the display <NUM>, the camera <NUM>, and other peripheral devices. Some embodiments of the distributed remote IOMMU network <NUM> are implemented using the secondary IOMMUs <NUM>, <NUM> shown in <FIG>. The SOC device translation block <NUM> also includes a translation cache <NUM> for translating addresses associated with requests generated by the GFX <NUM>. A virtual-to-physical manager <NUM> is used to support peripheral devices such as the display <NUM>, the camera <NUM>, and any other peripheral devices.

In operation, a kernel mode driver or memory manager <NUM> provides signaling <NUM> to configure address translation tables such as page tables that are used to translate virtual addresses to GPU physical addresses (or domain physical addresses), e.g., using a first layer of address translation performed by a GPU VM and associated TLB. The memory manager <NUM> also provides virtual addresses <NUM> such as GPU virtual addresses to the virtual-to-physical manager <NUM>. A hypervisor or hypervisor abstraction layer (HAL) <NUM> manages system physical page tables and access permissions stored in the DRAM <NUM>. The HAL <NUM> also configures the primary IOMMU <NUM> in the SOC device translation layer <NUM>. The GFX <NUM> attempts to translate virtual addresses using the translation cache <NUM>. If the attempt hits in the translation cache <NUM>, the returned address translation is used for further processing. If the attempt misses in the translation cache <NUM>, the request is forwarded to the primary IOMMU <NUM>, which handles the subsequent address translation as discussed herein. The primary IOMMU <NUM> and the distributed remote IOMMU network <NUM> are also able to access the DRAM <NUM> to perform page table walks, as discussed herein.

<FIG> is a flow diagram of a method <NUM> of selectively performing address translations and a primary IOMMU or a secondary IOMMU according to some embodiments. The method <NUM> is implemented in a networked IOMMU such as some embodiments of the networked IOMMU <NUM> shown in <FIG>, the networked IOMMU <NUM> shown in <FIG>, the second level translation <NUM> shown in <FIG>, the networked IOMMU <NUM> shown in <FIG>, and the SOC device translation block <NUM> shown in <FIG>.

At block <NUM>, the networked IOMMU receives a memory access request from a device of a particular type. Examples of device types include a graphics processor type, a peripheral device type, and the like. In some embodiments, the memory access request is received from a first address translation layer, e.g., an address translation layer that is implemented using a GPU VM and TLB, and the memory access request includes a domain physical address generated by the first address translation layer.

At decision block <NUM>, the networked IOMMU determines the type of device that issued the memory access request, e.g., based on information included in the request. If the type of device that issued the memory access request is a peripheral device, the method <NUM> flows to block <NUM>. If the type of device that issued the memory access request is a GPU device, the method <NUM> flows to block <NUM>.

At block <NUM>, the primary IOMMU in the networked IOMMU bypasses address translation for memory access requests from peripheral device types. The method <NUM> then flows to block <NUM> and the primary IOMMU forwards the memory access request to a secondary IOMMU associated with the requesting device. For example, the primary IOMMU forwards the memory access request to a secondary IOMMU integrated in display circuitry in response to the memory access request being from a display. The secondary IOMMU then performs the address translation at block <NUM>.

At block <NUM>, the primary IOMMU in the networked IOMMU performs the requested address translation for memory access requests from the GPU.

A computer readable storage medium includes any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media include, but are not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc , magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media.

In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software includes the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium includes, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium are in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

Claim 1:
An apparatus, comprising:
a networked input/output memory management unit, IOMMU, (<NUM>) comprising a plurality of IOMMUs (<NUM>, <NUM>, <NUM>),
wherein the networked IOMMU is configured to:
receive a memory access request that includes a domain physical address generated by a first address translation layer; and
selectively translate the domain physical address into a physical address in a system memory (<NUM>) using one of the plurality of IOMMUs that is selected based on a type of a device that generated the memory access request.