PASID GRANULARITY RESOURCE CONTROL FOR IOMMU

An embodiment of an integrated circuit may comprise memory to store respective resource control descriptors in correspondence with respective identifiers, and an input/output (JO) memory management unit (IOMMU) communicatively coupled to the memory, the IOMMU including circuitry to determine resource control information for an IO transaction based on a resource control descriptor stored in the memory that corresponds to an identifier associated with the IO transaction, and control utilization of one or more resources of the IOMMU based on the determined resource control information. Other embodiments are disclosed and claimed.

BACKGROUND

1. Technical Field

This disclosure generally relates to processor technology, and input/output memory management unit technology.

2. Background Art

An input/output (IO) memory management unit (IOMMU) connects direct memory access (DMA) capable IO buses to system memory (e.g., main memory). A central processor unit (CPU) memory management (MMU) translates CPU-visible virtual addresses to physical addresses. An IOMMU brokers any DMA request on behalf of an IO device, translating IO virtual addresses much the same way as the processor MMU complex performs translation of a virtual address to physical address. Some IOMMUs may also provide memory protection from faulty or malicious devices.

INTEL VIRTUALIZATION TECHNOLOGY (INTEL VT) provides hardware support for efficient virtual machines. INTEL VIRTUALIZATION TECHNOLOGY FOR DIRECTED I/O (VT-d) provides support for IO-device virtualization.

DETAILED DESCRIPTION

Embodiments discussed herein variously provide techniques and mechanisms for resource control in an input/output (IO) memory management unit (IOMMU). The technologies described herein may be implemented in one or more electronic devices. Non-limiting examples of electronic devices that may utilize the technologies described herein include any kind of mobile device and/or stationary device, such as cameras, cell phones, computer terminals, desktop computers, electronic readers, facsimile machines, kiosks, laptop computers, netbook computers, notebook computers, internet devices, payment terminals, personal digital assistants, media players and/or recorders, servers (e.g., blade server, rack mount server, combinations thereof, etc.), set-top boxes, smart phones, tablet personal computers, ultra-mobile personal computers, wired telephones, combinations thereof, and the like. More generally, the technologies described herein may be employed in any of a variety of electronic devices including integrated circuitry which is operable to provide resource control in an IOMMU.

It is to be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are, for example, capable of operation in other orientations than those illustrated or otherwise described herein.

As used throughout this description, and in the claims, a list of items joined by the term “at least one of” or “one or more of” can mean any combination of the listed terms. For example, the phrase “at least one of A, B or C” can mean A; B; C; A and B; A and C; B and C; or A, B and C. It is pointed out that those elements of a figure having the same reference numbers (or names) as the elements of any other figure can operate or function in any manner similar to that described, but are not limited to such.

In addition, the various elements of combinatorial logic and sequential logic discussed in the present disclosure may pertain both to physical structures (such as AND gates, OR gates, or XOR gates), or to synthesized or otherwise optimized collections of devices implementing the logical structures that are Boolean equivalents of the logic under discussion.

Quality of Service (QoS) is important in a virtualization environment to control/restrict resources that can be used by the virtual machines (VMs) or containers that the tenant owns. Some resources in a server may be shared by different VMs/containers and different QoS requirements may be applied in a manner to provide suitable performance for the tenants and avoid problems from noisy neighbors. Shared resources exist in some computer architectures in all or most layers including, for example, CPU core resources, un-core resources (e.g., IOMMU, L3 cache, internal bus, etc.), and peripheral IO device resources. To ensure equitable allocation of resources in virtualization, QoS policies may be implemented in various components. For example, in IO virtualization, a Virtual Function (VF) (e.g., for single root IO virtualization (SRIOV)) or an Assignable Device Interface (ADI) (e.g., for scalable IO virtualization (Scalable IOV)) may represent a slice of hardware that may implement the QoS per VF/ADI or other granularity. When a hardware slice is assigned to a VM/Container, a hypervisor can use the QoS configuration interface provided by the VF/ADI to control/restrict the shared resource within the IO device utilized by the VM/Container.

In some computer architectures, the IO data-path may include more than just IO devices and the CPU core. An IOMMU may work in conjunction with applications and IO devices to perform IO workloads, providing important functions for virtualization like DMA remapping, IRQ remapping, shared virtual memory, etc. A problem is that the IOMMU provides shared resource/functions that all applications/cores/tenants/devices competitively share, potentially causing contention. For example, IOMMU shared resources may include an IO Translation Lookaside Buffer (IOTLB) for IO/Virtual address translation, a Page Request Service, etc. Some embodiments may provide technology to control/restrict utilization of IOMMU resources on a per process address space identifier (PASID) basis and/or a per domain basis. Advantageously, some embodiments may implement QoS policies within the IOMMU at a PASID granularity, thereby enabling equitable utilization of IOMMU resources and mitigating or preventing problems with noisy neighbors.

With reference toFIG.1, an embodiment of an integrated circuit100may include memory101to store respective resource control descriptors in correspondence with respective identifiers, and an IOMMU103communicatively coupled to the memory101. The IOMMU103may include circuitry105configured to determine resource control information for an JO transaction based on a resource control descriptor stored in the memory101that corresponds to an identifier associated with the IO transaction, and control utilization of one or more resources of the IOMMU103based on the determined resource control information. For example, the identifier may correspond to a PASID associated with the IO transaction, a domain identifier associated with the IO transaction, etc.

In some embodiments, the memory101may also store a table of entries indexed by respective PASIDs, and the circuitry105may be configured to determine if an entry in the table indexed by a PASID associated with the IO transaction includes a field that points to a resource control descriptor that corresponds to the PASID, and, if so determined, to determine the resource control information for the IO transaction based on the resource control descriptor stored at a location in the memory101indicated by the field. Otherwise, if the entry does not include the field that points to the resource control descriptor, the circuitry105may be configured to determine the resource control information for the IO transaction based on a resource control descriptor that corresponds to a domain identifier associated with the IO transaction. In some embodiments, the circuitry105may be further configured to determine if PASID-granularity control is disabled, and, if so determined, to determine the resource control information for the IO transaction based on the resource control descriptor that corresponds to the domain identifier associated with the IO transaction (e.g., even if the entry includes the field that points to the resource control descriptor associated with the PASID).

In some embodiments, the resource control information may indicate a threshold for a number of entries for the identifier in an IOTLB and, in response to an IOTLB miss for the IO transaction, the circuitry105may be further configured to determine if a count of entries in the IOTLB associated with the identifier exceeds the threshold for the number of entries for the identifier in the IOTLB, and, if so determined, to identify a least recently used (LRU) entry in the IOTLB associated with the identifier and invalidate the identified entry. Additionally, or alternatively, the resource control information may indicate a threshold for a number of inflight page requests for the identifier and, in response to a page request for the IO transaction, the circuitry105may be further configured to determine if a count of inflight page requests associated with the identifier exceeds the threshold for the number of inflight page requests for the identifier, and, if so determined, to reject the page request associated with the identifier.

With reference toFIGS.2A to2D, an embodiment of a method200may include storing respective resource control descriptors in correspondence with respective identifiers at box221, determining resource control information for an IO transaction based on a stored resource control descriptor that corresponds to an identifier associated with the IO transaction at box222, and controlling utilization of one or more resources of an IOMMU based on the determined resource control information at box223. For example, the identifier may correspond to a PASID associated with the IO transaction at box224, or a domain identifier associated with the IO transaction at box225(e.g., or other identifier associated with the JO transaction, such as a stream identifier, etc.).

Some embodiments of the method200may further include storing a table of entries indexed by respective PASIDs at box226, determining if an entry in the table indexed by a PASID associated with the IO transaction includes a field that points to a resource control descriptor that corresponds to the PASID at box227and, if so determined, determining the resource control information for the IO transaction based on the resource control descriptor stored at a location indicated by the field at box228, and, otherwise, determining the resource control information for the IO transaction based on a resource control descriptor that corresponds to a domain identifier associated with the IO transaction at box229. Prior to box227, some embodiments of the method200may optionally determine if PASID-granularity control is disabled at box230, and, if so determined, proceed to determining the resource control information for the IO transaction based on the resource control descriptor that corresponds to the domain identifier associated with the IO transaction at box229.

In some embodiments of the method200, the resource control information may indicate a threshold for a number of entries for the identifier in an IOTLB at box231, and, in response to an IOTLB miss for the IO transaction at box232, the method200may further include determining if a count of entries in the IOTLB associated with the identifier exceeds the threshold for the number of entries for the identifier in the IOTLB at box233, and, if so determined, identifying a LRU entry in the IOTLB associated with the identifier at box234and invalidating the identified entry at box235. Additionally, or alternatively, the resource control information may indicate a threshold for a number of inflight page requests for the identifier at box236, and, in response to a page request for the IO transaction at box237, the method200may further include determining if a count of inflight page requests associated with the identifier exceeds the threshold for the number of inflight page requests for the identifier at box238, and, if so determined, rejecting the page request associated with the identifier at box239.

With reference toFIG.3, an embodiment of an apparatus300may include a core341, a memory management unit (MMU)343communicatively coupled to the core341, memory345communicatively coupled to the MMU343to store respective resource control descriptors in correspondence with respective identifiers, and an IOMMU347communicatively coupled to the memory345. The IOMMU347may include circuitry349to determine resource control information for an IO transaction based on a resource control descriptor stored in the memory345that corresponds to an identifier associated with the IO transaction, and control utilization of one or more resources of the IOMMU347based on the determined resource control information. For example, the identifier may correspond to a PASID associated with the IO transaction, a domain identifier associated with the IO transaction, etc.

In some embodiments, the memory345may also store a table of entries indexed by respective PASIDs, and the circuitry349may be configured to determine if an entry in the table indexed by a PASID associated with the IO transaction includes a field that points to a resource control descriptor that corresponds to the PASID, and, if so determined, to determine the resource control information for the IO transaction based on the resource control descriptor stored at a location in the memory345indicated by the field. Otherwise, if the entry does not include the field that points to the resource control descriptor, the circuitry349may be configured to determine the resource control information for the IO transaction based on a resource control descriptor that corresponds to a domain identifier associated with the IO transaction. In some embodiments, the circuitry349may be further configured to determine if PASID-granularity control is disabled, and, if so determined, to determine the resource control information for the IO transaction based on the resource control descriptor that corresponds to the domain identifier associated with the IO transaction (e.g., even if the entry includes the field that points to the resource control descriptor associated with the PASID).

In some embodiments, the resource control information may indicate a threshold for a number of entries for the identifier in an IOTLB and, in response to an IOTLB miss for the IO transaction, the circuitry349may be further configured to determine if a count of entries in the IOTLB associated with the identifier exceeds the threshold for the number of entries for the identifier in the IOTLB, and, if so determined, to identify a LRU entry in the IOTLB associated with the identifier and invalidate the identified entry. Additionally, or alternatively, the resource control information may indicate a threshold for a number of inflight page requests for the identifier and, in response to a page request for the IO transaction, the circuitry349may be further configured to determine if a count of inflight page requests associated with the identifier exceeds the threshold for the number of inflight page requests for the identifier, and, if so determined, to reject the page request associated with the identifier.

Some embodiments may provide an extension to IOMMU technology for a PASID-granular QoS engine. A conventional IOMMU does not include QoS capabilities that prevents a tenant from overutilizing resources of the IOMMU in a way that might degrade the performance of other tenants and potentially prevent the system from meeting a QoS requirement for the other tenants. Some embodiments may provide IOMMU technology to define QoS at a per PASID granularity or a per domain granularity. Advantageously, some embodiments may provide centralized QoS and flow control at a platform level to maintain performance for all tenants at a desired QoS (e.g., and to prevent one tenant from overutilizing IOMMU resources to the detriment of other tenants). For example, two IOMMU resources that may benefit from such controlled access in accordance with some embodiments include the IOTLB and the Page Request Interface.

With reference toFIG.4, an embodiment of a virtualization environment400may include a plurality of VMs (VM-1 through VM-3), each with respective ADIs (ADI-1 through ADI-3). A processor device in a socket441may include a plurality of cores (Core-1 through Core-N, N>1) coupled to an IOMMU443. The IOMMU443includes an IOTLB that has respective table entries for each of ADI-1, ADI-2, and ADI-3. In a conventional multi-tenant environment, one tenant (e.g., VM-1) may have a chance to overutilize the IOTLB445with its device (e.g., ADI-1). An LRU replacement policy in a conventional IOMMU may force eviction of TLB caches used by other VMs, causing extra page table walk and reducing IO performance of other tenants (e.g., that may have higher priority and/or other QoS requirements).

In some embodiments, the IOMMU443includes technology to apply respective QoS policies (e.g., on a per PASID or per domain basis) to limit the VMs' utilization of the IOTLB. As illustrated inFIG.4, the brackets A, B, and C represent respective limits on the number of IOTLB table entries (e.g., where A, B, and C came from respective QoS policies). For example, transactions from each VM may have a unique identifier (e.g., PASID, domain identifier, etc.) that may be utilized to apply the appropriate QoS policy. As illustrated inFIG.4, the shaded areas next to the respective brackets A, B, and C may represent available entries in the IOTLB445for the corresponding ADIs. In this example, ADI-1 has reached the limit for the number of entries in the IOTLB445allotted to ADI-1, while ADI-2 and ADI-3 have not. In accordance some embodiments, if ADI-1 requires more entries in the IOTLB445, existing entries for ADI-1 in the IOTLB445are evicted to make room for the new ADI-1 entries. Advantageously, the entries in the IOTLB445for ADI-2 and ADI-3 are unaffected. Those skilled in the art will appreciate that, in addition to the IOTLB445, there are many other caches in the IOMMU443(e.g., Page Walk Cache, etc.) that are competitively shared and subject to similar contention and that may benefit from embodiments of resource control for QoS as described herein.

With reference toFIG.5, an embodiment of a virtualization environment500may include a plurality of VFs (VF-1 through VF-3), each attached to respective VMs (VM-1 through VM-3), that send page requests to an IOMMU551. The IOMMU551manages the page requests through messages in a page request queue553. System software (e.g., a kernel)557includes a page fault handler in a memory management module559that services the page request queue553. An example Page Request Interface is defined by PCI-SIG (pci-sig.com) to allow a PCI device interacting with un-pinned system memory for DMA transactions. In the event of page fault (e.g., not allocated yet, or swapped out), the PCI device can send page request message to the IOMMU551. The IOMMU551will raise a page fault request to the system software557to recover the memory region so that the PCI device can continue DMA operations. For SR-IOV, PCI-SIG defines that a single Page Request Interface is permitted for a physical function (PF) and the Page Request Interface is shared between the PF and the associated VFs. Moreover, a Page Request specification defines outstanding page request messages that the associated Page Request Interface of the PF physically supports which is shared by the PF and all its deriving VFs.

The number of page requests that can be processed by the IOMMU551is limited, and a page request is rejected if the number of inflight page requests from a PF and all its VFs reach a limitation allowed for the physical device. In a conventional multi-tenant environment, one tenant (e.g., VM-2) may have a chance to overutilize the Page Request Interface with its VFs (e.g., VF-2). If a number of inflight page request messages from VF-2 causes the Page Request Interface to reach the limit (e.g., outstanding page request messages) of the PF, page requests sent from VFs belonging to other tenants (e.g., that are sharing the same PF) will be rejected by a conventional IOMMU due to the overflow of inflight page request messages, thereby reducing performance of the other tenants (e.g., which may have higher priority and/or other QoS requirements).

In some embodiments, the IOMMU551includes technology to apply respective QoS policies (e.g., on a per PASID or per domain basis) to limit the VFs' utilization of the Page Request Interface. As illustrated inFIG.5, the brackets A, B, and C represent respective limits on the number of inflight page requests corresponding to respective QoS policies for the VMs/VFs. For example, each VF may have a unique identifier (e.g., PASID, domain identifier, etc.) that may be utilized to apply the appropriate QoS policy. As illustrated inFIG.5, the shaded areas next to the respective brackets A, B, and C may represent a number of page requests that can be made by the VF before it reaches the limit for the VF. In this example, VF-2 has reached the limit for the number of inflight page requests allotted to VF-2, while VF-1 and VF-3 have not. In accordance some embodiments, if VF-2 issues another page request, the page request is rejected by the IOMMU551(e.g., even if the limitation allowed for the physical device is not reached). Advantageously, page requests for VF-1 and VF-3 are unaffected. Those skilled in the art will appreciate that, in addition to the Page Request Interface, there are other resources in the IOMMU551that are competitively shared and subject to similar contention and that may benefit from embodiments of resource control for QoS as described herein.

With reference toFIG.6, an embodiment of a virtualization environment600includes virtualization tables661stored in system memory663(e.g., host physical address space). The virtualization tables661include one or more PASID-QoS descriptors665. For example, the PASID-QoS descriptor665is created by a hypervisor in the system memory663to specify a compound QoS strategy for a PASID that maps to an application in a host or a VM. Some embodiments expand a Scalable Mode IOMMU PASID table entry structure667with an additional field669, referred to herein as QoS_PTR, that points to the physical address of the PASID-QoS descriptor665in the system memory663. The PASID-QoS descriptor665includes QoS settings for multiple shared resources of an IOMMU or CPU, including IOTLB, Page Request, page walk cache, etc. If new shared resources are added to the IOMMU or CPU, QoS settings for such new resources may also be included in the PASID-QoS descriptor665. In some embodiments (e.g., where an IOMMU may not support Scalable Mode or where a Scalable Mode feature is disabled), a context table may provide a QoS_PTR that points to a more generic DOMAIN-QoS descriptor, that may function in the same manner as a PASID-QoS descriptor. Advantageously, embodiments of an IOMMU and CPU can apply flow control and provide QoS on a per process or per domain basis. Some embodiments provide technology for an IOMMU to reduce or prevent contention for shared resources of the IOMMU/CPU, advantageously limiting overutilization of a resource by one tenant and reducing or preventing cross-tenant or cross-domain performance degradation.

In addition to the data structure expansion for the QoS_PTR669and the PASID-QoS descriptor665(e.g., or a DOMAIN-QoS descriptor), an embodiment of an IOMMU is configured to implement additional flow control to apply the QoS policies indicated by the QoS descriptors. For example, an IOMMU flow control unit applies the QoS upon receiving an upstream request (e.g., DMA, Page Request Message, etc.) from a PCI device using a PASID-QoS descriptor that corresponds to a PASID in the request.

With reference toFIG.7, an embodiment of a method700starts with the IOMMU receiving a request (e.g., a DMA request, a page request, etc.) at box771. When the IOMMU receives an upstream PCI packet, the IOMMU retrieves a requester identifier (ID) including Bus, Device, Function, and PASID information at box772. The IOMMU then locates a PASID context entry at box773, using the requester ID (Bus:Dev:Func and PASID).

If the IOMMU determines that no PASID-QoS descriptor is available at box774(e.g., and in some embodiments if no DOMAIN-QoS is available), then the method700may progress in legacy mode at box775. Otherwise (e.g., if the PASID context entry contains a QoS_PTR field), the IOMMU retrieves the PASID-QoS descriptor from the location indicated by the QoS_PTR (e.g., from memory, internal cache, etc.). The PASID-QoS descriptor may selectively contain QoS settings for IOTLB, Page Request, etc. The flow control unit in the IOMMU uses the corresponding QoS settings to apply flow control.

The method700then includes determining the packet type at box776. For a DMA request, if the DMA request contains a virtual address (SVM Mode) that does not have corresponding IOTLB entry (e.g., an IOTLB miss), the IOMMU needs to do a page table walk at box777to fill the IOTLB. At this point the flow control unit keeps track of how many entries in IOTLB are already consumed by this PASID. If the IOTLB entries consumed by the PASID already reached the limit allowed by this PASID at box778(e.g., as defined by PASID-QoS→IOTLB_QoS), the IOMMU invalidates another entry consumed by the same PASID (e.g., using a LRU algorithm) at box779and then populates the IOTLB with an entry for the newly translated address at box781. Otherwise, the IOMMU allocates a new IOTLB entry for the PASID at box783(e.g., and increments the count of IOTLB entries consumed by this PASID). Advantageously, embodiments of the method700prevent one VM or application from preempting entries in the IOTLB consumed by other VMs or applications.

For a page request at box785, the method700includes determining if the inflight page requests of the PASID already reached a limit defined by the PASID-QoS at box787. If not, the IOMMU may forward the page request to the software at box789(e.g., and increment the count of inflight page requests for the PASID). If the limit is reached at box787, the IOMMU may reject the page request message at box791. Those skilled in the art will appreciate that the method700may readily be extended to other resources with appropriately configured flow to control utilization of those resources.

Those skilled in the art will appreciate that a wide variety of devices may benefit from the foregoing embodiments. The following exemplary core architectures, processors, and computer architectures are non-limiting examples of devices that may beneficially incorporate embodiments of the technology described herein.

Exemplary Core Architectures

In-Order and Out-of-Order Core Block Diagram

InFIG.8A, a processor pipeline900includes a fetch stage902, a length decode stage904, a decode stage906, an allocation stage908, a renaming stage910, a scheduling (also known as a dispatch or issue) stage912, a register read/memory read stage914, an execute stage916, a write back/memory write stage918, an exception handling stage922, and a commit stage924.

FIG.8Bshows processor core990including a front end unit930coupled to an execution engine unit950, and both are coupled to a memory unit970. The core990may be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As yet another option, the core990may be a special-purpose core, such as, for example, a network or communication core, compression engine, coprocessor core, general purpose computing graphics processing unit (GPGPU) core, graphics core, or the like.

The front end unit930includes a branch prediction unit932coupled to an instruction cache unit934, which is coupled to an instruction translation lookaside buffer (TLB)936, which is coupled to an instruction fetch unit938, which is coupled to a decode unit940. The decode unit940(or decoder) may decode instructions, and generate as an output one or more micro-operations, micro-code entry points, microinstructions, other instructions, or other control signals, which are decoded from, or which otherwise reflect, or are derived from, the original instructions. The decode unit940may be implemented using various different mechanisms. Examples of suitable mechanisms include, but are not limited to, look-up tables, hardware implementations, programmable logic arrays (PLAs), microcode read only memories (ROMs), etc. In one embodiment, the core990includes a microcode ROM or other medium that stores microcode for certain macroinstructions (e.g., in decode unit940or otherwise within the front end unit930). The decode unit940is coupled to a rename/allocator unit952in the execution engine unit950.

The execution engine unit950includes the rename/allocator unit952coupled to a retirement unit954and a set of one or more scheduler unit(s)956. The scheduler unit(s)956represents any number of different schedulers, including reservations stations, central instruction window, etc. The scheduler unit(s)956is coupled to the physical register file(s) unit(s)958. Each of the physical register file(s) units958represents one or more physical register files, different ones of which store one or more different data types, such as scalar integer, scalar floating point, packed integer, packed floating point, vector integer, vector floating point, status (e.g., an instruction pointer that is the address of the next instruction to be executed), etc. In one embodiment, the physical register file(s) unit958comprises a vector registers unit, a write mask registers unit, and a scalar registers unit. These register units may provide architectural vector registers, vector mask registers, and general purpose registers. The physical register file(s) unit(s)958is overlapped by the retirement unit954to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using a reorder buffer(s) and a retirement register file(s); using a future file(s), a history buffer(s), and a retirement register file(s); using a register maps and a pool of registers; etc.). The retirement unit954and the physical register file(s) unit(s)958are coupled to the execution cluster(s)960. The execution cluster(s)960includes a set of one or more execution units962and a set of one or more memory access units964. The execution units962may perform various operations (e.g., shifts, addition, subtraction, multiplication) and on various types of data (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some embodiments may include a number of execution units dedicated to specific functions or sets of functions, other embodiments may include only one execution unit or multiple execution units that all perform all functions. The scheduler unit(s)956, physical register file(s) unit(s)958, and execution cluster(s)960are shown as being possibly plural because certain embodiments create separate pipelines for certain types of data/operations (e.g., a scalar integer pipeline, a scalar floating point/packed integer/packed floating point/vector integer/vector floating point pipeline, and/or a memory access pipeline that each have their own scheduler unit, physical register file(s) unit, and/or execution cluster—and in the case of a separate memory access pipeline, certain embodiments are implemented in which only the execution cluster of this pipeline has the memory access unit(s)964). It should also be understood that where separate pipelines are used, one or more of these pipelines may be out-of-order issue/execution and the rest in-order.

The set of memory access units964is coupled to the memory unit970, which includes a data TLB unit972coupled to a data cache unit974coupled to a level 2 (L2) cache unit976. In one exemplary embodiment, the memory access units964may include a load unit, a store address unit, and a store data unit, each of which is coupled to the data TLB unit972in the memory unit970. The instruction cache unit934is further coupled to a level 2 (L2) cache unit976in the memory unit970. The L2 cache unit976is coupled to one or more other levels of cache and eventually to a main memory.

By way of example, the exemplary register renaming, out-of-order issue/execution core architecture may implement the pipeline900as follows: 1) the instruction fetch938performs the fetch and length decoding stages902and904; 2) the decode unit940performs the decode stage906; 3) the rename/allocator unit952performs the allocation stage908and renaming stage910; 4) the scheduler unit(s)956performs the schedule stage912; 5) the physical register file(s) unit(s)958and the memory unit970perform the register read/memory read stage914; the execution cluster960perform the execute stage916; 6) the memory unit970and the physical register file(s) unit(s)958perform the write back/memory write stage918; 7) various units may be involved in the exception handling stage922; and 8) the retirement unit954and the physical register file(s) unit(s)958perform the commit stage924.

Specific Exemplary In-Order Core Architecture

FIG.9Ais a block diagram of a single processor core, along with its connection to the on-die interconnect network1002and with its local subset of the Level 2 (L2) cache1004, according to embodiments of the invention. In one embodiment, an instruction decoder1000supports the x86 instruction set with a packed data instruction set extension. An L1 cache1006allows low-latency accesses to cache memory into the scalar and vector units. While in one embodiment (to simplify the design), a scalar unit1008and a vector unit1010use separate register sets (respectively, scalar registers1012and vector registers1014) and data transferred between them is written to memory and then read back in from a level 1 (L1) cache1006, alternative embodiments of the invention may use a different approach (e.g., use a single register set or include a communication path that allow data to be transferred between the two register files without being written and read back).

FIG.9Bis an expanded view of part of the processor core inFIG.9Aaccording to embodiments of the invention.FIG.9Bincludes an L1 data cache1006A part of the L1 cache1006, as well as more detail regarding the vector unit1010and the vector registers1014. Specifically, the vector unit1010is a 16-wide vector processing unit (VPU) (see the 16-wide ALU1028), which executes one or more of integer, single-precision float, and double-precision float instructions. The VPU supports swizzling the register inputs with swizzle unit1020, numeric conversion with numeric convert units1022A-B, and replication with replication unit1024on the memory input. Write mask registers1026allow predicating resulting vector writes.

FIG.10is a block diagram of a processor1100that may have more than one core, may have an integrated memory controller, and may have integrated graphics according to embodiments of the invention. The solid lined boxes inFIG.10illustrate a processor1100with a single core1102A, a system agent1110, a set of one or more bus controller units1116, while the optional addition of the dashed lined boxes illustrates an alternative processor1100with multiple cores1102A-N, a set of one or more integrated memory controller unit(s)1114in the system agent unit1110, and special purpose logic1108.

The memory hierarchy includes one or more levels of respective caches1104A-N within the cores1102A-N, a set or one or more shared cache units1106, and external memory (not shown) coupled to the set of integrated memory controller units1114. The set of shared cache units1106may include one or more mid-level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of cache, a last level cache (LLC), and/or combinations thereof. While in one embodiment a ring based interconnect unit1112interconnects the integrated graphics logic1108, the set of shared cache units1106, and the system agent unit1110/integrated memory controller unit(s)1114, alternative embodiments may use any number of well-known techniques for interconnecting such units. In one embodiment, coherency is maintained between one or more cache units1106and cores1102-A-N.

In some embodiments, one or more of the cores1102A-N are capable of multi-threading. The system agent1110includes those components coordinating and operating cores1102A-N. The system agent unit1110may include for example a power control unit (PCU) and a display unit. The PCU may be or include logic and components needed for regulating the power state of the cores1102A-N and the integrated graphics logic1108. The display unit is for driving one or more externally connected displays.

Exemplary Computer Architectures

Referring now toFIG.11, shown is a block diagram of a system1200in accordance with one embodiment of the present invention. The system1200may include one or more processors1210,1215, which are coupled to a controller hub1220. In one embodiment the controller hub1220includes a graphics memory controller hub (GMCH)1290and an Input/Output Hub (IOH)1250(which may be on separate chips); the GMCH1290includes memory and graphics controllers to which are coupled memory1240and a coprocessor1245; the IOH1250couples input/output (I/O) devices1260to the GMCH1290. Alternatively, one or both of the memory and graphics controllers are integrated within the processor (as described herein), the memory1240and the coprocessor1245are coupled directly to the processor1210, and the controller hub1220in a single chip with the IOH1250.

The optional nature of additional processors1215is denoted inFIG.11with broken lines. Each processor1210,1215may include one or more of the processing cores described herein and may be some version of the processor1100.

The memory1240may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of the two. For at least one embodiment, the controller hub1220communicates with the processor(s)1210,1215via a multi-drop bus, such as a frontside bus (FSB), point-to-point interface such as QuickPath Interconnect (QPI), or similar connection1295.

In one embodiment, the coprocessor1245is a special-purpose processor, such as, for example, a high-throughput MIC processor, a network or communication processor, compression engine, graphics processor, GPGPU, embedded processor, or the like. In one embodiment, controller hub1220may include an integrated graphics accelerator.

There can be a variety of differences between the physical resources1210,1215in terms of a spectrum of metrics of merit including architectural, microarchitectural, thermal, power consumption characteristics, and the like.

In one embodiment, the processor1210executes instructions that control data processing operations of a general type. Embedded within the instructions may be coprocessor instructions. The processor1210recognizes these coprocessor instructions as being of a type that should be executed by the attached coprocessor1245. Accordingly, the processor1210issues these coprocessor instructions (or control signals representing coprocessor instructions) on a coprocessor bus or other interconnect, to coprocessor1245. Coprocessor(s)1245accept and execute the received coprocessor instructions.

Referring now toFIG.12, shown is a block diagram of a first more specific exemplary system1300in accordance with an embodiment of the present invention. As shown inFIG.12, multiprocessor system1300is a point-to-point interconnect system, and includes a first processor1370and a second processor1380coupled via a point-to-point interconnect1350. Each of processors1370and1380may be some version of the processor1100. In one embodiment of the invention, processors1370and1380are respectively processors1210and1215, while coprocessor1338is coprocessor1245. In another embodiment, processors1370and1380are respectively processor1210coprocessor1245.

Processors1370and1380are shown including integrated memory controller (IMC) units1372and1382, respectively. Processor1370also includes as part of its bus controller units point-to-point (P-P) interfaces1376and1378; similarly, second processor1380includes P-P interfaces1386and1388. Processors1370,1380may exchange information via a point-to-point (P-P) interface1350using P-P interface circuits1378,1388. As shown inFIG.12, IMCs1372and1382couple the processors to respective memories, namely a memory1332and a memory1334, which may be portions of main memory locally attached to the respective processors.

Processors1370,1380may each exchange information with a chipset1390via individual P-P interfaces1352,1354using point to point interface circuits1376,1394,1386,1398. Chipset1390may optionally exchange information with the coprocessor1338via a high-performance interface1339and an interface1392. In one embodiment, the coprocessor1338is a special-purpose processor, such as, for example, a high-throughput MIC processor, a network or communication processor, compression engine, graphics processor, GPGPU, embedded processor, or the like.

Chipset1390may be coupled to a first bus1316via an interface1396. In one embodiment, first bus1316may be a Peripheral Component Interconnect (PCI) bus, or a bus such as a PCI Express bus or another third generation I/O interconnect bus, although the scope of the present invention is not so limited.

As shown inFIG.12, various I/O devices1314may be coupled to first bus1316, along with a bus bridge1318which couples first bus1316to a second bus1320. In one embodiment, one or more additional processor(s)1315, such as coprocessors, high-throughput MIC processors, GPGPU's, accelerators (such as, e.g., graphics accelerators or digital signal processing (DSP) units), field programmable gate arrays, or any other processor, are coupled to first bus1316. In one embodiment, second bus1320may be a low pin count (LPC) bus. Various devices may be coupled to a second bus1320including, for example, a keyboard and/or mouse1322, communication devices1327and a storage unit1328such as a disk drive or other mass storage device which may include instructions/code and data1330, in one embodiment. Further, an audio I/O1324may be coupled to the second bus1320. Note that other architectures are possible. For example, instead of the point-to-point architecture ofFIG.12, a system may implement a multi-drop bus or other such architecture.

Referring now toFIG.13, shown is a block diagram of a second more specific exemplary system1400in accordance with an embodiment of the present invention. Like elements inFIGS.12and13bear like reference numerals, and certain aspects ofFIG.12have been omitted fromFIG.13in order to avoid obscuring other aspects ofFIG.13.

FIG.13illustrates that the processors1370,1380may include integrated memory and I/O control logic (“CL”)1472and1482, respectively. Thus, the CL1472,1482include integrated memory controller units and include I/O control logic.FIG.13illustrates that not only are the memories1332,1334coupled to the CL1472,1482, but also that I/O devices1414are also coupled to the control logic1472,1482. Legacy I/O devices1415are coupled to the chipset1390.

Referring now toFIG.14, shown is a block diagram of a SoC1500in accordance with an embodiment of the present invention. Similar elements inFIG.10bear like reference numerals. Also, dashed lined boxes are optional features on more advanced SoCs. InFIG.14, an interconnect unit(s)1502is coupled to: an application processor1510which includes a set of one or more cores1102A-N and shared cache unit(s)1106; a system agent unit1110; a bus controller unit(s)1116; an integrated memory controller unit(s)1114; a set or one or more coprocessors1520which may include integrated graphics logic, an image processor, an audio processor, and a video processor; an static random access memory (SRAM) unit1530; a direct memory access (DMA) unit1532; and a display unit1540for coupling to one or more external displays. In one embodiment, the coprocessor(s)1520include a special-purpose processor, such as, for example, a network or communication processor, compression engine, GPGPU, a high-throughput MIC processor, embedded processor, or the like.

FIG.15is a block diagram contrasting the use of a software instruction converter to convert binary instructions in a source instruction set to binary instructions in a target instruction set according to embodiments of the invention. In the illustrated embodiment, the instruction converter is a software instruction converter, although alternatively the instruction converter may be implemented in software, firmware, hardware, or various combinations thereof.FIG.15shows a program in a high level language1602may be compiled using an x86 compiler1604to generate x86 binary code1606that may be natively executed by a processor with at least one x86 instruction set core1616. The processor with at least one x86 instruction set core1616represents any processor that can perform substantially the same functions as an Intel processor with at least one x86 instruction set core by compatibly executing or otherwise processing (1) a substantial portion of the instruction set of the Intel x86 instruction set core or (2) object code versions of applications or other software targeted to run on an Intel processor with at least one x86 instruction set core, in order to achieve substantially the same result as an Intel processor with at least one x86 instruction set core. The x86 compiler1604represents a compiler that is operable to generate x86 binary code1606(e.g., object code) that can, with or without additional linkage processing, be executed on the processor with at least one x86 instruction set core1616. Similarly,FIG.15shows the program in the high level language1602may be compiled using an alternative instruction set compiler1608to generate alternative instruction set binary code1610that may be natively executed by a processor without at least one x86 instruction set core1614(e.g., a processor with cores that execute the MIPS instruction set of MIPS Technologies of Sunnyvale, CA and/or that execute the ARM instruction set of ARM Holdings of Sunnyvale, CA). The instruction converter1612is used to convert the x86 binary code1606into code that may be natively executed by the processor without an x86 instruction set core1614. This converted code is not likely to be the same as the alternative instruction set binary code1610because an instruction converter capable of this is difficult to make; however, the converted code will accomplish the general operation and be made up of instructions from the alternative instruction set. Thus, the instruction converter1612represents software, firmware, hardware, or a combination thereof that, through emulation, simulation or any other process, allows a processor or other electronic device that does not have an x86 instruction set processor or core to execute the x86 binary code1606.

ADDITIONAL NOTES AND EXAMPLES

Techniques and architectures for per process or per domain granularity resource control for an IOMMU are described herein. In the above description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of certain embodiments. It will be apparent, however, to one skilled in the art that certain embodiments can be practiced without these specific details. In other instances, structures and devices are shown in block diagram form in order to avoid obscuring the description.