Patent Description:
GPUs exploit large amounts of thread-level parallelism to achieve high instruction throughput. This high throughput has helped make GPUs an essential computational resource in many different paradigms. In many types of computing environments, multiple applications share a single processing unit. Executing simultaneous multiple applications from one or more Operating Systems requires various support mechanisms in hardware and in software. One key mechanism is virtual memory, which manages and protects the address space of each application running on the machine. However, modern GPUs lack support for parallel execution of multiple applications. As a result, GPU suffer from high performance overheads when running multiple concurrent applications. <CIT> relates to techniques for allocating resources to a task from a shared hardware structure.

In the following description, numerous specific details are set forth to provide a thorough understanding of the methods and mechanisms presented herein. However, one having ordinary skill in the art should recognize that the various implementations may be practiced without these specific details. In some instances, well-known structures, components, signals, computer program instructions, and techniques have not been shown in detail to avoid obscuring the approaches described herein. For example, the dimensions of some of the elements may be exaggerated relative to other elements.

Various systems, apparatuses, and methods for abstracting tasks in virtual memory identifier (VMID) containers are disclosed herein. A processor coupled to a memory executes a plurality of concurrent tasks including a first task. Responsive to detecting one or more instructions of the first task which correspond to a first operation, the processor retrieves a first identifier (ID) which is used to uniquely identify the first task, wherein the first ID is transparent to the first task. Then, the processor maps the first ID to a second ID and/or a third ID. The processor completes the first operation by using the second ID and/or the third ID to identify the first task to at least a first data structure. In one implementation, the first operation is a memory access operation and the first data structure is a set of page tables. Also, in one implementation, the second ID identifies a first application of the first task and the third ID identifies a first operating system (OS) of the first task.

Referring now to <FIG>, a block diagram of one implementation of a computing system <NUM> is shown. In one implementation, computing system <NUM> includes at least processors 105A-N, input/output (I/O) interfaces <NUM>, bus <NUM>, memory controller(s) <NUM>, network interface controller (NIC) <NUM>, and memory device(s) <NUM>. In other implementations, computing system <NUM> includes other components and/or computing system <NUM> is arranged differently. Processors 105A-N are representative of any number of processors which are included in system <NUM>. In one implementation, processor 105A is a general purpose processor, such as a central processing unit (CPU). In one implementation, processor 105N is a data parallel processor with a highly parallel architecture. Data parallel processors include graphics processing units (GPUs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and so forth. In some implementations, processors 105A-N include multiple data parallel processors.

Memory controller(s) <NUM> are representative of any number and type of memory controllers accessible by processors 105A-N and I/O devices (not shown) coupled to I/O interfaces <NUM>. Memory controller(s) <NUM> are coupled to any number and type of memory devices(s) <NUM>. Memory device(s) <NUM> are representative of any number and type of memory devices. For example, the type of memory in memory device(s) <NUM> includes Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), NAND Flash memory, NOR flash memory, Ferroelectric Random Access Memory (FeRAM), or others.

I/O interfaces <NUM> are representative of any number and type of I/O interfaces (e.g., peripheral component interconnect (PCI) bus, PCI-Extended (PCI-X), PCIE (PCI Express) bus, gigabit Ethernet (GBE) bus, universal serial bus (USB)). Various types of peripheral devices (not shown) are coupled to I/O interfaces <NUM>. Such peripheral devices include (but are not limited to) displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, and so forth. Network interface controller (NIC) <NUM> receives and sends network messages across network <NUM>.

Network <NUM> is representative of any type of network or combination of networks, including wireless connection, direct local area network (LAN), metropolitan area network (MAN), wide area network (WAN), an Intranet, the Internet, a cable network, a packet-switched network, a fiber-optic network, a router, storage area network, or other type of network. Examples of LANs include Ethernet networks, Fiber Distributed Data Interface (FDDI) networks, and token ring networks. In various implementations, network <NUM> further includes remote direct memory access (RDMA) hardware and/or software, transmission control protocol/internet protocol (TCP/IP) hardware and/or software, router, repeaters, switches, grids, and/or other components.

In various implementations, computing system <NUM> is a computer, laptop, mobile device, game console, server, streaming device, wearable device, or any of various other types of computing systems or devices. It is noted that the number of components of computing system <NUM> varies from implementation to implementation. For example, in other implementations, there are more or fewer of each component than the number shown in <FIG>. It is also noted that in other implementations, computing system <NUM> includes other components not shown in <FIG>. Additionally, in other implementations, computing system <NUM> is structured in other ways than shown in <FIG>.

Turning now to <FIG>, a block diagram of another implementation of a computing system <NUM> is shown. In one implementation, system <NUM> includes GPU <NUM>, system memory <NUM>, and local memory <NUM>. System <NUM> also includes other components which are not shown to avoid obscuring the figure. GPU <NUM> includes at least command processor <NUM>, scheduling unit <NUM>, compute units 255A-N, memory controller <NUM>, global data share <NUM>, level one (L1) cache <NUM>, and level two (L2) cache <NUM>. Although not shown in <FIG>, in one implementation, compute units 255A-N also include one or more caches and/or local memories within each compute unit 255A-N. In other implementations, GPU <NUM> includes other components, omits one or more of the illustrated components, has multiple instances of a component even if only one instance is shown in <FIG>, and/or is organized in other suitable manners.

In various implementations, computing system <NUM> executes any of various types of software applications. In one implementation, as part of executing a given software application, a host CPU (not shown) of computing system <NUM> launches tasks to be executed on GPU <NUM>. Command processor <NUM> receives tasks from the host CPU and issues tasks to scheduling unit <NUM> for scheduling on compute units 255A-N. In one implementation, when scheduling unit <NUM> schedules a given task on compute units 255A-N, scheduling unit <NUM> generates a unique container identifier (ID) for the given task by hashing together an operating system (OS) or virtual function (VF) ID with a virtual memory (VM) ID of the given task. Threads within tasks executing on compute units 255A-N access various data structures internal and external to computing system <NUM>. When a thread accesses a data structure external to computing system <NUM>, the container ID is mapped to a VFID and/or VMID, with the VFID and/or VMID used to identify the thread to the external data structure.

Turning now to <FIG>, a block diagram of one implementation of the virtual environment <NUM> of a GPU is shown. A GPU is a massively parallel machine which is able to support multiple concurrent tasks. GPUs are employed to handle graphics and/or compute workloads in a variety of applications. For graphics workloads, the GPU includes a deep, mixed pipeline which includes fixed function hardware and programmable shaders. A typical workload source hierarchy for GPUs includes the levels of operating system (OS) and virtual function (VF) 310A-B, applications 308A-N, queue(s) 306A, draw calls, and dispatches. Each queue 306A includes any number of graphics task(s) 302A and/or compute task(s) 304A. When supporting multi-tasking, there are potentially multiple workloads from various sources that are active on the GPU concurrently.

When executing multiple different types of concurrent applications on a GPU, each different workload is tagged to identify and manage the workload to implement various functions. For example, each different workload is tagged for synchronization and reset purposes, to implement memory management techniques and structures, to manage interrupts, and to implement other functionality. In one implementation, each different workload is identified using a container ID to differentiate the workload from the other workloads. In one implementation, the container ID is created from a hash of the OS or VF ID and the VMID. This allows the GPU to support multiple concurrent queues, applications, draws, and dispatches from different guest OS's. In other implementations, the container ID is created from other types of functions and/or from other types of values.

While several of the discussions presented herein identify characteristics of a GPU, it should be understood that the same techniques apply to other types of processors with parallel execution capabilities (e.g., multi-core CPUs, FPGAs, ASICs, DSPs). Accordingly, throughout this disclosure, when a technique is described as being performed by or implemented on a GPU, it should be understood that the technique is also able to be performed on other types of processors.

Turning now to <FIG>, a block diagram of one implementation of a GPU <NUM> is shown. In one implementation, GPU <NUM> includes graphics pipeline <NUM> coupled to memory subsystem <NUM>. In one implementation, graphics pipeline <NUM> includes multi-tasking support in each pipeline stage. Frontends <NUM> includes various queues for storing incoming tasks which have been forwarded to graphics pipeline <NUM>. Geometry engine <NUM> performs rendering of primitives using draw call commands for the different tasks being executed on graphics pipeline <NUM>. Shader engine <NUM> implements the various shader stages involved in rendering graphics. In one implementation, pixel engine <NUM> is invoked to compute output information and cause results to be written to output surfaces after an image being rendered is divided into a grid of bins or tiles in screen space. In some implementations, pixel engine <NUM> calculates the values of vertex attributes that are to be interpolated across a rasterized object. In other implementations, graphics pipeline <NUM> includes other stages or engines and/or the individual engines perform other types of operations.

Typically, a GPU includes pipelined support for multi-tasking to handle the order and dependency requirements in each stage. This refers to each stage of graphics pipeline <NUM> executing a separate task (e.g., task 450A on geometry engine <NUM>, task 450B on shader engine <NUM>, task 450C on pixel engine <NUM>). In one implementation, graphics pipeline <NUM> is configured to support independent and concurrent graphics tasks. In this implementation, the source hierarchy details are obscured from each stage in the GPU. The support for independent and concurrent graphics tasks of graphics pipeline is supported by having a container ID used to distinguish each task from the other concurrent tasks which are executing on the stage.

Referring now to <FIG>, a block diagram of one implementation of a processor <NUM> with a compute pipeline <NUM> is shown. In one implementation, processor <NUM> includes at least compute pipeline <NUM> coupled to memory subsystem <NUM>. In one implementation, processor <NUM> is a GPU. In other implementations, processor <NUM> is any of various other types of processing units (e.g., FPGA, ASIC, DSP, multi-core CPU).

Compute pipeline <NUM> includes frontends <NUM> coupled to shader engines <NUM>. Compute pipeline <NUM> includes vertical and/or horizontal partitioning to enable concurrent execution of tasks 525A-D. For example, task 525A is launched onto shader engines <NUM> in one implementation, followed by other tasks in subsequent clock cycles. In another implementation, tasks 525B-D are launched and executed concurrently on shader engines <NUM>. Shader engines <NUM> are partitioned vertically in this implementation to enable the concurrent execution of multiple tasks. The vertical partitioning refers to the ability of the processing units and processing logic of shader engines <NUM> to maintain state of multiple tasks and/or operate on multiple tasks in the same clock cycle. In one implementation, the separate tasks are identified using container IDs, with each different task having a unique container ID.

In one implementation, each stage of the compute pipeline <NUM> is vertically partitioned. For example, frontends <NUM> includes queues which are partitioned to allow multiple different tasks to be launched and run concurrently. Also, shader engines <NUM> include vertical partitions to allow multiple compute tasks to execute concurrently. Frontends <NUM> and shader engines <NUM> are able to execute workloads from numerous sources simultaneously. In one implementation, each stage or partition in compute pipeline <NUM> is unaware of the source hierarchy of the tasks when executing these individual tasks. In this implementation, the source hierarchy details are only utilized when boundaries of processor <NUM> are traversed by a task.

Turning now to <FIG>, a block diagram of one implementation of abstracting tasks in virtual memory identifier (VMID) containers is shown. In one implementation, VMIDs <NUM> are typically used for memory management operations to differentiate applications which are executing on a processor. However, in another implementation, task container IDs <NUM> are used in place of VMIDs <NUM> to abstract the source hierarchy of executing tasks. In various implementations, the processor does not need to discern the source of a task during execution. Rather, the processor only needs to discern the source when interacting with external components or external data structures. In one implementation, the container IDs <NUM> are used to bundle memory data structures. In this implementation, the processor provides memory protection by not allowing one container ID to access the memory contents of another container ID.

In one implementation, container IDs <NUM> are managed by hardware mechanisms and are transparent to the software hierarchy above the hardware execution level. The rest of the software hierarchy continues using existing mechanisms of source tracking. These existing mechanisms include OS ID, process ID, queue ID, and so on. In one implementation, task tagging and source hierarchy are relevant only at boundaries from the processor to external components. For example, synchronization tasks such as end-of-pipe and reset operations will reference the source of a task. Also, memory transactions outside of the processor will use the source of a task to map to the correct memory management data structures (e.g., page tables). Additionally, interrupt handling is tracked on a per source basis to identify which of the currently executing tasks to interrupt. Still further, peripheral component interconnect express (PCI-e) bus-device-function resolution is tracked on a per source basis to determine the source of a virtual or physical function. Other types of operations which are performed at the boundaries of the processor to track the source of a task are possible and are contemplated.

In one implementation, a mapping table <NUM> is maintained at each boundary between the processor and external components. The mapping table maps the container ID <NUM> of the task to operating system (OS) ID or virtual function (VF) ID <NUM>, process ID, or otherwise. Using a container ID <NUM> to identify a source of a task enables concurrent draws and dispatches, queues, applications, virtual functions, and OS's to execute on the processor. Entries <NUM>, <NUM>, <NUM>, and <NUM> are representative of any number of entries in mapping table <NUM>. In one implementation, mapping table <NUM> has <NUM> entries while in other implementations, mapping table <NUM> includes other numbers of entries.

Referring now to <FIG>, a block diagram of one implementation of a computing system <NUM> is shown. In one implementation, computing system <NUM> uses container IDs to track tasks in a multi-task concurrent execution environment. In order to use container IDs to track tasks, system <NUM> includes multiple mapping tables (MTs) 725A-D to map container IDs to the source hierarchy of the tasks. Each MT 725A-D includes any number of entries, with the number of entries varying from implementation to implementation.

In one implementation, system <NUM> includes system on chip (SoC) <NUM> coupled to memory subsystem <NUM> and input/output (I/O) devices 740A-B. I/O devices 740A-B are representative of any number and type of peripheral devices. It is noted that in other implementations, system <NUM> also includes other components which are not shown to avoid obscuring the figure. SoC <NUM> includes at least processor cores 710A-N, I/O fabric <NUM>, memory management unit <NUM>, and interrupt controller <NUM>. In one implementation, I/O fabric <NUM> includes mapping tables 725A-B which correspond to I/O devices 740A-B. Also, MMU <NUM> and interrupt controller <NUM> include mapping tables 725C-D, respectively. Mapping table 725C is used to map container IDs to the source hierarchy of tasks to enable access to the appropriate sets of page tables while mapping table 725D is used to map container IDs to corresponding sources for delivering interrupts. In one implementation, the different MTs 725A-D are synchronized so that the entries of each MT match the entries of the other MTs. Accordingly, in this implementation, when an existing entry is evicted from a given MT 725A-D to make room for a new entry for a different container ID, then the other MTs 725A-D are notified and updated to match the given MT 725A-D.

Turning now to <FIG>, one implementation of a mapping table <NUM> for mapping container IDs to OS or VF IDs is shown. In one implementation, a processor (e.g., GPU <NUM> of <FIG>) includes one or more copies of mapping table <NUM> for mapping container IDs to OS/VF IDs and VMIDs. Mapping table <NUM> includes container ID field <NUM>, OS/VF ID field <NUM>, and VMID field <NUM>. In other implementations, mapping table <NUM> includes other fields and/or is organized differently. The container ID of a given task is used to perform a lookup of mapping table <NUM> to find a matching OS/VF ID and VMID for the given task.

When the source of a task needs to be identified to access a given data structure or to interact with an external component, the processor uses mapping table <NUM> to resolve a given container ID to an OS or VF ID and VMID of the task. The OS or VF ID and/or VMID is then used to identify the source of a task to complete a particular type of operation. For example, a virtual-to-physical address translation is performed by accessing a specific set of page tables for a given container ID. In another example, a particular task is identified for determining where to deliver an interrupt by using mapping table <NUM> to resolve a container ID to an OS/VF ID and a VMID.

In one implementation, mapping table <NUM> has a limited number of entries. If mapping table <NUM> is full and a new task is initiated on the processor, then the processor will evict, suspend or wait for completion of one of the existing entries from mapping table <NUM> to make room for a new entry for the new task. In some implementations, multiple copies of mapping table <NUM> are maintained by the processor, with one copy of mapping table <NUM> stored at each boundary point of the processor. In these implementations, the processor synchronizes the mapping tables to ensure they have the same entries for the various container IDs of the currently executing tasks.

Referring now to <FIG>, one implementation of a method <NUM> for abstracting tasks using container IDs is shown. For purposes of discussion, the steps in this implementation and those of <FIG> are shown in sequential order. However, it is noted that in various implementations of the described methods, one or more of the elements described are performed concurrently, in a different order than shown, or are omitted entirely. Other additional elements are also performed as desired. Any of the various systems or apparatuses described herein are configured to implement method <NUM>.

A processor executes a first task and one or more other tasks concurrently (block <NUM>). In one implementation, the processor is a GPU. While executing the first task, the processor detects one or more instructions which correspond to a first operation (block <NUM>). In one implementation, the first operation is a memory access operation. In another implementation, the first operation is an operation which targets a component external to the processor. In a further implementation, the first operation is an operation which requires the source of the first task to be identified. In a still further implementation, the first operation is a graphics operation which involves rendering one or more pixels for display. In this implementation, the processor generates one or more pixels to drive to a display as part of the first operation. In other implementations, the first operation is any of various other types of operations.

In response to detecting the one or more instructions of the first task which correspond to the first operation, the processor receives a first identifier (ID) which uniquely identifies the first task, wherein the first ID does not identify a source hierarchy of the first task (block <NUM>). In other words, the first ID is transparent to the first task. In one implementation, the first ID is a container ID. Next, the processor maps the first ID to a second ID which identifies a source hierarchy of the first task (block <NUM>). In another implementation, the processor maps the first ID to a second ID and to a third ID which together identify the source hierarchy of the first task. In this implementation, the second ID identifies a first application of the first task and the third ID identifies a first operating system (OS) of the first task. In this implementation, the source hierarchy refers to an application, OS, and/or virtual function (VF) of the first task. In comparison, the first ID abstracts the source hierarchy of the first task.

Next, the processor completes the first operation by performing an access to a first data structure using the second ID to identify the first task (block <NUM>). In one implementation, the first data structure is a set of virtual to physical address translation page tables. In another implementation, the first data structure is an interrupt table. After block <NUM>, method <NUM> ends. It is noted that method <NUM> is able to be performed in parallel for multiple tasks which are being executed concurrently on the processor. These multiple tasks include tasks from two or more different guest OS's.

Turning now to <FIG>, one implementation of a method <NUM> for a scheduler generating container IDs for tasks is shown. A scheduler of a processor (e.g., GPU <NUM> of <FIG>) receives a task to be scheduled on one or more compute units of the processor (block <NUM>). The scheduler retrieves a virtual function (VF) ID and a virtual memory (VM) ID associated with the task (block <NUM>). In another implementation, the scheduler retrieves an operating system (OS) ID associated with the task. Next, the scheduler creates a container ID for the task from the VFID and VMID associated with the task, wherein the container ID uniquely identifies the task (block <NUM>). In another implementation, the scheduler creates a unique container ID for the task from the OS ID and VMID associated with the task. In one implementation, the scheduler creates the container ID by generating a hash of the VFID (or OS ID) with the VMID. Then, the scheduler tags the task with the unique container ID (block <NUM>). In other words, the task is now associated with the unique container ID rather than being associated with the VFID, OS ID, or VMID of the task. This allows the processor to support multiple concurrent tasks from different guest OS's or even from the same OS.

Next, the scheduler schedules the task for execution on one or more compute units responsive to determining the task is ready (block <NUM>). During execution of the task, the processor accesses one or more data structures by mapping the container ID to a VFID and/or VMID (block <NUM>). Alternatively, the processor maps the container ID to an OS ID and/or VMID in block <NUM>. Depending on the implementation, the one or more data structures include page tables, interrupt tables, and/or other data structures. After block <NUM>, method <NUM> ends. It is noted that in one implementation, method <NUM> is performed in parallel for multiple tasks which are being executed concurrently on the processor. These multiple tasks include tasks from two or more different guest OS's.

In various implementations, program instructions of a software application are used to implement the methods and/or mechanisms described herein. For example, program instructions executable by a general or special purpose processor are contemplated. In various implementations, such program instructions are represented by a high level programming language. In other implementations, the program instructions are compiled from a high level programming language to a binary, intermediate, or other form. Alternatively, program instructions are written that describe the behavior or design of hardware. Such program instructions are represented by a high-level programming language, such as C. Alternatively, a hardware design language (HDL) such as Verilog is used. In various implementations, the program instructions are stored on any of a variety of nontransitory computer readable storage mediums. The storage medium is accessible by a computing system during use to provide the program instructions to the computing system for program execution. Generally speaking, such a computing system includes at least one or more memories and one or more processors configured to execute program instructions.

Claim 1:
A graphics processing unit (<NUM>) comprising:
a scheduler unit (<NUM>); and
a plurality of compute units (255A-255N);
wherein the graphics processing unit, GPU, is configured to:
receive, from a central processing unit, a first task and a second task to be scheduled for execution on the GPU;
retrieve an original identifier, ID, of the first task that identifies at least a portion of a source hierarchy associated with the first task, and an original identifier of the second task that identifies at least a portion of a source hierarchy associated with the second task;
create a first ID for the first task and a second ID for the second task, where the first ID and second ID uniquely identify each respective task without identifying the at least a portion of the source hierarchy associated with each respective task;
execute the first task and the second task (<NUM>) concurrently on the plurality of compute units (255A-255N) using the first ID and the second ID;
responsive to detecting one or more instructions of the first task which correspond to a first operation (<NUM>) and determining the first operation is an operation which requires a source of the first task to be identified:
receive the first ID for the first task;
map the first ID to the original ID of the first task which identifies the at least a portion of the source hierarchy of the first task (<NUM>); and
complete the first operation by performing an access to a first data structure using the original ID of the first task to identify the first task (<NUM>).