Patent ID: 12254527

DETAILED DESCRIPTION

The overall framework of the fixed function hardware blocks defines the shape of a conventional graphics pipeline and determines a maximal throughput of the graphics pipeline. For example, the maximal throughput is typically determined by a subset of the hardware blocks that are bottlenecks for the processing flow. Different applications can generate bottlenecks at different blocks in the graphics pipeline, which can lead to pipeline imbalances that result in some stages (or hardware blocks) idling without any tasks to perform while other stages (or hardware blocks) are bottlenecks because they are unable to keep up with their assigned tasks. Furthermore, new algorithms for scene generation and new requirements to display image fidelity are likely to completely change the distribution of processing bottlenecks across multistage graphics pipelines.

Furthermore, processing objects in a graphics pipeline that is defined by the fixed function hardware blocks reduces the flexibility of the graphics pipeline and can lead to redundant processing that unnecessarily consumes resources of the graphics pipeline. For example, virtual reality techniques such as light field rendering or holographic rendering can require rendering portions of a scene from thousands of different perspectives. Each object must be processed through the entire graphics pipeline for each perspective. However, processing of objects in the programmable shaders in the geometry front-end, the fixed function tessellator, and other programmable shaders or fixed function hardware are independent of the rendering perspective. Consequently, operations of these shaders or fixed function hardware are unnecessarily repeated during rendering of each of the different perspectives. For another example, foveation is used to compress information that represents rendered images provided to different eyes by reducing the image resolution at larger distances from the points of gaze of the eyes. Processing the same object through the entire conventional graphics pipeline once for each eye (i.e., once for each of the different points of gaze) unnecessarily duplicates operations of many of the programmable shaders and fixed function hardware in the graphics pipeline.

Virtualization of a graphics processing unit (GPU) so that it functions as a shared resource is conventionally used to support a multi-user environment in workstations and data centers. In software-based virtualization technologies, graphics device drivers on client machines (instead of the embedded GPUs) communicate with special hypervisor software that manages access to the plurality of shared or virtualized GPUs. Shared virtual GPUs could be implemented remotely (e.g., in data centers) or locally (e.g., in desktop workstations). An example of an industry technology for sharing virtual GPUs (vGPUs) across multiple virtual desktop and applications instances is GPU NVIDIA GRID™. The AMD Multiuser GPU also works with hypervisor software to provide ease of installation of client virtual graphics support environment. Unlike the pure software virtualization approach implemented in the NVIDIA GPU, AMD's hardware-based virtualization solution makes it even more difficult for a hacker to break in at the hardware level.

All current GPUs used in virtual shared mode suffer from significant overhead on user/context switch between different clients that are sharing the resources of the GPUs. In some cases, the overhead becomes comparable to the resources consumed by processing runs due to the significant increase of computational power. Consequently, relying on a fixed configuration of GPUs that operates as a single device may cause performance/power inefficiency when it is shared between clients with different processing profiles. As discussed herein, GPU virtualization can be extended to overcome such problems. For example, GPU complexity is increasing significantly and the available physical resources of the GPU can include billions of transistor blocks, which requires moving beyond conventional device-level virtualization towards internal GPU block-level virtualization. Reconfigurable GPUs with virtualized pipelines components, such as described herein, can support numerous different processing configurations that provide optimal power/performance for different execution profiles of different virtual clients that are sharing the same physical GPU device. Providing the user with the capability to shape a virtual pipeline by defining the components and configuration enables the resources of the GPU to be shared by different tasks with user-defined dispatch and synchronization. The reconfigurable GPU with virtualized pipeline components is therefore a significant change in the conventional GPU usage paradigm for both graphics and compute applications.

Physical resources of a graphics processing unit (GPU) such as shader engines and fixed function hardware units are used to implement user-defined reconfigurable virtual pipelines that share the physical resources of the GPU. Each virtual pipeline is fed via one or more queues that hold commands that are to be executed in the virtual pipeline and a context that defines the operational state of the virtual pipeline. Some embodiments of the queues are implemented as ring buffers using a head pointer and a tail pointer. The commands include draw commands and compute commands. The draw commands include state information or geometry data including information associated with vertices of primitives. The compute commands include kernel code or a reference (such as a pointer or an index) to code, arguments, barriers, and the like.

Virtual pipelines are composed of user-defined reconfigurable fragments including a super-pipe fragment (SPF), a meta-pipe fragment (MPF), and one or more virtual pipe fragments (VPFs) that represent configured shaders and fixed function hardware or, in some embodiments, emulations of fixed function hardware. The SPF implements an upper-level state machine that is used to dispatch/manage multiple queues of command packets for the virtual pipeline, as well as the interaction between application threads and the physical resources that are allocated to the virtual pipeline via an operating system (OS) or low level driver (LLD). The MPF fetches command packets from the queue or, in the case of an indirect fetch, the MPF fetches a pointer or an index from the queue that indicates another location that stores the command packet. After dereferencing of the command and data flows, the MPF provides the retrieved commands and data to virtual pipeline. Each VPF implements user-configurable functionality using an allocated set of physical resources of the GPU such as shader engines, compute units, micro-engine cores, fixed function hardware units, and the like. The VPFs can also be mapped to memory hierarchy resources in the GPU. The physical resources that are available for allocation to the VPFs are referred to as physical processing pipe fragments (PPFs), which include processing resources and associated buffers or interfaces. Any number of VPFs can be chained together and configured to form the virtual pipeline based on requirements of the application or thread that is to be executed using the virtual pipeline.

The reconfigurable graphics pipeline can be shaped as one single powerful virtual graphics pipeline or multiple virtual graphics pipelines of different configurations that operate concurrently using the same pool of shared graphics processing resources. For example, the processing resources of a unified shader pool, such as multiple graphics processing cores, can be allocated as SPFs, MPFs, VPF and PPFs to support a plurality of virtual pipelines. Dynamic reconfiguration of the graphics pipeline can also be used to alleviate bottlenecks in the processing flow. In some embodiments, fixed function hardware becomes a bottleneck in the virtual pipeline, in which case one or more VPFs can be used to emulate the functionality of the fixed function hardware to provide additional processing resources to unclog the bottleneck and avoid idling of other portions of the graphics pipeline. Arbitration is used to decide whether to process objects using the fixed function hardware or the emulation and provide the ability to share PPFs between multiple virtual pipelines.

FIG.1depicts a first example a state-of-the-art graphics processing system compliant with graphics API specifications. The first example graphics processing system includes a graphics pipeline100that is capable of processing high-order geometry primitives to generate rasterized images of three-dimensional (3-D) scenes at a predetermined resolution. The graphics pipeline100has access to storage resources101such as a hierarchy of one or more memories or caches that are used to implement buffers and store vertex data, texture data, and the like. An input assembler102is configured to access information from the storage resources101that is used to define objects that represent portions of a model of a scene. A vertex shader103, which can be implemented in software, logically receives a single vertex of a primitive as input and outputs a single vertex. Some embodiments of shaders such as a vertex shader103implement massive single-instruction-multiple-data (SIMD) processing so that multiple vertices can be processed concurrently. The graphics pipeline100implements the concept of unified shader model so that all the shaders included in the graphics pipeline100have the same execution platform on the shared massive SIMD compute units. The shaders, including the vertex shader103, are therefore implemented using a common set of resources that is referred to herein as the unified shader pool104. A hull shader105operates on input high-order patches or control points that are used to define the input patches. The hull shader105outputs tessellation factors and other patch data.

Primitives generated by the hull shader105can optionally be provided to a tessellator106. The tessellator106receives objects (such as patches) from the hull shader105and generates information identifying primitives corresponding to the input object, e.g., by tessellating the input objects based on tessellation factors provided to the tessellator106by the hull shader105. Tessellation subdivides input higher-order primitives such as patches into a set of lower-order output primitives that represent finer levels of detail, e.g., as indicated by tessellation factors that specify the granularity of the primitives produced by the tessellation process. A model of a scene can therefore be represented by a smaller number of higher-order primitives (to save memory or bandwidth) and additional details can be added by tessellating the higher-order primitive. The granularity of the tessellation can be configured based on a required level of detail, which is typically determined by the relative position of the object represented by the higher-order primitives and a camera that represents the viewpoint used to render the image of the scene including the object. Objects that are closer to the camera require higher levels of detail and objects that are further from the camera require lower levels of detail. Depending on the required level of detail, tessellation can increase the number of lower-order primitives in the graphics pipeline by orders of magnitude relative to the number of input higher-order primitives. Some of the primitives produced by the tessellator106are micropolygons that represent an area that is less than or approximately equal to the area of a single pixel on the image space or the screen used to display the rendered image.

A domain shader107inputs a domain location and (optionally) other patch data. The domain shader107operates on the provided information and generates a single vertex for output based on the input domain location and other information. A geometry shader108receives an input primitive and outputs up to four primitives that are generated by the geometry shader108based on the input primitive. One stream of primitives is provided to a rasterizer109and up to four streams of primitives can be concatenated to buffers in the storage resources101. The rasterizer109performs shading operations and other operations such as clipping, perspective dividing, scissoring, and viewport selection, and the like.

A pixel shader110inputs a pixel flow and outputs zero or another pixel flow in response to the input pixel flow. An output merger block111performs blend, depth, stencil, or other operations on pixels received from the pixel shader110.

The first example of the graphics processing system includes a single graphics pipeline (i.e., graphics pipeline100) that is implemented using a unified shader pool104that includes one or more SIMD compute processing cores for executing appropriate shader programs. For example, the vertex shader103, the hull shader105, the domain shader107, the geometry shader108, and the pixel shader110can be implemented using shader programs executing on the SIMD-type processing cores in the unified shader pool104. Other elements of the graphics pipeline100, such as the input assembler102, the tessellator106, the rasterizer109, and the output merger block111, are implemented using fixed-function hardware that is configured to perform a single function or set of functions. However, the number of stages (which are also referred to herein as “fragments”) of the graphics pipeline100is static, which leads to some of the stages being redundant and unused by some applications. Furthermore, bottlenecks in the fixed-function hardware can reduce the overall throughput of the graphics pipeline100and leave a large proportion of the computational power of the unified shader pool104unused.

FIG.2is a block diagram of an example of graphics processing system200that includes a configurable graphics pipeline201according to some embodiments. The graphics processing system200can be implemented as part of a graphics processing unit (GPU) and software driver environment that accesses commands from a queue202that is configured to store one or more command buffers. The command buffers store rendering commands (e.g., Draw with geometry commands) or compute commands (e.g., shader code) that are targeted to one or more of the shader engines in the graphics pipeline201. Command buffers are generated by driver software and added to the queue202. When the graphics pipeline201is ready to process another command, and input assembler (IA)204pulls command buffers from the queue202and provides them to other shader engines in the graphics pipeline201for execution.

The configurable graphics pipeline201includes a set of required shader stages that include shader engines and fixed function hardware units. The required shader engines include a vertex shader (VS)206and a pixel shader (PS)208. The required fixed function hardware units include the input assembler204and a rasterizer (RS)210. The configurable graphics pipeline201also includes a set of optional shader stages that include shader engines and fixed function hardware units. The optional shader stages include a hull shader (HS)212, a domain shader (DS)214, and a graphics shader (GS)216. The optional fixed function hardware units include a tessellator (TESS)218, a depth stencil test and output unit (DB)220, and a color blender and output unit (CB)222. As discussed herein, the shader stages can be implemented using the resources of a unified shader pool. The functionality of the shader engines and fixed function hardware units in the configurable graphics pipeline201corresponds to the functionality of corresponding elements discussed herein, e.g., with regard to the graphics pipeline100shown inFIG.1.

Operation of the fixed function hardware units in the graphics pipeline201is configured and controlled based on dynamic state information that is provided to the graphics pipeline201in conjunction with commands that are executed by the fixed function hardware units. In some embodiments, the dynamic state information includes viewport dynamic state information224that defines the viewport for the object or fragment that is being processed by the rasterizer210, rasterizer dynamic state information226that defines the state of the rasterizer210, multi-sample antialiasing (MSAA) dynamic state information228that defines the state of the rasterizer210to reduce aliasing, color blender dynamic state information230that defines the state of the color blender and output unit222, and depth stencil dynamic state information232that defines the state of the depth stencil test and output unit220. Index data234is provided to the input assembler204to identify the indices of the objects, primitives, or fragments that are processed by the graphics pipeline201.

Operation of the shader engines is configured or controlled on the basis of dynamic memory views236that are accessible by the shader engines. The dynamic memory views236include primitive index data. A static memory view238is also accessible as part of a descriptor set240. As used herein, the term “descriptor set” refers to a special state object that conceptually can be viewed as an array of shader resources, sampler object descriptors, or pointers to other descriptor sets. Some embodiments of the descriptor set240also include image views242. One or more different descriptor sets are available to the graphics pipeline201. Shader resources and samplers that are referenced in the descriptor sets240are shared by all the shader engines in the graphics pipeline201. Color targets244for the object, primitive, or fragment are accessible by the color blender222. Depth stencil targets246for the object, primitive, or fragment are accessible by the depth stencil test and output unit220.

The graphics pipeline201is configurable using different combinations of the shader engines and fixed function hardware units. In some embodiments, the valid graphics pipelines can be built by following a set of rules such as: (1) a vertex shader206is required, (2) a pixel shader208is required for color output and blending but is optional for depth-only rendering, and (3) a hull shader212and a domain shader214are required to enable tessellation in graphics pipelines that include the tessellator218. Various configurations of the graphics pipeline201can then be generated in different circumstances, as shown in Table 2. However, other configurations of the graphics pipeline201can be generated based on the above set of rules.

The graphics pipeline201is an example of a monolithic pipeline object that defines a large part of the state associated with a 3D pipeline using a single bind point. The state associated with the single bind point includes state information for all of the shader engines in the graphics pipeline201, as well as fixed function states that impact shader execution in various configurations of the graphics pipeline201. Implementing the graphics pipeline201as a monolithic pipeline object, allows a reduction in API overhead by enabling up-front shader optimization at compile time. Embodiments of the graphics pipeline201also make the CPU performance of the pipeline driver more predictable, since shader compilation is not kicked off by the driver at draw time outside of the application's control. The monolithic pipeline representation is bound to the state of the graphics pipeline100in command buffers.

TABLE 2Examples of valid configurations of the graphics pipeline 201Pipeline configurationDescriptionIA−>VS−>RS−>DBDepth-stencil only rendering pipeline.IA−>VS−>RS−>PS−>DBDepth/stencil only rendering pipeline with pixelshader (for example, using pixel shader for alphatest).IA−>VS−>RS−>PS−>CBColor only rendering pipeline.IA−>VS−>RS−>PS−>DB−>CBColor and depth-stencil rendering pipeline.IA−>VS−>GS−>RS−>PS−>DB−>CBRendering pipeline with geometry shader.IA−>VS−>HS−>TESS−>DS−>RS−>PS−>DB−>CBRendering pipeline with tessellation.IA−>VS−>HS−>TESS−>DS−>GS−>RS−>PS−>DB−>CBRendering pipeline with tessellation andgeometry shader.

Although there are many advantages to implementing the graphics pipeline201as a monolithic pipeline, the graphics pipeline201cannot be reconfigured to support processing by the shader engines or the fixed function hardware units in different orders. Only valid configurations that conform to the above set of rules (such as the example shown in Table 2) can be used for graphics and other data type processing in the graphics pipeline201.

FIG.3is a block diagram of a graphics processing system300that supports reconfigurable virtual graphics pipelines according to some embodiments. The reconfigurable graphics processing system300is implemented using one or more graphics and general compute processing cores that can be used to support a plurality of unified shader engines, programmable RISC micro-engines, one or more fixed function hardware units, memory elements that are used to store data or instructions, and other hardware circuitry, as discussed herein. Although some embodiments of the reconfigurable graphics processing system300are utilized to perform graphics processing (e.g., using configurable virtual graphics pipelines), the virtual graphics pipelines in the graphics processing system300can also be used for general purpose calculations, image processing with involved dedicated fixed function units implementing FFT, convolution, and other specific functions. For example, virtual graphics pipelines can be configured to support deep learning neural networks that are implemented on a GPU platform.

The reconfigurable graphics processing system300includes a configuration and control block302that supports management, control, arbitration, and synchronization of multiple reconfigurable virtual graphics pipelines. The configuration and control block302receives system input or user input that is used to configure the virtual graphics pipelines. Configuring the virtual graphics pipelines can include resource allocation or mapping of resources to the virtual pipelines in fully static, semi-static, semi-dynamic, and fully dynamic modes. For example, the configuration and control block302can dynamically configure or reconfigure the virtual graphics pipelines in response to system events or user input indicating that a new virtual graphics pipeline is to be instantiated, an existing virtual graphics pipeline is to be reconfigured, or an existing virtual graphics pipeline is to be removed or terminated, e.g., due to completion of the thread that was being executed by the existing virtual graphics pipeline.

A set of queues304include commands that are to be executed by one of the virtual graphics pipelines. Some embodiments of the queues304are implemented as ring buffers in a memory. Each of the queues304is able to be in an “active” state or an “on hold” state depending on application activity that writes the data to a tail of the queue304. The application is also able to send doorbell signals with head and tail pointer values to context status descriptor registers or memory locations. The number of virtual graphics pipeline that can be supported by the graphics processing system300is determined by the maximum number of supported context descriptor sets. In some embodiments, the reconfigurable graphics processing system300implements a reconfigurable structure that supports mapping a flexible number of context descriptor sets into a memory hierarchy.

A routing, queuing, and mapping (RQM) element306receives commands from the set of queues304. The RQM element306is configured to map the queues304to different virtual graphics pipelines. The RQM element306can then queue commands from the queues304for the corresponding virtual graphics pipelines. The commands are routed by the RQM element306to the virtual graphics pipelines for execution. In the illustrated embodiment, the RQM element306provides the commands to one or more super pipe fragments (SPF)310,311,312, which are collectively referred to herein as “the SPFs310-312.” Each of the SPFs310-312is part of a different virtual graphics pipeline and each of the SPFs310-312processes commands for the corresponding virtual graphics pipeline in accordance with descriptors in a descriptor set associated with the commands. Some embodiments of the SPFs310-312are described in more detail below.

An RQM element314receives commands from the SPFs310-312. The RQM element314is configured to map the SPFs310-312to meta-pipe fragments (MPFs)315,316,317that are part of the same virtual graphics pipeline as one of the corresponding SPFs310-312. For example, commands received from the SPF310can be mapped to the MPF315. The RQM element314can then queue commands from the SPFs310-312for the MPFs315-317in the corresponding virtual graphics pipelines. The commands are then routed by the RQM element314to the corresponding MPFs315-317for execution. The MPFs315-317could be implemented using RISC micro-engines for executing metacommand resolution and processing threads. Some embodiments of the MPFs315-317are described in more detail below.

An RQM element318receives commands from the MPFs315-317. The RQM element318is configured to map the MPFs310-312to corresponding virtual pipe fragments (VPFs)320,321,322that are part of the same virtual graphics pipeline as one of the corresponding MPFs315-317. As a result of processing in MPFs315-317, the metacommand flow is converted to command/data flow that can be properly interpreted by following VPFs320-322. For example, command and data flow received from the MPF315can be mapped to the VPF320. The RQM element318can then queue commands and data from the MPFs315-317for the VPFs320-322in the corresponding virtual graphics pipelines. The commands and data are then routed by the RQM element318to the corresponding VPFs320-322for execution. The VPFs320-322process commands and data in conjunction with corresponding physical pipe fragments (PPFs)326,327,328. Some embodiments of the VPFs320-322are described in more detail below.

An RQM element324receives commands and data from the VPFs320-322. The RQM element324is configured to map the VPFs320-322to the corresponding PPF326-328that are part of the same virtual graphics pipeline as one of the corresponding VPFs320-322. For example, commands and data received from the VPF320can be mapped to the PPF326. The PPFs326-328implement fixed function hardware and/or perform compute data processing in allocated resources of a unified shader pool. Some embodiments of the PPFs326-328are described in more detail below. The RQM element324can then queue commands from the VPFs320-322for the PPFs326-328in the corresponding virtual graphics pipelines. The commands and data are then routed by the RQM element324to the corresponding PPFs326-328for execution.

An RQM element330receives commands from the PPFs326-328. The RQM element330is configured to map the PPFs326-328to corresponding data pipe fragments (DPFs)332,333,334to support the physical processing functions in the virtual graphics pipeline. For example, commands and data received from the PPF326can be mapped to the DPF332. The RQM element330can then queue commands and data from the PPFs326-328for the DPFs332-334in the corresponding virtual graphics pipelines. The commands and data are then routed by the RQM element330to the corresponding DPFs332-334for execution. The DPFs332-334perform multiple types of operations on data including packet generation and data re-arrangement. Some embodiments of the DPFs332-334are described in more detail below. Packets, rearranged data, or other feedback can be provided from the RQM element330or the DPFs332-334to the set of queues304, the RQM element314, the RQM element318, or the RQM element324.

Stages of the virtual graphics pipelines are configured and operated under the control of corresponding control elements340,341,342,343,344, which are collectively referred to herein as “the control elements340-344.” Each of the control elements340-344can receive data or instructions from the configuration and control unit302. The control elements340-344are also configured to provide configuration and control signaling to their corresponding stages of the virtual graphics pipelines. For example, the control element340can provide configuration and control signaling to the RQM element306and the SPFs310-312.

The control element340performs queue status monitoring configuration and provides configuration signaling to the RQM element306or the SPFs310-312to support interaction between multiple parallel running applications that are concurrently updating their context status descriptors. In some embodiments, each potential active application is associated with its own descriptor set. The control element340can then allocate a set of registers to store a context queue status for each active application which can be monitored by dedicated hardware. The allocated registers can be stored in dedicated memory blocks or shared buffers that are mapped to a memory space.

The control element341performs configuration of the command packet resolution functionality and controls command packet (metacommand) resolution for the MPF stage. The control element341operates on active queues detected at the previous SPF stage. In some embodiments, one or more VPFs320-322can be created and associated with one or more of the MPFs315-317, perhaps in combination with dedicated hardware blocks or selected threads (micro-threads) that are implemented on one or more of the RISC micro-engine MPFs315-317. As used herein, the phrase “micro-engine VPF” refers to pipeline fragments that are established using a micro-engine as a base for the portion of the virtual graphics pipeline. RISC micro-engine VPFs are able to fetch and decode queue entries and associated DMA buffers to create the tasks for on-chip processing in the virtual graphics pipelines.

The control element342performs configuration and control of the processing front-end. Some embodiments of the control element342distribute the tasks prepared on previous MPF stage to allocated resources of the graphics processing system300. The control element342is also able to schedule the tasks for execution on processing VPFs320-322or PPFs326-328. Front-end VPFs implementing such functionality can be implemented on different platforms depending on minimal latency requirements. RISC micro-engine VPFs can be used in case of high latency tolerance and hardware-based state machines in the case of low latency tolerance.

The control element343performs configuration and control for processing VPFs320-322or PPFs326-328. For example, the control element343can define configuration of all computing VPFs320-322contained in multiple virtual pipes pipelines as well as their internal connectivity with different type of resources. Computing VPFs320-322can be configured to contain PPFs326-328implemented as programmable shader kernels or fixed function hardware computation blocks or combination of both.

The control element344performs configuration and control of the data output stages, e.g., the DPFs332-334. Some embodiments of the control element344define one or more types of data output that can be exported to following VPFs320-322via internal routing, queueing, and mapping (e.g., the RQM elements314,318,324) or to external queues such as the set of queues304. The control element344in combination with other control elements340-343is also able to create virtual graphics pipelines of any shape and complexity to match application requirements, as discussed herein.

FIG.4is a block diagram of a graphics and compute processing system400that implements multiple queues that provide commands and data to multiple graphics pipelines according to some embodiments. The graphics processing system400includes multiple CPU-type processor cores401,402,403,404(collectively referred to herein as “the CPU cores401-404”) that generate commands and data for execution by one or more graphics pipelines. Some embodiments of the CPU cores401-404are multithreaded processor cores that are implemented as part of a central processing unit (CPU). The CPU cores401-404provide the commands and data to queues405,406,407,408,409,410(collectively referred to herein as “the queues405-410”) that are part of a set415. The queues405-410include entries that store commands and data or associated contexts that are used to configure the graphics processing system400to process the commands.

One or more shader engines (SE)420,421,422,423,424,425(collectively referred to herein as “the shader engines420-425”) are implemented using shared hardware resources of the graphics processing system400. Some embodiments of the shader engines420-425can be used to implement shaders in the graphics processing system300shown inFIG.3. The shared hardware resources include asynchronous compute engines (ACE)430,431,432,433,434(collectively referred to herein as “the asynchronous compute engines430-434”) for executing general compute metacommands, a graphics processing engine435configured to execute graphics metacommands, and a video processing engine438configured to execute video metacommands. The asynchronous compute engines430-434are distinct functional hardware blocks that are capable of executing general compute metacommands concurrently with other asynchronous compute engines430-434.

A command processor440fetches metacommands from the queues405-410and routes the metacommands to the appropriate shared hardware resources. For example, the command processor440can fetch commands from the queue405and route the commands to the asynchronous compute engine430. The command processor440can also be used to map the queues405-410to the corresponding shared hardware resources. For example, the command processor440can map the queue409to the graphics engine435and subsequently route metacommands from the queue409to the graphics engine435based on the mapping. A resource allocator445is used to allocate shared resources to the shader engines420-425to implement the graphics pipelines, a pipe scheduler450is used to schedule commands for execution by the shared resources that are used to implement the graphics pipelines, a synchronizer455is used to synchronize execution of metacommands by the graphics pipelines, and a context switch module460is configured to perform context switching to allow the graphics pipelines to operate on different threads of commands using different states that are specified by the different contexts. The context switch module460also supports preemption of threads that are executing on the graphics pipelines for different clients providing shared GPU resource mode.

FIG.5is a block diagram of a command processor500that is configured to fetch commands from one or more application queues according to some embodiments. The command processor500is implemented using fixed function hardware blocks and RISC micro-engine cores. The command processor500also includes firmware to fetch commands from command buffers, allocate resources, and perform graphics pipeline and compute pipeline scheduling and synchronization. The command processor500is subdivided into three portions: (1) a graphics engine505, (2) an asynchronous compute engine510, and (3) a scheduling engine515. A cache interface520provides an interface to one or more caches or a cache hierarchy that caches metacommands or state information used to define the contexts of the metacommands. The shader interface525provides an interface to one or more shader engines, which are implemented using shared resources of a unified shader pool. The shader interface525can provide support for allocating tasks to the different shader engines.

The scheduling engine505includes a queue block530that provides queues, buffers, and other hardware. For example, the queue block530can include a primary queue (PQ), a heterogeneous system architecture (HSA) Interface Queue (IQ), one or more Indirect Buffers (IB), and End-Of-Pipe (EOP) hardware support. The queue block530also supports command buffers and data fetch from memory hierarchy via the cache interface520. The scheduling engine505also includes a scheduler531. Some embodiments of the scheduler531perform scheduling of commands stored in a command buffer for subsequent execution. The scheduled commands can then be provided to the shader interface525. Scheduling of the commands for execution can be performed by firmware running on a hardware RISC micro-engine that is used to implement the scheduler531

The asynchronous compute engine510includes a queue block535to provide queues, buffers, and other hardware such as a primary queue, and HSA interface queue, indirect buffers, or EOP (end of packet) hardware support. The asynchronous compute engine510also includes a compute engine536that can perform processing of metacommands received from the queue block535. A dispatch controller537propagates command task execution to shader processor input and task allocation (via the shader interface525) for further execution. For example, the dispatch controller537can dispatch commands to shader pool resources for execution as part of a set of connected threads (or waves) that use the same program counter value (e.g., using single program multiple data, SPMD, techniques). In some cases, multiple asynchronous compute engines can be implemented as multiple firmware threads running on one or more micro-engines.

The graphics engine515includes a queue block540to provide queues, buffers, and other hardware. The queue block540is configured to store commands or context information requested by a prefetch parsing block541that is configured to prefetch the commands, data or context information from the caches via the cache interface520. The graphics engine515also includes a RISC micro-engine542that can perform metacommand and data processing using commands or data that are prefetched from the queue block541and a dispatch controller543that propagates command task execution to shader processor input and task allocation via the shader interface525. The graphics engine515includes another queue block545that stores commands or context information for execution by a constant engine546, which can be implemented as a RISC micro-engine. The constant engine546is coupled to a memory such as a random access memory (RAM)547, which is dedicated to support graphics constant management. Commands or context information stored in the RAM547can be accessed via the cache interface520.

FIG.6is a block diagram of a graphics processing system600that supports multiple reconfigurable virtual graphics pipelines (or virtual GPUs) according to some embodiments. The graphics processing system600includes a command processor605(such as the command processor500shown inFIG.5) that is configured to route, queue, or map commands associated with different threads to corresponding asynchronous compute engines610,611,612, which are collectively referred to herein as “the asynchronous compute engines610-612”. Each of the asynchronous compute engines610-612is able to support a different virtual graphics pipeline and the virtual graphics pipelines concurrently execute commands in different threads generated by different applications without context switching or preemption. This feature provides seamless virtualization of GPU for multiple clients. The virtual graphics pipeline can be tailored to requirements of the different applications, e.g., by an application developer, and multiple instantiations of virtual graphics pipelines coexist on the physical resources of a graphics processing system. Thus, the graphics processing system600can host multiple virtual graphics pipelines with different pipeline shapes on shared hardware or firmware resources of the graphics processing system. The graphics processing system600also provides better memory access locality by reducing the size of tasks to small batches or objects that can be distributed among the multiple virtual graphics pipelines with manageable level of internal memory footprint.

The graphics processing system600includes a first virtual graphics pipeline615that is supported by the asynchronous compute engine610and a second virtual graphics pipeline630that is supported by the asynchronous compute engine612. The first virtual graphics pipeline615includes an input assembler620, a vertex shader621, a tessellator622, a domain shader623, a geometry shader624, a rasterizer625, a pixel shader626, a depth stencil test unit627, and a color blender and output unit628. The second virtual graphics pipeline630shows a closer view of the virtual pipeline implementation as a multistage shader software stack and includes a data assembler631, a vertex shader kernel632, a hull shader kernel633, a tessellator634, a domain shader kernel635, a geometry shader kernel636, a rasterizer637, a pixel shader kernel638, a depth stencil test block639, and a color merge block640. Although not shown inFIG.6, some embodiments of the graphics processing system600also include additional virtual graphics pipelines that can be supported by the asynchronous compute engines similar to610-612.

To support implementations of a reconfigurable GPU, the graphics processing system600also includes shared fixed function hardware blocks641,642,643,644,645, which are collectively referred to herein as “the shared fixed function hardware blocks641-645.” Some embodiments of the shared fixed function hardware blocks641-645are used as common shared resources with arbitrated access so that any kernel in a software stage of the virtual graphics pipelines615,630is able to request a dedicated fixed function hardware block to execute a particular function and return the resulting data to the requesting software stage. For example, the data assembler631can transmit a request (or call) to the dedicated fixed function hardware block641to perform an operation and the results of the operation can be returned to the vertex shader kernel632. For another example, the tessellator634can transmit a request to the dedicated fixed function hardware block642to perform an operation and the results of the operation can be returned to the kernel domain shader635. In the interest of clarity, requests and responses transmitted by stages of the virtual graphics pipeline615are not shown inFIG.6. Moreover, the fixed function hardware blocks641-645(and their association with stages of the virtual graphics pipeline630) are intended to be illustrative. Other stages in the virtual graphics pipeline615,630can request access to, and receive responses from. more or fewer fixed function hardware blocks.

The fixed function hardware blocks641-645can be emulated using corresponding shader firmware. For example, emulations of the fixed function hardware blocks641-645can be instantiated in shader firmware in response to detecting bottlenecks in the fixed function hardware blocks641-645. The shader firmware emulations of the fixed function hardware blocks641-645can then be used to perform the requested operations instead of using the actual hardware, thereby alleviating the bottlenecks in the fixed function hardware blocks641-645.

Some embodiments of the graphics processing system600have a number of advantages over conventional graphics pipelines. For example, the graphics processing system600utilizes fixed-function hardware within the compute domain and many compute shaders or virtual GPUs can be scheduled concurrently and load balanced by the asynchronous compute engines610-612, perhaps in combination with software supported by an HSA (Heterogeneous System Architecture) stack. Multiple virtual graphics pipelines are also able to perform millions of small scene texture-space renders per second and process a number of small command buffers created by multithreaded parallel applications running on power multicore CPU platforms. The graphics processing system600is also able to remove front-end bottlenecks on context switch and preemption, minimize data movement by enabling persistent producer-consumer threads, and maintain the option to keep code and data local to a compute unit and iterate as well as use a local on-chip memory hierarchy such as a cache hierarchy, ring buffers, or other parts of the memory hierarchy to stream data to or from the fixed function hardware blocks641-645. The need to flush caches to communicate between GPU-local processes can also be avoided.

FIG.7is a block diagram of a graphics processing system700including multiple reconfigurable virtual graphics pipelines according to some embodiments. The graphics processing system700is used to implement some embodiments of the graphics processing system300shown inFIG.3, the graphics processing system400shown inFIG.4, or the graphics processing system600shown inFIG.6.

The graphics processing system700includes feedback queues701that receives commands or context information that is fed back from other portions of the graphics processing system700, as discussed herein. Some embodiments of the feedback queues701are defined by the virtual graphics pipelines that create data queues or buffers for processing by other virtual graphics pipelines. The graphics processing system700also includes user-defined queues702for receiving commands or context information, e.g., the queues702can be created by users via an HSA software stack. The graphics processing system700also includes application/context queues703that receives commands or context information associated with concurrent application tasks, scene decomposition, application programming interfaces (APIs), or device drivers. The application/context queues703hold tasks defined by multithreading system software.

The graphics processing system700implements a shared command processor resource pool705that is used in some embodiments of the command processor440shown inFIG.4, the command processor500shown inFIG.5, or the command processor605shown inFIG.6. The shared command processor resource pool705supports a front-end710that is configured to perform resource allocation, scheduling, and task synchronization for the tasks being performed by the graphics processing system700. The shared command processor resource pool705also supports asynchronous compute engines711,712,713,714,715,716,717,718,719,720(collectively referred to herein as “the asynchronous compute engines711-720”), a graphics engine721, a video engine722, and a fetch engine723that is configured to fetch data or commands from the queues701-703and buffers with commands and data that are indicated by the pointers.

The shared command processor resource pool705is connected to an allocation block725that is configured to allocate an array of shader kernels to one or more virtual graphics pipelines726,727,728,729, which are collectively referred to herein as “the virtual graphics pipelines726-729.” The shared command processor resource pool705is also connected to a set730of state management registers, buses, and state machines. For example, the front-end710can provide context information associated with the commands via buses in the set730to configure the registers or state machines in the set730to determine an operational state of one or more of the virtual graphics pipelines726-729. The buses in the set730are also able to convey information between the registers or state machines in the set730and the virtual graphics pipelines726-729.

The asynchronous compute engines711-720are configured to pull data or commands from the queues701-703and create tasks that are provided to the virtual graphics pipelines726-729. In conjunction with allocation of the tasks to the virtual graphics pipeline726-729, the front-end710provides the pipeline state data to the set730for distribution to the corresponding virtual graphics pipeline726-729. Distribution of the state data is performed synchronously with allocation of the tasks to the virtual graphics pipelines726-729so that the operational states of the virtual graphics pipelines726-729are consistent with the tasks that are to be performed.

The virtual graphics pipelines726-729are reconfigurable and user-definable and each of the virtual graphics pipelines726-729can be formed of a different combination of shader engines, fixed function hardware, or firmware emulations of the fixed-function hardware. For example, the virtual graphics pipeline726can include a first shader kernel740, a fixed function hardware call/return block741, a second shader kernel742, a second fixed function hardware call/return block743, a third shader kernel744, a third fixed function hardware call/return block745, a fourth shader kernel746, as well as other shader engines, fixed function hardware, or firmware emulations that form the set740-746. The virtual graphics pipeline727is formed of a set750-756of shader engines, fixed function hardware, or firmware emulations, the virtual graphics pipeline728is formed of a set760-766of shader engines, fixed function hardware, or firmware emulations, and the virtual graphics pipeline729is formed of a set770-776of shader engines, fixed function hardware, or firmware emulations. Some embodiments of the graphics processing system700include more or fewer virtual graphics pipelines than the four virtual graphics pipelines726-729shown inFIG.7and the number of virtual graphics pipelines (as well as the composition of the virtual graphics pipelines) can change dynamically, e.g., in response to user input or system events such as completion of the tasks in a thread assigned to a virtual graphics pipeline.

The virtual graphics pipeline726-729are implemented using shared on-chip hardware resources780. The resources780include a set781of shader engines and corresponding caches. The shader engines are implemented using one or graphics processing cores. The caches can include a memory hierarchy that is formed of L1 caches, L2 caches, L3 caches, and the like. The number of levels in the memory hierarchy can be larger or smaller than the three levels used to implement the L1 caches, L2 caches, and L3 caches. Some embodiments of the memory hierarchy include DRAM, registers, queues, buffers, and the like. The resources780also include a set782of specialized memory buffers and first-in-first-out (FIFO) buffers that provide buffering of data during interactions between the shader engines and fixed function hardware units783,784,785,786, which are collectively referred to herein as “the fixed function hardware units783-786.” Access to the shared on-chip hardware resources is controlled by blocks787,786perform access arbitration, scheduling, and queuing of tasks such as commands that are to be executed by the shader engines in the set781or the fixed function hardware units783-786.

FIG.8is a block diagram of a graphics processing system800illustrating interaction of compute units with shared fixed function hardware according to some embodiments. The graphics processing system800includes one or more compute units805(only one shown in the interest of clarity) that are part of a shared resource pool that is used to execute shader kernels (programs).

The shader kernel executing on the compute unit805issues a call808to a fixed function hardware block810. The call808is received at an arbiter815that performs access arbitration between the call808and other calls that are received from other shader kernels. The arbiter815is able to provide the call808to a fixed function hardware scheduler820that schedules tasks for execution by the fixed function hardware block810or a fixed function emulations scheduler825that schedules tasks for execution by a shader firmware emulation830of the fixed function hardware block810. The arbiter815chooses between the schedulers820,825based on the operational status of the fixed function hardware block810. If the block810is busy, e.g., if a buffer or queue835that holds tasks that are scheduled for the fixed function hardware block810is full, the arbiter815directs the call808to the emulation scheduler825, which can schedule the call808and provide the call808to a queue840. If the block810is not busy, e.g., if the queue835is not full, the arbiter815directs the call808to the fixed function hardware scheduler820.

The arbiter815also provides virtual pipeline state information845to the fixed function hardware block810or the emulation830. The state information845is provided in a manner that is synchronous with the call808so that the state information845can be used to define the appropriate state of the fixed function hardware block810or the emulation830when it is performing the operation requested by the call808. In some cases, different virtual pipelines maintain different sets of state data packages to configure the operational states of the fixed function hardware block810or the emulation830. The state information845can be provided via pointers to memory locations and actual data retrieval might take several clock cycles. Pointer resolution can be performed or conducted while the call808is waiting in the firmware queue840.

The fixed function hardware block810and the shader firmware emulation kernels830are both able to access a shared local memory850. The shared local memory850includes a memory hierarchy that can be used to implement shared registers, buffers, and a cache hierarchy including L1 caches, L2 caches, and the like. Access arbitration for the shared local memory850is performed using an arbiter855that can include one or more queues to facilitate the arbitration process. In some cases, the fixed function hardware block810retrieves data from shader export registers implemented in the memory850and returns results directly to shared memory buffers or shader register file portions that are allocated in the local memory850for use by the shader kernel executing on the compute unit805. In some cases, results generated by the fixed function hardware block810have variable or unpredictable sizes. This type of result can be written to ring buffers allocated in cache and memory hierarchy of the shared local memory850by passing a pointer to the caller shader kernel.

Some embodiments of the firmware emulation830perform single-instruction-multiple-data (SIMD) wave rearrangement if data is not immediately available in the registers, caches, or buffers of the shared local memory when the firmware emulation830is ready to execute a requested operation. If data is available immediately in the shared local memory850, then the call808is provided directly to the emulation834execution. If input data is not available immediately or needs SIMD wave data rearrangement, then the call808is queued and the kernel thread/wave goes to a thread execution queue (which can be implemented in the shared local memory850) in a manner that is similar to a cache miss event and data retrieval from DRAM memory. Some embodiments of the emulation830call an export shader type of kernel to perform data rearrangement in conjunction with the emulation830.

FIG.9is a diagram900illustrating a functional hierarchy and control domains for reconfigurable virtual graphics pipeline according to some embodiments. The diagram900illustrates the functional hierarchy and control domains that are implemented in some embodiments of the graphics processing system300shown inFIG.3, the graphics processing system400shown inFIG.4, the graphics processing system600shown inFIG.6, and the graphics processing system700shown inFIG.7.

The application/driver level905includes one or more user applications and supporting user mode drivers (UMD) and low level drivers (LLD) that are configured to populate and manage multiple queues or ring buffers in system memory. User software or API software implemented at the application/driver level905can manage queues in the memory. Some embodiments of the queues are configured for promotion to graphics processing system context queues by mapping the queue into a GPU queue status descriptor domain, which can be monitored and controlled by hardware or firmware implemented in a super pipe domain910.

The super pipe domain910supports queue status monitoring, as well as queue activation and synchronization between applications based on their priority and detected activity information. The super pipe domain910supports instantiation of one or more super pipe fragments such as the super pipe fragments310-312shown inFIG.3. Some embodiments of a front-end of a command processor (such as the front-end710shown inFIG.7) are implemented in the super pipe domain910. The front-end can be implemented on a RISC micro-engine in combination with dedicated fixed function hardware blocks. Some embodiments of the front-end implemented in the super pipe domain910are configured to manage the activity of a meta-pipe domain915. Some embodiments of the front-end are also able to execute multiple concurrent threads to make multiple GPU devices visible to one or more applications.

The meta-pipe domain915provides active queue data processing with packet fetch, decode, and execution dispatch. The meta-pipe domain915also supports retrieval of associated data from a memory hierarchy. The meta-pipe domain915supports instantiation of meta-pipe fragments such as the meta-pipe fragments315-317shown inFIG.3. The meta-pipe domain915can support allocation of multiple meta-pipes for active GPU contexts. The meta-pipes can be executed concurrently to process command packets (metacommands) and perform related data acquisition for virtual pipes that are defined in the virtual pipe domain920. The meta-pipe domain915also provides synchronization and arbitration based on explicit codes in command packets (metacommands). Some embodiments of the meta-pipe domain915are implemented as a core part of a command processor in conjunction with firmware or software running on one or more RISC micro-engines in conjunction with dedicated fixed function hardware blocks. Some embodiments of the meta-pipe domain915are also configured to manage the activity of the virtual pipe domain920. One or more parallel threads can be executed concurrently in the RISC micro-engines running meta-pipe software to serve multiple independent command packet queues or virtual pipes in the virtual pipe domain920.

The virtual pipe domain920provides mapping and chaining of virtual graphics or compute pipelines to shared resources necessary for processing context information in associated queues. The virtual pipe domain920supports instantiation of virtual pipe fragments (VPF) such as the virtual pipe fragments320-322shown inFIG.3. Virtual pipe fragments can be mapped to particular physical processing fragments (PPF) and a shared memory hierarchy. Multiple virtual pipe fragments can be concurrently instantiated, arranged, and interconnected to process one or more metacommands in parallel based on a data flow that is fetched from one or more active context queues via meta-pipes. The virtual pipe domain920is functionally equivalent to task pipelining and monitors the status of engaged virtual pipeline fragments that are being executed on dedicated graphics processing resources. Some embodiments of the virtual pipe domain920can be implemented as a back-end part of a command processor including firmware or software executing on one or more RISC micro-engines in conjunction with one or dedicated fixed function hardware blocks. The virtual pipe domain920is able to configure virtual pipeline fragments to create different pipeline shapes and manage the activity of preconfigured virtual pipeline fragments with appropriate dispatch and synchronization between virtual pipeline fragments. The virtual pipe domain920can also be implemented as multiple concurrent threads in each RISC micro-engine running virtual pipe software to serve multiple virtual pipes and monitor underlying activity in a physical processing domain925.

The physical processing domain925contains shared resource pools or clusters of fixed function hardware-based physical processing pipe fragments or physical processing pipe fragments that are implemented using a unified shader resource pool or fixed function hardware cluster. The physical processing domain925supports instantiation of the physical processing pipe fragments326-328shown inFIG.3. The virtual pipe fragment configuration and control blocks, as well as block/memory mapping and routing circuitry, can be considered as a resource pool to use by multiple virtual pipes. The physical processing domain925can be implemented using a combination fixed function hardware, RISC micro-engine firmware threads, or shader kernels combined with appropriate buffers and interfaces.

The outermost domain930represents hardware circuit resources that are configured to perform data processing and implement data migration pipes. Some embodiments of the data processing and migration pipes are used to implement data paths for fixed function hardware programmable arithmetic logic units (ALUs). The outermost domain930also includes hardware circuit resources that include control and arbitration circuitry, interfaces, and data fabrics with predetermined functionality and timing.

FIG.10is a block diagram illustrating a hierarchy1000for context management, task queueing, dispatch, and scheduling in domains of a reconfigurable graphics processing system according to some embodiments. The hierarchy1000is implemented in accordance with some embodiments of the functional hierarchy and control domains shown inFIG.9.

One or more application contexts1001,1002,1003(collectively referred to herein as “the contexts1001-1003”) are executing on one or more graphics processing cores or compute units in an OS/application/driver domain such as the domain905shown inFIG.9. The OS/application/driver domain operates with millisecond delays considering low reactivity of the corresponding software stack. Application context management with scheduling, synchronization is usually implemented by using special OS utilities and application program components. The contexts1001-1003provide commands to corresponding queues1005,1006,1007, which are implemented in a low level driver domain. The low level driver domain works with hundreds of microseconds delay and manages all GPU queues1005-1007and ring buffers creating command packet flows based on API user mode driver requests and other requests. GPU resources are not involved in this domain. The queues1005include HSA and other format queues, the queues1006include draw command queues, and the queues1007include video, image, and other commands.

A compute arbiter1010is used to monitor descriptor sets and arbitrate between the commands in the queues1005. A graphics arbiter1011is used to monitor and arbitrate between commands in the queues1006. An “other apps” arbiter1012is used to monitor and arbitrate between commands in the queues1007. The arbiters1010-1012are implemented in a super pipe domain such as the super pipe domain910shown inFIG.9. Queue status monitoring in the super pipe domain as a latency of 100 s-1000 s clock cycles and manages the reaction of GPU front end to message signaling and doorbell interrupts coming from multiple running application contexts to GPU. The super pipe domain initiates command processor to start activity with one of the queues1005-1007in response to “doorbell” type of notification from application that activates the use of the queue.

A command processor1015is implemented in a meta-pipe domain such as the meta-pipe domain915shown inFIG.9. The meta-pipe domain operates with a latency of 100 s-1000 s clock cycles for queuing, arbitration and synchronization in metacommand/command packet processing and resolution. Programmable RISC-type microengines supported by some HW blocks can be used for queue processing, synchronization and data fetch arrangements. The command processor1015is a multichannel command processor that includes programmable RISC micro-engines that can be combined with one or more fixed function hardware blocks. The command processor1015is configured for processing, synchronization, and dispatch of queued or buffered packets (metacommands).

A distribution, allocation, and scheduling block1018is implemented in a virtual pipe domain such as the virtual pipe domain920shown inFIG.9. The virtual pipe domain operates with delays of 10-20 s clock cycles for queuing, arbitration and synchronization. Some embodiments of the virtual pipe domain use dedicated hardware or microcode controllers depending on the timing requirement. The block1018distributes, allocates, and schedules tasks to a shader input1020, a graphics pipeline1021, or a custom pipeline1022that are implemented in the virtual pipe domain.

A physical pipeline scheduling block1025is implemented in a physical processing pipe domain such as the physical processing pipe domain925shown inFIG.9. The physical processing pipe domain operates with delays of between two clock cycles and tens of clock cycles and uses special hardwired state machines and sequencers to provide arbitrated and synchronized access to programmable data path and fixed function hardware resources. The block1025performs scheduling an excuse arbitration for the physical pipeline fragments in the physical pipe domain. The block1025can provide tasks or commands to a shader engine that generates shader waves1026, data paths for the graphics fixed function hardware1027, or custom pipelines for fixed function hardware or firmware emulations1028.

A microcode and fixed function hardware scheduling block1030is implemented in a data processing pipe domain such as the domain930shown inFIG.9. The data processing pipe domain operates with delays of 1-3 clock cycles. Some embodiments of the data processing pipe domain are therefore implemented hardware to perform queuing, arbitration and synchronization via dedicated state machines. A compiler can be used to help synchronize compute data path operations and avoid data hazards. The block1030performs arbitration and scheduling of commands or tasks for compute data paths1031, graphics fixed function hardware data paths1032, or custom fixed function hardware or firmware emulations data paths1033, which are also implemented in the data processing pipe domain.

A memory and bus scheduling block1035is implemented in a data migration pipe domain that can be included in some embodiments of the domain930shown inFIG.9. The data migration pipe domain operates with delays of between two clock cycles and tens of clock cycles and uses special hardwired controllers to perform queuing, arbitration and synchronization of access to memory and I/O resources/busses. The block1035is configured to perform access arbitration and scheduling of access to memory or input/output buses. The arbitration and scheduling is performed for shader engine compute units1036, graphics pipeline fixed function hardware units1037, or custom pipeline fixed function hardware or firmware emulation units1038.

FIG.11is a block diagram illustrating a set1100of contexts that are used by applications to generate packets for execution according to some embodiments. The contexts1101,1102,1103(collectively referred to herein as “the contexts1101-1103”) are created by applications that are executing in graphics processing systems including some embodiments of the graphics processing system300shown inFIG.3, the graphics processing system400shown inFIG.4, the graphics processing system600shown inFIG.6, and the graphics processing system700shown inFIG.7.

The contexts1101-1103create corresponding queues1105,1106,1107, which are collectively referred to herein as “the queues1105-1107.” The queues1105-1107are implemented as memory ring buffers that are specified by head pointers1110,1111,1112that point to the head of the corresponding queues1105-1107and tail pointers1113,1114,1115that point to the tail of the corresponding queues1105-1107. For example, the context1101can create the queue1105for storing elements such as pointers1120,1121and meta-commands1122,1123. Processing elements can read entries in the queues1105-1107from the slots indicated by the head pointers1110-1112and the application can fill the queues1105-1107by adding entries at the position indicated by the tail pointers1113-1115. Some embodiments of the pointers are direct memory access (DMA) memory pointers that indicate locations in the memory hierarchy or a DMA buffer. Some embodiments of the metacommands are command packets that include attached DMA buffer pointers. Draw commands1125,1126,1127in the command packets contain state or primitive geometry information, vertex information, and the like. Compute commands can include kernel code1130or a reference to code, workgroup arguments1131, barriers1132, and the like. The queues1105-1107can be monitored in response to processing requests or status changes.

Some embodiments of the queues1105-1107are managed by an application/driver domain and lowest level driver (LLD) domain functionality. For example, application or driver software can allocate memory for the queues1105-1107and create a Unified Queue Descriptor (UQD) to retain queue information. The software can then register the queues1105-1107with LLD for processing on the hardware, store handles and doorbell address return from LLD to update the hardware write (tail) pointers1113-1115, and enter data in the queues1105-1107with flow control base on the queue owner's read (head) pointers1110-1112. If one of the queues1105-1107is filled, the application waits for hardware to drain it or move the data to a bigger queue. The application software can also set up pipeline state based on desired type of processing and perform queue coordination on the pipeline or across the pipelines.

The LLD Driver software is configured to perform queue registration and tracking, allocate memory for Queue Run List (QRL), allocate memory for a descriptor, and create a MQD (Memory Queue Descriptor) and initialize queue data. When finished, the LLD driver software can send a request to the queue to de-queue and clean up. The LLD driver software can also perform queue scheduling including assigning queues to Hardware Queue Descriptors (HQD) for hardware processing, setting up doorbell addresses and performing handshaking to establish a good initial write pointer. The LLD driver software can also program an HQD and then assign the HQD for hardware processing, as well as setting up and maintaining pipeline priorities, establishing virtualization methods for the queues1105-1107, and the like.

FIG.12is a block diagram of a command processor1200for reconfigurable graphics pipelines according to some embodiments. The command processor1200is implemented in some embodiments of the command processor440shown inFIG.4, the command processor500shown inFIG.5, the command processor605shown inFIG.6, or the command processor1015shown inFIG.10. The command processor1200includes a front-end1205that supports a super pipe domain, a processor core1210that supports a meta-pipe domain, and a backend1215that supports a virtual pipe domain.

The front-end1205accesses one or more descriptor sets1220,1221,1222that are collectively referred to herein as “the descriptor sets1220-1222.” The descriptor sets1220-1222are assigned to different applications and originate processing for different instances of virtual graphics pipelines. Some embodiments of the front-end1205are implemented using parallel hardware and firmware components that monitor and arbitrate multiple input queues via interaction with multiple sets of queue/context status descriptors (or registers) in the descriptor sets1220-1222. The status of any queue can be updated by the applications and confirmed using special message signaling or doorbells which could be detected by command processor1200using monitoring hardware in the front-end1205. An arbitrated dispatch block1225is used to dispatch commands to the processor core1210. For example, the arbitrated dispatch block1225can dispatch active queue processing requests after detecting multiple active queue processing requests and performing arbitrations on the multiple active queue processing requests.

The processing core1210is associated with a packet fetch block1230, which can be implemented using one or more parallel micro-engines that can execute multiple concurrent threads to provide fetching and decoding of command packets from application/agent queues and respective DMA buffers. Some embodiments of the processing core1210provide synchronizations via barriers and semaphores between multiple command streams before pushing the command packets to the back-end1215for execution dispatch. In some embodiments, the synchronization primitives are encoded in respective command packets such as kernel barriers. The processing core1210routes one or more streams of command packets to appropriate virtual pipes in the back-end1215.

The backend1215is configured to arrange one or more streams of command packets and associated data for dispatch and execution in one or more virtual graphics pipelines that are implemented using preconfigured virtual pipe fragments that are mapped to the resources of one or more physical processing fragments. In some embodiments, a virtual pipe configuration, distribution, and dispatch block1235is used to configure the corresponding virtual pipes and then distribute commands to the configured virtual pipes.

FIG.13is a block diagram of a super pipe fragment1300according to some embodiments. The super pipe fragment1300is implemented in a super pipe domain such as the super pipe domain910shown inFIG.9and is used to implement some embodiments of the super pipe fragments310-312shown inFIG.3. The super pipe fragment1300operates as a front-end of a corresponding virtual graphics pipeline and provides interaction between the allocated resources of the virtual graphics pipeline and one or more application threads that are assigned to the virtual graphics pipeline. Some embodiments of the super pipe fragment1300are implemented on an application level and operate within the super pipe domain910shown inFIG.9.

The super pipe fragment1300includes a set1305of queues or ring buffers that include slots that can be filled by tasks or commands generated by application threads running on one or more processor cores, e.g., processor cores implemented in a CPU1310. Some embodiments of the application threads running on the processor cores are supported by OS/LLD drivers, as discussed herein. Some or all of the queues in the set1305are mapped to descriptors1315, such as GPU queue descriptors. The queues in the set1305are monitored by a shared super pipe engine and queue state machine1320, which is implemented in hardware or firmware. Access to the descriptors1315can be arbitrated by an arbiter1325. Threads generated by the CPU1310can activate the queues in the set1305by changing a status of one of the descriptors1315that is a queue descriptor for the corresponding queue. The super pipe engine1320can initiate a data fetch1330from queues in the set1305via a memory interface1332. The super pipe engine1320then places the fetched queue slots1335in a special buffer for further processing by a meta-pipe fragment (not shown inFIG.13), as indicated by the arrow1340. Some embodiments of the arbiter1325generate inter-queue synchronization signals, tokens, or semaphores to manage processing of the fetched queue slots1335by the meta-pipe fragment, as indicated by the arrow1345.

FIG.14is a block diagram of a state machine1400that is implemented in a super pipe fragment according to some embodiments. The state machine1400is used to implement some embodiments of the queue state machine1320shown inFIG.13.

In response to powering on or a reset command1401, the state machine1400is placed in an idle state1405and waits for a corresponding queue or ring buffer (such as the queues or ring buffers in the set1305shown inFIG.13) to become active. As long as the ring buffer is not active, the state machine1400remains in the idle state1405, as indicated by the arrow1406.

The state machine1400transitions from the idle state1405to the active state1410in response to detecting activation of the ring buffer, as indicated by the arrow1411. The state machine1400can request access to the virtual graphics pipeline if the queue is not empty when the state machine1400is in the active state1410. The state machine1400transitions back to the idle state1405in response to the ring buffer becoming inactive, as indicated by the arrow1412. The state machine1400transitions from the active state1410to a pre-fetch state1415, as indicated by the arrow1413, if the ring buffer has previously been connected to the virtual graphics pipeline. Otherwise, the state machine1400transitions directly from the active state1410to a connected state1420, as indicated by the arrow1414. In the pre-fetch state1415, the state machine1400reestablishes a persistent state associated with the previous connection before initiating processing of the commands from the ring buffer. The state machine1400then transitions from the pre-fetch state1415to the connected state1420, as indicated by the arrow1416.

In the connected state1420, the state machine1400instructs a fetcher such as the packet fetch block1230shown inFIG.12to retrieve entries from the ring buffer and use them to populate a queue that provides tasks or commands to a subsequent meta-pipe fragment in the virtual graphics pipeline, as discussed herein. The state machine1400can transition from the connected state1420to a switch state1425if it is necessary to rewind the state for synchronization or indirection, as indicated by the arrow1426. While in the switch state1425, the state machine1400can handle indirection with continue or wait actions. The state machine1400can transition back to the connected state1420, as indicated by the arrow1428, or the state machine1400can transition to a waiting state1430(as indicated by the arrow1431) if a wait action or a semaphore is generated. In the waiting state1430, the state machine1400waits for a predetermined time or services semaphores that are used for synchronization.

The state machine1400is also able to transition from the connected state1420or the waiting state1430to a de-queue state1435. The state machine1400transitions from the connected state1420to the de-queue state1435(as indicated by the arrow1436) in response to determining that the ring buffer is empty and the queue for the meta-pipe fragment is also empty. The state machine1400can also evaluate whether it has reached an end of a time slice or a packet boundary, whether the ring buffer or queue has been removed by OS/LLD functionality, or whether the corresponding application has been terminated. The state machine1400transitions from the connected state1420to the de-queue state1435in response to any of these conditions being satisfied. The state machine1400transitions from the waiting state1430to the de-queue state1435(as indicated by the arrow1438) in response to the ring buffer or queue being removed by the OS/LLD functionality.

In the de-queue state1435, the state machine1400handles time slice, empty, or OS/LLD removal of the corresponding ring buffer or queue. The state machine1400then transitions from the de-queue state1435to the idle state1405, as indicated by the arrow1440.

FIG.15is a block diagram of a meta-pipe fragment1500according to some embodiments. The meta-pipe fragment1500is used to implement some embodiments of the meta-pipe fragments315-317shown inFIG.3. The meta-pipe fragment1500operates in the meta-pipe domain915shown inFIG.9. The meta-pipe fragment1500fetches command packets from ring buffers or queues that are activated and promoted by a preceding super pipe fragment such as the super pipe fragment1400shown inFIG.14.

Ring buffers1505represent the ring buffers from the preceding super pipe fragment that have been activated and promoted. The meta-pipe fragment1500also receives key entries for descriptors of the activated queues in the ring buffer1505from the preceding super pipe fragment.

A first prefetch parser1510transmits instructions to a fetch engine1515to begin prefetching command packets or metacommands from the ring buffers1505. In some embodiments, the ring buffer1505includes commands with pointers or indices that are used for indirect fetching of the command packets or metacommands. In that case, the meta-pipe fragment1500performs multiple levels of fetching. Each level of indirection requires a memory access latency compensation buffer and an additional prefetch parser engine to decode data retrieval metacommands and initiate DMA fetch for the application data stream. For example, the meta-pipe fragment1500can include an indirect access buffer1520to compensate for the memory access latency by buffering the pointers or indices used for indirect fetching. The meta-pipe fragment1500also includes a second prefetch parser1525that transmits instructions to the fetch engine1515to prefetch command packets or metacommands indicated by the pointers or indices. Indirection can be performed using a complete address pointer that is retrieved from a primary metacommand or indirection can be performed using an input assembler index that is used to address 3-D graphics data via a surface base register in a state register pool.

In the illustrated embodiment, the second prefetch parser1525is able to prefetch command packets or metacommands from a memory hierarchy1530, which includes memory elements, caches, registers, and the like. The second prefetch parser1525can also initiate fetching of information in state registers by transmitting instructions to a state register fetch block1535.

The meta-pipe fragment1500includes a metacommand buffer1540that is used to hide latency of a metacommand parser1545by buffering metacommands that are fetched or prefetched by other elements of the meta-pipe fragment1500. The metacommand parser1545decodes the fetched or prefetched metacommands. Some embodiments of the metacommand parser1545also initiate memory access to retrieve data that can be used by a dispatch block1550to dispatch command packets or metacommands for processing by one or more subsequent virtual pipe fragments. The dispatch block1550organizes and dispatches data in a format that is determined based on the processing requirements of the subsequent virtual pipe fragments, which differ between different types of applications that implement different programming models. The dispatch block1550can implement grid dispatching or workgroup dispatching for compute and graphics data in 3-D graphics data processing. Some embodiments of the meta-pipe fragment1500also support interrupts and context preemption.

FIG.16is a block diagram illustrating virtual pipe fragments in a virtual graphics pipeline according to some embodiments. The virtual pipe fragments1601,1602,1603(collectively referred to herein as “the virtual pipe fragments1601-1603”) are used to implement some embodiments of the virtual pipe fragments320-322shown inFIG.3. The virtual pipe fragments1601-1603are implemented in the virtual pipe domain920shown inFIG.9.

The virtual pipe fragments1601-1603are associated with a virtual graphics pipeline that includes a super pipe fragment such as the super pipe fragment1400shown inFIG.14and a meta-pipe fragment such as the meta-pipe fragment1500shown in FIG.15. The virtual pipe fragment1601receives data dispatched by a dispatch block in the meta-pipe fragment such as the dispatch block1550shown inFIG.15. The data is processed by the virtual pipe fragment1601, routed to the virtual pipe fragment1602by the router1605, processed by the virtual pipe fragment1602, routed to the virtual pipe fragment1603by the router1610, and then processed by the virtual pipe fragment1603. Some embodiments of the virtual graphics pipeline include more or fewer virtual pipe fragments and corresponding routers.

The virtual pipe fragments1601-1603are configured by selecting physical pipe fragments1604to implement the virtual pipe fragments1601-1603and defining the processing requirements for applications that are being executed by the virtual graphics pipeline. The physical pipe fragments1604can be implemented as either a firmware thread that is executed on programmable micro-engines or a shader kernel that is executed on respective shader computing units. The physical pipe fragments1604can also be implemented as fixed function hardware blocks or a sequence of fixed function hardware blocks that are configured using corresponding state information and allocated to support the functionality of the virtual pipe fragments1601-1603. The virtual pipe fragments1601-1603schedule tasks for execution by the corresponding physical pipe fragments1604. For example, the virtual pipe fragments1602includes a physical pipe fragment dispatch/scheduling block1615. The block1615can be internal to the virtual pipe fragment1602, external to the virtual pipe fragment1602, or mixed internal/external depending on the mapping to the micro-engine, shader compute unit, or fixed function hardware or firmware resources.

The virtual pipe fragment1602also includes a data input buffer1620for buffering data that is input to the physical pipe fragment1604and a data output buffer1625for buffering data that is output from the physical pipe fragment1604. The buffers1620,1625can be implemented as separate entities or pre-allocated registers or they can be allocated space in a memory hierarchy for other buffer. Input and output command queues or state queues for the physical pipe fragment1604are also implemented in some embodiments of the virtual pipe fragment1602. The virtual pipe fragment1602also includes input and output command/state/ID queues1630,1635for holding information that is provided to a physical pipe fragment control block1640that provides operation control and sequencing functionality for the physical pipe fragment1604.

Some embodiments of the routers1605,1610implement join or fork functionality that allows additional virtual pipe fragments to be joined to the virtual pipeline or fork from the virtual pipe fragment. For example, the router1605can use join functionality to join another virtual pipe fragment (not shown) to the virtual graphics pipeline at the virtual pipe fragment1602. For another example, the router1610can use fork functionality to fork the output data stream from the virtual pipe fragment1602to another virtual pipe fragment (not shown) in the virtual graphics pipeline. The joint-fork functionality allows the on-chip virtual graphics pipelines to be arranged in multiple possible shapes, which can be used to match load balancing requirements between the virtual graphics and compute pipelines.

FIG.17is a block diagram illustrating allocation of resources of a graphics processing system to a virtual pipe fragment1700according to some embodiments. The virtual pipe fragment1700is used to implement some embodiments of the virtual pipe fragment1600shown inFIG.16. The virtual pipe fragment1700therefore includes physical pipe fragments1705and a physical pipe fragment control block1710that correspond to the physical pipe fragments1604and the physical pipe fragment control block1640shown inFIG.16. The virtual pipe fragment1700also includes input and output buffers1715,1716and input and output command/state/ID queues1720,1721that correspond to the same entities in the virtual pipe fragment1600shown inFIG.16.

In the illustrated embodiment, the graphics processing system includes shared resources such as hardware shader resources1735including one or more graphics processing cores in a unified shader pool, firmware threads1740that are executed on one or more hardware RISC micro-engines, a pool1745of fixed function hardware that are configured to perform different functions, and a memory hierarchy1750that includes memory elements such as a DRAM, queues, buffers, registers, caches, and the like. A resource allocation and mapping block1730is used to allocate the resources of the graphics processing system to the virtual pipe fragment1700and establish the mapping between the resources and the entities in the virtual pipe fragment1700. For example, shader resources1735, firmware threads1740, fixed function hardware units1745, or a combination thereof can be allocated to implement the physical pipe fragments1705. For another example, resources of the memory hierarchy1750can be allocated to implement the input and output buffers1715,1716and the input and output command/state/ID queues1720,1721. Some embodiments of the resource allocation and mapping block1730are implemented in the VPF control block342shown inFIG.3.

The resources of the graphics processing system also include shader kernel code1755, firmware microcode1760, state registers1765, and hardware state machines1770. These resources can be allocated to implement the physical pipe fragment control block1710. For example, shader kernel code1755, firmware microcode1760, or a combination thereof can be used to implement the functionality of the physical pipe fragment control block1710. The state registers1765and the hardware state machines1770can then be configured to store and utilize state information to determine the current operational state of the physical pipe fragment control block1710, which can provide control signaling to the physical pipe fragment1705based on its operational state and other information available to the physical pipe fragment control block1710. In some embodiments, control and sequencing in the physical pipe fragment1705depends on processing data-path selection: executable kernel code1755provides control and sequencing for shader resources1735, firmware microcode1760provides control and sequencing for the firmware threads1740executing on micro-engines, the hardware state machines1770or hardwired microcode provide control and sequencing for the fixed function hardware blocks1745. In some cases, state control bit fields in reconfigurable GPU state registers of a synchronous flow are used to determine the control and sequencing signals.

FIG.18is a block diagram of a graphics processing system1800that includes a configurable number of virtual graphics pipelines that are each implemented using a configurable number of pipeline fragments according to some embodiments. The graphics processing system1800is used to implement some embodiments of the graphics processing system300shown inFIG.3.

The graphics processing system includes a set of ring buffers1801,1802,1803,1804that are collectively referred to herein as “the ring buffers1801-1804.” As discussed herein with regard toFIG.11, the ring buffers1801-1804include a set of entries specified by a head pointer and a tail pointer. The entries in the ring buffers1801-1804can hold command packets, metacommands, pointers to commands, indices for commands, and the like. In the illustrated embodiment, the ring buffer1801is reserved for holding information related to 3-D graphics processing. Information is provided to the ring buffers1801-1804by processor cores such as cores implemented in a CPU. The ring buffers1801-1804can be dynamically allocated (or de-allocated) from a memory hierarchy in response to the instantiation of virtual graphics pipelines or the removal of virtual graphics pipelines. The ring buffers1801-1804are associated with corresponding context status descriptors.

A block1805includes a command processor, a memory hierarchy, and a memory resource cluster that is used to instantiate and allocate resources to virtual graphics pipelines1810,1811,1812, which are collectively referred to herein as “the virtual graphics pipelines1810-1812.” The block1805maps the ring buffers1801-1804to different virtual graphics pipelines1810-1812and the context or status of the virtual graphics pipelines1810-1812is determined by the corresponding context status descriptors. The block1805can retrieve commands or data from the ring buffers1801-1804and route the commands or data to the appropriate virtual graphics pipelines1810-1812based on the mapping. The block1805can also remove one or more of the virtual graphics pipelines1810-1812, e.g., in response to the virtual graphics pipeline completing tasks in a thread allocated to the virtual graphics pipeline. The block1805then deallocates resources of the removed virtual graphics pipeline.

The virtual graphics pipelines1810-1812include configuration/control blocks, super pipe fragments, meta-pipe fragments virtual pipe fragments, and a mapping to allocated physical pipe fragments and memory resources. For example, the virtual graphics pipeline1810includes a configuration/control block1820, a super pipe fragment1821, a meta-pipe fragment1822, one or more virtual pipe fragments1823, and a mapping1824to the physical pipe fragments and memory resources that are allocated to the virtual graphics pipeline1810. For another example, the virtual graphics pipeline1811includes a configuration/control block1830, a super pipe fragment1831, a meta-pipe fragment1832, one or more virtual pipe fragments1833, and a mapping1834to the physical pipe fragments and memory resources that are allocated to the virtual graphics pipeline1811. For yet another example, the virtual graphics pipeline1812includes a configuration/control block1840, a super pipe fragment1841, a meta-pipe fragment1842, one or more virtual pipe fragments1843, and a mapping1844to the physical pipe fragments and memory resources that are allocated to the virtual graphics pipeline1812.

The virtual graphics pipelines1810-1812share fixed function hardware resources1850and shader engines from a unified shader engine pool1855. The fixed function hardware resources1850and the unified shader engine pool1855contain multiple physical processing blocks or physical pipe fragments that can be allocated to any pipeline fragments in one of the virtual graphics pipeline1810-1812. Thus, the number of virtual graphics pipelines1810-1812that are instantiated at any given time can be larger or smaller than the number of physical processing blocks or physical pipe fragments in the fixed function hardware resources1850or the unified shader engine pool1855.

The virtual graphics pipelines1810-1812access the shared fixed function hardware resources1850via access arbitration1860that arbitrates between access requests by the different virtual graphics pipelines1810-1812, as discussed herein with regardFIG.8. The shared fixed function hardware resources1850can include fixed function hardware units configured to perform pixel shading, scan conversion, primitive assembly, vertex shading, graphics shading, hull shading, local shading, and the like.

The virtual graphics pipelines1810-1812access the unified shader pool1855via a set of queues1865. Some embodiments of the queues1865include thread group queues that are used to support multiple shader pipes that concurrently issue executable compute kernels to an arbitration and dispatch unit1870. The arbitration and dispatch unit1870can perform arbitration in the manner discussed herein with regard toFIG.8. Some embodiments of the arbitration and dispatch unit1870form thread group queues that hold SIMD compute waves for dispatch to the unified shader pool1855. The multiple shader pipes can be referred to as asynchronous compute engines, as discussed herein. Memory hierarchy and on-chip buffer resources are shared by all active processing SIMD computing and fixed function hardware blocks in the fixed function hardware resources1850and the unified shader engine pool1855.

FIG.19is a flow diagram of a method1900instantiating a virtual graphics pipeline according to some embodiments. The method1900is implemented in some embodiments of the graphics processing system300shown inFIG.3, the graphics processing system400shown inFIG.4, the graphics processing system600shown inFIG.6, and the graphics processing system700shown inFIG.7.

At block1905, the graphics processing system determines a number of queues and pipeline fragments for a virtual graphics pipeline. The number of queues or the number of pipeline fragments can be determined based on the requirements of an application that is generating the thread for execution on the virtual graphics pipeline. The number of pipeline fragments can include a super pipe fragment, a meta-pipe fragment, and one or more virtual pipe fragments. The queues and pipeline fragments can be allocated in response to user input, e.g., in response to a user initiating an application that generates the thread or in response to a system event such as an operating system or application generating a new thread.

At block1910, the graphics processing system allocates shared resources to support the queues and pipeline fragments of the virtual graphics pipeline. The shared resources include graphics processing cores that can be a part of a unified shader pool and fixed function hardware, as discussed herein. The allocated resources can be referred to as physical pipe fragments. The resources of the graphics processing system are shared with other virtual graphics pipelines so that the virtual graphics pipelines can concurrently execute commands using the shared resources.

At block1915, the operational states of the pipeline fragments are configured using state information associated with the virtual graphics pipeline. As discussed herein, the state information can be stored in and accessed from queues, registers, ring buffers, caches, or other memory elements.

At block1920, the virtual graphics pipeline executes one or more commands that are retrieved from the queues that are associated with the virtual graphics pipeline. The commands are executed using the shared hardware resources that are allocated to the virtual graphics pipeline. The virtual graphics pipeline executes the commands concurrently with other virtual graphics pipelines executing other commands using the shared resources of the graphics processing system.

At block1925, the graphics processing system de-allocates the shared resources that were allocated to the virtual graphics pipeline. The shared resources can be de-allocated in response to the graphics processing system terminating the virtual graphics pipeline e.g., because the virtual graphics pipeline has completed executing the commands in the thread associated with the virtual graphics pipeline. De-allocating the shared resources includes deallocating resources of the shared graphics cores, the fixed function hardware, or firmware emulations of the fixed function hardware.

FIG.20is a flow diagram of a method2000for selectively emulating fixed function hardware using a firmware emulation according to some embodiments. The method2000is implemented in some embodiments of the graphics processing system300shown inFIG.3, the graphics processing system400shown inFIG.4, the graphics processing system600shown inFIG.6, and the graphics processing system700shown inFIG.7.

At block2005, the graphics processing system instantiates a virtual graphics pipeline including resources of a fixed function hardware unit. For example, a virtual pipe fragment can transmit calls to the fixed function hardware unit to request that the fixed function hardware unit perform a particular operation and return results of the operation. As discussed herein, the fixed function hardware unit can be shared by multiple virtual graphics pipelines or pipeline fragments within a single virtual graphics pipeline.

At block2010, the graphics processing system monitors a throughput of the fixed function hardware unit or other indication of the loading of the fixed function hardware units such as queue statuses, buffer occupancy, and the like.

At decision block2015, the graphics processing system determines whether there is a bottleneck at the fixed function hardware unit. For example, the throughput of the fixed function hardware unit can be compared to a threshold value. If the throughput falls below the threshold value, the graphics processing system detects a bottleneck at the fixed function hardware unit. The other indicators such as the queue status or the buffer occupancy can also be compared to corresponding thresholds to detect bottlenecks in the fixed function hardware unit. As long as the graphics processing system does not detect a bottleneck at the fixed function hardware unit, the graphics processing system continues to monitor throughput of the fixed function hardware unit at block2010. If the graphics processing system detects a bottleneck, the method2000flows to block2020.

At block2020, the graphics processing system instantiates a firmware emulation of the fixed function hardware unit, e.g., using shader firmware that is executing on one or more graphics processing kernels in the graphics processing system. At block2025, the graphics processing system routes pipeline traffic to the firmware emulation of the fixed function hardware unit instead of routing the pipeline traffic to the fixed function hardware unit. Re-routing the pipeline traffic to the firmware emulation can alleviate the bottleneck at the fixed function hardware unit and improve overall performance of the graphics processing system.

In some embodiments, the apparatus and techniques described above are implemented in a system comprising one or more integrated circuit (IC) devices (also referred to as integrated circuit packages or microchips), such as the graphics processing system including configurable virtual graphics pipelines described above with reference toFIGS.1-20. Electronic design automation (EDA) and computer aided design (CAD) software tools are used in the design and fabrication of these IC devices. These design tools typically are represented as one or more software programs. The one or more software programs comprise code executable by a computer system to manipulate the computer system to operate on code representative of circuitry of one or more IC devices so as to perform at least a portion of a process to design or adapt a manufacturing system to fabricate the circuitry. This code can include instructions, data, or a combination of instructions and data. The software instructions representing a design tool or fabrication tool typically are stored in a computer readable storage medium accessible to the computing system. Likewise, the code representative of one or more phases of the design or fabrication of an IC device is stored in and accessed from the same computer readable storage medium or a different computer readable storage medium.

A computer readable storage medium includes any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer readable storage medium can be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

In some embodiments, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing software. The software comprises one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer readable storage medium can include, for example, a magnetic or optical disk storage device, solid state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer readable storage medium are in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.