Patent Description:
<CIT>, Reconfigurable Virtual Graphics and Compute Processor Pipeline, discloses a graphics processing unit (GPU) including a plurality of programmable processing cores configured to process graphics primitives and corresponding data and a plurality of fixed-function hardware units. The plurality of processing cores and the plurality of fixed-function hardware units are configured to implement a configurable number of virtual pipelines to concurrently process different command flows. Each virtual pipeline includes a configurable number of fragments and an operational state of each virtual pipeline is specified by a different context. The configurable number of virtual pipelines can be modified from a first number to a second number that is different than the first number. An emulation of a fixed-function hardware unit can be instantiated on one or more of the graphics processing cores in response to detection of a bottleneck in a fixed-function hardware unit. One or more of the virtual pipelines can then be reconfigured to utilize the emulation instead of the fixed-function hardware unit. <CIT>, Dynamic Load Balancing Apparatus and Method for Graphic Processing Unit (GPU), discloses a GPU including at least one shader processor which may assign a vertex shader task and a pixel shader task to the at least one shader processor.

Processing on a GPU is typically initiated by application programming interface (API) calls (e.g., draw calls) that are processed by a CPU. A draw call is a command that is generated by the CPU and transmitted to the GPU to instruct the GPU to render an object (or a portion of an object) in a frame. The draw call includes information defining textures, states, shaders, rendering objects, buffers, and the like that are used by the GPU to render the object or portion thereof. In response to receiving a draw call, the GPU renders the object to produce values of pixels that are provided to a display, which uses the pixel values to display an image that represents the rendered object. The object is represented by primitives such as triangles, patches, or other polygons that include multiple vertices connected by corresponding edges. An input assembler fetches the vertices based on topological information indicated in the draw call. The vertices are provided to a graphics pipeline for shading according to corresponding commands that are stored in a command buffer prior to execution by the GPU. The commands in the command buffer are written to a queue (or ring buffer) and a scheduler schedules the command buffer at the head of the queue for execution on the GPU.

The hardware used to implement the GPU is typically configured based on the characteristics of an expected workload. For example, if the workload processed by the GPU is expected to produce graphics at <NUM> resolution, the GPU processes up to eight primitives per clock cycle to guarantee a target quality of service and level of utilization. For another example, if the workload processed by the GPU is expected to produce graphics at a much lower 1080p resolution, the GPU guarantees a target quality of service and level of utilization when processing workloads at the lower 1080p resolution. Although conventional GPUs are optimized for a predetermined type of workload, many GPUs are required to process workloads that have varying degrees of complexity and output resolution. For example, a flexible cloud gaming architecture includes servers that implement sets of GPUs for concurrently executing a variety of games at different levels of user experience that potentially range from 1080p resolution all the way up to <NUM> resolution depending on the gaming application and the level of experience requested by the user. Although a lower-complexity or lower-resolution game can execute on a GPU that is optimized for higher complexity or resolution, a difference between the expected complexity or resolution of an optimized GPU and the actual complexity or resolution required by the application often leads to underutilization of the resources of the higher performance GPU. For example, serial dependencies between commands in a lower complexity/resolution game executing on a higher performance GPU reduce the amount of pixel shading that is performed in parallel, which results in underutilization of the resources of the GPU.

<FIG> disclose embodiments of a reconfigurable graphics processing unit (GPU) that includes front end (FE) circuitry and shader engines that are spatially partitioned to execute multiple concurrent graphics streams having different characteristics. The FE circuitry fetches primitives for geometry workloads, performs scheduling of the geometry workloads for execution on the shader engines and, in some cases, handles serial synchronization, state updates, draw calls, cache activities, and tessellation of primitives. The shader engines shade the vertices of the primitives (as scheduled by the FE circuitry) and shade the pixels generated based on the shaded primitives. In some embodiments, the FE circuitry includes multiple FE circuits that selectively schedule the geometry workloads for concurrent execution on corresponding subsets of the shader engines. Using different FE circuits to schedule workloads for execution on different subsets of the shader engines is referred to herein as "spatial partitioning" of the shader engines.

The amount of spatial partitioning that is available in a reconfigurable GPU depends on the number of independent FE circuits implemented in the FE circuitry. For example, if the FE circuitry includes two FE circuits, a first FE circuit schedules the geometry workloads for all the shader engines in a first operational mode. In a second (partitioned) operational mode, the first FE circuit schedules the geometry workloads for execution on a first subset of the shader engines and a second FE circuit schedules the geometry workloads for execution on a second subset of the shader engines concurrently with execution of the geometry workloads on the first subset. In some embodiments, the multiple FE circuits are configured based on different levels of user experience corresponding to different complexities or graphics resolutions. For example, a GPU including four shader engines include a first FE circuit that is optimized for high complexity/resolution, two second FE circuits that are optimized for medium complexity/resolution, and a third FE circuit that is optimized for low complexity/resolution. The GPU is therefore reconfigurable to support one high complexity/resolution application (such as a game that provides <NUM> resolution) using the first FE circuit, two medium complexity/resolution applications (such as games that provide <NUM> resolution) using the two second FE circuits, or four low complexity/resolution applications (such as games that provide 1080p resolution) using the first, second, and third FE circuits. In some embodiments, one or more of the multiple FE circuits support multiple concurrent threads using time division multiplexing.

<FIG> is a block diagram of a processing system <NUM> that implements spatial partitioning in a multi-tenancy graphics processing unit (GPU) <NUM> according to some embodiments. The processing system <NUM> includes one or more central processing units (CPUs) <NUM>, <NUM>. Although two CPUs <NUM>, <NUM> are shown in <FIG>, some embodiments of the processing system <NUM> include more or fewer CPUs. A scalable data fabric (SDF) <NUM> supports data flows between endpoints within the processing system <NUM>. Some embodiments of the SDF <NUM> support data flows between connecting points such as peripheral component interface (PCI) physical layers, memory controllers, universal serial bus (USB) hubs, computing and execution units including the GPU <NUM> and the CPUs <NUM>, <NUM>, as well as other endpoints. In the illustrated embodiment, the SDF <NUM> is connected to an input/output (I/O) hub <NUM>, which is in turn connected to a PCI express (PCI-E) bus <NUM> and an NBIF <NUM>. The processing system <NUM> also includes a scalable control fabric (SCF) <NUM> is a control communication plane that conveys system control signals within the processing system <NUM>. Examples of system control signals are control signals used to support thermal and power management, tests, security, and the like.

The GPU <NUM> includes a set of shader engines (SE) <NUM>, <NUM>, <NUM>, <NUM> (collectively referred to herein as "the SE <NUM>-<NUM>") that are used to execute commands concurrently or in parallel. Some embodiments of the SE <NUM>-<NUM> are configured using information in draw calls received from one of the CPUs <NUM>, <NUM> to shade vertices of primitives that represent a model of a scene. The SE <NUM>-<NUM> also shade the pixels generated based on the shaded primitives and provide the shaded pixels to a display for presentation for user, e.g., via the I/O hub <NUM>. Although four shader engines are shown in <FIG>, some embodiments of the GPU <NUM> include more or fewer shader engines. The SE <NUM>-<NUM> are connected to a graphics L2 cache <NUM> that stores frequently used data and instructions. In some embodiments, the L2 cache <NUM> is connected to one or more L1 caches that are implemented in the SE <NUM>-<NUM> and one or more L3 caches (or other last level caches) implemented in the processing system <NUM>. The caches form a cache hierarchy that includes the L2 cache <NUM>. The other caches in the cache hierarchy are not shown in <FIG> in the interest of clarity.

Front end (FE) circuitry in the GPU <NUM> fetches primitives for geometry workloads, performs scheduling of the geometry workloads for execution on the shader engines and, in some cases, handles serial synchronization, state updates, draw calls, cache activities, and tessellation of primitives. The FE circuitry in the GPU <NUM> includes FE circuits <NUM>, <NUM>, although some embodiments of the FE circuitry are partitioned to include additional FE circuits, as discussed herein. The FE circuits <NUM>, <NUM> include command processors <NUM>, <NUM> that receives command buffers for execution on the SE <NUM>-<NUM>. The FE circuits <NUM>, <NUM> also include graphics register bus managers (GRBMs) <NUM>, <NUM> that act as hubs for register read and write operations that support multiple masters and multiple slaves.

The GPU <NUM> operates in either a first mode or a second, spatially partitioned mode. In the first mode, the FE circuit <NUM> schedules geometry workloads for the SE <NUM>-<NUM>. In the second mode, the FE circuit <NUM> schedules geometry workloads for a first subset of the SE <NUM>-<NUM> and the FE circuit <NUM> schedules geometry workloads for a second subset of the SE <NUM>-<NUM>. The first subset includes the SE <NUM>, <NUM> and the second subset includes the SE <NUM>, <NUM>, although other groupings of the SE <NUM>-<NUM> into subsets are used in some embodiments. The GPU <NUM> includes a partition switch <NUM> that selectively connects the FE circuits <NUM>, <NUM> to the first and second subsets of the SE <NUM>-<NUM> depending on whether the GPU <NUM> is operating in the first mode or the second mode. In the illustrated embodiment, the partition switch <NUM> determines the operational status of the GPU <NUM>. If the GPU <NUM> is operating in the first mode, the partition switch <NUM> connects the FE circuit <NUM> to the SE <NUM>, <NUM> so that the FE circuit <NUM> schedules operations to all the SE <NUM>-<NUM>. If the GPU <NUM> is operating in the second mode, the partition switch <NUM> connects the FE circuit <NUM> to the SE <NUM>, <NUM> so that the FE circuit <NUM> schedules operations to the SE <NUM>, <NUM> and the FE circuit <NUM> schedules operations to the SE <NUM>, <NUM>.

<FIG> is a block diagram of a mapping <NUM> of FE circuits <NUM>, <NUM> to a set of SE <NUM>, <NUM>, <NUM>, <NUM> for a GPU operating in a first mode according to some embodiments. The mapping <NUM> indicates a mapping of some embodiments of the FE circuits <NUM>, <NUM> to the SE <NUM>-<NUM> in the GPU <NUM> shown in <FIG>. The GPU is operating in the first mode and the FE circuit <NUM> is mapped to all the SE <NUM>-<NUM>. The FE circuit <NUM> therefore schedules commands for concurrent execution on the SE <NUM>-<NUM>. The FE circuit <NUM> is not mapped to any of the SE <NUM>-<NUM> and therefore does not schedule commands for execution on any of the SE <NUM>-<NUM>, as indicated by the dashed outline of the box representing the FE circuit <NUM>.

<FIG> is a block diagram of a mapping <NUM> of FE circuits <NUM>, <NUM> to a set of SE <NUM>, <NUM>, <NUM>, <NUM> for a GPU operating in a second mode according to some embodiments. The mapping <NUM> indicates a mapping of some embodiments of the FE circuits <NUM>, <NUM> to the SE <NUM>-<NUM> in the GPU <NUM> shown in <FIG>. The GPU is operating in the second mode and the FE circuit <NUM> is mapped to a first subset of the SE <NUM>-<NUM> that includes the SE <NUM>, <NUM>. The FE circuit <NUM> therefore schedules commands for execution on the SE <NUM>, <NUM>. The FE circuit <NUM> is mapped to a second subset of the SE <NUM>-<NUM> that includes the SE <NUM>, <NUM>. The FE circuit <NUM> therefore schedules commands for execution on the SE <NUM>, <NUM>. The FE circuit <NUM>, <NUM> schedule commands for concurrent execution on their corresponding first and second subsets of the SE <NUM>-<NUM>.

<FIG> is a block diagram of a GPU <NUM> that includes a set of FE circuits that are configured based on different characteristics of applications that provide instructions for execution by the GPU according to some embodiments. The GPU <NUM> includes a set of SE <NUM>, <NUM>, <NUM>, <NUM>, which are collectively referred to herein a "the SE <NUM>-<NUM>" and execute instructions concurrently or in parallel. The GPU <NUM> also includes FE circuits <NUM>, <NUM>, <NUM>, <NUM>, which are collectively referred to herein as "the FE circuits <NUM>-<NUM>. " The FE circuits <NUM>-<NUM> are configured based on different levels of user experience corresponding to different complexities or graphics resolutions. In the illustrated embodiment, the FE circuit <NUM> is configured based upon the requirements of applications that have a high complexity or graphics resolution, such as a game that implements a sophisticated physics engine or provides <NUM> resolution. The FE circuits <NUM>, <NUM> are configured based upon the requirements of applications that have a medium complexity or graphics resolution, such as games that provide <NUM> resolution. The FE circuit <NUM> is configured based upon the requirements of applications that have low complexity or graphics solution resolution, such as games that provide 1080p resolution.

A partition switch <NUM> selectively maps subsets of the FE circuits <NUM>-<NUM> to corresponding subsets of the SE <NUM>-<NUM>. The map indicates connections between the FE circuits <NUM>-<NUM> and the SE <NUM>-<NUM>, as well as indicating which of the FE circuits <NUM>-<NUM> is responsible for scheduling commands to one or more of the SE <NUM>-<NUM>. Some embodiments of the partition switch <NUM> selectively map the subsets of the FE circuits <NUM>-<NUM> to the corresponding subsets of the SE <NUM>-<NUM> based on characteristics of applications that provide commands for execution on the SE <NUM>-<NUM>. For example, the GPU <NUM> can operate in one of a plurality of modes depending on the characteristics of the applications. The partition switch <NUM> determines the current operation mode based on either signaling associated with the GPU <NUM> or using other indications of the characteristics of the application. The partition switch <NUM> then selectively determines a mapping between the SE <NUM>-<NUM> and the FE circuits <NUM>-<NUM> based on the operating mode.

<FIG> is a block diagram of a mapping <NUM> of FE circuits <NUM>, <NUM>, <NUM>, <NUM> to a set of SE <NUM>, <NUM>, <NUM>, <NUM> for a GPU operating at a high level of user experience according to some embodiments. The mapping <NUM> indicates a mapping of some embodiments of the FE circuits <NUM>-<NUM> to the SE <NUM>-<NUM> in the GPU <NUM> shown in <FIG>. The GPU is executing commands provided by an application that requires a relatively high level of user experience, e.g., a high level of complexity or graphics resolution. The FE circuit <NUM> supports high levels of user experience and therefore the FE circuit <NUM> is mapped to the SE <NUM>-<NUM>. The FE circuit <NUM> schedules commands for concurrent execution on the SE <NUM>-<NUM>. The FE circuits <NUM>-<NUM> are not mapped to the SE <NUM>-<NUM> and therefore do not schedule commands for execution on the SE <NUM>-<NUM>, as indicated by the dashed boxes that represent the FE circuits <NUM>-<NUM>.

<FIG> is a block diagram of a mapping <NUM> of FE circuits <NUM>, <NUM>, <NUM>, <NUM> to a set of SE <NUM>, <NUM>, <NUM>, <NUM> for a GPU operating at a medium level of user experience according to some embodiments. The mapping <NUM> indicates a mapping of some embodiments of the FE circuits <NUM>-<NUM> to the SE <NUM>-<NUM> in the GPU <NUM> shown in <FIG>. The GPU is executing commands provided by an application that requires a medium level of user experience, e.g., a medium level of complexity or graphics resolution. The FE circuits <NUM>, <NUM> supports medium levels of user experience. In the illustrated embodiment, the FE circuit <NUM> is mapped to the SE <NUM>, <NUM> and the FE circuit <NUM> is mapped to the SE <NUM>, <NUM>. The FE circuits <NUM>, <NUM> therefore schedule commands for concurrent execution on the corresponding subsets of the SE <NUM>-<NUM>. The FE circuits <NUM>, <NUM> are not mapped to the SE <NUM>-<NUM> and therefore do not schedule commands for execution on the SE <NUM>-<NUM>, as indicated by the dashed boxes that represent the FE circuits <NUM>, <NUM>. However, in some embodiments, the FE circuit <NUM> is mapped to a subset of the SE <NUM>-<NUM> because the FE circuit <NUM> is capable of scheduling commands for applications requiring a medium level of user experience.

<FIG> is a block diagram of a mapping <NUM> of FE circuits <NUM>, <NUM>, <NUM>, <NUM> to a set of SE <NUM>, <NUM>, <NUM>, <NUM> for a GPU operating at a low level of user experience according to some embodiments. The mapping <NUM> indicates a mapping of some embodiments of the FE circuits <NUM>-<NUM> to the SE <NUM>-<NUM> in the GPU <NUM> shown in <FIG>. The GPU is executing commands provided by an application that requires a low level of user experience, e.g., a low level of complexity or graphic resolution. All the FE circuits <NUM>-<NUM> are capable of scheduling commands to the SE <NUM>-<NUM> from applications that require a low level of user experience. The FE circuits <NUM>-<NUM> are therefore mapped to corresponding SE <NUM>-<NUM>. For example, the FE circuit <NUM> is mapped to (and schedules commands for) the SE <NUM>, the FE circuit <NUM> is mapped to (and schedules commands for) the SE <NUM>, the FE circuit <NUM> is mapped to (and schedules commands for) the SE <NUM>, and the FE circuit <NUM> is mapped to (and schedules commands for) the SE <NUM>. The FE circuits <NUM>-<NUM> schedule commands for concurrent execution on the corresponding SE <NUM>-<NUM>.

<FIG> is a block diagram of a GPU <NUM> that includes a set of FE circuits that schedule instructions in time division multiplexed threads for execution by a set of SE in the GPU according to some embodiments. The GPU <NUM> represents some embodiments of the GPU <NUM> shown in <FIG>. The set of FE circuits includes a first FE circuit <NUM> and a second FE circuit <NUM>, although some embodiments of the GPU <NUM> include more FE circuits in the set. The first FE circuit <NUM> schedules commands for execution on one or more corresponding SE including the first SE <NUM>. In the illustrated embodiment, the first FE circuit <NUM> schedules commands for a first thread <NUM> during a first time interval and a third time interval. The first FE circuit <NUM> also schedules commands or a second thread <NUM> during a second time interval that is time division multiplexed with the first and third time intervals. The second FE circuit <NUM> schedules commands for execution on one or more corresponding SE including the second SE <NUM>. In the illustrated embodiment, the second FE circuit <NUM> schedules commands for a third thread <NUM> during fourth time interval and a fifth time interval. The second FE unit <NUM> also schedules commands for a fourth thread <NUM> during a sixth time interval that is time division multiplexed with the fourth and fifth time intervals. Thus, the FE circuits <NUM>, <NUM> schedule commands in the threads <NUM>, <NUM>, <NUM>, <NUM> for concurrent execution on the SE <NUM>, <NUM>.

<FIG> is a flow diagram of a method <NUM> of selectively allocating FE circuits to schedule commands for concurrent execution on a set of SE according to some embodiments. The method <NUM> is implemented in some embodiments of the GPU <NUM> shown in <FIG>.

At block <NUM>, the GPU determines characteristics of one or workloads (or threads) that are provided for execution on the GPU. In some embodiments, the characteristics include, but are not limited to, complexity of the workloads or graphics resolutions required (or specified or preferred) by the workloads. The characteristics are determined based on information provided in the workload (or thread) or using other information that configures the GPU for execution of the workload (or thread).

At decision block <NUM>, the GPU determines whether one or more workloads (or threads) are to be executed concurrently. Examples of workloads that are executed concurrently include workloads having a complexity or graphics resolution that is less than or equal to a complexity or graphics resolution that is used to configure multiple FE circuitry implemented in the GPU, as discussed herein. If only a single workload is to be executed by the GPU, the method <NUM> flows to block <NUM>. If multiple workloads are to be scheduled concurrently, the method <NUM> flows to block <NUM>.

At block <NUM>, one FE circuit is allocated to schedule commands for concurrent execution on the set of SE. The other FE circuits that are available in the GPU are not allocated to schedule commands for execution on any of the set of SE.

At block <NUM>, a set of FE circuits are allocated to schedule commands for concurrent execution by corresponding subsets of the set of SE. At block <NUM>, the set of FE circuits schedule commands for concurrent execution by the corresponding subsets. For example, if two FE circuits are allocated, a first FE circuit schedules commands for execution on a first subset of the set of SE and a second FE circuit schedules commands for execution on a second subset of the set of SE. The first and second subsets execute the scheduled commands concurrently.

A computer readable storage medium includes any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. The computer readable storage medium can be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

In some embodiments, certain aspects of the techniques described above can implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer readable storage medium.

Claim 1:
An apparatus comprising:
a plurality of shader engines (<NUM>-<NUM>); and
a first front end, FE, circuit (<NUM>) and at least one second FE circuit (<NUM>),
wherein the first FE circuit is configured to schedule geometry workloads for the plurality of shader engines (<NUM>-<NUM>) in a first mode, and wherein the first FE circuit is configured to schedule geometry workloads for a first subset of the plurality of shader engines and the at least one second FE circuit is configured to schedule geometry workloads for a second subset of the plurality of shader engines in a second mode, the first subset being disjoint from the second subset.