PATENT DOCUMENT

Publication Number: US-10795730-B2
Application Number: US-201816145573-A
Country: US
Kind Code: B2

Title: Graphics hardware driven pause for quality of service adjustment

Abstract:
In general, embodiments are disclosed for tracking and allocating graphics processor hardware resources. More particularly, a graphics hardware resource allocation system is able to generate a priority list for a plurality of data masters for graphics processor based on a comparison between a current utilizations for the data masters and a target utilizations for the data masters. The graphics hardware resource allocation system designate, based on the priority list, a first data master with a higher priority to submit work to the graphics processor compared to a second data master. The graphics hardware resource allocation system determines a stall counter value for the data master and generates a notification to pause work for the second data master based on the stall counter value.

Claims:
The invention claimed is: 
     
       1. A non-transitory program storage device, readable by one or more processors and comprising instructions stored thereon to cause the one or more processors to:
 generate a priority list for a plurality of data masters for graphics processor based on a comparison between a current utilizations for the data masters and a target utilizations for the data masters; 
 assign, based on the priority list, a first data master of the plurality of data masters as a designated data master, wherein the designated data master has a higher priority to submit work to the graphics processor compared to a second data master of the plurality of data masters; 
 determine a stall counter value for the designated data master, wherein the stall counter value is indicative of a number of time periods the designated data master has work to submit to the graphics processor, but is unable to submit the work; and 
 generate a notification to pause work for the second data master based on the stall counter value. 
 
     
     
       2. The non-transitory program storage device of  claim 1 , where the instructions further cause the one or more processors to:
 send priority information indicating that the designated data master has a higher priority to a cluster; 
 receive, based on the priority information, a cluster stall counter value indicative of a number of time periods the cluster is unable to process the work for the designated data master; and 
 determine the stall counter value based on the cluster stall counter value. 
 
     
     
       3. The non-transitory program storage device of  claim 2 , wherein the instructions to determine the stall counter value comprises instructions that cause the one or more processors to:
 receive a second cluster stall counter value from a second cluster; 
 determine a maximum stall counter value based on the second cluster stall counter value and the cluster stall counter value; and 
 set the maximum stall counter value as the stall counter value. 
 
     
     
       4. The non-transitory program storage device of  claim 2 , wherein the instructions to determine the stall counter value comprises instructions that cause the one or more processors to:
 receive a second cluster stall counter value from a second hardware resource cluster; and 
 averaging the second cluster stall counter value with the cluster stall counter value to generate the stall counter value. 
 
     
     
       5. The non-transitory program storage device of  claim 1 , wherein the number of time periods represents a number of graphics processor clock cycles during which the first data master has the work to submit the graphics processor, but is unable to submit the work. 
     
     
       6. The non-transitory program storage device of  claim 1 , where the instructions further cause the one or more processors to:
 update the priority list after generating the notification to pause the second data master, wherein the updated priority list indicates that a third data master of the plurality of data masters has been assigned a highest priority to submit work to the graphics processor amongst the plurality of data masters; 
 designate, based on the updated priority list, the third data master of the plurality of data masters with a higher priority to submit work to the graphics processor than the first data master and the second data master; and 
 resume executing work for the second data master based on designating the third data master with the higher priority. 
 
     
     
       7. The non-transitory program storage device of  claim 6 , where the instructions further cause the one or more processors to:
 determine an updated stall counter value for the third data master, wherein the updated stall counter value is indicative of a number of time periods the designated third data master has work to submit the graphics processor, but is unable to submit the work; 
 compare the updated stall counter value with a threshold value; and 
 generate a second notification to pause the first data master and the second data master based on a determination that the updated stall counter value satisfies the threshold value. 
 
     
     
       8. The non-transitory program storage device of  claim 1 , wherein that notification to pause is a hardware driven notification that pauses all other data masters except the first data master within the priority list. 
     
     
       9. The non-transitory program storage device of  claim 1 , wherein the first data master has the highest priority to submit work to the graphics processor amongst the plurality of data masters. 
     
     
       10. A system comprising:
 memory; and 
 a graphics processor that interacts with the memory and includes a plurality of graphics hardware interfaces, wherein the graphics processor is configured to:
 generate a priority list for the plurality of graphics hardware interfaces based on a comparison between actual utilizations for the graphics hardware interfaces and target utilizations for the graphics hardware interfaces 
 identify, based on the priority list, a first graphics hardware interface of the plurality of graphics hardware interfaces as a designated graphics hardware interface, wherein the designated graphics hardware interface has a higher priority to submit work to the graphics processor compared to a second graphics hardware interface of the plurality of graphics hardware interfaces; 
 determine an overall stall counter value for the designated graphics hardware interface, wherein the overall stall counter value is based on a set of stall counter values generated from a set of clusters within the graphics processor; 
 compare the overall stall counter value to a threshold value; and 
 generate a notification to pause execution of work for the second graphics hardware interface based on a determination that the overall stall counter value satisfies the threshold value. 
 
 
     
     
       11. The system of  claim 10 , wherein the first graphics hardware interface and the second graphics hardware interface are associated with a first quality of service (QoS) layer. 
     
     
       12. The system of  claim 11 , wherein a third graphics hardware interface of the plurality of graphics hardware interfaces are associated with a second QoS layer. 
     
     
       13. The system of  claim 12 , wherein the notification to pause execution of work for the second graphics hardware interface is generated after pausing execution of work for the third graphics hardware interface. 
     
     
       14. The system of  claim 10 , wherein the graphics processor is further configured to:
 update the priority list after generating the notification to pause the second graphics hardware interface; 
 designate, based on the updated priority list, a third graphics hardware interface of the plurality of graphics hardware interfaces with a higher priority to submit work to the graphics processor than the first graphics hardware interface and the second graphics hardware interface; and 
 resume executing work for the second graphics hardware interface based on designating the third graphics hardware interface with the higher priority. 
 
     
     
       15. The system of  claim 14 , wherein the graphics processor is further configured to:
 determine a second overall stall counter value for the third graphics hardware interface, wherein the second overall stall counter value is indicative of a number of time periods the designated third graphics hardware interface has work to submit the graphics processor, but is unable to submit the work; 
 compare the second overall stall counter value with the threshold value; and 
 generate a third notification to pause the first graphics hardware interface and the second graphics hardware interface based on a determination that the second overall stall counter value satisfies the threshold value. 
 
     
     
       16. The system of  claim 10 , wherein the graphics processor is further configured to:
 send priority information indicating that the designated graphics hardware interface has a higher priority to a first cluster of the set of clusters; and 
 receive, based on the priority information, a first stall counter value of the set of stall counter values, wherein the first stall counter value is indicative of a number of time periods the first cluster is unable to process the work for the designated graphics hardware interface. 
 
     
     
       17. A computer-implemented method comprising:
 generating a priority list for a plurality of data masters for graphics processor based on a comparison between a current utilizations for the data masters and a target utilizations for the data masters; 
 selecting, based on the priority list, a first data master of the plurality of data masters as a designated data master, wherein the designated data master has a higher priority to submit work to the graphics processor compared to a second data master of the plurality of data masters; 
 determining a stall counter value for the designated data master, wherein the stall counter value is indicative of a number of time periods the designated data master has work to submit to the graphics processor, but is unable to submit the work; and 
 generating a notification to pause work for the second data master based on the stall counter value. 
 
     
     
       18. The method of  claim 17 , further comprising:
 updating the priority list after generating the notification to pause the second data master; 
 designating, based on the updated priority list, a third data master of the plurality of data masters with a higher priority to submit work to the graphics processor than the first data master and the second data master; and 
 resuming executing work for the second data master based on designating the third data master with the higher priority. 
 
     
     
       19. The method of  claim 18 , further comprising:
 determining an updated stall counter value for the third data master, wherein the updated stall counter value is indicative of a number of time periods the designated third data master has work to submit the graphics processor, but is unable to submit the work; 
 comparing the updated stall counter value with a threshold value; and 
 generating a third notification to pause the first data master and the second data master based on a determination that the updated stall counter value satisfies the threshold value. 
 
     
     
       20. The method of  claim 17 , wherein the number of time periods represents a number of graphics processor clock cycles during which the first data master has the work to submit the graphics processor, but is unable to submit the work.

Description:
BACKGROUND 
     This disclosure relates generally to the field of graphics processing. More particularly, but not by way of limitation, this disclosure relates to utilizing hardware to track the execution of work on a graphics processor, such as a graphics processing unit (GPU). 
     One goal for managing graphics hardware resources for computing devices, such as GPUs, is to utilize the computing device as much as possible. One approach in increasing a computing device&#39;s hardware utilization is to simultaneously execute multiple processes in parallel and dynamically allocate the graphics hardware resources between them. In many cases, the underlying graphics hardware resources may not be allocated at a fine enough granularity to match a requested division of resources, and thus, causing the starvation of one or more processes (e.g., one or more lower priority processes). In addition, software systems issuing or generating such requests are often unable to detect when the underlying graphics hardware resources have been allocated differently from the requests. Each of these situations can result in hardware utilizations being sub-optimal. 
     SUMMARY 
     The following summary is included in order to provide a basic understanding of some aspects and features of the claimed subject matter. This summary is not an extensive overview and as such it is not intended to particularly identify key or critical elements of the claimed subject matter or to delineate the scope of the claimed subject matter. The sole purpose of this summary is to present some concepts of the claimed subject matter in a simplified form as a prelude to the more detailed description that is presented below. 
     In one embodiment, a method for utilizing graphics hardware resources. The example method generates a priority list for a plurality of data masters for a graphics processor based on a comparison between current utilizations for the data masters and target utilizations for the data masters. The example method designates, based on the priority list, a first data master with a higher priority to submit work to the graphics processor when compared to a second data master. The example method determines a stall counter value for the designated data master, where the stall counter value is indicative of a number of time periods the designated data master has work to submit to the graphics processor, but is unable to submit the work. The example method then generates a notification to pause work for the second data master based on the stall counter value. 
     In another embodiment, a method for utilizing graphics processing hardware. The example method generates a priority list for multiple graphics hardware interfaces of a graphics processor based on a comparison between measured utilizations for the graphics hardware interfaces and target utilizations for the graphics hardware interfaces. Each of the graphics hardware interfaces processes commands from an application process. The example method designates, based on the priority list, a first graphics hardware interface with a higher priority to submit work to the graphics processor compared to a second graphics hardware interface of the plurality of graphics hardware interfaces. The example method determines an overall stall counter value for the designated graphics hardware interface. The overall stall counter value is indicative of a number of time periods the designated first graphics hardware interface has work to submit the graphics processor, but is unable to submit the work. The example method compares the overall stall counter value to a threshold value and generates a notification to pause the second graphics hardware interface based on a determination that the overall stall counter value satisfies the threshold value. 
     In another embodiment, a graphics processor comprising a director circuit and multiple data masters. The director circuit generates a priority list of data masters based on a current utilization measurement and a target utilization. Based on the priority list, the director circuits designates a high priority data master and determines an overall stall counter value based on stall counter values received from hardware resource clusters. The stall counter values indicate the number of clock cycles the designated data master is unable to schedule work for a graphics processor to execute. The director circuit generates a notification for the hardware resource clusters to pause work for non-designated data masters by comparing the stall counter value to a threshold value. 
     In one embodiment, each of the above described methods, and variation thereof, may be implemented as a series of computer executable instructions. Such instructions may use any one or more convenient programming language. Such instructions may be collected into engines and/or programs and stored in any media that is readable and executable by a computer system or other programmable control device. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       While certain embodiments will be described in connection with the illustrative embodiments shown herein, this disclosure is not limited to those embodiments. On the contrary, all alternatives, modifications, and equivalents are included within the spirit and scope of this disclosure as defined by the claims. In the drawings, which are not to scale, the same reference numerals are used throughout the description and in the drawing figures for components and elements having the same structure, and primed reference numerals are used for components and elements having a similar function and construction to those components and elements having the same unprimed reference numerals. 
         FIG. 1  is a diagram of a graphics processing path where embodiments of the present disclosure may operate. 
         FIG. 2  is a block diagram of an embodiment of a graphics hardware resource allocation system, which corresponds to the graphics hardware resource allocation system shown in  FIG. 1 . 
         FIG. 3  is a block diagram of another embodiment of a graphics hardware resource allocation system, which also corresponds to the graphics hardware resource allocation system shown in  FIG. 1 . 
         FIG. 4  depicts an embodiment of an input QoS stack that a graphics hardware resource allocation systems can dynamically modify to generate an output QoS stack. 
         FIG. 5  is a block diagram illustrating another embodiment of a graphics hardware resource allocation system, which corresponds to the graphics hardware resource allocation system shown in  FIG. 1 . 
         FIG. 6  is a flow diagram of operation for allocating graphics hardware resources to prevent graphics processor starvation. 
         FIG. 7  is a block diagram illustrating one embodiment of a computing device that monitors and controls, in real-time, a process&#39;s quality of service (QoS). 
         FIG. 8  is block diagram illustrating an embodiment of a computing system that includes at least a portion of a processing circuit hardware resource allocation system. 
         FIG. 9  is a simplified block diagram illustrating one embodiment of a graphics processor. 
     
    
    
     DETAILED DESCRIPTION 
     This disclosure includes various example embodiments that track and prevent graphics processor starvation and delays. In one embodiment, a graphics hardware resource allocation system includes clusters that process kicks submitted to a graphics processor. Each cluster maintains its own hardware stall counter to track when a designated data master is unable to schedule work (e.g., part of a kick) because other data masters are utilizing the available graphics hardware resources. Each hardware stall counter increments for each graphics processor clock cycle the designated data master has work to execute, but is unable to launch into the respective cluster. A director circuit aggregates the counts of the hardware stall counters from each cluster and generates an overall count value. For example, the director circuit can determine the overall count value based on a weighted average or a maximum count value from the different hardware stall counters. The director circuit then compares the overall count value to a threshold value. If the overall count value exceeds the threshold value, the director circuit notifies the clusters and/or data masters to pause the execution of work for non-designated data masters. By pausing work associated with non-designated data masters, the clusters are then able to execute work from the designated data master. The clusters and/or data masters continue to pause the execution of work from non-designated data masters until the director circuit identifies a new designated data master. 
     As used herein, the term “kernel” in this disclosure refers to a computer program that is part of a core layer of an operating system (e.g., Mac OSX™) typically associated with relatively higher or the highest security level. The “kernel” is able to perform certain tasks, such as managing hardware interaction (e.g., the use of hardware drivers) and handling interrupts for the operating system. To prevent application programs or other processes within a user space from interfering with the “kernel,” the code for the “kernel” is typically loaded into a separate and protected area of memory. Within this context, the term “kernel” may be interchangeable throughout this disclosure with the term “operating system kernel.” 
     The disclosure also uses the term “compute kernel,” which has a different meaning and should not be confused with the term “kernel” or “operating system kernel.” In particular, the term “compute kernel” refers to a program for a graphics processor (e.g., GPU, DSP, or FPGA). In the context of graphics processing operations, programs for a graphics processor are classified as a “compute kernel” or a “shader.” The term “compute kernel” refers to a program for a graphics processor that performs general compute operations (e.g., compute commands), and the term “shader” refers to a program for a graphics processor that performs graphics operations (e.g., render commands). 
     As used herein, the term “application program interface (API) call” in this disclosure refers to an operation an application is able to employ using a graphics application program interface (API). Examples of API calls include draw calls for graphics operations and dispatch calls for computing operations. Examples of graphics API include OpenGL®, Direct3D®, or Metal® (OPENGL is a registered trademark of Silicon Graphics, Inc.; DIRECT3D is a registered trademark of Microsoft Corporation; and METAL is a registered trademark of Apple Inc.). Generally, a graphics driver translates API calls into commands a graphics processor is able to execute. 
     For the purposes of this disclosure, the term “processor” refers to a programmable hardware device that is able to process data from one or more data sources, such as memory. One type of “processor” is a general-purpose processor (e.g., a CPU or microcontroller) that is not customized to perform specific operations (e.g., processes, calculations, functions, or tasks), and instead is built to perform general compute operations. Other types of “processors” are specialized processor customized to perform specific operations (e.g., processes, calculations, functions, or tasks). Non-limiting examples of specialized processors include GPUs, floating-point processing units (FPUs), DSPs, FPGAs, application-specific integrated circuits (ASICs), and embedded processors (e.g., universal serial bus (USB) controllers). 
     As used herein, the term “graphics processor” refers to a specialized processor for performing graphics processing operations. Examples of “graphics processors” include, but are not limited to, a GPU, DSPs, FPGAs, and/or a CPU emulating a GPU. In one or more implementations, graphics processors are also able to perform non-specialized operations that a general-purpose processor is able to perform. Examples of general compute operations are compute commands associated with compute kernels. 
       FIG. 1  is a diagram of a graphics processing path  100  where embodiments of the present disclosure may operate.  FIG. 1  illustrates an example in which the graphics processing path  100  utilizes a processor resource  110  and a graphics processor resource  112 . The processor resource  110  includes one or more general-purpose processors (e.g., CPUs), where each processor has one or more cores. The processor resource  110  can also contain and/or communicate with memory, other microcontrollers, and/or any other graphics hardware resources a processor may utilize to process commands for graphics processor resource  112  to execute. The graphics processor resource  112  includes one or more graphics processors (e.g., GPUs), where each graphics processor has one or more execution cores and other computing logic for performing graphics and/or general compute operations. Stated another way, the graphics processor resource  112  may also encompass and/or communicate with memory (e.g., memory cache), and/or other graphics hardware resources to execute programs, such as shaders or compute kernels. 
       FIG. 1  illustrates that application  101  generates graphics API calls for the purpose of encoding commands for the graphics processor resource  112  to execute. To generate the graphics API calls, application  101  includes code written with a graphics API. The graphics API (e.g., Metal®) represents a published and/or standardized graphics library and framework that define functions and/or other operations that application  101  is able to have with a graphics processor. For example, the graphics API allows application  101  to be able to control the organization, processing, and submission of render and compute commands, as well as the management of associated data and resources for those commands. 
     In one or more implementations, application  101  is a graphics application that invokes the graphics API to convey a description of a graphics scene. Specifically, the user space driver  102  receives graphics API calls from application  101  and maps the graphics API calls to operations understood and executable by the graphics processor resource  112 . For example, the user space driver  102  can translate the API calls into commands encoded within command buffers before being transferred to kernel driver  103 . The translation operation may involve the user space driver  102  compiling shaders and/or compute kernels into commands executable by the graphics processor resource  112 . The command buffers are then sent to the kernel driver  103  to prepare the commands for execution on the graphics processor resource  112 . As an example, the kernel driver  103  may perform memory allocation and scheduling of the commands to be sent to the graphics processor resource  112 . For the purpose of this disclosure and to facilitate ease of description and explanation, unless otherwise specified, the user space driver  102  and the kernel driver  103  are collectively referred to as a graphics driver. 
       FIG. 1  illustrates that the graphics processor firmware  104  obtains commands from processor resource  110 . The graphics processor firmware  104  can perform a variety of operations to manage the graphics processor hardware  105  that includes powering on and off the graphics processor hardware  105  and/or scheduling the order of commands that the graphics processor hardware  105  receives for execution. With reference to  FIG. 1  as an example, the graphics processor firmware  104  can be implemented by a graphics microcontroller that boots up firmware. Specifically, the graphics microcontroller could be embedded in the same package as a graphics processor within the graphic processor resource  112  and setup to pre-process commands for the graphics processor. In other implementations, the graphics microcontroller is physically separated from the graphics processor. 
     After scheduling the commands, in  FIG. 1 , the graphics processor firmware  104  sends command streams (e.g., multiple kicks) to the graphics processor hardware  105 . The graphics processor hardware  105  then executes the kicks within the command streams according to the order the graphics processor hardware  105  receives the kicks. The graphics processor hardware  105  includes graphics hardware resources that are able to execute a number of received kicks in parallel. The graphics processor hardware  105  then outputs rendered frames to frame buffer  106 . In one implementation, the frame buffer  106  is a portion of memory, such as a memory buffer, that contains a bitmap that drives display  107 . Display  107  subsequently accesses the frame buffer  106  and converts (e.g., using a display controller) the rendered frame (e.g., bitmap) to a video signal for display. 
     In one or more embodiments, the graphics processor hardware  105  includes a graphics hardware resource allocation system  108  that allocates graphics hardware resources (not shown in  FIG. 1 ) to execute kicks processor resource  110  submits to the graphics processor resource  112 . The graphics hardware resource allocation system  108  receives the kicks and breaks the kicks down into work that the graphics hardware resources (e.g., vertex shaders, fragment shaders, united shader clusters, registers, or computational units) are able to execute. As used herein, the term “kick” in this disclosure refers to a discrete unit of instructions that are submitted to a graphics processor. For example, the user kernel driver  103  splits commands committed for a graphics processor to execute into a finer granularity of instructions for the graphics processor to execute. The finer granularity of instructions represent kicks, where each kick can target a specialized type of data master (e.g., a pixel data master, fragment data master, or compute data master). For the purpose of this disclosure, the term “work” represents a finer unit of instructions than a “kick.” Stated another way, the graphics hardware resource allocation system  108  receives “kicks” and breaks the “kicks” down into a smaller granularity of instructions referenced as “work.” 
     The graphics hardware resource allocation system  108  tracks and prevents starvation and/or reduces delays in utilizing the graphics hardware resources. In one embodiment, the graphics hardware resource allocation system  108  includes hardware stall counters that track when one or more designated data masters are unable to schedule work. In some situations, non-designated data masters may prevent designated data masters from utilizing available graphics hardware resources. The hardware stall counters are setup to increment at specified time periods. For example, the hardware stall counters could increment every graphics processor clock cycle when the designated data master has available work to execute but the graphics hardware resources are unable to actually execute the work. Based on the counter information from the hardware stall counters, the graphics hardware resource allocation system  108  generates a hardware driven pause notification to pause non-designated data masters from submitting work and/or pause the processing of submitted work from the non-designated data master. By implementing the hardware pause, the clusters are then able to execute work from the designated data masters. The hardware driven pause may pause work from non-designated data master until graphics hardware resource allocation system  108  selects a new designated data master to track with the hardware stall counters. 
       FIG. 2  is a block diagram of an embodiment of a graphics hardware resource allocation system  200 , which corresponds to the graphics hardware resource allocation system  108  shown in  FIG. 1 . As shown in  FIG. 2 , the graphics hardware resource allocation system  200  is part of the graphics processor  215  that communicates with a CPU  205  (on which process  210  executes). The graphics hardware resource allocation system  200  includes data maters  220 A-W, director circuit  225  and clusters  230 A-X. Each cluster further includes graphics hardware resources organized into “slots”  235 A-Y and hardware resource utilization sensors  240 A-Z. Examples of graphics hardware resources include universal shaders, vertex shaders, fragment shaders, computational units, registers, and the like. Each of slots  235 A-Y represent a portion of a cluster&#39;s graphics hardware resources. At run-time, process  210  may issue a series of commands to be executed by graphics processor  215 . The commands are broken into kicks that are then sent to data masters  220 A-W. With reference to  FIG. 1 , application  101  may generate process  210  and/or other processes (not shown in  FIG. 2 ) that provide kicks to graphics processor  215 . 
     Each data master  220 A-W represents a graphics hardware interface for submitting work to a graphics processor  215 . Within this disclosure, the term “graphics hardware interface” may be used interchangeably with the term “data master.” Data masters  220 A-W may include multiple types of data masters  220  within the graphics hardware resource allocation system  200 . There could also be multiple instances of the same type of data master  220  associated with a graphics processor  215 . In tile based deferred rendering (TBDR) GPU architectures, for example, where graphics rendering may be divided into geometry and pixel phases, there may be one or more vertex data masters  220 , one or more pixel data masters  220  and one or more compute data masters  220 . In immediate mode rendering GPU architectures, where graphics rendering may be grouped by draw commands, different data masters  220  may be used for different objects (wherein each object is responsible for processing its own vertex and pixel data). As such, data masters  220  may be considered heterogeneous in the sense each type of data master  220  can have different characteristics for acquiring resources and being dispatched to a graphics processor  215 . 
     Each data master  220 A-W may break kicks received from process  210  into finer granularity work and submit the work to one or more director circuits  225 . One or more data masters  220 A-W could receive kicks from other processes not shown in  FIG. 2 . Recall that a kick represents instructions at a certain granularity level that can be submitted to graphics processor  215  for processing. Director circuit  225  ensures that each kick is allocated a specified amount of cluster resources (e.g., in units of slots  235 A-Y) in accordance with priority. As shown in  FIG. 2 , each of the clusters  230 A-Y have hardware resource utilization sensors  240 A-Z used to track the utilization of a single (unique) kick on its corresponding cluster  230 . That is, if a cluster  230  (e.g., cluster  230 A) could execute a single kick at a time, one hardware resource utilization sensor  240  (e.g., hardware resource utilization sensor  240 A) may be used for that cluster  230  (e.g., cluster  230 A). If a cluster  230  (e.g., cluster  230 B) could execute ‘J’ unique kicks at a time, ‘J’ hardware resource utilization sensors  240  (e.g., sensors  240 A-J) would be needed for the cluster (e.g.,  230 B). In another example, there may be as many different kicks executing on a cluster  230  as the cluster has slots  235 . 
       FIG. 2  also illustrates that the graphics hardware resource allocation system  200  includes multiple hardware stall counters  245 A-X. Each cluster  230 A-X within the graphics processor  215  includes a corresponding hardware stall counter  245 A-X. Using  FIG. 2  as an example, cluster  230 A contains hardware stall counter  245 A, and cluster  230 X includes hardware stall counter  245 X. The graphics processor  215  can generate and supply one or more graphics processor clock signals for the hardware stall counters  245 A- 245 X. Each hardware stall counter  245 A-X may increment based on the received graphics processor clock signal and when the corresponding cluster  230 A-X satisfies a delay increment condition. As an example, the hardware stall counter  245 A can encounter a delay increment condition when a designated data master  220  (e.g., data master  220 A) has work to submit, but was unable to utilize an allocated set of slots  235  within cluster  230 A for a given time period (e.g., one or more graphics processor clock cycles). As a result, the hardware stall counter  245 A increments by one. The hardware stall counter  245 A continues to increment when the delay increment condition occurs in subsequent time periods (e.g., one or more subsequent graphics processor clock cycles). 
     In  FIG. 2 , each hardware stall counters  245 A-X independently tracks a data master  220  the director circuit  225  designates (e.g., a designated highest priority data master  220 ). Because the hardware stall counters  245 A-X are distributed amongst clusters  230 A-X, when one of the cluster  230 A-X fails to launches work for the designated data master  220  the respective hardware stall counters  245  increments without affecting other hardware stall counters  245 . As an example, when cluster  230 A does not process work for the designated data master  220  (e.g., data master  220 A) within a specified time period (e.g., one or more graphics processor clock cycles), then the corresponding hardware stall counter  245 A increments by one. Having cluster  230 A satisfy the delay increment condition does not affect the counts on the other hardware stall counters  245 B- 245 X. The other hardware stall counters  245 B- 245 X increment when their respective clusters  230 B- 230 X separately satisfy the delay increment condition. 
     After designating data master  220  for tracking with the hardware stall counters  245 , the director circuit  225  can receive and aggregate the count values from the hardware stall counters  245 A-X. By aggregating the count values, the director circuit  225  determines whether to generate a hardware driven pause notification. For example, director circuit  225  can obtain count values from the hardware stall counters  245 A-X, and subsequently compute an average or weighted average from the count values. The average or weight average may be designated as an overall count value such that the director circuit  225  compares the overall count value to one or more thresholds. Based on the comparison, the director circuit  225  determines whether to generate a hardware driven pause notification. In another example, the director circuit  225  may determine a maximum count value received from the hardware stall counters  245 A-X and set the maximum count value as the overall count value. Other embodiments of the director circuit  225  could use other statistical operations (e.g., median) to determine an overall count value for generating hardware driven pause notifications. 
     If the director circuit  225  determines that the overall count value satisfies the threshold, the director circuit  225  generates a hardware driven pause notification. In one embodiment, the hardware driven pause notification can provide instructions to pause all non-designated data masters  220 , and thereby, allow the clusters  230  to execute the designated data masters  220 . Using  FIG. 2  as an example, if the director circuit designates data master  220 W for tracking, then the hardware driven pause notification can cause all of the non-designated data masters  220 A- 220 V to pause. In another embodiment, the hardware driven pause notification can cause one of the non-designated data master  220  to pause. For example, the hardware driven pause notification could provide instructions to pause the non-designated data master  220  with the lowest priority of the non-designated data master  220 . Other embodiments of the director circuit  225  could generate the hardware driven pause notification to pause some, but not all of the non-designated data master. 
     The director circuit  225  can communicate hardware driven pause notifications to one or more data masters  220  and/or to the clusters  230 . In situations where the director circuit  225  provides a hardware driven pause notification to a non-designated data master  220  (e.g., data master  220 A), the non-designated data master  220  stops submitting work to clusters  230  in response to receiving the hardware driven pause notification. Preventing a data master  220  from submitting work to clusters  230  allows clusters  230  to execute work from the designated data master  220 . Alternatively or additionally, director circuit  225  is able to provide the hardware driven pause notifications to one or more clusters  230 . When clusters  230  receive the hardware driven pause notification from the director circuit, clusters  230  pause the execution of work submitted from non-designated data masters  220 . By doing so, clusters  230  are also able to execute work for the designated data masters  220 . 
     The hardware driven pause may pause the non-designated data masters  220  until the director circuit  225  identifies a new designated data master  220  and/or the designated data master  220  completes its outstanding work. For example, after pausing the non-designated data masters  220 , clusters  230  execute work for the designated data master  220 . As clusters  230  execute work for the designated data master  220 , the director circuit  225  may continue to track graphics hardware resource utilization information for the graphics processor  215 . Based on the graphics hardware resource utilization information, the director circuit  225  and/or clusters  230  may determine to un-pause and resume the execution of work associated with the non-designated data masters  220 . The director circuit  225  and/or clusters  230  may resume execution of work for the non-designated data masters  220  when the director circuit  225  assigns a new designated data master  220  to track with the hardware stall counters  245 . In another example, the non-designated data masters  220  may un-pause when the designated data master  220  completes its outstanding work. Utilizing graphics hardware resource utilization information is discussed in more detail with reference to  FIGS. 4 and 5 . 
       FIG. 3  is a block diagram of another embodiment of a graphics hardware resource allocation system  300 , which also corresponds to the graphics hardware resource allocation system  108  shown in  FIG. 1 . Graphics hardware resource allocation system  300  is similar to graphics hardware resource allocation system  200  shown in  FIG. 2  except that each of the clusters  230  include multiple hardware stall counters  302  and  304 . In  FIG. 3 , the director circuit  225  is able to designate multiple data masters  220  for tracking, where each hardware stall counter  302  and  304  is setup to track one of the designated data masters  220 . Using  FIG. 3  as an example, the director circuit  225  may designate data master  220 A and  220 W for tracking such that hardware stall counter  302  tracks data master  220 A, and hardware stall counter  304  tracks data master  220 W. Although  FIG. 3  illustrates that each cluster  230  includes two hardware stall counters  302  and  304 , other embodiments could have each cluster  230  include more than two hardware stall counters  302 . 
     Similar to  FIG. 2 , the director circuit  225  receives and aggregates the count values corresponding to the hardware stall counters  302  and  304  to generate overall count values. Recall that an overall count value can be determined from a variety of statistical operations (e.g., average or weighted average). In contrast to the graphics hardware resource allocation system  200  shown in  FIG. 2 , the director circuit  225  could generate multiple overall count values. Each overall count value corresponds to one of the designated data masters  220 . As an example, the director circuit  225  could generate one overall count value for hardware stall counter  302 , which tracks data master  220 A and another overall count value for hardware stall counters  304 , which track data master  220 W. The overall count values could then be compared to one or more threshold values to determine whether the director circuit  225  generates one or more hardware driven pause notifications. 
     The threshold values for evaluating the overall count values may differ and depend on which designated data master  220  the overall count value corresponds to. Continuing with  FIG. 3  as an example, the director circuit could designate data master  220 A as the first priority data master and data master  220 W as the second priority data master. In other words, designated data master  220 A has a higher priority to access slots  235  within clusters  230  over designated data master  220 W. Because of the difference in priority, the threshold value associated with designated data master  220 A could be less than the threshold value associated with designated data master  220 B. In another example, a single threshold value may be setup for evaluating both overall count values regardless if whether the designated data masters  220  have the same or different priority level. 
     Similar to  FIG. 2 , the director circuit  225  can generate hardware driven pause notifications that can be supplied to the data masters  220  and/or clusters  230 . In one embodiment, similar to  FIG. 2 , the hardware driven pause notification could pause work for non-designated data masters  220 . In another embodiment, the hardware driven pause notifications could include instructions to not just pause non-designated data masters  220 B- 220 V, but also other lower-priority, designated data masters  220 . As an example, in  FIG. 3 , if designated data master  220 A has a higher priority to access slots  235  over designated data master  220 W, then a hardware driven pause notification for designated data master  220 A could not only pause non-designated data masters  220 B- 220 V, but also designated data master  220 W. The hardware driven pause notification for designated data master  220 W may include instructions to pause non-designated data masters  220 B- 220 V, but not designated data master  220 A since data master  220 A has a higher priority. 
     With reference  FIGS. 2 and 3 , the utilization of hardware stall counters  245 ,  302 , and  304  within graphics hardware resource allocation system  200  and  300  allow for monitoring and controlling, in real-time, a process&#39;s QoS. As used herein, real-time means during graphics processor operations involving the process who&#39;s QoS is being measured and controlled. The concept of QoS as it applies to the graphics hardware resource allocation systems  200  and  300  involves dynamically generating an output QoS based on an input QoS. An input QoS refers to an input priority a process and/or application assigns to commands. Based on the input QoS, the graphics hardware resource allocation systems  200  and  300  initially allocates a certain amount of graphics hardware resources to execute work, which initially represents a target utilization of graphics hardware resources. The graphics hardware resource allocation systems  200  and  300  generate an output QoS to ensure that work (e.g., a kick) sent to a graphics processor  215  actually receives the allocated amount of resources during its execution, which represents a current utilization of graphics hardware resource. A process&#39;s ability to utilize its allocated graphics hardware resources (e.g., slots  235 A-Y), in turn, may be described in terms of its current utilization of those allocated resources on a kick-by-kick basis. Additionally or alternatively, output QoS may also refer to a related group or collection of work (e.g., as generated by process  210  executing on CPU  205 ). In other embodiments, output QoS refers to the current resource utilization of a group or collection of processes (e.g., process  210  and other processes executing on CPU  205 ). Input QoS and output QoS are described in more detail with reference to  FIG. 4 . 
       FIG. 4  depicts an embodiment of an input QoS stack  400  that a graphics hardware resource allocation system  108  can dynamically modify to generate an output QoS stack  410 . The input QoS stack  400  can initially represents an absolute priority that a process and/or application assigns to commands. As an example, a process may assign a command for a system user interface to QoS level  402  of the QoS stack  400  while assigning commands for data mining operations to QoS level  408 . In  FIG. 4 , the input QoS stack  400  include QoS levels  402 ,  404 ,  406 , and  408  that are arranged based on the order of priority level. The top most QoS level  402  represents the highest level of priority within the input QoS stack  400 ; QoS level  404  represents the next level of priority, where QoS level  404  has a lower priority level than QoS level  402  but a higher priority level than QoS level  406 ; QoS level  406  represents a priority level that is less than QoS level  404  and higher than QoS level  408 ; and QoS level  408  represents the lowest level of priority for the input QoS stack  400 . Over time, the input QoS stack  400  could represent priorities the graphics hardware resource allocation system  108  previously assigns to data masters  220  as the graphics hardware resource allocation system  108  dynamically adjusts priorities for data masters  220 . 
     As shown in  FIG. 4 , different data masters  220  are associated with the different QoS levels  402 ,  404 ,  406 , and  408  within the input QoS stack  400 . In  FIG. 4 , data masters  220 A-F belong to the QoS level  402 ; data masters  220 G-L belong to the QoS level  404 ; data masters  220 M-T belong to QoS level  406 ; and data masters  220 U-Z belong to QoS level  408 . The graphics hardware resource allocation system  108  initially utilizes the input QoS stack  400  to define a target utilization of graphics hardware resources for each of the data masters  220 A-Z. Specifically, the different QoS levels  402 ,  404 ,  406 , and  408  can cause the graphics hardware resource allocation system  108  to assign different target utilization of graphics hardware resources for the data masters  220 . For example, data masters  220 A-F associated with QoS level  402  can be allocated a higher target utilization than data masters  220 G-Z. 
     To generate an output QoS stack  410 , the graphics hardware resource allocation system  108  dynamically prioritizes and readjusts the QoS levels  402 ,  404 ,  406 , and  408  for data masters  220 A-Z. By comparing a current utilization of graphics hardware resources to the target utilization for each of the data masters  220 , a graphics hardware resource allocation system  108  determines an actual or effective utilization of graphics hardware resources. As shown in  FIG. 4 , the output QoS stack  410  includes a group or collection of data masters  220  at each QoS levels  402 ,  404 ,  406 , and  408 . The graphics hardware resource allocation system  108  generates the output QoS stack  410  by dynamically readjusting the QoS levels for data masters  220  based on how effective the data masters  220  utilize the graphics hardware resources. When data masters  220  are unable to utilize allocated graphics hardware resources, the deviation between current utilization and target utilization increases for those data masters  220 . By determining the changes in deviation, the graphics hardware resource allocation system reassigns the data masters  220  to different QoS levels  402 ,  404 ,  406 , and  408 . Managing deviations between current utilization and target utilize are described in more detail in U.S. Patent Application Publication No. 2018/0173560, filed Dec. 21, 2016 by Gokhan Avkarogullari et al. and entitled “Processing Circuit Hardware Resource Allocation System,” which is herein incorporated by reference in its entirety. 
       FIG. 4  illustrates the output QoS stack  410  reassigns the QoS levels for data masters  220 A-F and  202 G-L to prevent graphics processor starvation. According to the input QoS stack  400 , data masters  220 G-L initially has a priority level lower than data masters  220 A-F. During execution, the deviation between current utilization and target utilization of graphics hardware resources for data masters  220 G-L eventually exceeds the deviation experienced by data masters&#39;  220 A-F. Because of the greater deviation, which represents a less effective use of allocated graphics hardware resources, the graphics hardware resource allocation system  108  dynamically adjusts the priority levels. In  FIG. 4 , the output QoS stack  410  moves up the priority level of data masters  220 G-L to QoS level  402  and moves down the priority level of data masters  220 A-F to QoS level  404 . By placing data masters  220 G-L at a higher priority level, the graphics hardware resource allocation system attempts to reduce the deviation between actual utilization and target utilization of graphics hardware resources for data masters  220 G-L. 
     In one embodiment, if within a certain period of time data masters  220 G-L continue to experience a sizeable deviation and/or the deviation continues to increase, a processor (e.g., CPU) may implement a software-based pause on data masters  220  with lower priority levels according to the output QoS stack  410 . Specifically, a graphics processor driver could issue a software-based pause notification that affects data masters  220  and/or other graphics processor hardware (e.g., clusters). With this, the graphics processor driver is able to perform a throttle operation that allows the graphics processor driver to directly control data masters  220  to match current utilization with the target utilization of graphics hardware resources. In particular, the software-based pause notification operation that pauses the execution of work for data masters  220  associated with lower QoS levels within the output QoS stack  410 . Using  FIG. 4  as an example, the software-based pause notification could pause execution of work for data masters  220  (e.g., data masters  220 A-F and data masters  220 M-Z) associated with QoS levels  404 ,  406 , and/or  408  of the output QoS stack  410 . The graphics processor driver could pause the data masters  220  until data masters  220 G-L reaches the target utilization and/or a certain deviation. Implementing software-based pause notifications is described in more detail in U.S. patent application Ser. No. 15/615,412, filed Jun. 6, 2017 by Tatsuya Iwamoto et al. and entitled “GPU Resource Tracking,” which is herein incorporated by reference in its entirety. 
     In some situations, commands that complete in a relatively short amount of time (e.g., hundreds of microseconds) may suffer from delays that could cause frame drops even when utilizing software-based pause notification. Often time the graphics processor driver&#39;s turnaround time is relative longer and is unable to prevent graphics processor starvation for commands that complete in a relatively short amount of time. In other words, the graphics processor driver may be unable to issue software-based pause notifications within the needed time period to prevent graphics processor starvation. To avoid starvation for commands with relative short amount of times, the graphics hardware resource allocation system  108  is able to maintain hardware stall counters that track one or more designated data masters  220  as described in  FIGS. 2 and 3 . 
     To avoid starvation, a hardware driven pause may prioritize data masters  220  that belong to the same QoS level  402 ,  404 ,  406 , and  408  of the output QoS stack  410 . Using  FIG. 4  as an example, a hardware driven pause may avoid starvation that may occur amongst data masters  220 G-L, which all belong to QoS level  402  of the output QoS stack  410 . In one example, the graphics hardware resource allocation system  108  may have previously detected graphics processor starvation and issued a software-based pause notification that paused data masters  220  (e.g., data masters  220 A-F and data masters  220 M-Z) with the other QoS levels  404 ,  406 , and  408 . However, even with the software-based pause notification, one or more of the data masters  220 G-L continue to experience delays and/or starvation. The graphics hardware resource allocation system  108  is able to prevent delays and/or starvation for QoS level  402  by designating one or more data masters  220  (e.g., data masters  220 G and H) for tracking using hardware stall counters. Recall that tracking and pausing the execution of work for data masters  220  are discussed in detail with reference to  FIGS. 2 and 3 . 
       FIG. 5  is a block diagram illustrating another embodiment of a graphics hardware resource allocation system  500 , which corresponds to the graphics hardware resource allocation system  108  shown in  FIG. 1 . With reference to  FIGS. 2 and 3 , cluster  230 A and director circuit  225 A shown in  FIG. 5  may be part of a larger processing system, and for clarity&#39;s sake, various portions of a complete system are not shown. In the illustrated embodiment, cluster  230 A includes graphics hardware resources  505 , hardware resource arbitration circuit  510 , hardware resource utilization sensor  515 , and process priority list  522 . The director circuit  225 A includes utilization accumulation circuit  526 , target utilization circuit  530 , comparator circuit  535 , process priority adjustment circuit  540 , and switching circuit  545 . In some embodiments, cluster  230 A may include multiple instances of hardware resource utilization sensor  515 , corresponding to various director circuits  225 . In another embodiment, rather than process priority adjustment circuit  540  communicating with multiple clusters  230 , director circuit  225 A may include multiple instances of process priority adjustment circuit  540 . In some embodiments, other clusters  230 , director circuit  225 , or both may not include various respective illustrated portions of cluster  230 A and/or director circuit  225 A. For example, target utilization circuit  530  may correspond to both director circuit  225 A and another director circuit  225 B (not shown in  FIG. 5 ). In still other embodiments, each cluster  230  includes one or more hardware resource utilization sensors  515  that contain counters. 
     As previously described, cluster  230 A may receive work from one or more processes via data masters  220 . Using  FIGS. 2 and 3  as an example, a set of data masters (not shown in  FIG. 5 ) may be assigned to break down commands from a particular process into work. The clusters  230  may execute the work by utilizing graphics hardware resources  505  (e.g., registers, execution cores, logic units, cache entries, program state storage circuitry such as that used as a program counter, etc.). Work may request more graphics hardware resources than are available. Accordingly, hardware resource arbitration circuit  510  may, via resource allocation information  550 , allocate graphics hardware resources  505  between the processes based on QoS information  562  received from process priority list  522 . Hardware resource utilization sensor  515  may monitor utilization of the allocated graphics hardware resources  505  by one or more of the processes and may, in response thereto, generate cluster utilization indication  520 A. Cluster utilization indication  520 A may indicate a portion of the allocated graphics hardware resources  505  that were actually utilized during a given time period (e.g., a sample interval). In some embodiments, some portions of graphics hardware resources  505  (e.g., registers) may be weighted differently from other portions of graphics hardware resources  505  (e.g., execution cores). In the illustrated embodiment, hardware resource utilization sensor  515  may periodically send cluster utilization indication  520 A to director circuit  225 A (e.g., after every sample interval). Cluster utilization indication  520 A may represent a utilization of graphics hardware resources  505  over a specified amount of time (e.g., 1 millisecond, 1 second, or a lifetime of a corresponding process) or a utilization of graphics hardware resources  505  at a specific time. 
     Director circuit  225 A may receive cluster utilization indications  520 A or other information from clusters  230  (e.g., cluster  230 A). The cluster utilization indications  520 A may indicate utilization of graphics hardware resources by one or more processes at the respective cluster. In the illustrated embodiment, director circuit  225 A may receive cluster utilization indication  520 A at switching circuit  545 . Switching circuit  545  may, in turn, output cluster utilizations as current utilization  555  based on cluster selection  560 . In some embodiments, switching circuit  545  may comprise one or more multiplexers. Current utilization  555  may be sent to utilization accumulation circuit  526  and to comparator circuit  535 . Utilization accumulation circuit  526  may determine the utilization of graphics hardware resources (e.g., at clusters  230 A) by a process over a particular amount of time (e.g., an epoch interval). In the illustrated embodiment, utilization accumulation circuit  526  may output an indication of the utilization of the graphics hardware resources to target utilization circuit  530 . 
     Target utilization circuit  530  may use the utilization of the graphics hardware resources to identify a target utilization  565  for a particular cluster  230  (e.g., cluster  230 A). By way of example, target utilization circuit  530  may indicate a target utilization of graphics hardware resources  505  for a process monitored by hardware resource utilization sensor  515  when current utilization  555  corresponds to cluster utilization indication  520 A. Target utilization  565  may indicate a number of resources to be given to the process during a next specified period of time (e.g., until target utilization  565  is recalculated for graphics hardware resources  505 ). In some embodiments, target utilization circuit  530  may determine target utilization  565  based on a utilization of graphics hardware resources by one or more other processes (e.g., received at cluster  230 A from process queues other than the process corresponding to director circuit  225 A). In other embodiments, target utilization circuit  530  may determine target utilization  565  by tracking a number of threads of the process that are consumed. In still other embodiments, one or more software components (e.g., executing at director circuit  225 A or at one or more processors external to director circuit  225 A) may be used to determine target utilization  565 . 
     Comparator circuit  535  may compare current utilization  555  to target utilization  565  and may output a result to execute priority adjustment circuit  540 . Additionally, in some embodiments, comparator circuit  535  may convert current utilization  555  into a format appropriate for target utilization  565  (e.g., a percentage). In one embodiment, the result may indicate a difference between current utilization  555  and target utilization  565 . The result may indicate that a difference between current utilization  555  and target utilization  565  is within a specified range (e.g., current utilization  555  is at least 10% larger than target utilization  565 , current utilization  555  and target utilization  565  are less than 10% of each other, or current utilization is at least 10% smaller than target utilization  565 ). In other embodiments, several ranges may be used (e.g., current utilization  555  is 10-20% larger target utilization  565 , current utilization  555  is 21-30% larger target utilization  565 , etc.). In still other embodiments, an output of comparator circuit  535  may indicate a number of credits. As used herein, the number of credits may indicate a specified amount of graphics hardware resources allocated to the process per a specified number of execution cycles, as compared to an expected amount of graphics hardware resources allocated to the process per the specified number of execution graphics processor clock cycles. 
     Process priority adjustment circuit  540  may determine whether to dynamically adjust, via priority signal(s)  525 , a priority of one or more processes at one or more clusters  230  based on the result from comparator circuit  535 . In some cases, at least some of the one or more clusters  230  where the priority is adjusted may be different from the cluster  230  corresponding to current utilization  555 . As noted above, the result may indicate that a difference between current utilization  555  and target utilization  565  is within a specified range (or outside a specified range). In response to the difference being within the specified range, process priority adjustment circuit  540  may determine not to adjust the priority of the process at one or more of the clusters  230 . In some other embodiments, priority signal  525 A may be sent to process priority list  522 , indicating no adjustment to the priority should be made. In other embodiments, priority signal  525 A may not be sent. In response to the result being outside the specified range and current utilization  555  being larger than target utilization  565 , process priority adjustment circuit  540  may reduce the priority of the process at one or more clusters  230  (e.g., via priority signal  525 A). In response to the result being outside the specified range and current utilization  555  being smaller than target utilization  565 , process priority adjustment circuit  540  may increase the priority of the process at one or more clusters (e.g., via priority signal  525 A). The priority may be adjusted, for example, by a fixed amount or may be based on the difference between current utilization  555  and target utilization  565 . 
     In some cases, process priority adjustment circuit  540  may track a total difference for the process based on outputs from comparator circuit  535  (e.g., multiple outputs corresponding to a single cluster, outputs corresponding to multiple clusters, or both). As noted above, in some embodiments, the results from comparator circuit  535  may indicate a number of credits. Process priority adjustment circuit  540  may track a total number of credits for a process. Additionally, process priority adjustment circuit  540  may adjust the priority of the process based on the total number of credits exceeding or falling below various specified thresholds. The adjusted priority may be used by hardware resource arbitration circuit  510  in future allocation cycles to reallocate graphics hardware resources  505 . As discussed above, in some embodiments the priority may be adjusted such that allocation of graphics hardware resources  505  to executes at cluster  230 A trends towards a specified ratio over a period of time (e.g., 1 millisecond or 1 second), as opposed to the allocation being the specified ratio. 
     In still other embodiments, process priority adjustment circuit  540  may use additional information to adjust the priority. For example, process priority adjustment circuit  540  may receive results from comparator circuits corresponding to other processes (e.g., received at cluster  230 A from other process queues than the process corresponding to director circuit  225 A). As another example, process priority adjustment circuit  540  may save information from previous results provided by comparator circuit  535 . As a third example, process priority adjustment circuit  540  may receive an indication of a number of graphics hardware resources requested by the process at one or more of clusters  230 . As noted above, in some cases, various processes may have specified ranges of priorities. The specified ranges may be based on the processes themselves (e.g., based on a process type), based on a priority requested by the process, based on a process queue from which the process was received, or based on other factors. The specified ranges may differ at different clusters. In light of these differences, process priority adjustment circuit  540  may adjust priorities based on the specified ranges such that the adjusted priorities are in the specified ranges. 
     In some cases, process priority adjustment circuit  540  may identify the process as being ill-behaved or hung, which indicates the possibility of graphics processor starvation. For example, in response to determining the current utilization  555  for a first process exceeds target utilization  565 , determining that the priority of the process is already the lowest priority that can be assigned, and determining that one or more other processes are receiving an insufficient number of resources, process priority adjustment circuit  540  may identify the first process as being ill-behaved. As another example, in response to determining that a second process is failing to utilize an allocated portion of graphics hardware resources  505  despite being allocated a requested portion of graphics hardware resources  505  for a particular amount of time, process priority adjustment circuit  540  may identify the second process as being hung. The process may be identified as ill-behaved or hung based on a difference between current utilization  555  and target utilization  565  exceeding one or more specified amounts. In various embodiments where credits are used, a process may be identified as being ill-behaved or hung in response to the number of credits exceeding or falling below respective specified thresholds. In some embodiments, in response to identifying a process as being ill-behaved or hung, process priority adjustment circuit  540  may indicate to one or more of clusters  230  that a context switch should occur for the process or that the process should be terminated. The indication may be sent via one or more of priority signals  525  (e.g., setting the priority to a particular value) or to one or more other devices (e.g., to hardware resource arbitration circuit  510  directly). 
       FIG. 5  illustrates that the hardware resource arbitration circuit  510  includes a hardware stall counter  245 A for tracking designated data masters. Recall that a set of data masters may be assigned to receive commands from a given process. The director circuit  225 A may designate one of the data masters to be tracked with the hardware stall counter  245 A. In  FIG. 5 , the process priority adjustment circuit  540  may include within the priority signal  525 A the designated data master. The process priority adjustment circuit  540  determines the designated data master based on the comparison between the current utilization  555  and target utilization  565 . As an example, the process priority adjustment circuit  540  may assign the data master that has the largest difference between the current utilization  555  and target utilization  565  as the designated data masters. 
     After the process priority list  522  receives the priority signal  525 A identifying the designated data master, the process priority list  522  sends QoS information  562  to the hardware resource arbitration circuit  510 . In one embodiment, the QoS information  562  may include a list of data masters sorted by priority, where the priority indicates which data master is the designated data master. For example, in FIG.  5 , the highest priority data master within the list could indicate the designated data master. In embodiments where the list includes multiple designated data masters, the list could indicate that data masters with the higher priorities in the list are designated data masters. Each time the designated data master changes within the list, the count value for the hardware stall counter  145  is reset back to zero. Using  FIG. 2  as an example, at a first point in time, QoS information  562  sent to the hardware resource arbitration circuit  510  indicates that data master  202 A has the highest priority, and thus, is the designated data master. Afterwards, at a second point in time, the process priority list  522  sends QoS information  562  to the hardware resource arbitration circuit  510  that changes in the highest priority data master to data master  220 W. When this occurs, the count value for hardware stall counter  245 A is reset back to zero. 
     At each specified time period (e.g., one or more graphics processor clock cycles), the hardware resource arbitration circuit  510  receives graphics hardware resource utilization information  564  from the hardware resource utilization sensor  515 . The hardware resource arbitration circuit  510  uses the graphics hardware resource utilization information  564  to determine whether the designated data master was able to execute its available work using the graphics hardware resource  505 . If the designated data master was unable to submit its work, the hardware stall counter  245  increments. If the designated data master was able to execute work, the hardware stall counter  245  does not change. The count value  568  for hardware stall counter  245  may then be sent to the director circuit  225 A. The hardware resource arbitration circuit  510  may send the count value to the director circuit  225 A periodically and/or each time the count value for the hardware stall counter  245  changes. Although  FIG. 5  illustrates that the hardware stall counter  245  is part of the hardware resource arbitration circuit  510 , other embodiments could have the hardware stall counter  245  separate from the hardware resource arbitration circuit  510  and/or belong within other components of the cluster  230 A (e.g., hardware resource utilization sensor  515 ). 
     In  FIG. 5 , a hardware pause comparator circuit  566  receives the count value  568  from hardware stall counter  245 A and other hardware stall counters  245  located in other clusters  230 . The hardware pause comparator circuit  566  aggregates the count values  568  received from the hardware stall counters  245  to generate an overall count value. As discussed with reference to  FIG. 2 , the overall count value can be computed based on one or more statistical operations. The hardware pause comparator circuit  566  then compares the overall count value to one or more thresholds to determine whether to generate a hardware driven pause notification  570 . In  FIG. 5 , the hardware pause comparator circuit  566  sends hardware driven pause notifications  570  back to the hardware resource arbitration circuit  510 . In response, the hardware resource arbitration circuit  510  can dynamically modify the resource allocation information  550  sent to graphics hardware resources  505 . 
     After implementing the hardware driven pause and/or software-based pause, the process priority adjustment circuit  540  may dynamically adjust the priorities and/or assign a new designated data master as current utilization  555  updates. In particular, current utilization  555  will change as work for the non-designated and/or lower priority data masters do not execute. Based on the change in current utilization  555 , process priority adjustment circuit  540  sends priority signals  525 A that updates QoS information  562  to include newly designated data masters. In one embodiment, when hardware resource arbitration circuit  510  receives the new designated data master, the hardware resource arbitration circuit  510  may un-pause and resume executing work associated with the non-designated and/or lower priority data master. Recall that the hardware stall counter  245 A can reset to a specified reset count value (e.g., a count value of zero) because of the new designated data master. 
       FIG. 6  is a flow diagram of operation  600  for allocating graphics hardware resources to prevent graphics processor starvation. In some embodiments, operation  600  may be initiated or performed by one or more processors in response to one or more instructions stored in a computer-readable storage medium. For example, operation  600  may be performed by the graphics hardware resource allocation system  108  shown in  FIG. 1 . The use and discussion of  FIG. 6  is only an example to facilitate explanation and is not intended to limit the disclosure to this specific example. For example, although  FIG. 6  illustrates that the blocks within operation  600  are implemented in a sequential order, operation  600  is not limited to this sequential order. For example blocks, 
     As noted briefly above, because different data masters have different characteristics of acquiring resources and dispatching their kicks to the graphics processor, they may be considered heterogeneous. With reference to  FIGS. 2, 3, and 5 , a side-effect of this is that regardless of what priority a data master assigns to a kick (e.g., data master  220 A), the director circuit  225  to which that kick is assigned (e.g., director circuit  225 ) may grant a lower priority kick from a different data master  220  more resources (e.g., data master  220 W). The phenomenon of lower priority kicks being allocated more resources than higher priority kicks is referred to herein as “sneaking.” Sneaking is a side effect of arbitrating graphics processor resources across or through heterogeneous data masters  220 . 
     For illustrative purposes only, assume a director circuit is capable of arbitrating and granting resources to one data master every graphics processor clock cycle whenever a slot is available. Consider a first data master that issues high priority kicks at a low rate from a shallow queue. Consider next a second data master that issues lower priority kicks at a higher rate and which requires a block grant of a cluster&#39;s slots. In such cases, when slots becomes available even if a high priority kick from the first data master was able to claim the first slots offered, it could soon run out of work to fill subsequently available slots due to its low rate of production and shallow queue. When there is no contention for taking the slots, the data master issuing the lower priority kicks will claim the available slots and lock out the first (higher priority) data master due to the block grant. 
     Operation  600  reduces the effects of locking out data masters by implementing hardware driven pauses. At block  602 , operation  600  may initially obtain QoS information that identifies a new designated data masters. In reference to  FIG. 5 , operation  600  may receive priority signals  525 A that generates an updated list from the process priority list  522  that prioritizes the data masters for a process. In one embodiment, the highest priority data master is the designate data master. Afterwards, operation  600  continues to block  604  and resumes execution of work for data masters affected by a previous hardware driven pause notification. For example, operation  600  may resume execution of work from non-designated data masters and/or data masters with lower priorities. Operation  600  then moves to block  606  and resets a hardware stall counter for tracking the new designated data master to a reset count value. The reset count value can be a discrete value, for example, the numerical value zero. 
     Operation  600  then moves to block  608  and determines whether the designated data masters has work available for execution for a specified time period, such as a graphics processor clock cycle. Persons of ordinary skill in the art are aware that a graphics processor clock cycle represents the amount of time between two pulses within a clock signal generated from a graphics processor&#39;s clock generator (e.g., oscillator). The clock speed of the graphics processor&#39;s clock generator can be measured in hertz (Hz), for example, GHz. If operation  600  determines that no work is available for execution, operation  600  moves to block  612  and does not change the count value on the hardware stall counter. Returning back to block  608 , if operation  600  determines there is available work for execution for the given time period, operation  600  proceeds to block  610  and determines whether the graphics hardware resources actually executed the available work within the specified time period (e.g., graphics processor clock cycle). 
     At block  610 , if operation  600  determines that the graphics hardware resources actually executed the available work, then operation  600  continues to block  612  and does not adjust the count value on the hardware stall counter. After completing block  612 , operation  600  moves back to block  608  for further monitoring. Returning to block  610 , if operation  600  does not actually execute the work, then operation  600  moves to block  614  and increments the hardware stall counter. Operation  600  then moves to block  616  and determines whether to generate a hardware driven pause notification based on the updated count value for the hardware stall counter. 
     Other Illustrative Systems 
       FIG. 7  is a block diagram illustrating one embodiment of a computing device  702  that monitors and controls, in real-time, a process&#39;s QoS.  FIG. 7  illustrates process queues  700 A-K, clusters  230 A-M, and director circuits  225 A-N, where clusters  230 A-M and director circuits  225 A-N are part of the graphics hardware resource allocation system  108 . Although process queues  700 A-K, clusters  230 A-M, and director circuits  225 A-N are interconnected in a particular manner in  FIG. 7 , in other embodiments process queues  700 A-K, clusters  230 A-M, and director circuits  225 A-N may be connected in other manners (e.g., process queue  700 K may not be connected to cluster  230 A). In various embodiments, different numbers of at least one of process queues  700 A-K, clusters  230 A-M, or director circuits  225 A-N may be present. In various other embodiments, some or all of the elements shown in  FIG. 7  may be part of one or more components of the graphics hardware resource allocation system  108 . 
     Process queues  700 A-K may store data for respective processes and may provide the data to clusters  230 A-M as process data  715 A-K. Process data of a single process queue may be provided to a single cluster or to multiple clusters. Process data provided to multiple clusters may be the same or different. Additionally, multiple process queues may provide process data to a single cluster. For example, process queue  700 A may provide a first portion of process data  715 A (e.g., first kick) to cluster  230 A and a second portion of process data  715 A (e.g., second kick) to cluster  230 M. Further, during a single execution cycle, process queue  700 B may provide a first portion of process data  715 B (e.g., a third kick) to cluster  230 M and a second portion of process data  715 B (e.g., a fourth kick) to cluster  230 B. Process queues  700 A-K may correspond to different functional aspects of the system. For example, in some embodiments, process queues  700 A-K may correspond to various data master functions of a graphics processor. Processes may be allocated to execute process queues  700 A-K based on the functions performed by the processes. In the illustrated embodiment, process data  715 A includes data for only a single process. In some cases, the data may correspond to multiple threads of a single process. In other embodiments, process data  715 A may include data for multiple processes. In still other embodiments, process queues  700 A-K may be software queues. In other embodiments, process queues  700 A-K may be hardware queues. In yet other embodiments, some of process queues  700 A-K may be software queues while others may be hardware queues. 
     Clusters  230 A-M may include graphics hardware resources used to perform various computing actions using process data. As noted above, in some cases clusters  230 A-M may receive process data from multiple processes. For example, cluster  230 M may receive a portion of process data  715 A and a portion of process data  715 B. When process data corresponding to multiple processes is received, clusters  230 A-M may allocate respective graphics hardware resources to the processes based on priorities of the processes and the determined hardware utilization. In various embodiments, the priorities may be determined based on at least one of a process type, a priority requested by the process queue, or a queue from which the process is received. For example, processes relating to a user interface may have a specified range of priorities (e.g., at least one of a specified minimum priority, a specified maximum priority, or a specified initial priority). As another example, processes received from a vertex queue may also have a specified range of priorities. In some cases, the graphics hardware resources of clusters  230 A-M may not be utilized as indicated by the priorities. In accordance with one or more embodiments, clusters  230 A-M may periodically indicate utilization of the graphics hardware resources by the various processes to director circuits  225 A-N via cluster utilizations  720 A-M (e.g., based on utilization sensor output over one or more sample intervals and/or one or more epoch intervals). Cluster utilizations  720 A-M may represent a utilization of graphics hardware resources for a particular amount of time (e.g., an epoch interval) or may represent an instantaneous utilization of graphics hardware resources. In response to cluster utilizations  720 A-M, clusters  230 A-M may receive priority signals  725 A-M, which may modify one or more priorities at clusters  230 A-M. Clusters  230 A-M may reallocate the graphics hardware resources based on the modified priorities. In some embodiments, the graphics hardware resources may be reallocated to be within a specified range over a specified amount of time. As an example, in some embodiments, cluster  230 A may include twenty registers and may further include requests from a first process and a second process. The priorities of the processes may indicate that the first process should receive eighty percent of the registers (sixteen registers) and the second process should receive twenty percent of the registers (four registers). However, the first process may be unable to proceed with fewer than ten registers and the second process may be unable to proceed with fewer than six registers. Because, in this example, the initially allocated four registers for the second process is insufficient for it to execute, cluster utilizations  720 A-M may indicate that the second process is not utilizing its allocated registers. In response, priority signals  725 A-M may adjust the priorities so the second process is not allocated any of the registers half of the time and receives forty percent of the registers (eight registers) the other half of the time. Under this allocation, the first process receives 10 registers half the time and 20 registers the other half of the time while the second process receives 10 registers half the time and no registers the other half of the time. As a result, this adjustment may allow both processes to make progress. 
     Director circuits  225 A-N may receive cluster utilizations  720 A-M and may determine whether to adjust the priorities at clusters  230 A-M. In particular, as described further below, director circuits  225 A-N may determine, for a particular process, its actual utilization over a given time period (e.g., an instantaneous utilization, a utilization based on one or more sample intervals, or a utilization based on one or more epoch intervals). Based on a comparison between a target utilization and a current or actual utilization, one or more of director circuits  225 A-N may adjust a priority of a process at one or more of clusters  230 A-M. As a result, processes may receive an allocated amount of graphics hardware resources over a window of time (e.g., an interval). Additionally, director circuits  225 A-N may detect that one or more processes are ill-behaved (e.g., requesting resources and failing to utilize them) or hung (e.g., failing to continue execution). In some cases, director circuits  225 A-N may indicate, via priority signals  725 A-M or via another signal that a context switch should occur with regard to a process, removing the process from clusters  230 A-M. In some embodiments, each director circuit  225 A-N corresponds to a different process. Accordingly, where each of process queues  700 A-K sends process data for a single process to one of clusters  230 A-M at a time, director circuits  225  may correspond to different process queues  700 . 
     Turning next to  FIG. 8 , a block diagram illustrating an exemplary embodiment of a computing system  800  that includes at least a portion of a graphics hardware resource allocation system. The computing system  800  includes graphics processor  215  of  FIG. 2 . In some embodiments, graphics processor  215  includes one or more of the circuits described above with reference to  FIG. 2 , including any variations or modifications described previously with reference to  FIGS. 1-7 . For example, in the illustrated embodiment, graphics processor  215  includes cluster(s)  230  and director circuit(s)  225  of  FIGS. 2 and 3 . In some embodiments, some or all elements of the computing system  800  may be included within a system on a chip (SoC). In other embodiments, computing system  800  may be included in a mobile device. Accordingly, in at least some embodiments, area and power consumption of the computing system  800  may be important design considerations. In the illustrated embodiment, the computing system  800  includes communication&#39;s fabric  805 , graphics processor  215 , compute complex  810 , input/output (I/O) bridge  815 , cache/memory controller  820 , and display unit  825 . Although the computing system  800  illustrates graphics processor  215  as being connected to fabric  805  as a separate device of computing system  800 , in other embodiments, graphics processor  215  may be connected to or included in other components of the computing system  800 . 
     Additionally, the computing system  800  may include multiple graphics processors  215 . The multiple graphics processors  215  may correspond to different embodiments or to the same embodiment. Further, although in the illustrated embodiment, cluster(s)  230  and director circuit(s)  225  are part of graphics processor  215 , in other embodiments, cluster(s)  230 , director circuit(s)  225 , or both may be a separate device or may be included in other components of computing system  800 . Fabric  805  may include various interconnects, buses, MUXes, controllers, etc., and may be configured to facilitate communication between various elements of computing system  800 . In some embodiments, portions of fabric  805  are configured to implement various different communication protocols. In other embodiments, fabric  805  implements a single communication protocol and elements coupled to fabric  805  may convert from the single communication protocol to other communication protocols internally. 
     In the illustrated embodiment, compute complex  810  includes bus interface unit (BIU)  830 , cache  835 , and cores  840  and  845 . In some embodiments, cores  840  and  845  may correspond to execution cores of clusters  230 . In various embodiments, compute complex  810  includes various numbers of cores and/or caches. For example, compute complex  810  may include 1, 2, or 4 processor cores, or any other suitable number. In some embodiments, cores  840  and/or  845  include internal instruction and/or data caches. In some embodiments, a coherency unit (not shown) in fabric  805 , cache  835 , or elsewhere in computing system  800  is configured to maintain coherency between various caches of computing system  800 . BIU  830  may be configured to manage communication between compute complex  810  and other elements of computing system  800 . Processor cores such as cores  840  and  845  may be configured to execute instructions of a particular instruction set architecture (ISA), which may include operating system instructions and user application instructions. I/O bridge  815  may include various elements configured to implement: universal serial bus (USB) communications, security, audio, and/or low-power always-on functionality, for example. I/O bridge  815  may also include interfaces such as pulse-width modulation (PWM), general-purpose input/output (GPIO), serial peripheral interface (SPI), and/or inter-integrated circuit (I2C), for example. Various types of peripherals and devices may be coupled to computing system  800  via I/O bridge  815 . 
     In some embodiments, graphics processor  215  may be coupled to computing system  800  via I/O bridge  815 . Cache/memory controller  820  may be configured to manage the transfer of data between fabric  805  and one or more caches and/or memories (e.g., non-transitory computer readable mediums). For example, cache/memory controller  820  may be coupled to an L3 cache, which may, in turn, be coupled to a system memory. In other embodiments, cache/memory controller  820  is directly coupled to a memory. In some embodiments, the cache/memory controller  820  includes one or more internal caches. In some embodiments, the cache/memory controller  820  may include or be coupled to one or more caches and/or memories that include instructions that, when executed by one or more processors (e.g., compute complex  810  and/or graphics processor  215 ), cause the processor, processors, or cores to initiate or perform some or all of the operations described above with reference to  FIGS. 1-7 . Display unit  825  may be configured to read data from a frame buffer and provide a stream of pixel values for display. Display unit  825  may be configured as a display pipeline in some embodiments. Additionally, display unit  825  may be configured to blend multiple frames to produce an output frame. Further, display unit  825  may include one or more interfaces (e.g., MIPI or embedded display port, eDP) for coupling to a user display (e.g., a touchscreen or an external display). 
     As used herein, the term “coupled to” may indicate one or more connections between elements, and a coupling may include intervening elements. For example, in  FIG. 8 , display unit  825  may be described as “coupled to” compute complex  810  through fabric  805 . In contrast, in the illustrated embodiment of  FIG. 8 , display unit  825  is “directly coupled” to fabric  805  because there are no intervening elements. 
     Referring to  FIG. 9 , a simplified block diagram illustrating one embodiment of a graphics processor  900  is shown. In the illustrated embodiment, graphics processor  900  includes vertex pipe  905 , fragment pipe  910 , programmable shader  915 , texture processing unit (TPU)  920 , image write buffer  925 , and memory interface  930 . In some embodiments, graphics unit  900  may be configured to process both vertex and fragment data using programmable shader  915 , which may be configured to process data (e.g., graphics data) in parallel using multiple execution pipelines or instances. In other embodiments, the multiple execution pipelines correspond to a plurality of execution units of a processing circuit hardware resource allocation system. 
     Vertex pipe  905  may include various fixed-function hardware configured to process vertex data. Vertex pipe  905  may be configured to communicate with programmable shader  915  to coordinate vertex processing, and to send processed data to fragment pipe  910  and/or programmable shader  915  for further processing. Fragment pipe  910  may include various fixed-function hardware configured to process pixel data. Fragment pipe  910  may be configured to communicate with programmable shader  915  in order to coordinate fragment processing. Fragment pipe  910  may also be configured to perform rasterization on polygons received from vertex pipe  905  and/or programmable shader  915  so as to generate fragment data. Vertex pipe  905  and/or fragment pipe  910  may be coupled to memory interface  930  (coupling not shown) in order to access graphics data. 
     Programmable shader  915  may be configured to receive vertex data from vertex pipe  905  and fragment data from fragment pipe  910  and/or TPU  920 . Programmable shader  915  may be further configured to perform vertex processing tasks on vertex data, including various transformations and/or adjustments of vertex data. By way of example, programmable shader  915  may also be configured to perform fragment processing tasks on pixel data such as texturing and shading. Programmable shader  915  may include multiple execution instances for processing data in parallel. In various embodiments, portions (e.g., execution units, registers, arithmetic logic units, memory locations, etc.) of programmable shader  915  may be usable by multiple processes (e.g., vertex processing tasks, compute processing tasks and fragment processing tasks). In practice, different portions of programmable shader  915  may be allocated to different processes during execution of those processes. Programmable shader  915  in one or more embodiments may be coupled in any of various appropriate configurations to other programmable and/or fixed-function elements in a graphics unit. The configuration shown in  FIG. 9  is illustrative only. 
     TPU  920  may be configured to schedule fragment processing tasks from programmable shader  915 . In some embodiments, TPU  920  may be configured to pre-fetch texture data and assign initial colors to fragments for further processing by programmable shader  915  (e.g., via memory interface  930 ). In other embodiments, TPU  920  may be configured to provide fragment components in one or more normalized integer formats or one or more floating-point formats. In still other embodiments, TPU  920  may be configured to provide fragments in groups of four (a “fragment quad”) in a 2×2 format to be processed by a group of four execution pipelines in programmable shader  915 . Image write buffer  925  may be configured to store processed tiles of an image and may further perform final operations to a rendered image before it is transferred to a frame buffer (e.g., in a system memory via memory interface  930 ). Memory interface  930  may facilitate communication between graphics unit  900  and one or more of various memory hierarchies in various embodiments. 
     Within this disclosure, different entities (which may variously be referred to as “units,” “circuits,” other components, etc.) may be described or claimed as “configured” to perform one or more tasks or operations. This formulation—[entity] configured to [perform one or more tasks]—is used herein to refer to structure (i.e., something physical, such as an electronic circuit). More specifically, this formulation is used to indicate that this structure is arranged to perform the one or more tasks during operation. A structure can be said to be “configured to” perform some task even if the structure is not currently being operated. A “memory device configured to store data” is intended to cover, for example, an integrated circuit that has circuitry that performs this function during operation, even if the integrated circuit in question is not currently being used (e.g., a power supply is not connected to it). Thus, an entity described or recited as “configured to” perform some task refers to something physical, such as a device, circuit, memory storing program instructions executable to implement the task, etc. This phrase is not used herein to refer to something intangible. The term “configured to” is not intended to mean “configurable to.” An un-programmed field-programmable gate array (FPGA), for example, would not be considered to be “configured to” perform some specific function, although it may be “configurable to” perform that function after programming. 
     As used herein, the term “based on” is used to describe one or more factors that affect a determination. This term does not foreclose the possibility that additional factors may also affect the determination. That is, a determination may be solely based on specified factors or based on the specified factors as well as other, unspecified factors. Consider the phrase “determine A based on B.” This phrase specifies that B is a factor that is used to determine A or that affects the determination of A. This phrase does not foreclose the situation in which the determination of A may also be based on some other factor, such as C. This phrase is also intended to cover an embodiment in which A is determined based solely on B. As used herein, the phrase “based on” is synonymous with the phrase “based at least in part on.” 
     As used herein, the phrase “in response to” describes one or more factors that trigger an effect. This phrase does not foreclose the possibility that additional factors may affect or otherwise trigger the effect. That is, an effect may be solely in response to those factors, or may be in response to the specified factors as well as other, unspecified factors. Consider the phrase “perform A in response to B.” This phrase specifies that B is a factor that triggers the performance of A. This phrase does not foreclose the situation in which the performance of A may also be in response to some other factor, such as C. This phrase is also intended to cover an embodiment in which A is performed solely in response to B. 
     As used herein, the terms “first,” “second,” etc. are used as labels for nouns that they precede, and do not imply any type of ordering (e.g., spatial, temporal, logical, etc.), unless stated otherwise. For example, in a processing circuit that includes six clusters, the terms “first cluster” and “second cluster” can be used to refer to any two of the six clusters, and not, for example, to two specific clusters (e.g., logical clusters 0 and 1). 
     In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the disclosed concepts. As part of this description, some of this disclosure&#39;s drawings represent structures and devices in block diagram form in order to avoid obscuring the novel aspects of the disclosed concepts. In the interest of clarity, not all features of an actual implementation may be described. Further, as part of this description, some of this disclosure&#39;s drawings may be provided in the form of flowcharts. The boxes in any particular flowchart may be presented in a particular order. It should be understood however that the particular sequence of any given flowchart is used only to exemplify one embodiment. In other embodiments, any of the various elements depicted in the flowchart may be deleted, or the illustrated sequence of operations may be performed in a different order, or even concurrently. In addition, other embodiments may include additional steps not depicted as part of the flowchart. Moreover, the language used in this disclosure has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter, resort to the claims being necessary to determine such inventive subject matter. Reference in this disclosure to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosed subject matter, and multiple references to “one embodiment” or “an embodiment” should not be understood as necessarily all referring to the same embodiment. 
     It will be appreciated that in the development of any actual implementation (as in any software and/or hardware development project), numerous decisions must be made to achieve a developers&#39; specific goals (e.g., compliance with system- and business-related constraints), and that these goals may vary from one implementation to another. It will also be appreciated that such development efforts might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the design and implementation of graphics processing systems having the benefit of this disclosure. 
     It is to be understood that the above description is intended to be illustrative, and not restrictive. The material has been presented to enable any person skilled in the art to make and use the disclosed subject matter as claimed and is provided in the context of particular embodiments, variations of which will be readily apparent to those skilled in the art (e.g., some of the disclosed embodiments may be used in combination with each other. The scope of the invention therefore should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. In the appended claims, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.”

Metadata:
Filing Date: 20180928
Publication Date: 20201006
Grant Date: 20201006
Priority Date: 20180928
Inventors: BANERJEE, KUTTY
BOWMAN, BENJAMIN
POTTER, TERENCE M.
IWAMOTO, TATSUYA
AVKAROGULLARI, GOKHAN
Assignee: APPLE INC
CPC Classifications: [{"code": "G06F9/5038", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F9/4881", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F9/542", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/485", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/505", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/3877", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/542", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/3877", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/505", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F9/5038", "inventive": true, "first": true, "tree": "[]"}]
Family ID: 69946940