FLEXIBLE MULTI-USER GRAPHICS ARCHITECTURE

A technique for operating a processor that includes multiple cores is provided. The technique includes determining a number of active applications, selecting a processor configuration for the processor based on the number of active applications, configuring the processor according to the selected processor configuration, and executing the active applications with the configured processor.

BACKGROUND

Graphics processing hardware accelerates graphics rendering tasks for applications. Server-size hardware-based rendering is becoming increasingly common and improvements to such rendering are frequently being made.

DETAILED DESCRIPTION

A technique for operating a processor that includes multiple cores is provided. The technique includes determining a number of active applications, selecting a processor configuration for the processor based on the number of active applications, configuring the processor according to the selected processor configuration, and executing the active applications with the configured processor.

FIG. 1Ais a block diagram of a cloud gaming system101, according to an example. A server103communicates with one or more clients105. The server103executes gaming applications at least partly using graphics hardware. The server103receives inputs from the one or more clients105, such as button presses, mouse movements, and the like. The server103provides these inputs to the applications executing on the server103, which processes the inputs and generates video data for transmission to the clients105. The server103transmits this video data to the clients105for display and the clients105display the video data.

FIG. 1Bis a block diagram of an example device100in which one or more features of the disclosure can be implemented. In various implementations, the server103and/or client105ofFIG. 1Aare implemented as the device100. In the server, a graphics processor107is included. In different implementations, the clients105do or do not include the graphics processor107. In various implementations, the device100includes, for example, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, or a tablet computer. The device100includes a processor102, a memory104, a storage106, one or more input devices108, and one or more output devices110. The device100also optionally includes an input driver112and an output driver114. It is understood that the device100can include additional components not shown inFIG. 1B.

In various alternatives, the processor102includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, wherein each processor core can be a CPU or a GPU. In various alternatives, the memory104is be located on the same die as the processor102, or is located separately from the processor102. The memory104includes a volatile or non-volatile memory, for example, random access memory (RAM), dynamic RAM, or a cache.

The input driver112communicates with the processor102and the input devices108, and permits the processor102to receive input from the input devices108. The output driver114communicates with the processor102and the output devices110, and permits the processor102to send output to the output devices110. The output driver114includes a graphics processor107. The graphics processor107is configured to accept graphics rendering commands from processor102, to process those compute and graphics rendering commands, and to provide pixel output to a display device for display.

FIG. 1Cillustrates additional details of the server103, according to an example. The processor102is configured to support a virtualization scheme in which multiple virtual machines execute on the processor102. Each virtual machine (“VM”) “appears” to software executing in that VM as a completely “real” hardware computer system, but in reality comprises a virtualized computing environment that may be sharing the device100with other virtual machines. Virtualization may be supported fully in software, partially in hardware and partially in software, or fully in hardware. The graphics processor107supports virtualization, meaning that the graphics processor107can be shared among multiple virtual machines executing on the processor102, with each VM “believing” that the VM has full ownership of a real hardware graphics processor107. The graphics processor107supports virtualization by assigning a different graphics core116of the graphics processor107to each active guest VM204. Each graphics core116performs graphics operations for the associated guest VM204and not for any other guest VM204.

The processor102supports multiple virtual machines, including one or more guest VMs204and, in some implementations, a host VM202. The host VM202performs one or more aspects related to managing virtualization of the graphics processor107for the guest VMs204. A hypervisor206provides virtualization support for the virtual machines, by performing a wide variety of functions such as managing resources assigned to the virtual machines, spawning and killing virtual machines, handling system calls, managing access to peripheral devices, managing memory and page tables, and various other functions. In some implementations, the host VM202provides an interface for an administrator or administrative software to control configuration operations of the graphics processor107related to virtualization. In some systems, the host VM202is not present, with the functions of the host VM202described herein performed by the hypervisor206instead (which is why the GPU virtualization driver121is illustrated in dotted lines in the hypervisor206).

The host VM202and the guest VMs204have operating systems120. The host VM202has management applications123and a GPU virtualization driver121. The guest VMs204have applications126, an operating system120, and a GPU driver122. These elements control various features of the operation of the processor102and the graphics processor107.

The GPU virtualization driver121of the host VM202is not a traditional graphics driver that simply communicates with and sends graphics rendering (or other) commands to the graphics processor107, without understanding aspects of virtualization of the graphics processor107. Instead, the GPU virtualization driver121communicates with the graphics processor107to configure various aspects of the graphics processor107for virtualization. In some examples, in addition to performing the configuration functions, the GPU virtualization driver121issues traditional graphics rendering commands to the graphics processor107or other commands not directly related to configuration of the graphics processor107.

The guest VMs204include an operating system120, a GPU driver122, and applications126. The operating system120is any type of operating system that could execute on processor102. The GPU driver122is a “native” driver for the graphics processor107in that the GPU driver122controls operation of the graphics processor107for the guest VM204on which the GPU driver122is running, sending tasks such as graphics rendering tasks or other work to the graphics processor107for processing. The native driver may be an unmodified or slightly modified version of a device driver for a GPU that would exist in a bare-bones non-virtualized computing system.

Although the GPU virtualization driver121is described as being included within the host VM202, in other implementations, the GPU virtualization driver121is included in the hypervisor instead206. In such implementations, the host VM202may not exist and functionality of the host VM202may be performed by the hypervisor206.

The operating systems120of the host VM202and the guest VMs204perform standard functionality for operating systems in a virtualized environment, such as communicating with hardware, managing resources and a file system, managing virtual memory, managing a network stack, and many other functions. The GPU driver122controls operation of the graphics processor107for any particular guest VM204by, for example, providing an application programming interface (“API”) to software (e.g., applications126) to access various functionality of the graphics processor107. In some implementations, the driver122also includes a just-in-time compiler that compiles programs for execution by processing components (such as the SIMD units138discussed in further detail below) of the graphics core116. For any particular guest VM204, the GPU driver122controls functionality on the graphics core116related to that guest VM204, and not for other VMs.

The graphics processor107includes multiple graphics cores116, a shared data fabric144, a shared physical interface142, a shared cache140, a shared multimedia processor146, and a shared graphics processor memory118.

The graphics cores116of the graphics processor107are individually assignable to different guest VMs204. More specifically, the GPU virtualization driver121assigns a physical graphics core116exclusively to a particular guest VM204for use in performing processing tasks such as graphics processing and compute processing.

The shared multimedia processor146, graphics processor memory118, shared cache140, shared physical interface142, and shared data fabric144are all shareable between the different graphics cores.

The graphics processor memory118includes multiple memory portions. In some configurations, the graphics processor memory118is divided into portions, each of which is assigned to a different graphics core116. In such configurations, the GPU virtualization driver121assigns particular portions of the graphics processor memory118to particular graphics cores116. In such configurations, a graphics core116is able to access portions of the graphics processor memory118that are assigned to that graphics core116and a graphics core116is unable to access portions of the graphics processor memory118that are not assigned to that graphics core116. In some implementations, the portions that are assignable to different graphics cores116are physical subdivisions of the graphics processing memory118, such as specific memory banks. In some implementations, more than one portion of memory is assigned to a single graphics core116. In some implementations, all (or multiple) graphics cores116

The shared cache140is shareable in that different graphics cores116are able to cache data in any portion of the shared cache140. In alternative implementations, however, the shared cache140is configured differently. More specifically, in one implementation, the cache140is partitioned into portions and each portion is assigned to a graphics core116(e.g., for exclusive use). In another implementation, the entire cache140is shared between the graphics cores116to reduce external memory traffic if the graphics cores116access the same data. The shared physical interface142is an input/output interface to components external to the graphics processor107. The shared physical interface142is shareable between the graphics cores116in that the shared physical interface142is capable of routing data and commands for each graphics core116to components external to the graphics processor107. The shared data fabric114routes memory transactions between the graphics cores116and the graphics processor memory118. The shared data fabric114is shareable between the different graphics cores116in that each graphics core116interfaces with the shared data fabric114to access the portions of the graphics processor memory118assigned to that graphics core116.

In various configurations, the graphics cores116are operable at different performance levels. In some implementations, one or more of the graphics cores116differs from one or more of the other graphics cores116in terms of the number of resources physically present within that graphics core. In some examples, these resources include one or more of amount of memory, amount of cache memory, and/or number of compute units134.

In some examples, the graphics cores116are switchable between different performance levels at runtime. In some implementations, each graphics core116has an adjustable performance level in terms of one or more of clock speed, or number of components enabled. In some implementations, a higher clock speed applied to a graphics core116or a higher number of components enabled for a graphics core116results in a greater power usage for the graphics core116and/or a greater amount of heat dissipation for the graphics core116. In general, a higher performance level for a graphics core116is associated with a higher amount of power usage and heat dissipation.

In some examples, the hypervisor206configures the device103for use by a certain number of active guest VMs204. Depending on the number of guest VMs204that are active and the performance requirements of the guest VM204, the hypervisor206configures the performance levels of the different graphics cores116. In some implementations, the hypervisor206identifies a power budget and a thermal budget for the graphics processor107overall and sets the performance levels of the enabled graphics cores116based on the total power budget and the total thermal budget. Thus, in some implementations, in situations where more guest VMs204are enabled, the hypervisor206sets the performance levels of one or more graphics cores116to a lower performance level than in situations where fewer guest VMs204are enabled.

In some implementations, the graphics processor107is switchable between a set of a fixed number of configurations. Each such configuration indicates a number of graphics cores116that are enabled and indicates a specific performance level for each enabled graphics core116.

In some implementations, the set of fixed configurations includes at least one configuration in which a first graphics core116is enabled and a second graphics core116is disabled and another configuration in which the first graphics core116and the second graphics core116are both enabled, where in the first configuration, the first graphics core has a higher performance level than the first graphics core in the second configuration.

The graphics processor memory118has a certain amount of bandwidth to the graphics cores116. In configurations in which multiple graphics cores116are enabled, the bandwidth is divided between the different graphics cores116. When one graphics core116is enabled, that graphics core116has access to all of the memory bandwidth. In some configurations, it is possible for each graphics core116to access the entirety of the graphics processor memory118. In some configurations, all of the components of the graphics processor107are included on a single die. In some implementations, each graphics core116, the shared cache140, the shared physical interface142, the shared data fabric144, the shared multimedia processor146, and the graphics processor memory118have their own individually adjustable clock.

FIG. 2is a block diagram illustrating details of a graphics core116, according to an example. The graphics core116executes commands and programs for selected functions, such as graphics operations and non-graphics operations that may be suited for parallel processing. The graphics core116can be used for executing graphics pipeline operations such as pixel operations, geometric computations, and rendering an image to display device based on commands received from the processor102. The graphics core116also executes compute processing operations that are not directly related to graphics operations, such as operations related to video, physics simulations, computational fluid dynamics, or other tasks, based on commands received from the processor102. A command processor213accepts commands from the processor102(or another source), and delegates tasks associated with those commands to the various elements of the graphics core116such as the graphics processing pipeline134and the compute units132.

The graphics core116includes compute units132that include one or more SIMD units138that are configured to perform operations at the request of the processor102in a parallel manner according to a SIMD paradigm. The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and program counter and thus execute the same program but are able to execute that program with different data. In one example, each SIMD unit138includes sixteen lanes, where each lane executes the same instruction at the same time as the other lanes in the SIMD unit138but can execute that instruction with different data. Lanes can be switched off with predication if not all lanes need to execute a given instruction. Predication can also be used to execute programs with divergent control flow. More specifically, for programs with conditional branches or other instructions where control flow is based on calculations performed by an individual lane, predication of lanes corresponding to control flow paths not currently being executed, and serial execution of different control flow paths allows for arbitrary control flow.

The basic unit of execution in compute units132is a work-item. Each work-item represents a single instantiation of a program that is to be executed in parallel in a particular lane. Work-items can be executed simultaneously as a “wavefront” on a single SIMD processing unit138. One or more wavefronts are included in a “work group,” which includes a collection of work-items designated to execute the same program. A work group can be executed by executing each of the wavefronts that make up the work group. In alternatives, the wavefronts are executed sequentially on a single SIMD unit138or partially or fully in parallel on different SIMD units138. A scheduler136is configured to perform operations related to scheduling various workgroups and wavefronts on different compute units132and SIMD units138.

The parallelism afforded by the compute units132is suitable for graphics related operations such as pixel value calculations, vertex transformations, and other graphics operations. Thus in some instances, a graphics pipeline134, which accepts graphics processing commands from the processor102, provides computation tasks to the compute units132for execution in parallel.

The compute units132are also used to perform computation tasks not related to graphics or not performed as part of the “normal” operation of a graphics pipeline134(e.g., custom operations performed to supplement processing performed for operation of the graphics pipeline134). An application126or other software executing on the processor102transmits programs that define such computation tasks to the graphics core116for execution.

As described elsewhere herein, the graphics processor107includes multiple graphics cores116. Each graphics core116has its own command processor213. Therefore, each graphics core116independently processes a command stream received from a guest VM204assigned to that graphics core116. Thus, the operation of a particular graphics core116does not affect the operation of another graphics core116. For example, if a graphics core116becomes unresponsive or experiences a stall or slowdown, that unresponsiveness, stall, or slowdown does not affect a different graphics core116within the same graphics processor107.

The description herein describes the graphics cores116as being associated with, and used by, a single guest VM204in a virtualized computing scheme. However, it should be understood that other implementations are possible. More specifically, any implementation in which the server103includes multiple independent server-side entities, each of which communicates with a different client105, each of which is associated with a particular graphics core116, and each of which transmits command streams to the associated graphics core116and transmits the results of such command streams (e.g., pixels) to the associated client105, falls within the scope of the present disclosure. Generically, such server-side entities are referred to herein as server applications. In some examples, one or more server applications are video games and the server103assigns each such video game a different graphics core116of the graphics processor107.

In addition, the description herein describes the configuration of the graphics processor107as being controlled by a hypervisor206. However, any other component (implemented as hardware, software, or a combination thereof) of the server103could alternatively control the configurations of the graphics processor107. Generically, such component is referred to herein as the graphics processor configuration controller.

FIG. 3is a block diagram showing additional details of the graphics processing pipeline134illustrated inFIG. 2. The graphics processing pipeline134includes stages that each performs specific functionality. The stages represent subdivisions of functionality of the graphics processing pipeline134. Each stage is implemented partially or fully as shader programs executing in the compute units132, or partially or fully as fixed-function, non-programmable hardware external to the compute units132.

The input assembler stage302reads primitive data from user-filled buffers (e.g., buffers filled at the request of software executed by the processor102, such as an application126) and assembles the data into primitives for use by the remainder of the pipeline. The input assembler stage302can generate different types of primitives based on the primitive data included in the user-filled buffers. The input assembler stage302formats the assembled primitives for use by the rest of the pipeline.

The vertex shader stage304processes vertexes of the primitives assembled by the input assembler stage302. The vertex shader stage304performs various per-vertex operations such as transformations, skinning, morphing, and per-vertex lighting. Transformation operations include various operations to transform the coordinates of the vertices. These operations include one or more of modeling transformations, viewing transformations, projection transformations, perspective division, and viewport transformations. Herein, such transformations are considered to modify the coordinates or “position” of the vertices on which the transforms are performed. Other operations of the vertex shader stage304modify attributes other than the coordinates.

The vertex shader stage304is implemented partially or fully as vertex shader programs to be executed on one or more compute units132. The vertex shader programs are provided by the processor102and are based on programs that are pre-written by a computer programmer. The driver122compiles such computer programs to generate the vertex shader programs having a format suitable for execution within the compute units132.

The hull shader stage306, tessellator stage308, and domain shader stage310work together to implement tessellation, which converts simple primitives into more complex primitives by subdividing the primitives. The hull shader stage306generates a patch for the tessellation based on an input primitive. The tessellator stage308generates a set of samples for the patch. The domain shader stage310calculates vertex positions for the vertices corresponding to the samples for the patch. The hull shader stage306and domain shader stage310can be implemented as shader programs to be executed on the compute units132.

The geometry shader stage312performs vertex operations on a primitive-by-primitive basis. A variety of different types of operations can be performed by the geometry shader stage312, including operations such as point sprint expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. In some instances, a shader program that executes on the compute units132perform operations for the geometry shader stage312.

The rasterizer stage314accepts and rasterizes simple primitives and generated upstream. Rasterization consists of determining which screen pixels (or sub-pixel samples) are covered by a particular primitive. Rasterization is performed by fixed function hardware.

The pixel shader stage316calculates output values for screen pixels based on the primitives generated upstream and the results of rasterization. The pixel shader stage316may apply textures from texture memory. Operations for the pixel shader stage316are performed by a shader program that executes on the compute units132.

The output merger stage318accepts output from the pixel shader stage316and merges those outputs, performing operations such as z-testing and alpha blending to determine the final color for a screen pixel.

FIG. 4is a flow diagram of a method400for operating a graphics processor107with multiple graphics cores116, according to an example. Although described with respect to the system ofFIGS. 1A-3, those of skill in the art will understand that any system, configured to perform the steps of the method400in any technically feasible order, falls within the scope of the present disclosure.

The method400begins at step402, where a graphics processor configuration controller (such as the hypervisor206) determines a number of active server applications (such as guest VMs204). An active server application is a server application that is configured to request that work be performed by an associated graphics core116. In some examples, the graphics processor configuration controller receives a request from another entity such as a workload scheduler for a cloud gaming system to configure the processor102to execute a certain number of active server applications and the same number of graphics cores116of the graphics processor107. In various examples, this request is based on the number of clients105using the services of the cloud gaming system.

At step404, the graphics processor configuration controller selects a graphics processor configuration based on the number of active server applications. In some examples, the graphics processor configuration controller is capable of varying the performance levels of one or more graphics cores116based on the number of active server applications and thus based on the number of active graphics cores116. In some examples, graphics processor configurations differ in that, in configurations with fewer graphics cores116that are enabled, more of the available power and thermal budget is available for those fewer graphics cores116than in configurations with a greater number of graphics cores116enabled. Therefore, in configurations with fewer graphics cores116enabled, at least one graphics core is afforded a higher performance level than that same graphics core116is afforded in a graphics processor configuration with a greater number of graphics cores116enabled. In various examples, performance levels define one or more of the clock frequency of a graphics core116, the amount of memory bandwidth available for the graphics core116, the amount of memory or cache that is available for use by the graphics core116, or other features that define the performance level of the graphics core116.

At step406, the graphics processor configuration controller configures the graphics processor107according to the selected graphics processor configuration. Specifically, the graphics processor configuration controller enables the graphics cores116that are deemed to be enabled according to the selected graphics processor configuration and sets the performance levels of each of the enabled graphics cores116according to the selected graphics processor configuration.

At step408, the graphics processor configuration controller causes the active server applications to execute with the configured graphics processor107. Executing a server application includes causing the server application to forward a stream of commands for processing by an associated graphics core116of the graphics processor107. More specifically, as described elsewhere herein, each server application is assigned a particular graphics core116. Each server application transmits a command stream to the graphics core116associated with that server application. In any particular graphics core116, the command processor213of that graphics core executes that command stream to process commands and data through the graphics processing pipeline134and/or to process compute commands.

It should be understood that many variations are possible based on the disclosure herein. Although features and elements are described above in particular combinations, each feature or element can be used alone without the other features and elements or in various combinations with or without other features and elements. It should be understood that although the graphics cores116are described as including a graphics processing pipeline134that, in some implementations, includes fixed function components, a graphics core116with a graphics processing pipeline134fully implemented through shaders without fixed function hardware, or a graphics core116with general purpose compute capabilities but not graphics processing capabilities is contemplated herein. In other words, in the present disclosure, the graphics cores116may be substituted with graphics cores that do not include fixed function elements (and thus are implemented fully as programmable shader programs), or may be substituted with general purpose compute cores that include the compute units132but not the graphics processing pipeline134and can perform general purpose compute operations.

Any of the disclosed functional blocks are implementable as hard-wired circuitry, software executing on a processor, or a combination thereof. The methods provided can be implemented in a general purpose computer, a processor, or a processor core. Suitable processors include, by way of example, a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), and/or a state machine. Such processors can be manufactured by configuring a manufacturing process using the results of processed hardware description language (HDL) instructions and other intermediary data including netlists (such instructions capable of being stored on a computer readable media). The results of such processing can be maskworks that are then used in a semiconductor manufacturing process to manufacture a processor which implements features of the disclosure.