TRUSTED PROCESSOR FOR SAVING GPU CONTEXT TO SYSTEM MEMORY

A trusted processor saves and restores context and data stored at a frame buffer of a GPU concurrent with initialization of a CPU of the processing system. In response to detecting that the GPU is powering down, the trusted processor accesses the context of the GPU and data stored at a frame buffer of the GPU via a high-speed bus. The trusted processor stores the context and data at a system memory, which maintains the context and data while the GPU is powered down. In response to detecting that the GPU is powering up again, the trusted processor restores the context and data to the GPU, which can be performed concurrently with initialization of the CPU.

BACKGROUND

Processing units including but not limited to processors such as graphics processing units (GPUs), massively parallel processors, single instruction multiple data (SIMD) architecture processors, and single instruction multiple thread (SIMT) architecture processors can improve performance or conserve power by transitioning between different power management states. For example, a processing unit can conserve power by idling when there are no instructions to be executed by the processing unit. When a processing unit becomes idle, power management hardware or software may reduce dynamic power consumption. In some cases, a processing unit may be power gated (i.e., may have power removed from it) or partially power gated (i.e., may have power removed from parts of it) if the processing unit is predicted to be idle for more than a predetermined time interval. Power gating a processing unit is referred to as placing the processing unit into a deep sleep, or powered down, state. Powering down a GPU requires saving content stored at a frame buffer or other power gated areas of the GPU to system memory. Transitioning the GPU from a low power state (such as an idle or power gated or partially power gated state) to an active state exacts a performance cost in reinitializing the GPU and copying back content stored at system memory to the frame buffer.

DETAILED DESCRIPTION

A parallel processor is a processor that is able to execute a single instruction on a multiple data or threads in a parallel manner. Examples of parallel processors include processors such as graphics processing units (GPUs), massively parallel processors, single instruction multiple data (SIMD) architecture processors, and single instruction multiple thread (SIMT) architecture processors for performing graphics, machine intelligence or compute operations. In some implementations, parallel processors are separate devices that are included as part of a computer. In other implementations such as advance processor units, parallel processors are included in a single device along with a host processor such as a central processor unit (CPU). Although the below description uses a graphics processing unit (GPU), for illustration purposes, the embodiments and implementations described below are applicable to other types of parallel processors.

A GPU is a processing unit that is specially designed to perform graphics processing tasks. A GPU may, for example, execute graphics processing tasks required by an end-user application, such as a video game application. Typically, there are several layers of software between the end-user application and the GPU. For example, in some cases the end-user application communicates with the GPU via an application programming interface (API). The API allows the end-user application to output graphics data and commands in a standardized format, rather than in a format that is dependent on the GPU.

Many GPUs include a plurality of internal engines and graphics pipelines for executing instructions of graphics applications. A graphics pipeline includes a plurality of processing blocks that work on different steps of an instruction at the same time. Pipelining enables a GPU to take advantage of parallelism that exists among the steps needed to execute the instruction. As a result, a GPU can execute more instructions in a shorter period of time. The output of the graphics pipeline is dependent on the state of the graphics pipeline. The state of a graphics pipeline is updated based on state packages (e.g., context-specific constants including texture handlers, shader constants, transform matrices, and the like) that are locally stored by the graphics pipeline. Because the context-specific constants are locally maintained, they can be quickly accessed by the graphics pipeline.

To perform graphics processing, a central processing unit (CPU) of a system often issues to a GPU a call, such as a draw call, which includes a series of commands instructing the GPU to draw an object according to the CPU's instructions. As the draw call is processed through the GPU graphics pipeline, the draw call uses various configurable settings to decide how meshes and textures are rendered. A common GPU workflow involves updating the values of constants in a memory array and then performing a draw operation using the constants as data. A GPU whose memory array contains a given set of constants may be considered to be in a particular state or have a particular context. These constants and settings, referred to as context (also referred to as “context state”, “rendering state”, “GPU state”, or “GPU context”), affect various aspects of rendering and include information the GPU needs to render an object. The context provides a definition of how meshes are rendered and includes information such as the current vertex/index buffers, the current vertex/pixel shader programs, shader inputs, texture, material, lighting, transparency, and the like. The context contains information unique to the draw or set of draws being rendered at the graphics pipeline. The GPU context also includes compute, video, display, and machine learning contexts. Each internal GPU engine includes a context. “Context” therefore refers to the required GPU pipeline state to correctly draw something as well as the compute, video, display, and machine learning contexts for each internal GPU engine of the GPU.

The context is locally maintained at a GPU memory (i.e., a frame buffer) for quick access by the graphics pipeline. The frame buffer also stores additional data such as firmware, application data, and GPU configurational data (collectively referred to as “data”). In addition, each of the internal GPU engines (microprocessors) includes firmware, registers and a static random access memory (SRAM). The GPU is also connected to a non-volatile memory such as an electrically erasable programmable read-only memory (EEPROM) by a relatively slow serial bus. The EEPROM is configured to store microcontroller firmware for each of the internal GPU engines, GPU subsystem specific data, and sequence instructions on how to initialize the GPU. In a normal boot sequence that occurs when the GPU is powered up after being placed in a fully or partially power gated state, the GPU retrieves the microcontroller firmware over the slow serial bus interface and follows the initialization sequences, including subsystem training, calibration, and set up, which is typically a relatively lengthy process. A driver is then invoked to carry some of the microcontroller firmware and load the microcontroller firmware to the internal GPU engines from the CPU. The driver also initializes the internal GPU engines.

However, accessing the microcontroller firmware via the serial bus and invoking the driver to initialize the internal GPU engines is time-consuming and therefore limits the opportunities for placing the GPU is a powered down mode. In addition, the driver is invoked by an operating system of the processing system, which is unavailable when the CPU is also powered down or busy serving other devices in the processing system.

FIGS.1-7illustrate techniques for using a trusted processor of a processing system to save and restore context and content of a GPU concurrent with initialization of a CPU of the processing system. In response to detecting that the GPU is powering down (i.e., transitioning to a fully or partially power gated state), the trusted processor accesses the context of the GPU, including all initialization settings, and data stored at a frame buffer of the GPU before the GPU enters a low power state. In some embodiments, the trusted processor accesses the context via a high-speed bus such as a peripheral component interconnect express (PCIe) high-speed serial bus. The trusted processor also saves data such as the firmware, registers, and SRAM from the internal GPU engines that are being power gated to system memory. The trusted processor stores the context and data at off-chip memory such as system memory dynamic random-access memory (DRAM), which maintains the context and data while the GPU is powered down. In response to detecting that the GPU is powering up again, the trusted processor restores the context directly to the internal GPU engines in lieu of reinitialization, retraining, recalibration, and re-setup when the GPU exits the low power state. In addition, the trusted processor restores the data such as firmware, registers, and SRAM to the internal GPU engines when the internal GPU engines exit the low power state before the CPU can trigger the driver to reinitialize. Thus, restoration of the context and data to the internal GPU engines is independent of driver initialization or CPU scheduling and can be performed concurrently with initialization of the CPU.

In some embodiments, the trusted processor detects tampering of the context and data prior to restoring the context and data to the GPU. The trusted processor protects the context and data from tampering by hashing the context and data to generate a first hash value and encrypting the context and data prior to storing the context and data at the system memory. In response to detecting that the GPU is powering up, the trusted processor accesses the encrypted context and data and hashes the context and data to generate a second hash value. The trusted processor compares the first hash value to the second hash value to detect tampering prior to decrypting and restoring the context and data to the GPU.

In some embodiments, the system memory includes a pre-reserved portion for storing the GPU context and data. If the system memory does not include a pre-reserved portion for storing the GPU context and data, in some embodiments, a driver dynamically allocates a portion of the system memory for storing the context and data in response to the GPU powering down.

By leveraging the trusted processor to save and restore the context and data to the GPU in response to the GPU powering down and then powering up again, the GPU can bypass the reinitialization process when the GPU powers up. In addition, the trusted processor can restore the GPU context and data in parallel with the CPU powering up, without having to wait for the operating system to invoke the driver. The trusted processor further detects tampering of the context and data, providing security for the GPU data. The techniques described herein are, in different embodiments, employed at any of a variety of parallel processors (e.g., vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), non-scalar processors, highly-parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multithreaded processing units, and the like).

FIG.1illustrates a processing system100including a trusted processor120to save and restore context155and content (illustrated as data160) of a graphics processing unit (GPU)110concurrent with initialization of a CPU105in accordance with some embodiments. The GPU110is part of a GPU subsystem102that includes the GPU110, a frame buffer115, and a non-volatile memory135that is connected to the GPU110via a serial bus165. In some embodiments, the components of the GPU subsystem102are soldered to a printed circuit board (PCB) (not shown). The processing system100also includes a power management controller150, a system memory140, a driver130, and an interconnect125. The processing system100is generally configured to execute sets of instructions (e.g., applications) that, when executed, manipulate one or more aspects of an electronic device in order to carry out tasks specified by the sets of instructions. Accordingly, in different embodiments the processing system100is part of one of a variety of electronic devices, such as desktop computer, laptop computer, server, smartphone, tablet, game console, and the like.

In various embodiments, the CPU105includes one or more single- or multi-core CPUs. In various embodiments, the GPU110includes any cooperating collection of hardware and/or software that perform functions and computations associated with accelerating graphics processing tasks, data parallel tasks, nested data parallel tasks in an accelerated manner with respect to resources such as conventional CPUs, conventional graphics processing units (GPUs), and combinations thereof. In the embodiment ofFIG.1, the GPU subsystem102is an add-in card to the processing system100such that a user can add or replace the GPU subsystem102. It should be appreciated that processing system100may include more or fewer components than illustrated inFIG.1. For example, processing system100may additionally include one or more input interfaces, non-volatile storage, one or more output interfaces, network interfaces, and one or more displays or display interfaces.

Access to system memory140is managed by a memory controller (not shown), which is coupled to system memory140. For example, requests from the CPU105or other devices for reading from or for writing to system memory140are managed by the memory controller. In some embodiments, one or more applications (not shown) include various programs or commands to perform computations that are also executed at the CPU105. The CPU105sends selected commands for processing at the GPU110. The operating system145and the interconnect125are discussed in greater detail below. The processing system100further includes a device driver130and a memory management unit, such as an input/output memory management unit (IOMMU) (not shown). Components of processing system100are implemented as hardware, firmware, software, or any combination thereof. In some embodiments the processing system100includes one or more software, hardware, and firmware components in addition to or different from those shown inFIG.1.

Within the processing system100, the system memory140includes non-persistent memory, such as DRAM (not shown). In various embodiments, the system memory140stores processing logic instructions, constant values, variable values during execution of portions of applications or other processing logic, or other desired information. For example, in various embodiments, parts of control logic to perform one or more operations on CPU105reside within system memory140during execution of the respective portions of the operation by CPU105. During execution, respective applications, operating system functions, processing logic commands, and system software reside in system memory140. Control logic commands that are fundamental to operating system145generally reside in system memory140during execution. In some embodiments, other software commands (e.g., a set of instructions or commands used to implement a device driver130) also reside in system memory140during execution of processing system100. In some embodiments, the GPU subsystem102includes additional non-volatile memory, or dedicated memory that is either on-chip or off-chip with a dedicated power rail such that the memory remains powered up when the GPU110is powered down (i.e., fully or partially power gated) that the GPU context and data can be saved to and restored from.

In various embodiments, the communications infrastructure (referred to as interconnect125) interconnects the components of processing system100. Interconnect125includes (not shown) one or more of a peripheral component interconnect (PCI) bus, extended PCI (PCI-E) bus, advanced microcontroller bus architecture (AMBA) bus, advanced graphics port (AGP), or other such communication infrastructure and interconnects. In some embodiments, interconnect125also includes an Ethernet network or any other suitable physical communications infrastructure that satisfies an application's data transfer rate requirements. Interconnect125also includes the functionality to interconnect components, including components of processing system100.

A driver, such as driver130, communicates with a device (e.g., GPU110) through an interconnect or the interconnect125. When a calling program invokes a routine in the driver130, the driver130issues commands to the device. Once the device sends data back to the driver130, the driver130invokes routines in an original calling program. In general, device drivers are hardware-dependent and operating-system-specific to provide interrupt handling required for any necessary asynchronous time-dependent hardware interface. In various embodiments, the driver130controls operation of the GPU110by, for example, providing an application programming interface (API) to software (e.g., applications) executing at the CPU105to access various functionality of the GPU110.

The CPU105includes (not shown) one or more of a control processor, field programmable gate array (FPGA), application specific integrated circuit (ASIC), or digital signal processor (DSP). The CPU105executes at least a portion of the control logic that controls the operation of the processing system100. For example, in various embodiments, the CPU105executes the operating system145, the one or more applications, and the device driver130. In some embodiments, the CPU105initiates and controls the execution of the one or more applications by distributing the processing associated with one or more applications across the CPU105and other processing resources, such as the GPU110.

The GPU110executes commands and programs for selected functions, such as graphics operations and other operations that are particularly suited for parallel processing. In general, GPU110is frequently used for executing graphics pipeline operations, such as pixel operations, geometric computations, and rendering an image to a display. In some embodiments, GPU110also executes compute processing operations (e.g., those operations unrelated to graphics such as video operations, physics simulations, computational fluid dynamics, etc.), based on commands or instructions received from the CPU105. For example, such commands include special instructions that are not typically defined in the instruction set architecture (ISA) of the GPU110. In some embodiments, the GPU110receives an image geometry representing a graphics image, along with one or more commands or instructions for rendering and displaying the image. In various embodiments, the image geometry corresponds to a representation of a two-dimensional (2D) or three-dimensional (3D) computerized graphics image.

The power management controller (PMC)150carries out power management policies such as policies provided by the operating system145implemented in the CPU105. The PMC150controls the power states of the GPU110by changing an operating frequency or an operating voltage supplied to the GPU110or compute units implemented in the GPU110. Some embodiments of the CPU105also implement a separate PMC (not shown) to control the power states of the CPU105. The PMC150initiates power state transitions between power management states of the GPU110to conserve power, enhance performance, or achieve other target outcomes. Power management states can include an active state, an idle state, a power-gated state, and some other states that consume different amounts of power. For example, the power states of the GPU110can include an operating state, a halt state, a stopped clock state, a sleep state with all internal clocks stopped, a sleep state with reduced voltage, and a power down state. Additional power states are also available in some embodiments and are defined by different combinations of clock frequencies, clock stoppages, and supplied voltages.

If both the CPU105and GPU110are in a power down state and the PMC150transitions the CPU105and GPU110to an active state, conventionally a bootloader (not shown) performs initialization of the hardware of the CPU105and loads the operating system (OS)145. The bootloader then hands control to the OS145, which initializes itself and configures the processing system100hardware by, for example, setting up memory management, setting timers and interrupts, and loading the device driver130. In some embodiments, the bootloader includes boot code170such as a Basic Input/Output System (BIOS) and a hardware configuration (not shown) indicating the hardware configuration of the CPU105.

The non-volatile memory135is implemented by flash memory, EEPROM, or any other type of memory device and is connected to the GPU110via a serial bus165. Conventionally, when the GPU110is powered up after being placed in a fully or partially power gated state, the GPU110retrieves microcontroller firmware stored at the non-volatile memory135over the serial bus165and follows initialization sequences, including subsystem training, calibration, and set up, which is typically a relatively lengthy process. The CPU105then invokes the driver130to carry some of the microcontroller firmware and load the microcontroller firmware to the internal GPU engines (not shown) from the CPU105and initialize the internal GPU engines.

The trusted processor120acts as a hardware root of trust for the GPU110. The trusted processor120includes a microcontroller or other processor responsible for creating, monitoring and maintaining the security environment of the GPU110. For example, in some embodiments the trusted processor manages the boot process, initializes various security related mechanisms, and monitors the GPU110for any suspicious activity or events and implementing an appropriate response.

To facilitate a faster resume time for power state transitions of the GPU110, the processing system uses the trusted processor120to directly access system memory140to save and restore GPU context155and data160without involvement of the driver130running on the CPU105. In response to detecting that the GPU110is powering down, the trusted processor120accesses the context155of the GPU110and data160stored at a frame buffer115of the GPU110via the interconnect125. The trusted processor120stores the context155and data160at the system memory140. The system memory140maintains the context155and data160during the time when the GPU110is powered down. In response to detecting that the GPU110is powering up again, the trusted processor120restores the context155and data160to the GPU110. In some embodiments, the trusted processor120is implemented in the GPU110and is powered down with the GPU110in the event the GPU110is fully powered down. When power is ungated, the trusted processor120wakes up and executes the restore sequence. For example, in some embodiments, the trusted processor120issues a direct memory access command to the system memory140to transfer the context155and data160in response to waking up. Because the trusted processor120performs direct memory accesses to the system memory140independent of the driver130, the trusted processor120is able to restore the context155and data160to the GPU110such that the GPU110can resume operations in a powered up data concurrently with initialization of the CPU105. By facilitating a faster resume time for the GPU110, the trusted processor120provides the PMC150with more opportunities to power down the GPU110, resulting in higher efficiency for the processing system100without the expense of adding more persistent memory to the processing system100.

In some embodiments, rather than storing the context155and data160at the system memory140when the GPU110is partially or fully power gated, the trusted processor120stores the context155and data160at another memory of the processing system100. For example, in some embodiments, the trusted processor120stores the context155and data160at additional non-volatile memory (not shown), or dedicated memory (not shown) that is either on-chip or off-chip with a dedicated power rail (not shown) such that the memory remains powered up when the GPU110is powered down (i.e., fully or partially power gated).

In some embodiments, the trusted processor120detects tampering of the context155and data160prior to restoring the context155and data160to the GPU110. The trusted processor hashes the context155and data160to generate a first hash value (not shown) and encrypting the context155and data160prior to storing the context155and data160at the system memory140. In response to detecting that the GPU110is powering up, the trusted processor120accesses the encrypted context155and data160and hashes the context155and data160to generate a second hash value (not shown). The trusted processor120compares the first hash value to the second hash value to detect tampering prior to decrypting and restoring the context155and data160to the GPU110.

FIG.2is a block diagram of the trusted processor120saving context155and data160of the GPU110to system memory140in response to the GPU110powering down in accordance with some embodiments. The trusted processor120includes a direct memory access (DMA) engine210that reads or write blocks of information from the system memory140. The DMA engine210generates addresses and initiates memory read or write cycles. Thus, the trusted processor210reads information from the system memory140and write information to the system memory140via the DMA engine210. In some embodiments, the DMA engine210is implemented in the trusted processor120and in other embodiments the DMA engine210is implemented as a separate entity from the trusted processor120. The trusted processor120can perform other operations concurrently with the data transfers being performed by the DMA engine210, which may provide an interrupt to the trusted processor120to indicate that the transfer is complete.

In the illustrated example, in response to detecting that the GPU110is powering down, the trusted processor120retrieves the context155and the contents (data160) of the frame buffer115of the GPU110. The DMA engine210writes the context155and data160to the system memory140. In some embodiments, the trusted processor120authenticates the context155and data160by, for example, appending a signature215to the context155and data160.

FIG.3is a block diagram of the trusted processor120restoring the context155and content160of the GPU110from the system memory140to the GPU110in response to the GPU110powering up in accordance with some embodiments. In the illustrated example, in response to detecting that the GPU110is powering up, the DMA engine210retrieves the context155and data160from the system memory140. In some embodiments, the trusted processor120authenticates the context155and data160by, for example, verifying that a signature315appended to the context155and data160matches an expected signature320when the trusted processor120retrieves the context155and data160in response to the GPU110powering up.

Once the trusted processor120has authenticated the context155and data160by verifying that the signature315matches the expected signature320, the trusted processor120restores the context155to the GPU110and restores the data160to the frame buffer115. In some embodiments, if the trusted processor120determines that the signature315does not match the expected signature320, the trusted processor120does not provide the context155and data160to the GPU110. If the trusted processor120does not provide the context155and data160to the GPU110such that the GPU110can be restored, the trusted processor120triggers the full GPU110initialization sequence from the non-volatile memory135. The driver130, in turn, initializes the internal GPU engines (not shown) that it manages.

FIG.4is a block diagram of the trusted processor120encrypting and hashing the context155and data160of the GPU110in response to the GPU110powering down, prior to storing the data160and context155at the system memory140in accordance with some embodiments. To provide for cryptographic protection of the context155and data160, the trusted processor120includes an encryption module410configured to encrypt and decrypt information according to a specified cryptographic standard. In some embodiments, the encryption module410is configured to employ Advanced Encryption Standard (AES) encryption and decryption, but in other embodiments the encryption module410may employ other encryption/decryption techniques. The encryption module410employs a key425to encrypt the context155and data160and provides the encrypted context455and encrypted data460to the system memory140for storage.

In some embodiments, the trusted processor120validates the encrypted context455and the encrypted data460using a validation protocol such as calculating a cryptographic hash (referred to as “hash”)415, or other protocol to determine whether the encrypted context455and the encrypted data460are valid. In some embodiments, the trusted processor120calculates the hash415of the encrypted context455and encrypted data460using the key425and then sends the hash415, the encrypted context455and encrypted data460to the system memory140.

Calculating the hash415refers to a procedure in which a variable amount of data is processed by a function to produce a fixed length result, referred to as a hash value. A hash function should be deterministic, such that the same data, presented in the same order should always produce the same hash value. A change in the order of the data or of one or more values of the data should produce a different hash value. A hash function may use a key word, or “hash key,” such that the same data hashed with a different key produces a different hash value. Since the hash value may have fewer unique values that the potential combinations of input data, different combinations of data input may result in the same hash value. For example, a 16-bit hash value will have 65536 unique values, whereas four bytes of data may have over four billion unique combinations. Therefore, a hash value length may be chosen that minimizes the potential duplicate results while not being so long as to make the hash function too complicated or time consuming.

FIG.5is a block diagram of the trusted processor120verifying that the context155and data160are untampered in accordance with some embodiments. In response to detecting that the GPU110is powering up, the trusted processor120retrieves the encrypted context455, the encrypted data460, the signature215, and the hash415from the system memory via the interconnect125. The trusted processor120calculates a second hash505of the encrypted context455and the encrypted data460using the key425. The trusted processor120includes a comparator530configured to compare the hash415to the second hash505. If the values of the hash415to the second hash505match, then the trusted processor120verifies that the encrypted context455and the encrypted data460have not been tampered. In response to determining that the encrypted context455and the encrypted data460have not been tampered, the encryption module410decrypts the encrypted context455and the encrypted data460and restores the context155and data160to the GPU110.

FIG.6is a block diagram of the driver130allocating a portion610of system memory140for storing the context155and data160of the GPU110in accordance with some embodiments. In some embodiments, the system memory140includes a pre-reserved portion for storing the context155and data160(or encrypted context455and encrypted data460). If the system memory140does not include a pre-reserved portion for storing the context155and data160, in some embodiments, the driver130dynamically allocates a portion610of the system memory140for storing the context155and data160in response to the GPU110powering down. The driver130determines the size of the context155and data160and allocates a sufficient portion610of the system memory140to store the context155and data160. In some embodiments, the driver saves a notation of the address range of the portion610, referred to as address notation620, at non-volatile memory135. In other embodiments, the driver130saves the address notation620at another location of the processing system. When the trusted processor120detects that the GPU110is powering down, the trusted processor120accesses the address notation620to determine where in the system memory140to store the context155and data160that the trusted processor120retrieves from the GPU110.

FIG.7is a flow diagram illustrating a method700for saving and restoring context155and data160of the GPU110concurrent with initialization of the CPU105in accordance with some embodiments. At block702, the driver130allocates a portion610of the system memory140to store the context155and data160of the GPU110, if the portion610was not pre-reserved. At block704, the driver130stores the address notation620of the address range of the portion610at non-volatile memory135or another location of the processing system100.

At block706, the PMC150initiates a power state transition of the GPU110to power down the GPU110. At block708, in response to detecting that the GPU110is powering down, the trusted processor120accesses the context155of the GPU110and data160stored at the frame buffer115of the GPU110. In some embodiments, the trusted processor120encrypts the context155and data160and generates a hash415to secure the context155and data160and detect tampering. At block710, the trusted processor stores the context155and data160(or encrypted context455and encrypted data460) at the portion610of the system memory140.

At block712, the PMC150initiates a power state transition of the GPU110to power up the GPU110. At block714, in response to detecting that the GPU110is powering up, the trusted processor120retrieves the context155and data160(or encrypted context455and encrypted data460) from the portion610of the system memory140. In some embodiments, the trusted processor120generates a second hash505of the encrypted context455and encrypted data460and compares the hash415to the second hash505to determine if the encrypted context455and encrypted data460have been tampered. The trusted processor120decrypts the encrypted context455and encrypted data460and restores the context155and data160to the GPU110concurrently with initialization of the CPU105.