Patent ID: 12210891

DETAILED DESCRIPTION

The hardware resources of a GPU are partitioned according to SR-IOV using a physical function (PF) and one or more virtual functions (VFs). Each virtual function is associated with a single physical function. In a native (host OS) environment, a physical function is used by native user mode and kernel-mode drivers and all virtual functions are disabled. All the GPU registers are assigned to the physical function via trusted access. In a virtual environment, the physical function is used by a hypervisor (host VM) and the GPU exposes a certain number of virtual functions as per the PCIe SR-IOV standard, such as one virtual function per guest VM. Each virtual function is assigned to the guest VM by the hypervisor. Subsets of the GPU resources are mapped to the virtual functions and the subsets are partitioned to include a frame buffer, context registers, a doorbell aperture, and one or more mailbox registers used for VF-PF synchronization.

A GPU that operates according to SR-IOV implements an isolation policy to provide security and safeguard the integrity of the GPU state while concurrently executing multiple VFs. For example, the isolation policy prevents any of the VFs from modifying a state of the GPU because modifications to the GPU state by one VF can affect operation of another VF. Although SR-IOV is typically used to configure a GPU to support multiple VF, the GPU can also operate in a single-VF mode. For example, the single-VF mode is used by performance analysis tools that run as a VF on the GPU. Performance analysis tools assess the performance of the GPU under different conditions and in different states. Thus, performance analysis tools are required to modify the state of the GPU. In order to satisfy the isolation policy, the performance analysis tool sends a request for a state modification through a guest driver to a mailbox register associated with the VF and the host driver reads the request from the mailbox register. The host driver performs the requested modification of the GPU status or passes the requests (and any required device information) to microcode (running on the GPU) that performs the requested modification. An acknowledgment or result of the modification performed by the host driver or the microcode is then passed back to the performance analysis tool via the mailbox register by the guest driver. However, the isolation policy is unnecessary when the GPU is running in the single-VF mode because there are no other VF to be affected by the change in the GPU status. Thus, the handshake between the host driver and the guest driver introduces unnecessary overhead.

FIGS.1-5disclose systems and techniques that reduce or eliminate overhead incurred by the handshake between the host driver and the guest driver by allowing a virtual function (VF) executing on a processing unit (such as a GPU) to modify a state of the processing unit in response to determining that the processing unit is operating in a single-VF mode that only permits one VF to execute on the processing unit. An indication of the operating mode of the processing unit (e.g., multi-VF or single-VF) is stored in an SR-IOV configuration space of the processing unit. The host driver updates the indication when it configures the processing unit. Microcode (executing on the processing unit) accesses the operating mode indicator to determine whether the processing unit is operating in the single-VF mode. If so, the VF is permitted to issue interrupts to a system management unit (SMU) and the SMU is configured to respond to the interrupts. For example, the VF can issue an interrupt to the SMU to request modifications to the state of the processing unit. Microcode executing on the SMU monitors the requests and selectively approves the requests, e.g., based on risks associated with the requested modifications. In some embodiments, the single VF executing on the processing unit is allowed to modify one or more of a clock running on the processing unit, a dynamic power management (DPM) state of the processing unit, and the like. For example, a performance monitoring tool executed by the VF can modify a graphics clock, a memory clock, or a power state of the processing unit. The performance monitoring tool then performs measurements (e.g., of one or more performance counters) to assess the impact of the modification. If the microcode determines that the processing unit is operating in the multi-VF mode, the microcode disables interrupt notification between the VF and the SMU. Requests on the VF to the SMU are not signaled to SMU. The host driver also denies requests from the VF to change the processing unit state because the changes introduce unexpected results to the other VF running on the processing unit. Thus, in the multi-VF mode, the VF may attempt to change the GPU state either by sending state modification through mailbox register to host driver or sending state modification requests directly to the SMU. However, the host driver also denies these requests to modify the state of the GPU.

FIG.1is a block diagram of a processing system100that allows modifications to a device state when the device is operating in a single-VF mode according to some embodiments. The techniques described herein are, in different embodiments, employed at any of a variety of parallel processors, such as vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), non-scalar processors, highly-parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multithreaded processing units, and the like.FIG.1illustrates an example of a parallel processor, and in particular a graphics processing unit (GPU)105, in accordance with some embodiments. However, reference to a GPU herein will be understood to include any of a variety of parallel processors unless otherwise noted.

The GPU105includes one or more GPU cores106that independently execute instructions concurrently or in parallel and one or more shader systems107that support 3D graphics or video rendering. For example, the shader system107can be used to improve visual presentation by increasing graphics rendering frame-per-second scores or patching areas of rendered images where a graphics engine did not accurately render the scene. A memory controller108provides an interface to a frame buffer109that stores frames during the rendering process. Some embodiments of the frame buffer109are implemented as a dynamic random access memory (DRAM). However, the frame buffer109can also be implemented using other types of memory including static random access memory (SRAM), nonvolatile RAM, and the like. Some embodiments of the GPU105include other circuitry such as an encoder format converter, a multiformat video codec, display output circuitry that provides an interface to a display or screen, and audio coprocessor, an audio codec for encoding/decoding audio signals, and the like.

The processing system100also includes a central processing unit (CPU)115for executing instructions. Some embodiments of the CPU115include multiple processor cores120,121,122(collectively referred to herein as “the CPU cores120-122”) that can independently execute instructions concurrently or in parallel. In some embodiments, the GPU105operates as a discrete GPU (dGPU) that is connected to the CPU115via a bus125(such as a PCI-e bus) and a northbridge130. The CPU115also includes a memory controller135that provides an interface between the CPU115and a memory140. Some embodiments of the memory140are implemented as a DRAM, an SRAM, nonvolatile RAM, and the like. The CPU115executes instructions such as program code145stored in the memory140and the CPU115stores information150in the memory140such as the results of the executed instructions. The CPU115is also able to initiate graphics processing by issuing draw calls to the GPU105. A draw call is a command that is generated by the CPU115and transmitted to the GPU105to instruct the GPU105to render an object in a frame (or a portion of an object).

A southbridge155is connected to the northbridge130. The southbridge155provides one or more interfaces160to peripheral units associated with the processing system100. Some embodiments of the interfaces160include interfaces to peripheral units such as universal serial bus (USB) devices, General Purpose I/O (GPIO), SATA for a hard disk drive, serial peripheral bus interfaces like SPI, I2C, and the like.

The GPU105includes a GPU virtual memory management unit with address translation controller (GPU MMU ATC)165and the CPU115includes a CPU MMU ATC170. The GPU MMU ATC165and the CPU MMU ATC170provide of virtual memory address (VA) to physical memory address (PA) translation by using a multilevel translation logic and a set of translation tables maintained by operating system kernel-mode driver (KMD). Thus, application processes that execute on the main OS or in the guest OS each have their own virtual address space for CPU operations and GPU rendering. The GPU MMU ATC165and the CPU MMU ATC170therefore support virtualization of GPU and CPU cores. The GPU105has its own memory management unit (MMU) which translates per-process GPU virtual addresses to physical addresses. Each process has separate CPU and GPU virtual address spaces that use distinct page tables. The video memory manager manages the GPU virtual address space of all processes and oversees allocating, growing, updating, ensuring residency of memory pages, and freeing page tables.

The GPU105also includes one or more physical functions (PFs)175. In some embodiments, the physical function175is a hardware acceleration function such as multimedia decoding, multimedia encoding, video decoding, video encoding, audio decoding, and audio encoding. The virtual environment implemented in the memory140supports a physical function and a set of virtual functions (VFs) exposed to the guest VMs. The GPU105further includes a set of registers or other resources (not shown inFIG.1in the interest of clarity) that store information associated with processing performed by kernel mode units. In some embodiments, a subset of the set of resources stores information that represents a state (or operational state) of the GPU105. Other subsets of the set of resources are allocated to store information associated with the virtual functions. These subsets of the GPU resources are mapped to the virtual functions and the subsets are partitioned to include a frame buffer, context registers, a doorbell aperture, and one or more mailbox registers used for VF-PF synchronization. The physical function175executes on behalf of one of the virtual functions for one of the guest VMs based on the information stored in a corresponding one of the subsets, as discussed in detail herein.

Some embodiments of the GPU105execute a host driver that maintains or modifies information representing an operational state of the GPU105. For example, the GPU105can operate in different modes including a first mode that allows more than one VF to execute on the PF circuitry and a second mode that constrains the PF circuitry to executing a single VF. Thus, context switches or world switches between different VF are permitted in the first mode and are not needed in the second mode. The first mode is referred to as a multi-VF mode and the second mode is referred to as a single-VF mode. If the GPU105is operating in the single-VF mode, the single VF executing on the GPU105is permitted to modify the state of the GPU105. In some embodiments, the single VF modifies the state of the GPU105by writing or modifying information stored in the resources that represent the operational state of the GPU105. If the GPU105is operating in the multi-VF mode, the virtual functions executing on the PF circuitry are not permitted to modify the operational state of the GPU105because of the potential impact on the other virtual functions. Microcode executing on the GPU105can determine the operational state of the GPU105by accessing an operating mode indicator to determine whether the processing unit is operating in the first mode or the second mode. The host driver can modify the information stored by the operating mode indicator to indicate the different operational states of the GPU105.

FIG.2is a block diagram of a processing system200that provides selective access to resources by VFs according to some embodiments. The processing system200is used to implement some embodiments of the processing system100shown inFIG.1. In the illustrated embodiment, the processing system200implements a host driver205, physical function circuitry210, and one or more virtual functions (VFs)215that execute on the physical function circuitry210, as discussed herein. A mode indicator220stores values that represent an operational mode of the processing system200. In some embodiments, the mode indicator220as a first value that indicates that the processing system200is operating in a multi-VF mode that allows the physical function circuitry210to operate more than one VF215and a second value that indicates that the processing system is operating in a single-VF mode that constrains the physical function circuitry210to operate a single VF215.

The processing system200implements a set221of resources that are allocated to the one or more VFs215executing on the physical function circuitry210. In the illustrated embodiment, the set221is partitioned into subsets of resources that are allocated to the one or more VFs215. For example, the set221is partitioned into a subset225that is reserved for frame buffers, a subset230that is reserved for context registers, a subset235that is reserved for doorbells, and a subset240that is reserved for mailbox registers. The set221also includes a subset that is reserved for registers (or other resources) that represent the state245of the processing system200. Although the subset that represents the state245of the processing system200is part of the same set221that includes the subsets225,230,235,240that are allocated to the VF215, some embodiments implement these subsets in different locations or as part of different register sets in the processing system200.

The VF215is selectively enabled to modify state245of the processing system200based on the operational mode of the processing system200. In the illustrated embodiment, the host driver205determines the operational mode of the processing system200by updating the mode indicator220. If the host driver205determines and updates the mode indicator220to a first value indicating a multi-VF mode, then access to the state245by the VF215is disabled, even if only a single VF215is executing on the PF circuitry210. If the host driver determines and updates the mode indicator220to a second value indicating a single-VF mode, then access to the state245by the VF215is enabled, e.g., interrupts can be generated to GPU blocks (such as the SMU) by the VF215. Some embodiments of the host driver205modify the information stored in the mode indicator220to indicate the operational mode of the processing system200. For example, if the processing system200is configured to execute a performance monitoring tool that uses a single VF215, the host driver205writes information to the mode indicator220to indicate that the processing system200is operating in the single-VF mode.

FIG.3is a block diagram of a processing system300that selectively enables a VF to modify an operational state of a GPU according to some embodiments. The processing system300is used to implement some embodiments of the processing system100shown inFIG.1and the processing system200shown inFIG.2. In the illustrated embodiment, the processing system300includes a GPU305and a system management unit (SMU)310that is incorporated in the GPU305. Some embodiments of the SMU310are implemented as part of an accelerated processing unit (APU) that includes both the CPU and a GPU on one die. The SMU310is responsible for a variety of system and power management tasks during boot of the processing system300and at runtime for the processing system300. The SMU310implements a microcontroller using microcode315.

The GPU305executes one or more VFs320on corresponding PF circuitry (not shown inFIG.3in the interest of clarity) depending on the operational mode of the GPU305. A mode indicator325is used to indicate the operational mode of the GPU305. Some embodiments of the mode indicator325are set to a first value that indicates that the processing system300is operating in a multi-VF mode that allows the PF circuitry to operate more than one virtual function, e.g., by world/context switching between multiple instances of virtual functions. The mode indicator325is set to a second value to indicate that the processing system300is operating in a single-VF mode that constrains the PF circuitry to operate a single virtual function, e.g., world/context switches between different instances of virtual functions are not necessary.

A set of registers330(or other resources) stores information representing the state of the GPU305. In the illustrated embodiment, subsets of the registers330are used to configure features of the GPU305such as a clock335running on the GPU305, a dynamic power management (DPM) module340that controls the power state of the GPU305and the like. The microcode315running on the GPU305selectively enables the VF320to modify the state of the GPU305by modifying value stored in the registers330. The microcode315allows modification of the registers330by the VF320in response to the mode indicator325indicating that the processing system300is operating in the single-VF mode. The microcode315does not permit the VF320to modify the registers330in response to the mode indicator325indicating that the processing system300is operating in the multi-VF mode. For example, if the virtual function320implements a performance monitoring tool that places the GPU305into the single-VF mode, microcode315permits the VF320to modify the operation of the clock335, the DPM340, or other feature of the GPU305. The performance monitoring tool then performs measurements (e.g., of one or more performance counters) to assess the impact of the modification.

In the illustrated embodiment, the VF320modifies the registers330using interrupts that are transmitted to the SMU310. The microcode315permits the VF320to issue interrupts to the SMU310and configures the SMU310to respond to the interrupts in response to the GPU305being in the single-VF mode. The microcode315executing on the SMU310monitors the interrupts and selectively approves modifications requested by the interrupts, e.g., based on risks associated with the modifications. The microcode315executing on the GPU305disables interrupt notification between the first VF320and the SMU310in response to the GPU305being in the multi-VF mode. In some embodiments, a host driver implemented in the GPU305denies requests from the VF320to modify the state of the GPU305in response to the GPU305being in the multi-VF mode.

FIG.4is a flow diagram of a method400of selectively enabling a VF to modify a state of a GPU according to some embodiments. The method400is implemented in some embodiments of the processing system100shown inFIG.1, the processing system200shown inFIG.2, and the processing system300shown inFIG.3.

At block405, a host driver implemented in the GPU updates a mode indicator that indicates whether the GPU is operating in a single-VF mode or a multi-VF mode. At decision block410, the microcode running on the SMU determines whether the GPU is operating in the single-VF mode based on the information in the mode indicator. If the GPU is operating in the single-VF mode, the method400flows to block415. Otherwise, the method400flows to block425.

At block415, microcode executing on the GPU enables interrupts from the VF to an SMU in response to the GPU being in the single-VF mode. At block420, microcode executing on the SMU configures the SMU to respond to interrupts received from the VF.

At block425, interrupts from the VF to the SMU are disabled. In some embodiments, a host driver or microcode executing on the GPU disables the interrupts from the VF to the SMU. Requests from the VF to modify the state of the GPU are denied.

FIG.5is a flow diagram of a method500of managing a request from a VF to modify a state of a GPU according to some embodiments. The method500is implemented in some embodiments of the processing system100shown inFIG.1, the processing system200shown inFIG.2, and the processing system300shown inFIG.3. Some embodiments of the method500are implemented in microcode executing on an SMU, such as the microcode315executing on the SMU310.

At block505, the microcode receives an interrupt from a VF executing on the GPU, which is operating in a single-VF mode that permits the VF to issue the interrupt to the SMU. At block510, the microcode assesses the risk of the requested modification to the GPU. At decision block515, the microcode determines whether the risk is acceptable. For example, the microcode determines whether the risk of modifying a clock, a power management state, or other state of the GPU presents a risk that is below a threshold value. If so, the method500flows to block520and the microcode responds with information indicating that the VF is allowed to modify the state of the GPU. If the risk is unacceptable, e.g., if the risk is above a threshold value, the microcode responds (at block525) with information denying the request to modify the state of the GPU.

A computer-readable storage medium includes any non-transitory storage medium, or combination of non-transitory storage media, accessible by a computer system during use to provide instructions and/or data to the computer system. Such storage media can include, but is not limited to, optical media (e.g., compact disc (CD), digital versatile disc (DVD), Blu-Ray disc), magnetic media (e.g., floppy disc, magnetic tape, or magnetic hard drive), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or Flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer-readable storage medium may be embedded in the computing system (e.g., system RAM or ROM), fixedly attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disc or Universal Serial Bus (USB)-based Flash memory), or coupled to the computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

In some embodiments, certain aspects of the techniques described above may implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software can include the instructions and certain data that, when executed by the one or more processors, manipulate the one or more processors to perform one or more aspects of the techniques described above. The non-transitory computer-readable storage medium can include, for example, a magnetic or optical disk storage device, solid-state storage devices such as Flash memory, a cache, random access memory (RAM) or other non-volatile memory device or devices, and the like. The executable instructions stored on the non-transitory computer-readable storage medium may be in source code, assembly language code, object code, or other instruction format that is interpreted or otherwise executable by one or more processors.

Note that not all of the activities or elements described above in the general description are required, that a portion of a specific activity or device may not be required, and that one or more further activities may be performed, or elements included, in addition to those described. Still further, the order in which activities are listed are not necessarily the order in which they are performed. Also, the concepts have been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present disclosure as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present disclosure.

Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any feature(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature of any or all the claims. Moreover, the particular embodiments disclosed above are illustrative only, as the disclosed subject matter may be modified and practiced in different but equivalent manners apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design herein shown, other than as described in the claims below. It is therefore evident that the particular embodiments disclosed above may be altered or modified and all such variations are considered within the scope of the disclosed subject matter. Accordingly, the protection sought herein is as set forth in the claims below.