Selectively writing back dirty cache lines concurrently with processing

A graphics pipeline includes a cache having cache lines that are configured to store data used to process frames in a graphics pipeline. The graphics pipeline is implemented using a processor that processes frames for the graphics pipeline using data stored in the cache. The processor processes a first frame and writes back a dirty cache line from the cache to a memory concurrently with processing of the first frame. The dirty cache line is retained in the cache and marked as clean subsequent to being written back to the memory. In some cases, the processor generates a hint that indicates a priority for writing back the dirty cache line based on a read command occupancy at a system memory controller.

BACKGROUND

Processing systems including graphics processing units (GPUs) implement a cache hierarchy (or multilevel cache) that uses a hierarchy of caches of varying speeds to store frequently accessed data. Data that is requested more frequently is typically cached in a relatively high speed cache (such as an L1 cache) that is deployed physically (or logically) closer to a processor core or compute unit. Higher-level caches (such as an L2 cache, an L3 cache, and the like) store data that is requested less frequently. A last level cache (LLC) is the highest level (and lowest access speed) cache and the LLC reads data directly from system memory and writes data directly to the system memory. Caches differ from memories because they implement a cache replacement policy to replace the data in a cache line in response to new data needing to be written to the cache line. For example, a least-recently-used (LRU) policy replaces data in a cache line that has not been accessed for the longest time interval by evicting the data in the LRU cache line and writing new data to the LRU cache line. The GPU processes data on a frame-by-frame basis, e.g., a graphics pipeline in the GPU renders one frame at a time. Thus, the cache hierarchy used to cache data for the graphics pipeline evicts dirty data from the caches at the end of one frame and before the start of the subsequent frame. Evicting the dirty data requires writing the dirty cache lines back to system memory, which consumes a significant amount of bandwidth and leads to bottlenecks for traffic between the cache hierarchy and the system memory. The bottlenecks have a significant performance impact on the GPU at the start of the subsequent frame because of the constrained bandwidth for reading new data into the clean cache lines and writing dirty cache lines back to the system memory.

DETAILED DESCRIPTION

FIGS.1-4disclose systems and techniques for reducing bottlenecks in the available bandwidth between a last level cache (LLC) and system memory during frame transitions in a graphics processing unit (GPU) by selectively writing back the data in dirty cache lines of the LLC based on a read command occupancy that indicates a number of pending read commands for the system memory. The data that is written back to the system memory is retained in the dirty cache lines, which are marked to indicate that the data in the marked cache lines has been written back to the system memory so the marked cache line can be treated as a clean cache line, e.g., during the transition from a first frame to a second frame. In some embodiments, dirty cache lines are selectively written back to the system memory by comparing the read command occupancy to one or more thresholds. For example, if the read command occupancy is less than a first threshold, data in the dirty cache lines is transmitted to a system memory controller (SMC) that writes the data back to the system memory. If the read command occupancy is greater than a second threshold (which is larger than the first threshold), a request to write the dirty cache lines back to the system memory is sent to the SMC with a hint that indicates that writing the data back to the system memory is low priority. The SMC therefore serves the pending read requests before performing the low priority writes to the system memory. If the read command occupancy is greater than a third threshold (which is larger than the second threshold), requests to write the dirty cache lines back to the system memory are not transmitted to the SMC.

FIG.1is a block diagram of a processing system100that selectively generates writing back dirty cache lines concurrently with processing according to some embodiments. The processing system100includes or has access to a memory105or other storage component that is implemented using a non-transitory computer readable medium such as a dynamic random-access memory (DRAM). However, in some cases, the memory105is implemented using other types of memory including static random-access memory (SRAM), nonvolatile RAM, and the like. The memory105is referred to as an external memory since it is implemented external to the processing units implemented in the processing system100. The processing system100also includes a bus110to support communication between entities implemented in the processing system100, such as the memory105. Some embodiments of the processing system100include other buses, bridges, switches, routers, and the like, which are not shown inFIG.1in the interest of clarity.

The techniques described herein are, in different embodiments, employed at any of a variety of parallel processors (e.g., vector processors, graphics processing units (GPUs), general-purpose GPUs (GPGPUs), non-scalar processors, highly-parallel processors, artificial intelligence (AI) processors, inference engines, machine learning processors, other multithreaded processing units, and the like).FIG.1illustrates an example of a parallel processor, and in particular a graphics processing unit (GPU)115, in accordance with some embodiments. The graphics processing unit (GPU)115renders images for presentation on a display120. For example, the GPU115renders objects to produce values of pixels that are provided to the display120, which uses the pixel values to display an image that represents the rendered objects. The GPU115implements a plurality of compute units (CU)121,122,123(collectively referred to herein as “the compute units121-123”) that execute instructions concurrently or in parallel. In some embodiments, the compute units121-123include one or more single-instruction-multiple-data (SIMD) units and the compute units121-123are aggregated into workgroup processors, shader arrays, shader engines, and the like. The number of compute units121-123implemented in the GPU115is a matter of design choice and some embodiments of the GPU115include more or fewer compute units than shown inFIG.1. The compute units121-123can be used to implement a graphics pipeline, as discussed herein. Some embodiments of the GPU115are used for general purpose computing. The GPU115executes instructions such as program code125stored in the memory105and the GPU115stores information in the memory105such as the results of the executed instructions.

The processing system100also includes a central processing unit (CPU)130that is connected to the bus110and therefore communicates with the GPU115and the memory105via the bus110. The CPU130implements a plurality of processor cores131,132,133(collectively referred to herein as “the processor cores131-133”) that execute instructions concurrently or in parallel. The number of processor cores131-133implemented in the CPU130is a matter of design choice and some embodiments include more or fewer processor cores than illustrated inFIG.1. The processor cores131-133execute instructions such as program code135stored in the memory105and the CPU130stores information in the memory105such as the results of the executed instructions. The CPU130is also able to initiate graphics processing by issuing draw calls to the GPU115. Some embodiments of the CPU130implement multiple processor cores (not shown inFIG.1in the interest of clarity) that execute instructions concurrently or in parallel.

An input/output (I/O) engine145handles input or output operations associated with the display120, as well as other elements of the processing system100such as keyboards, mice, printers, external disks, and the like. The I/O engine145is coupled to the bus110so that the I/O engine145communicates with the memory105, the GPU115, or the CPU130. In the illustrated embodiment, the I/O engine145reads information stored on an external storage component150, which is implemented using a non-transitory computer readable medium such as a compact disk (CD), a digital video disc (DVD), and the like. The I/O engine145is also able to write information to the external storage component150, such as the results of processing by the GPU115or the CPU130.

In the illustrated embodiment, the compute units121-123in the GPU115include (or are associated with) one or more caches151,152,153, which are collectively referred to herein as “the caches151-153.” The caches151-153can include an L1 cache, an L2 cache, an L3 cache, or other caches in a cache hierarchy. Portions of the caches151-153are used to implement texture caches for a graphics pipeline that is executed on the compute units121-123. In the illustrated embodiment, the caches151-153are (or include) last level caches (LLC) that are the highest-level cache in the cache hierarchy. Thus, data is read directly from the memory105into the caches151-153and data is read directly back from the caches151-153to the memory105.

The processing system100also includes a system memory controller (SMC)155that receives memory access requests from entities in the processing system. The SMC155services the memory access requests using data stored in the memory105. In the illustrated embodiment, the compute units121-123process frames in the graphics pipeline. Processing of the frames includes writing data into cache lines in one or more of the caches151-153. Cache lines that include data written by the compute units121-123that has not yet been written back to the memory105are referred to as “dirty” cache lines. The dirty cache lines are evicted from the caches151-153during transitions between frames processed by the computers121-123. Evicting the dirty cache lines includes writing the data in the dirty cache lines back to the memory105. However, the bandwidth and processing power required to evict all the dirty cache lines in the caches151-153can significantly reduce the bandwidth and processing power available to begin fetching data into the caches151-153for the new frame and processing the data.

To address this problem, the compute units121-123write back one or more dirty cache lines from the caches151-153to the memory105concurrently with processing the corresponding frames. The dirty cache lines that have been written back to the memory105are also retained in the caches151-153so that the data in the dirty cache line is available for processing of the current frame. However, the dirty cache line is marked as clean subsequent to being written back to the memory so that the dirty cache line does not have to be written back to memory during a transition between frames, thereby conserving memory bandwidth and processing power during the transition. In some cases, the compute units121-123generate hints that indicate priorities for writing back the dirty cache lines based on a read command occupancy at the SMC155.

FIG.2depicts a graphics pipeline200configured to process high-order geometry primitives to generate rasterized images of three-dimensional (3D) scenes at a predetermined resolution according to some embodiments. The graphics pipeline200is implemented in some embodiments of the processing system100shown inFIG.1. The illustrated embodiment of the graphics pipeline200is implemented in accordance with the DX11 specification. Other embodiments of the graphics pipeline200are implemented in accordance with other application programming interfaces (APIs) such as Vulkan, Metal, DX12, and the like. The graphics pipeline200is subdivided into a geometry portion201that includes portions of the graphics pipeline200prior to rasterization and a pixel processing portion202that includes portions of the graphics pipeline200after rasterization.

The graphics pipeline200has access to storage resources205such as a hierarchy of one or more memories or caches that are used to implement buffers and store vertex data, texture data, and the like. In the illustrated embodiment, the storage resources205include local data store (LDS)206circuitry that is used to store data and caches207that are used to cache frequently used data during rendering by the graphics pipeline200. The storage resources205are implemented using some embodiments of the system memory105shown inFIG.1. As discussed herein, dirty cache lines in the caches207are selectively written back to system memory concurrently with processing frames using the data in the dirty cache lines to conserve memory bandwidth in graphics pipeline200.

An input assembler210accesses information from the storage resources205that is used to define objects that represent portions of a model of a scene. An example of a primitive is shown inFIG.2as a triangle211, although other types of primitives are processed in some embodiments of the graphics pipeline200. The triangle203includes one or more vertices212that are connected by one or more edges214(only one of each shown inFIG.2in the interest of clarity). The vertices212are shaded during the geometry processing portion201of the graphics pipeline200.

A vertex shader215, which is implemented in software in the illustrated embodiment, logically receives a single vertex212of a primitive as input and outputs a single vertex. Some embodiments of shaders such as the vertex shader215implement massive single-instruction-multiple-data (SIMD) processing so that multiple vertices are processed concurrently. The graphics pipeline200implements a unified shader model so that all the shaders included in the graphics pipeline200have the same execution platform on the shared massive SIMD compute units. The shaders, including the vertex shader215, are therefore implemented using a common set of resources that is referred to herein as the unified shader pool216.

A hull shader218operates on input high-order patches or control points that are used to define the input patches. The hull shader218outputs tessellation factors and other patch data. In some embodiments, primitives generated by the hull shader218are provided to a tessellator220. The tessellator220receives objects (such as patches) from the hull shader218and generates information identifying primitives corresponding to the input object, e.g., by tessellating the input objects based on tessellation factors provided to the tessellator220by the hull shader218. Tessellation subdivides input higher-order primitives such as patches into a set of lower-order output primitives that represent finer levels of detail, e.g., as indicated by tessellation factors that specify the granularity of the primitives produced by the tessellation process. A model of a scene is therefore represented by a smaller number of higher-order primitives (to save memory or bandwidth) and additional details are added by tessellating the higher-order primitive.

A domain shader224inputs a domain location and (optionally) other patch data. The domain shader224operates on the provided information and generates a single vertex for output based on the input domain location and other information. In the illustrated embodiment, the domain shader224generates primitives222based on the triangles211and the tessellation factors. A geometry shader226receives an input primitive and outputs up to four primitives that are generated by the geometry shader226based on the input primitive. In the illustrated embodiment, the geometry shader226generates the output primitives228based on the tessellated primitive222.

One stream of primitives is provided to one or more scan converters230and, in some embodiments, up to four streams of primitives are concatenated to buffers in the storage resources205. The scan converters230perform shading operations and other operations such as clipping, perspective dividing, scissoring, and viewport selection, and the like. The scan converters230generate a set232of pixels that are subsequently processed in the pixel processing portion202of the graphics pipeline200.

In the illustrated embodiment, a pixel shader234inputs a pixel flow (e.g., including the set232of pixels) and outputs zero or another pixel flow in response to the input pixel flow. An output merger block236performs blend, depth, stencil, or other operations on pixels received from the pixel shader234.

Some or all the shaders in the graphics pipeline200perform texture mapping using texture data that is stored in the storage resources205. For example, the pixel shader234can read texture data from the storage resources205and use the texture data to shade one or more pixels. The shaded pixels are then provided to a display for presentation to a user. As discussed herein, texture data used by shaders in the graphics pipeline200is cached using the cache207. Dirty cache lines in the cache207are written back concurrently with processing a frame in the graphics pipeline200using the data in the cache207.

FIG.3is a block diagram of a portion of a memory system300according to some embodiments. The memory system300is implemented in some embodiments of the processing system100shown inFIG.1and the graphics pipeline200shown inFIG.2. The memory system300includes a cache305that includes cache lines310,311,312,313, which are collectively referred to herein as “the cache lines310-313.” Data that is used by a graphics pipeline is fetched into one or more of the cache lines310-313using read/write circuitry320that sends requests325to an SMC330. The SMC330serves the request325by fetching the requested data from a corresponding memory and providing the requested data to the read/write circuitry320, which writes the requested data into one of the cache lines310-313.

The read/write circuitry320writes the data in dirty cache line310-313back to the memory via the SMC330during a transition between frames being processed in the graphics pipeline. The read/write circuitry320also writes data in some of the dirty cache lines310-313back to the memory via the SMC330concurrently with processing a frame using the data in the cache305. The data in the dirty cache line310-313is retained in the cache305and the dirty cache line310-313is marked to indicate that the data has been written back. The dirty cache line310-313is therefore treated as a clean cache line that does not need to be written back to memory during transitions between frames. In the illustrated embodiment, the cache305includes status markers335associated with the cache lines310-313. The status markers335indicate that the cache lines310and313are clean (i.e., the data in the cache lines310and313has not been modified during processing and therefore corresponds to the data currently stored at the associated address in the memory) and the cache line311is dirty (i.e., the data in the cache line311has been modified during processing but has not yet been written back to memory). The status markers335also indicate that the cache line312is in the clean/written back (CLEAN/WB) state, which indicates that the cache line312is dirty but the data in the cache line312has been written back to the memory so it can be treated as a clean cache line during frame transitions.

In some embodiments, the read/write circuitry320includes a hint with the request325to indicate a priority associated with the request to write back data from a dirty cache line. The hint is determined based on a read command occupancy, i.e., an occupancy of a queue or buffer in the SMC330that includes pending read commands that have not yet been serviced by the SMC330. If the read command occupancy is relatively low, the hint indicates that the request325to write data from the dirty cache line back to the memory should be served as soon as possible. However, if the read command occupancy is relatively high, the hint indicates that the request325has a relatively low priority. The SMC330therefore serves the pending read commands (instead of the low priority write request325) until the read command occupancy falls below a threshold. If the read command occupancy is above a maximum threshold, the read/write circuitry320bypasses transmission of requests325to write back information in dirty cache lines310-313.

FIG.4is a flow diagram of a method400of selectively writing back dirty cache lines concurrently with processing frames using data in the cache according to some embodiments. The method400is implemented in some embodiments of the processing system100shown inFIG.1, the graphics pipeline200shown inFIG.2, and the memory system300shown inFIG.3.

At block405, read/write circuitry determines a read command occupancy at an SMC in a memory subsystem including a cache. The read command occupancy indicates a fullness of a queue or a buffer used to hold pending read commands at the SMC.

At decision block410, the read/write circuitry determines whether the read command occupancy is less than a first threshold. If so, the method400flows to the block415, and the read/write circuitry sends requests for the SMC write back data in one or more dirty cache lines in the cache. If the read command occupancy is greater than the first threshold, the method400flows to the decision block420.

At decision block420, the read/write circuitry determines whether the read command occupancy is greater than the first threshold and less than a second threshold, which is greater than the first threshold. If so, the method400flows to the block425and the read/write circuitry requests that the SMC write back data in one or more dirty cache lines in the cache. The requests include a hint indicating that the request to write the data back is lower priority than continuing to process the requests in the read command queue or buffer. If the read command occupancy is greater than the second threshold, the method400flows to the block430and the read/write circuitry bypasses transmitting requests to write back dirty cache lines to the SMC (that is, bypasses writing back dirty cache lines).