Low latency dirty RAM for cache invalidation speed improvement

A technique for improving performance of a cache is provided. The technique involves maintaining indicators of whether cache entries are dirty in a random access memory (“RAM”) that has a lower latency to a cache controller than the cache memory that stores the cache entries. When a request to invalidate one or more cache entries is received by the cache controller, the cache controller checks the RAM to determine whether any cache entries are dirty and thus should be written out to a backing store. Using the RAM removes the need to check the actual cache memory for whether cache entries are dirty, which reduces the latency associated with performing such checks and thus with performing cache invalidations.

BACKGROUND

Cache memories are used in a wide variety of locations in computing devices. These memories aim to improve memory access speed by providing local copies of data likely to be used in the future. Because of their ubiquity, improvements to cache memories are constantly being made.

DETAILED DESCRIPTION

A technique for improving performance of a cache is provided. The technique involves maintaining indicators of whether cache entries are dirty in a random access memory (“RAM”) that has a lower latency to a cache controller than the cache memory that stores the cache entries. When a request to invalidate one or more cache entries is received by the cache controller, the cache controller checks the RAM to determine whether any cache entries are dirty and thus should be written out to a backing store. Using the RAM removes the need to check the actual cache memory for whether cache entries are dirty, which reduces the latency associated with performing such checks and thus with performing cache invalidations.

FIG. 1is a block diagram of an example device100in which one or more features of the disclosure can be implemented. The device100could be one of, but is not limited to, for example, a computer, a gaming device, a handheld device, a set-top box, a television, a mobile phone, a tablet computer, or other computing device. The device100includes a processor102, a memory104, a storage106, one or more input devices108, and one or more output devices110. The device100also includes one or more input drivers112and one or more output drivers114. Any of the input drivers112are embodied as hardware, a combination of hardware and software, or software, and serve the purpose of controlling input devices112(e.g., controlling operation, receiving inputs from, and providing data to input drivers112). Similarly, any of the output drivers114are embodied as hardware, a combination of hardware and software, or software, and serve the purpose of controlling output devices114(e.g., controlling operation, receiving inputs from, and providing data to output drivers114). It is understood that the device100can include additional components not shown inFIG. 1.

In various alternatives, the processor102includes a central processing unit (CPU), a graphics processing unit (GPU), a CPU and GPU located on the same die, or one or more processor cores, wherein each processor core can be a CPU or a GPU. In various alternatives, the memory104is located on the same die as the processor102, or is located separately from the processor102. The memory104includes a volatile or non-volatile memory, for example, random access memory (RAM), dynamic RAM, or a cache.

The input driver112and output driver114include one or more hardware, software, and/or firmware components that are configured to interface with and drive input devices108and output devices110, respectively. The input driver112communicates with the processor102and the input devices108, and permits the processor102to receive input from the input devices108. The output driver114communicates with the processor102and the output devices110, and permits the processor102to send output to the output devices110. The output driver114includes an accelerated processing device (“APD”)116which is coupled to a display device118, which, in some examples, is a physical display device or a simulated device that uses a remote display protocol to show output. The APD116is configured to accept compute commands and graphics rendering commands from processor102, to process those compute and graphics rendering commands, and to provide pixel output to display device118for display. As described in further detail below, the APD116includes one or more parallel processing units configured to perform computations in accordance with a single-instruction-multiple-data (“SIMD”) paradigm. Thus, although various functionality is described herein as being performed by or in conjunction with the APD116, in various alternatives, the functionality described as being performed by the APD116is additionally or alternatively performed by other computing devices having similar capabilities that are not driven by a host processor (e.g., processor102) and configured to provide graphical output to a display device118.

FIG. 2illustrates details of the device100and the APD116, according to an example. The processor102(FIG. 1) executes an operating system120, a driver122, and applications126, and may also execute other software alternatively or additionally. The operating system120controls various aspects of the device100, such as managing hardware resources, processing service requests, scheduling and controlling process execution, and performing other operations. The APD driver122controls operation of the APD116, sending tasks such as graphics rendering tasks or other work to the APD116for processing.

The APD116executes commands and programs for selected functions, such as graphics operations and non-graphics operations that may be suited for parallel processing. The APD116can be used for executing graphics pipeline operations such as pixel operations, geometric computations, and rendering an image to display device118based on commands received from the processor102. The APD116also executes compute processing operations that are not directly related to graphics operations, such as operations related to video, physics simulations, computational fluid dynamics, or other tasks, based on commands received from the processor102.

The APD116includes compute units132that include one or more SIMD units138that are configured to perform operations at the request of the processor102(or another unit) in a parallel manner according to a SIMD paradigm. The SIMD paradigm is one in which multiple processing elements share a single program control flow unit and program counter and thus execute the same program but are able to execute that program with different data. In one example, each SIMD unit138includes sixteen lanes, where each lane executes the same instruction at the same time as the other lanes in the SIMD unit138but can execute that instruction with different data. Lanes can be switched off with predication if not all lanes need to execute a given instruction. Predication can also be used to execute programs with divergent control flow. More specifically, for programs with conditional branches or other instructions where control flow is based on calculations performed by an individual lane, predication of lanes corresponding to control flow paths not currently being executed, and serial execution of different control flow paths allows for arbitrary control flow.

The basic unit of execution in compute units132is a work-item. Each work-item represents a single instantiation of a program that is to be executed in parallel in a particular lane. Work-items can be executed simultaneously (or partially simultaneously and partially sequentially) as a “wavefront” on a single SIMD processing unit138. One or more wavefronts are included in a “work group,” which includes a collection of work-items designated to execute the same program. A work group can be executed by executing each of the wavefronts that make up the work group. In alternatives, the wavefronts are executed on a single SIMD unit138or on different SIMD units138. Wavefronts can be thought of as the largest collection of work-items that can be executed simultaneously (or pseudo-simultaneously) on a single SIMD unit138. “Pseudo-simultaneous” execution occurs in the case of a wavefront that is larger than the number of lanes in a SIMD unit138. In such a situation, wavefronts are executed over multiple cycles, with different collections of the work-items being executed in different cycles. An APD scheduler136is configured to perform operations related to scheduling various workgroups and wavefronts on compute units132and SIMD units138.

The parallelism afforded by the compute units132is suitable for graphics related operations such as pixel value calculations, vertex transformations, and other graphics operations. Thus in some instances, a graphics pipeline134, which accepts graphics processing commands from the processor102, provides computation tasks to the compute units132for execution in parallel.

The compute units132are also used to perform computation tasks not related to graphics or not performed as part of the “normal” operation of a graphics pipeline134(e.g., custom operations performed to supplement processing performed for operation of the graphics pipeline134). An application126or other software executing on the processor102transmits programs that define such computation tasks to the APD116for execution.

FIG. 3is a block diagram showing additional details of the graphics processing pipeline134illustrated inFIG. 2. The graphics processing pipeline134includes stages that each performs specific functionality of the graphics processing pipeline134. Each stage is implemented partially or fully as shader programs executing in the programmable compute units132, or partially or fully as fixed-function, non-programmable hardware external to the compute units132.

The input assembler stage302reads primitive data from user-filled buffers (e.g., buffers filled at the request of software executed by the processor102, such as an application126) and assembles the data into primitives for use by the remainder of the pipeline. The input assembler stage302can generate different types of primitives based on the primitive data included in the user-filled buffers. The input assembler stage302formats the assembled primitives for use by the rest of the pipeline.

The vertex shader stage304processes vertices of the primitives assembled by the input assembler stage302. The vertex shader stage304performs various per-vertex operations such as transformations, skinning, morphing, and per-vertex lighting. Transformation operations include various operations to transform the coordinates of the vertices. These operations include one or more of modeling transformations, viewing transformations, projection transformations, perspective division, and viewport transformations, which modify vertex coordinates, and other operations that modify non-coordinate attributes.

The vertex shader stage304is implemented partially or fully as vertex shader programs to be executed on one or more compute units132. The vertex shader programs are provided by the processor102and are based on programs that are pre-written by a computer programmer. The driver122compiles such computer programs to generate the vertex shader programs having a format suitable for execution within the compute units132.

The hull shader stage306, tessellator stage308, and domain shader stage310work together to implement tessellation, which converts simple primitives into more complex primitives by subdividing the primitives. The hull shader stage306generates a patch for the tessellation based on an input primitive. The tessellator stage308generates a set of samples for the patch. The domain shader stage310calculates vertex positions for the vertices corresponding to the samples for the patch. The hull shader stage306and domain shader stage310can be implemented as shader programs to be executed on the compute units132, that are compiled by the driver122as with the vertex shader stage304.

The geometry shader stage312performs vertex operations on a primitive-by-primitive basis. A variety of different types of operations can be performed by the geometry shader stage312, including operations such as point sprite expansion, dynamic particle system operations, fur-fin generation, shadow volume generation, single pass render-to-cubemap, per-primitive material swapping, and per-primitive material setup. In some instances, a geometry shader program that is compiled by the driver122and that executes on the compute units132performs operations for the geometry shader stage312.

The rasterizer stage314accepts and rasterizes simple primitives (triangles) generated upstream from the rasterizer stage314. Rasterization consists of determining which screen pixels (or sub-pixel samples) are covered by a particular primitive. Rasterization is performed by fixed function hardware.

The pixel shader stage316calculates output values for screen pixels based on the primitives generated upstream and the results of rasterization. The pixel shader stage316may apply textures from texture memory. Operations for the pixel shader stage316are performed by a pixel shader program that is compiled by the driver122and that executes on the compute units132.

The output merger stage318accepts output from the pixel shader stage316and merges those outputs into a frame buffer, performing operations such as z-testing and alpha blending to determine the final color for the screen pixels.

Referring back toFIG. 2, various components of the APD116may access a high latency cache memory (which corresponds to the cache memory406ofFIG. 4). The high latency cache memory may be located at any location, including within the APD116or outside of the APD116. In some examples, at least a portion of the APD116is embodied in a single computer chip and the high latency memory is external to (but coupled to) that chip. In some examples, the high latency memory is on a printed circuit board along with, but external to, the APD116. In other examples, the high latency memory is within the APD116, but at least some of the units that access the high latency memory do so with a high latency. Herein, the term “high latency” means that the number of cycles that communication with the high latency cache takes is greater than the number of cycles that communication with a low latency cache takes.

Sometimes, a unit of the APD116requests (possibly at the request of the processor102or another entity) that entries of the high latency cache be invalidated. These requests may be part of flush requests, which are requests to invalidate ranges of memory, or may comprise individual invalidation requests to invalidate individual cache lines. A cache invalidation involves marking specified cache entries as being invalid. If the entries to be made invalid include dirty data, then the invalidation must also cause the dirty data to be written out to a backing memory (such as memory104or a memory of the APD116). Data is dirty if the copy of the data in the cache is different than that in a backing store (such as a higher level cache or system memory).

A cache controller is a hardware circuitry unit that controls access to a cache memory. More specifically, the cache controller performs functions such as receiving requests to access the cache, identifying whether the cache stores the requested entries, determining which set and way cache entries are in, accessing those entries via the set and way, and controlling other operations such as cache evictions, cache write-ins (bringing cache entries from a higher level cache or system memory into the cache), and other functions. As used herein, the term “cache line” is synonymous with “cache entry.” A cache line comprises the smallest unit at which data is written into a cache memory.

When performing invalidation operations, in order to determine whether to write out cache entries, a cache controller examines metadata. If the metadata indicates that an entry is dirty, then the cache controller causes that data to be written out to memory. If the metadata indicates that any entry is not dirty, then the cache controller marks the data as invalid without causing the data to be written to memory. With a high latency cache, the act of reading the metadata to determine whether a cache entry is dirty consumes a large number of clock cycles.

For this reason, a cache controller is disclosed herein that includes a dirty random access memory (“dirty RAM”) which stores data indicating whether entries of the high latency cache memory are dirty or not dirty.FIG. 4is a block diagram of a cache memory system400, including a cache controller404having a dirty RAM405, coupled to a high latency cache memory406, according to an example. In various implementations, the cache memory system400is a part of the APD116. Invalidation requestors402are also illustrated as being in communication with the cache controller404. The invalidation requestors402include one or more units configured to send requests to the cache controller404to invalidate one or more entries of the cache memory406. One example of an invalidation requestor is a command processor, which is part of the APD116, and which may be the same unit, or be a part of, the scheduler136. The command processor receives commands generated external to the APD116(such as by an application126executing in the processor102), and converts those commands to other commands in a format suitable for execution by the APD116. In an example, the processor102sends commands to invalidate certain cache entries stored in the cache memory406to the command processor. The command processor converts those commands into commands recognizable by the cache controller404, which then invalidates the data specified by the commands.

The cache memory406includes cache data408and cache metadata410. The cache data408is whatever data is placed into the cache memory406by virtue of units that utilize the cache memory406accessing that data. In examples, the cache data408includes cache lines, which are units of data that can be written into or read out of the cache memory406. The cache metadata410stores metadata related to the cache data408. In an example, the cache metadata410includes, on a per cache line basis, the following metadata: the memory address associated with the cache line; and multiple status bits that include a dirty bit, a valid bit, a cache line type, and/or any other status bits. In some implementations, each cache line includes multiple sectors, and the cache metadata410includes status bits for each sector.

The cache memory406may have high latency as compared with other cache memories. Thus, the cache controller404checking the cache metadata410in the cache memory406before determining whether cache entries for which invalidation has been requested are to be written out to a backing store takes a relatively long amount of time. To reduce the amount of time to be used for identifying whether particular cache entries are to be written out to a backing store, the cache controller404includes a dirty RAM405, which stores copies of the information indicating which cache lines in the cache memory406are dirty, but not any of the other status information of the cache metadata410. In implementations where the cache metadata410includes a dirty bit for each sector of a cache line, the dirty RAM405stores one dirty bit for each cache line that indicates whether any of the sectors of the cache line are dirty. Instead of examining the cache metadata410in the cache memory406, the cache controller404examines the dirty RAM405to determine whether to write out entries of the cache memory406. Then, for whichever entries are indicated as dirty, the cache controller404causes those entries to be written out to a backing store and invalidates those entries in the cache memory406.

The cache controller404maintains the dirty RAM405, updating the data in the dirty RAM405to indicates whether entries in the cache memory406are dirty or not dirty. When the cache controller404causes a new cache entry to be placed into the cache memory406, the cache controller404stores corresponding data for that entry in the dirty RAM405, with an indication that the entry in the cache memory406is not dirty. When the cache controller404causes an entry in the cache memory406to be written to, the cache controller404also updates the data in the dirty RAM405corresponding to that entry to indicate that the entry is dirty. When an entry is no longer resident (or invalid) in the cache memory406, the corresponding entry in the dirty RAM405is also made no longer resident (or invalid). In an example, a cache entry is invalidated and the corresponding entry in the dirty RAM405is also invalidated.

FIG. 5is a flow diagram of a method500for invalidating entries of a cache, according to an example. Although described with respect to the system ofFIGS. 1-4, any system, configured to perform the steps of method500in any technically feasible order, falls under the scope of the present disclosure.

The method500begins at step502, where the cache controller404receives a command to invalidate a cache entry. These commands come from an invalidation requestor402, which can be a command processor or an APD116or another entity that requests invalidation of cache entries. At step504, the cache controller404examines a local dirty RAM (random access memory)405that stores indications of whether cache entries (i.e., the cache data408) of a cache memory406are dirty. The local dirty RAM405is local to the cache controller404. Here, “local” means accessing the dirty RAM405by the cache controller404has lower latency (requires fewer clock cycles) than accessing the cache metadata410of the cache memory406. In some implementations, the local dirty RAM405is within the cache controller404. In other implementations, the local dirty RAM405is external to the cache controller404.

At step506, the cache controller404determines whether the dirty RAM405indicates that the cache entry requested to be invalidated is dirty or not. If the cache entry is dirty, then the method proceeds to step510and if the cache entry is not dirty, then the method proceeds to step508. At step510, because the cache line is dirty, the cache controller404causes the cache entry to be written out to a backing store (such as a higher level cache or system memory). To write the data back to the backing store, the cache controller404examines the cache metadata410to obtain the memory address for the data. Then, the cache controller404causes the data to be written to the obtained memory address. At step508, because the cache line is dirty, the cache controller404does not cause the cache entry to be written out to a backing store. After either step508or510, the method500proceeds to step512, where the cache controller404causes the cache line to be invalidated.

The techniques provided herein provide the benefit that the cache controller404does not need to read metadata (such as cache metadata410) in a high latency manner when invalidating cache entries. In some possible implementations, when a cache controller for a high latency cache is to invalidate cache entries, the cache controller would first read cache metadata stored in the same cache memory that stores the cache entries to determine whether the cache entries are dirty, and then would either write-out or not write-out those entries to a backing store depending on the metadata. Because the cache is high latency (meaning that accessing the cache by the cache controller takes a relatively large number of cycles—more cycles, for example, than other caches that the cache controller or another cache controller in an APD would take to access different caches), the first access to read the metadata is a slow operation. Addition of the dirty RAM speeds up this metadata lookup operation. In some situations, the cache controller is provided with a command to flush an entire high latency cache memory or an entire portion of a high latency cache memory, which spans multiple cache entries. In such situations, the cache controller would have to perform multiple read operations on the high latency cache in order to read the metadata for each of the entries that is to be invalidated. Maintaining the dirty RAM405reduces the amount of time necessary to perform invalidations.