Variable frequency output to one or more buffers

A system and method are presented by which data on a graphics processing unit (GPU) can be output to one or more buffers with independent output frequencies. In one embodiment, a GPU includes a shader processor configured to respectively emit a plurality of data sets into a plurality of streams in parallel. Each data is emitted into at least a portion of its respective stream. Also included is a first number of counters configured to respectively track the emitted data sets.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a processor with variable frequency output.

2. Background Art

Processors, such as graphics processing units (GPUs), can provide fast solutions for a variety of complex algorithms due to their highly parallel structure. However, although multiple threads (or programs) can run in parallel on a conventional GPU, their output data can only be grouped into a single output stream. This limits the performance of the processor when multiple threads line up to output data into the single output stream.

What is needed, therefore, is a technique for enabling output data from multiple threads to be grouped into multiple output streams.

SUMMARY

Embodiments of the present invention relate to processors (including, but not limited to, GPUs). In particular, embodiments of the present invention relate to systems and methods for outputting data with a variable frequency on a processor (e.g., GPU). In one embodiment, a GPU includes a shader processor and a number of counters. The shader processor can process data for one or more threads in parallel. For each thread, one or more streams are assigned to the processed data corresponding to the thread. The shader processor emits the data into the assigned streams. Each stream is associated with a counter tracking the emitted data for the stream.

In another embodiment, a method for outputting data using one or more streams on a processor is introduced. The processor can process data for one or more threads in parallel. In order to output the processed data, one or more streams are enabled. A shader processor on, for example, a GPU emits the data into the enabled streams. For each thread, the corresponding data is emitted into a subset of the enabled streams. A number of counters are associated with the enabled streams respectively. The amount of emitted data in each enabled stream is tracked in the associated counter. The enabled streams are then stored in one or more stream buffers for output.

By allowing output data corresponding to different threads in parallel, the data read-write throughput of the processor can be increased.

Further features and advantages of the invention, as well as the structure and operation of various embodiments of the invention, are described in detail below with reference to the accompanying drawings.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

Embodiments of the present invention relate to a system and method for outputting data on a GPU to one or more buffers with independent output frequencies. In one embodiment, multiple sets of data can be processed by the GPU in parallel with each set of data corresponding to a respective thread as it runs on the GPU. In one embodiment, a thread can be a program that can run on the GPU, such as vector or scalar processing. Although the embodiments illustrated and described relate to GPUs, other processors (such as CPUs, vector processors and others) may benefit from aspects of the invention. Additionally, while embodiments describing aspects of the invention suggest an embodiment with a discrete GPU, a GPU or other processor may be included as part of another device or package. For example, embodiments a GPU may form part of a northbridge device or be included in a CPU or in the same package as the CPU). Other embodiments are also possible.

The multiple sets of data can be emitted into streams simultaneously. Each set of data can be emitted into one or more streams, and each stream is output to a stream buffer. This allows the threads to run in parallel on the GPU and only serializes the data output afterwards. Such parallelization enables the GPU to run faster and makes it more suitable for general purpose applications.

According to one feature of the present invention, the workload of the GPU with multiple streams is easily parallelizable without requiring complicated programming. This keeps significant amounts of hardware busy and efficiently improves throughput.

FIG. 1is a block diagram illustration of an exemplary system diagram of a GPU100with variable frequency output, according to one embodiment of the invention.

In the illustration ofFIG. 1, GPU100includes a ring buffer110, a shader processor120, a first set of counters132,134,136, and138, a set of stream buffers142,144,146, and148, and a second set of counters152,154,156, and158. Ring buffer110can be a memory device that has sections to store data for threads running on GPU100. In one embodiment, ring buffer110is an on-chip memory of GPU100. In another embodiment, ring buffer100can be an off-chip memory of GPU100. The location of ring buffer110depends upon specific applications.

Shader processor120can process data input and output for one or more threads. Shader processor120emits data into streams and stores the streamed data to stream buffers. The streams can be streams112,114,116, and118. The stream buffers can be stream buffer142,144,146, and148. Note that in alternative embodiments, GPU100can have different numbers of streams and/or stream buffers. In one embodiment, only a selected number of streams are enabled. Shader processor then emits data into the enabled streams.

In an embodiment, shader processor120first writes data to ring buffer110before emitting the data into streams.FIG. 2is an illustration of an exemplary diagram illustrating how to emit data in ring buffer110into streams on a GPU, such as GPU100. The ring buffer110is divided into sections. Each thread running in shader processor120is assigned its own section in ring buffer110. For a thread, writes to the output memory device, e.g. ring buffer110, can run in parallel without ordering conflicts. Accordingly, the data corresponding to each thread can be written to the assigned section in the ring buffer in parallel. As shown inFIG. 2, ring buffer110stores data for thread1and thread2in different sections.

By way of example, thread1data and thread2data can be written to ring buffer110in parallel. Shader processor120can then retrieve the data in ring buffer110section by section. Each thread can be assigned a number of streams. The data corresponding to each thread can then be emitted into the assigned streams. For example, thread1data is emitted into four streams: streams0-3, as shown inFIG. 2. Streams0,1,2, and3correspond to streams112,114,116, and118of GPU100, respectively.

In one embodiment, a thread can declare the number of the streams to store its corresponding data. In another embodiment, the number of the streams for a thread can be decided by GPU100. Each stream can have a different amount of emitted data compared to the other streams.

In an alternative embodiment, GPU100can associate an individual ring buffer with each stream rather than interleave data for multiple streams in a single ring buffer.

GPU100can also include an emit controller122for tracking emitted data. In the embodiment ofFIG. 1, emit controller122is included in shader processor120. Emit controller122can associate a respective counter to each stream and track the data emitted to the stream using the associated counter. For example, counter132can be associated with stream112, counter134can be associated with stream114, counter136can be associated with stream116, and counter138can be associated with stream118. Emit controller122can use counters132,134,136, and138to record the amount of the data emitted into the corresponding streams respectively.

Each counter and each stream can be identified by a thread ID. When there is a single stream, as in a conventional system, the thread ID is always incremented by one before being assigned to the next thread. In one embodiment, a thread ID is assigned per stream rather than per thread. So if there are 4 streams, the counters and streams for the first thread are assigned thread IDs0-3. The counters and streams for the next thread are assigned thread IDs4-7, and so on. Different threads can be assigned same counters and streams with different assigned thread IDs.

In some embodiments, streams don't have to be enabled sequentially. For example, streams112,116,118can be enabled while stream114is disabled. In this case, thread ID's0-2can be assigned to the first thread,3-5to the second thread, and so on. Therefore, the streams are packed to maximize the use of the counters. The configuration indicating the enabled streams can be stored in a table stored in on-chip registers or in an off-chip memory. After the thread finishes, the stored configuration information can be used to determine which stream is associated with a particular thread ID.

Shader processor120can store the data from streams112,114,116, and118in stream buffers142,144,146, and148. In one embodiment, data from each stream is stored in a different stream buffer. The second set of counters152,154,156, and158can be used to track the streams stored in the stream buffers. In one embodiment, emit controller122can associate a respective counter to each stream buffer and record the amount of data stored in the stream buffer in the corresponding counter. For example, counter152can be associated with stream buffer142, counter154can be associated with stream buffer144, counter156can be associated with stream buffer146, and counter158can be associated with stream buffer148. Counters152,154,156, and158can be the same type as counters132,134,136, and138. In one embodiment, the counters are stored in on-chip registers of GPU100. In an alternative embodiment, the counters are stored in off-chip memory.

Data for one or more threads can be sent to GPU100via input102and processed in shader processor120. Shader processor120can store the processed data in Ring buffer110before emitting the data into streams. Shader processor then retrieves the data from ring buffer110and sends it to output160via streams and stream buffers. In one embodiment, there can be multiple threads running in shader processor120in parallel. Accordingly, multiple sets of data corresponding to these threads can also be processed in parallel.

FIG. 3is a flow chart of an exemplary process300for outputting data to stream buffers via multiple streams, according to one embodiment of the invention. In step310, shader processor120can select processed data corresponding to one or more of those threads for output. Once the processing of a set of data is finished, shader processor120can first send it to a temporary memory device, such as ring buffer110. Shader processor120then selects processed data from ring buffer110for output.

In step320, shader processor120assigns streams to the threads. In one embodiment, shader processor120assigns the streams based on the number of streams declared in each thread. Shader processor120also associates a respective counter for each assigned stream. In an alternative embodiment, emit controller122associates the counters to the streams. One or more streams can be assigned to a single thread. It is possible that not all available streams are used by shader processor120. Before assigning the streams to the threads, shader processor120can enable a subset of the available streams and only assigns the enabled streams to the threads depending on particular applications. Each assigned stream has a thread ID which is determined based on the corresponding thread.

FIG. 4is an illustration of an exemplary assignment400of streams to two threads. InFIG. 4, threads410and420are assigned two streams each. Streams112and114are assigned to thread410, and streams116and118are assigned to thread420. Each stream is assigned a unique thread id. For example, stream112has thread ID0, stream114has thread ID1, stream116has thread ID2, and stream118has thread ID3. Each stream is also associated with a respective counter. For example, counter132is associated with stream112, counter134is associated with stream114, counter136is associated with stream116, and counter138is associated with stream118. The thread IDs can also be used to identify the counters associated with the streams. These examples are illustrative and not intended to limit the invention.

In step330, shader processor120emits the data into the streams assigned to the corresponding threads. For example, data for thread410is emitted into streams112and114, which have assigned thread IDs0and1respectively. Data for thread420is emitted into streams116(thread ID2) and118(thread ID3). The emitted data in each stream is tracked in its associated counter. For example, counter132tracks data for stream112, counters134tracks data for stream114, counter136tracks data for stream116, and counter138tracks data for118.

In one embodiment, the counters can show the amount of emitted data in the corresponding streams. In another example, emit controller122calculates the next available address for each stream and records it in the associated counter. Shader processor120can use such address information when emitting additional data. In one embodiment, shader processor120can emit data from ring buffer110into the streams. In an alternative embodiment, shader processor120can emit data directly into the streams after processing.

Note that, in some embodiments, not all the streams are used by shader processor120for data emission. Shader processor120can just enable a subset of the streams depending upon specific applications.

Once the data are emitted into the streams, shader processor120can begin to output the streams via stream buffers. In step340, shader processor120assigns a respective stream buffer to each stream. For example, according toFIG. 1, stream buffer142is assigned to stream112, stream buffer144is assigned to stream114, stream buffer146is assigned to stream116, and stream buffer148is assigned to stream118. This example is illustrative and not intended to limit the invention.

In step350, shader processor can output streams112,114,116, and118to corresponding stream buffers142,144,146, and148. Emit controller122can use counters152,154,156, and158to track the stored stream data in stream buffers142,144,146, and148respectively. As mentioned above, counters152,154,156, and158can be as the same type as counters132,134,136, and138. In one embodiment, each stream is stored in a respective stream buffer. In some further embodiments, two or more streams can be stored in one stream buffer.

In step360, the data in stream buffers142,144,146, and148are then sent to output160.

FIGS. 5A and 5Bare illustrations of exemplary data flows500and502in GPU100, according to one embodiment of the invention. InFIG. 5A, shader processor120receives data for m threads, such as data510i,510j, and510m(0<i,j<m). Data510i,510j, and510mcan be processed by corresponding threads running in shader processor120. Data520i,520j, and520mare processed data based on data510i,510j, and510mrespectively. Shader processor120then output data520i,520j, and520mto ring buffer110. Each thread is assigned a section in ring buffer110. Data520i,520j, and520mcan be output to the corresponding assigned sections in ring buffer110in parallel or based upon processing progress. Shader processor120can then select data from ring buffer110for output.

FIG. 5Bshows an exemplary data flow for outputting data for ring buffer110to stream buffers via streams according to one embodiment of the invention. InFIG. 5B, shader processor selects one or more sets of data from ring buffer110and emits the selected data into four streams, stream112,114,116, and118. Note that, the number of streams for data emission is decided based on specific applications. In this example, four streams are data corresponding to one thread. Stream112,114,116, and118are then stored in stream buffers142,144,146, and148. The data in the streams and stream buffers are tracked by the associated counters respectively.

In this way, the data in ring buffer110can be ordered and output sequentially. By allowing the threads to run in the shader processor in parallel and only serializing data output after they finish, the GPU can run faster and be more general purpose.

According to a feature of the present invention, shader processor120can perform data expansion or compaction automatically after the oldest thread finishes. So the data can be consumed by other system components. For data expansion, some of the data is replicated to facilitate its emission into the streams.

The data compaction is implemented by having counters that keep track of how much data has been emitted into each stream of a thread. For example, addresses for the data can be generated in the counters associated with streams. When shader processor emits the data into streams, these addresses allow shader processor120to skip empty space in ring buffer110. These counters can be stored in on-chip registers of GPU100or in an off-chip memory. When the oldest thread (or group of threads) completes, emit controller122reads the associated counters and generate addresses for the corresponding data. These addresses are used by shader processor120for data emission.

With traditional rendering, there is a single stream of output data so there is one counter per thread. In order to support multiple streams, the counters are partitioned such that each thread is assigned the same number of counters as it is assigned streams. If the counters are stored in on-chip registers of GPU100, executing additional streams can reduce the total number of threads that can be executed in parallel. If the counters are stored in a buffer in an off-chip memory, additional streams can require more memory, but don't reduce the number of threads that can execute in parallel. In other embodiments, the counters are stored on-chip to avoid the delay in waiting for the values to be fetched from the off-chip memory. The decision to implement on-chip vs. off-chip counters is based on particular applications.

According to one feature of the present invention, the workload of the GPU is easily parallelizable without requiring complicated programming. This keeps significant amounts of hardware busy and efficiently hides latency.

The present invention has been described above with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined so long as the specified functions and relationships thereof are appropriately performed. For example, various aspects of the present invention can be implemented by software, firmware, hardware (or hardware represented by software such, as for example, Verilog or hardware description language instructions), or a combination thereof. Exemplary system100in which the present invention, or portions thereof, can be implemented as computer-readable code. After reading this description, it will become apparent to a person skilled in the relevant art how to implement the invention using other computer systems and/or computer architectures.

It should be noted that the simulation, synthesis and/or manufacture of the various embodiments of this invention can be accomplished, in part, through the use of computer readable code, including general programming languages (such as C or C++), hardware description languages (HDL) including Verilog HDL, VHDL, Altera HDL (AHDL) and so on, or other available programming and/or schematic capture tools (such as circuit capture tools). This computer readable code can be disposed in any known computer usable medium including semiconductor, magnetic disk, optical disk (such as CD-ROM, DVD-ROM) and as a computer data signal embodied in a computer usable (e.g., readable) transmission medium (such as a carrier wave or any other medium including digital, optical, or analog-based medium). As such, the code can be transmitted over communication networks including the Internet and intranets. It is understood that the functions accomplished and/or structure provided by the systems and techniques described above can be represented in a core (such as a GPU core) that is embodied in program code and can be transformed to hardware as part of the production of integrated circuits.