Abstract:
An apparatus and method are described for efficiently transferring data from a core of a central processing unit (CPU) to a graphics processing unit (GPU). For example, one embodiment of a method comprises: writing data to a buffer within the core of the CPU until a designated amount of data has been written; upon detecting that the designated amount of data has been written, responsively generating an eviction cycle, the eviction cycle causing the data to be transferred from the buffer to a cache accessible by both the core and the GPU; setting an indication to indicate to the GPU that data is available in the cache; and upon the GPU detecting the indication, providing the data to the GPU from the cache upon receipt of a read signal from the GPU.

Description:
BACKGROUND 
       [0001]    1. Field of the Invention 
         [0002]    This invention relates generally to the field of computer processors. More particularly, the invention relates to an apparatus and method for implementing a memory-hierarchy aware producer-consumer instruction. 
         [0003]    2. Description of the Related Art 
         [0004]    In a model where a CPU  101  and GPU  102  work in a producer-consumer mode with the CPU as the producer and GPU as the consumer, the data transfer between them is performed as illustrated in  FIG. 1 . The CPU in the illustrated example includes a multi-level cache hierarchy including a level 1 (L1) cache  110  (sometimes referred to as an Upper Level Cache (ULC)); a level 2 (L2) cache  111  (sometimes referred to as a Mid-Level Cache (MLC)); and a level 3 (L3) cache  112  (sometimes referred to as a Lower Level Cache (LLC)). Both the GPU  102  and the CPU  101  are coupled to the L3 cache and a main memory  100 . 
         [0005]    To provide data to the GPU, the CPU performs a non-temporal store to main memory. A non-temporal store in this context is a store using data which is not anticipated to be needed by the CPU in the near future. Consequently, the store is directed to main memory rather than one of the caches in the hierarchy. Non-temporal stores may be implemented using, for example, the Uncacheable Speculative Write Combining (USWC) memory type or non-temporal store instructions (e.g., MovNT store instructions). Using a USWC operation, the data is not cached but the CPU may combine data in internal Write Combining (WC) buffers in the CPU before transferring the data all the way out to main memory. USWC operations also allow reading of data from memory in an out of order manner. 
         [0006]    Non-temporal stores are by nature weakly ordered meaning that data may be accessed in an order deviating from the order specified in program execution. For example, the program may specify the operation order “store A and then store B,” but in operation the CPU may store B and then store A. Because of this characteristic of non-temporal stores, a Fence instruction is needed to force all stores to be ordered as per program execution. The Fence instruction enforces an ordering constraint on memory operations issued before and after the Fence instruction, thereby ordering all the weakly ordered instructions from the CPU. 
         [0007]    After the data has been successfully written to main memory and ordered using a Fence instruction, the Fence producer writes to a flag notifying the consumer (the GPU in the example) that the data is ready. The consumer observes that the flag has been written, either by polling or by other techniques such as an interrupt, and generates unordered data fetch transactions (reads) to read the data. 
         [0008]    The foregoing approach suffers from low latency and low bandwidth because the store operations by the CPU and the read operations by the GPU must go all the way out to main memory  100 . Consequently, a more efficient mechanism is needed for transferring data between a CPU and a GPU. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0009]    A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which: 
           [0010]      FIG. 1  illustrates a prior art processor architecture for exchanging data between a CPU and a GPU. 
           [0011]      FIG. 2  illustrates a processor architecture in accordance with one embodiment of the invention for exchanging data between a CPU and a GPU. 
           [0012]      FIG. 3  illustrates one embodiment of a method for exchanging data between a CPU and a GPU. 
           [0013]      FIG. 4  illustrates a computer system on which embodiments of the invention may be implemented. 
           [0014]      FIG. 5  illustrates another computer system on which embodiments of the invention may be implemented. 
           [0015]      FIG. 6  illustrates another computer system on which embodiments of the invention may be implemented. 
       
    
    
     DETAILED DESCRIPTION 
       [0016]    In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention described below. It will be apparent, however, to one skilled in the art that the embodiments of the invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the embodiments of the invention. 
         [0017]    In one embodiment, rather than storing the data all the way to main memory as in prior implementations, the highest cache level common to both the CPU and the GPU is used for the data exchange. For example, if both the CPU and the GPU have read/write access to the level 3 (L3) cache (also sometimes referred to as the lower level cache) then the L3 cache is used to exchange the data. The underlying principles of the invention, however, are not limited to the use of any particular cache level for exchanging data. 
         [0018]    As illustrated in  FIG. 2 , one embodiment of the invention is implemented within the context of a multi-core central processing unit (CPU)  201 . For simplicity, the details of this embodiment of the invention are shown for a single core  211 , but the underlying principles apply equally to all cores of the CPU  201  (e.g., Core 1  219 ), and/or to single core CPUs. CPU-GPU producer-consumer logic  211  implements the techniques described herein for exchanging data with a graphics processing unit (GPU)  220 . In operation, the CPU-GPU producer-consumer logic  211  initially writes the data to be exchanged to write combining buffers  210 . Caches (such as the L1, L2, and L3 caches  212 ,  213 , and  214 , respectively) work in cache lines which are a fixed size (64 bytes in one particular embodiment) whereas typical store operations can vary from 4 bytes to 64 bytes in size. In one embodiment, the write-combining buffers  210  are used to combine multiple stores until a complete cache line is filled and then the data is moved between cache levels. Thus, in the example shown in  FIG. 2 , the data is written to the write combining buffers  210  until an amount equal to a complete cache line is stored. An eviction cycle is then generated and the data is moved from the write-combining buffers  210  to the L2 cache  213  and then from the L2 cache to the L3 cache. However, in contrast to prior implementations, a signal from the CPU-GPU producer-consumer logic  211  instructs the L3 cache  214  to hold a copy of the data for the data exchange with the GPU (rather than writing the data all the way to main memory). 
         [0019]    The CPU-GPU producer-consumer logic  211  then writes a flag  225  to indicate that the data is ready for transfer. In one embodiment, the flag  225  is a single bit (e.g., with a ‘1’ indicating that the data is ready in the L3 cache). The GPU  220  reads the flag  225  to determine that the data is ready, either through periodic polling or an interrupt. Once it learns that data is ready in the L3 cache (or other highest common cache level shared with the CPU  201 ), the GPU  220  reads the data. 
         [0020]    A method in accordance with one embodiment of the invention is illustrated in  FIG. 3 . The method may be implemented within the context of the architecture shown in  FIG. 2 , but is not limited to any particular architecture. 
         [0021]    At  301 , the data is stored to the write-combing buffers within the CPU. As mentioned, a chunk of data equal to a complete cache line may be stored within the write-combining buffers. Once the buffer is full (e.g., by an amount equal to a cache line)  302 , an eviction cycle is generated at  303 . The eviction cycle persists until the data is stored within a cache level common to both the CPU and the GPU, determined at  304 . At  305 , a flag is set to indicate that the data is available for the GPU, and at  306 , the GPU reads the data from the cache. 
         [0022]    In one embodiment, the data is transferred to the write-combining buffers and then evicted to the L3 cache using a specific instruction, referred to herein as a MovNonAllocate (MovNA) instruction. As indicated in  FIG. 4   a , in one embodiment, individual MovNA instructions may be interleaved with one another and, as indicated by the arrows, with other write-back (WB) store instructions (i.e., write bypassing is permitted), thereby improving performance (i.e., the weaker the memory ordering semantics the faster the system can perform). In this implementation, strong ordering may be enforced by the user when required using the Fence instruction. As is understood by those of skill in the art, a fence instruction is a type of barrier and a class of instruction which causes a central processing unit (CPU) or compiler to enforce an ordering constraint on memory operations issued before and after the fence instruction. 
         [0023]    In an alternative implementation, illustrated in  FIG. 4   b , individual MovNA instructions may be interleaved with one another but, as indicated by the X through the arrows, may not be interleaved with other write-back (WB) store instructions (i.e., write bypassing is not permitted). While this implementation reduces performance (i.e., the stronger the memory ordering semantics the slower the system performs), it does not require the user to issue a fence instruction to ensure proper instruction ordering. 
         [0024]    Referring now to  FIG. 5 , shown is a block diagram of another computer system  400  in accordance with one embodiment of the present invention. The system  400  may include one or more processing elements  410 ,  415 , which are coupled to graphics memory controller hub (GMCH)  420 . The optional nature of additional processing elements  415  is denoted in  FIG. 5  with broken lines. 
         [0025]    Each processing element may be a single core or may, alternatively, include multiple cores. The processing elements may, optionally, include other on-die elements besides processing cores, such as integrated memory controller and/or integrated I/O control logic. Also, for at least one embodiment, the core(s) of the processing elements may be multithreaded in that they may include more than one hardware thread context per core. 
         [0026]      FIG. 5  illustrates that the GMCH  420  may be coupled to a memory  440  that may be, for example, a dynamic random access memory (DRAM). The DRAM may, for at least one embodiment, be associated with a non-volatile cache. 
         [0027]    The GMCH  420  may be a chipset, or a portion of a chipset. The GMCH  420  may communicate with the processor(s)  410 ,  415  and control interaction between the processor(s)  410 ,  415  and memory  440 . The GMCH  420  may also act as an accelerated bus interface between the processor(s)  410 ,  415  and other elements of the system  400 . For at least one embodiment, the GMCH  420  communicates with the processor(s)  410 ,  415  via a multi-drop bus, such as a frontside bus (FSB)  495 . 
         [0028]    Furthermore, GMCH  420  is coupled to a display  440  (such as a flat panel display). GMCH  420  may include an integrated graphics accelerator. GMCH  420  is further coupled to an input/output (I/O) controller hub (ICH)  450 , which may be used to couple various peripheral devices to system  400 . Shown for example in the embodiment of  FIG. 4  is an external graphics device  460 , which may be a discrete graphics device coupled to ICH  450 , along with another peripheral device  470 . 
         [0029]    Alternatively, additional or different processing elements may also be present in the system  400 . For example, additional processing element(s)  415  may include additional processors(s) that are the same as processor  410 , additional processor(s) that are heterogeneous or asymmetric to processor  410 , accelerators (such as, e.g., graphics accelerators or digital signal processing (DSP) units), field programmable gate arrays, or any other processing element. There can be a variety of differences between the physical resources  410 ,  415  in terms of a spectrum of metrics of merit including architectural, microarchitectural, thermal, power consumption characteristics, and the like. These differences may effectively manifest themselves as asymmetry and heterogeneity amongst the processing elements  410 ,  415 . For at least one embodiment, the various processing elements  410 ,  415  may reside in the same die package. 
         [0030]      FIG. 6  is a block diagram illustrating another exemplary data processing system which may be used in some embodiments of the invention. For example, the data processing system  500  may be a handheld computer, a personal digital assistant (PDA), a mobile telephone, a portable gaming system, a portable media player, a tablet or a handheld computing device which may include a mobile telephone, a media player, and/or a gaming system. As another example, the data processing system  500  may be a network computer or an embedded processing device within another device. 
         [0031]    According to one embodiment of the invention, the exemplary architecture of the data processing system  900  may used for the mobile devices described above. The data processing system  900  includes the processing system  520 , which may include one or more microprocessors and/or a system on an integrated circuit. The processing system  520  is coupled with a memory  910 , a power supply  525  (which includes one or more batteries) an audio input/output  540 , a display controller and display device  560 , optional input/output  550 , input device(s)  570 , and wireless transceiver(s)  530 . It will be appreciated that additional components, not shown in  FIG. 5 , may also be a part of the data processing system  500  in certain embodiments of the invention, and in certain embodiments of the invention fewer components than shown in FIG.  55  may be used. In addition, it will be appreciated that one or more buses, not shown in  FIG. 5 , may be used to interconnect the various components as is well known in the art. 
         [0032]    The memory  510  may store data and/or programs for execution by the data processing system  500 . The audio input/output  540  may include a microphone and/or a speaker to, for example, play music and/or provide telephony functionality through the speaker and microphone. The display controller and display device  560  may include a graphical user interface (GUI). The wireless (e.g., RF) transceivers  530  (e.g., a WiFi transceiver, an infrared transceiver, a Bluetooth transceiver, a wireless cellular telephony transceiver, etc.) may be used to communicate with other data processing systems. The one or more input devices  570  allow a user to provide input to the system. These input devices may be a keypad, keyboard, touch panel, multi touch panel, etc. The optional other input/output  550  may be a connector for a dock. 
         [0033]    Other embodiments of the invention may be implemented on cellular phones and pagers (e.g., in which the software is embedded in a microchip), handheld computing devices (e.g., personal digital assistants, smartphones), and/or touch-tone telephones. It should be noted, however, that the underlying principles of the invention are not limited to any particular type of communication device or communication medium. 
         [0034]    Embodiments of the invention may include various steps, which have been described above. The steps may be embodied in machine-executable instructions which may be used to cause a general-purpose or special-purpose processor to perform the steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components. 
         [0035]    Elements of the present invention may also be provided as a computer program product which may include a machine-readable medium having stored thereon instructions which may be used to program a computer (or other electronic device) to perform a process. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, propagation media or other type of media/machine-readable medium suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection). 
         [0036]    Throughout this detailed description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. In certain instances, well known structures and functions were not described in elaborate detail in order to avoid obscuring the subject matter of the present invention. Accordingly, the scope and spirit of the invention should be judged in terms of the claims which follow.