Source: https://patents.justia.com/patent/9990287
Timestamp: 2019-10-20 06:43:41
Document Index: 378201508

Matched Legal Cases: ['§ 371', 'Application No. 101142183', 'Application No. 101142183', 'Application No. 101146624', 'Application No. 201180075740', 'Application No. 201180075875', 'Application No. 201180075875', 'Application No. 201180075740', 'Application No. 201180075740', 'Application No. 201180075875']

US Patent for Apparatus and method for memory-hierarchy aware producer-consumer instruction Patent (Patent # 9,990,287 issued June 5, 2018) - Justia Patents Search
Justia Patents CoherencyUS Patent for Apparatus and method for memory-hierarchy aware producer-consumer instruction Patent (Patent # 9,990,287)
Dec 21, 2011 - Intel
An apparatus and method are described for efficiently transferring data from a core of a central processing unit (CPU) to a graphics processing unit (GPU). For example, one embodiment of a method comprises: writing data to a buffer within the core of the CPU until a designated amount of data has been written; upon detecting that the designated amount of data has been written, responsively generating an eviction cycle, the eviction cycle causing the data to be transferred from the buffer to a cache accessible by both the core and the GPU; setting an indication to indicate to the GPU that data is available in the cache; and upon the GPU detecting the indication, providing the data to the GPU from the cache upon receipt of a read signal from the GPU.
This patent application is a U.S. National Phase Application under 35 U.S.C. § 371 of International Application No. PCT/US2011/066674, filed Dec. 21, 2011, entitled APPARATUS AND METHOD FOR MEMORY-HIERARCHY AWARE PRODUCER-CONSUMER INSTRUCTION.
This invention relates generally to the field of computer processors. More particularly, the invention relates to an apparatus and method for implementing a memory-hierarchy aware producer-consumer instruction.
In a model where a CPU 101 and GPU 102 work in a producer-consumer mode with the CPU as the producer and GPU as the consumer, the data transfer between them is performed as illustrated in FIG. 1. The CPU in the illustrated example includes a multi-level cache hierarchy including a level 1 (L1) cache 110 (sometimes referred to as an Upper Level Cache (ULC)); a level 2 (L2) cache 111 (sometimes referred to as a Mid-Level Cache (MLC)); and a level 3 (L3) cache 112 (sometimes referred to as a Lower Level Cache (LLC)). Both the GPU 102 and the CPU 101 are coupled to the L3 cache and a main memory 100.
To provide data to the GPU, the CPU performs a non-temporal store to main memory. A non-temporal store in this context is a store using data which is not anticipated to be needed by the CPU in the near future. Consequently, the store is directed to main memory rather than one of the caches in the hierarchy. Non-temporal stores may be implemented using, for example, the Uncacheable Speculative Write Combining (USWC) memory type or non-temporal store instructions (e.g., MovNT store instructions). Using a USWC operation, the data is not cached but the CPU may combine data in internal Write Combining (WC) buffers in the CPU before transferring the data all the way out to main memory. USWC operations also allow reading of data from memory in an out of order manner.
Non-temporal stores are by nature weakly ordered meaning that data may be accessed in an order deviating from the order specified in program execution. For example, the program may specify the operation order “store A and then store B,” but in operation the CPU may store B and then store A. Because of this characteristic of non-temporal stores, a Fence instruction is needed to force all stores to be ordered as per program execution. The Fence instruction enforces an ordering constraint on memory operations issued before and after the Fence instruction, thereby ordering all the weakly ordered instructions from the CPU.
After the data has been successfully written to main memory and ordered using a Fence instruction, the Fence producer writes to a flag notifying the consumer (the GPU in the example) that the data is ready. The consumer observes that the flag has been written, either by polling or by other techniques such as an interrupt, and generates unordered data fetch transactions (reads) to read the data.
The foregoing approach suffers from low latency and low bandwidth because the store operations by the CPU and the read operations by the GPU must go all the way out to main memory 100. Consequently, a more efficient mechanism is needed for transferring data between a CPU and a GPU.
FIG. 1 illustrates a prior art processor architecture for exchanging data between a CPU and a GPU.
FIG. 2 illustrates a processor architecture in accordance with one embodiment of the invention for exchanging data between a CPU and a GPU.
FIG. 3 illustrates one embodiment of a method for exchanging data between a CPU and a GPU.
In one embodiment, rather than storing the data all the way to main memory as in prior implementations, the highest cache level common to both the CPU and the GPU is used for the data exchange. For example, if both the CPU and the GPU have read/write access to the level 3 (L3) cache (also sometimes referred to as the lower level cache) then the L3 cache is used to exchange the data. The underlying principles of the invention, however, are not limited to the use of any particular cache level for exchanging data.
As illustrated in FIG. 2, one embodiment of the invention is implemented within the context of a multi-core central processing unit (CPU) 201. For simplicity, the details of this embodiment of the invention are shown for a single core 211, but the underlying principles apply equally to all cores of the CPU 201 (e.g., Core 1 219), and/or to single core CPUs. CPU-GPU producer-consumer logic 211 implements the techniques described herein for exchanging data with a graphics processing unit (GPU) 220. In operation, the CPU-GPU producer-consumer logic 211 initially writes the data to be exchanged to write combining buffers 210. Caches (such as the L1, L2, and L3 caches 212, 213, and 214, respectively) work in cache lines which are a fixed size (64 bytes in one particular embodiment) whereas typical store operations can vary from 4 bytes to 64 bytes in size. In one embodiment, the write-combining buffers 210 are used to combine multiple stores until a complete cache line is filled and then the data is moved between cache levels. Thus, in the example shown in FIG. 2, the data is written to the write combining buffers 210 until an amount equal to a complete cache line is stored. An eviction cycle is then generated and the data is moved from the write-combining buffers 210 to the L2 cache 213 and then from the L2 cache to the L3 cache. However, in contrast to prior implementations, a signal from the CPU-GPU producer-consumer logic 211 instructs the L3 cache 214 to hold a copy of the data for the data exchange with the GPU (rather than writing the data all the way to main memory).
The CPU-GPU producer-consumer logic 211 then writes a flag 225 to indicate that the data is ready for transfer. In one embodiment, the flag 225 is a single bit (e.g., with a ‘1’ indicating that the data is ready in the L3 cache). The GPU 220 reads the flag 225 to determine that the data is ready, either through periodic polling or an interrupt. Once it learns that data is ready in the L3 cache (or other highest common cache level shared with the CPU 201), the GPU 220 reads the data.
At 301, the data is stored to the write-combing buffers within the CPU. As mentioned, a chunk of data equal to a complete cache line may be stored within the write-combining buffers. Once the buffer is full (e.g., by an amount equal to a cache line) 302, an eviction cycle is generated at 303. The eviction cycle persists until the data is stored within a cache level common to both the CPU and the GPU, determined at 304. At 305, a flag is set to indicate that the data is available for the GPU, and at 306, the GPU reads the data from the cache.
In one embodiment, the data is transferred to the write-combining buffers and then evicted to the L3 cache using a specific instruction, referred to herein as a MovNonAllocate (MovNA) instruction. As indicated in FIG. 4a, in one embodiment, individual MovNA instructions may be interleaved with one another and, as indicated by the arrows, with other write-back (WB) store instructions (i.e., write bypassing is permitted), thereby improving performance (i.e., the weaker the memory ordering semantics the faster the system can perform). In this implementation, strong ordering may be enforced by the user when required using the Fence instruction. As is understood by those of skill in the art, a fence instruction is a type of barrier and a class of instruction which causes a central processing unit (CPU) or compiler to enforce an ordering constraint on memory operations issued before and after the fence instruction.
In an alternative implementation, illustrated in FIG. 4b, individual MovNA instructions may be interleaved with one another but, as indicated by the X through the arrows, may not be interleaved with other write-back (WB) store instructions (i.e., write bypassing is not permitted). While this implementation reduces performance (i.e., the stronger the memory ordering semantics the slower the system performs), it does not require the user to issue a fence instruction to ensure proper instruction ordering.
1. A method for transferring a chunk of data from a core of a central processing unit (CPU) to a graphics processing unit (GPU), comprising:
executing a first instruction, the first instruction being a single instruction, wherein the first instruction comprises a MovNonAllocate store instruction, the executing comprising: responsive to the first instruction, writing data, without caching the data, to a buffer within the core of the CPU until a designated amount of data has been written, wherein the buffer combines multiple stores until the designated amount of data has been written, and upon detecting that the designated amount of data has been written, responsively generating an eviction cycle, the eviction cycle causing the data to be transferred from the buffer to a cache shared by both the core and the GPU, wherein the cache is a level 3 cache;
setting an indication to indicate to the GPU that data is available in the cache; and
upon the GPU detecting the indication, providing the data to the GPU from the cache upon receipt of a read signal from the GPU.
2. The method as in claim 1 wherein the indication comprises a flag writable by the core and readable by the GPU.
4. The method as in claim 1 wherein the GPU reads the indication via a polling technique in which the GPU periodically reads polls for the indication.
5. The method as in claim 1 wherein the GPU reads the indication in response to an interrupt signal.
8. The method as in claim 7 wherein the other instructions are write-back store instructions.
9. The method as in claim 1 wherein the buffer within the CPU is a write-back buffer.
10. The method as in claim 1 wherein the CPU comprises a plurality of cores, each capable of performing the method.
11. An instruction processing apparatus comprising:
at least one core of a central processing unit (CPU) and a cache shared by both the core and a graphics processing unit (GPU); and
the core comprising CPU-GPU producer-consumer logic configured to perform the operations of: executing a first instruction, the first instruction being a single instruction, wherein the first instruction comprises a MovNonAllocate store instruction, the executing comprising: writing data, without caching the data, to a buffer within the core of the CPU until a designated amount of data has been written, wherein the buffer combines multiple stores until the designated amount of data has been written, and upon detecting that the designated amount of data has been written, responsively generating an eviction cycle, the eviction cycle causing the data to be transferred from the buffer to the cache shared by both the core and the GPU, wherein the cache is a level 3 cache; and setting an indication to indicate to the GPU that data is available in the cache; wherein upon the GPU detecting the indication, the data is provided to the GPU from the cache upon receipt of a read signal from the GPU.
12. The instruction processing apparatus as in claim 11 wherein the indication comprises a flag writable by the core and readable by the GPU.
13. The instruction processing apparatus as in claim 12 wherein the flag comprises a binary value indicative having a first value indicating that the data is available in the cache and a second value indicating that data is not available in the cache.
14. The instruction processing apparatus as in claim 11 wherein the GPU reads the indication via a polling technique in which the GPU periodically reads polls for the indication.
15. The instruction processing apparatus as in claim 11 wherein the GPU reads the indication in response to an interrupt signal.
16. The instruction processing apparatus as in claim 11 wherein the CPU-GPU producer-consumer logic permits the first instruction to be interleaved with a plurality of other instructions of the same instruction type.
17. The instruction processing apparatus as in claim 11 wherein the CPU-GPU producer-consumer logic permits the first instruction to be interleaved with a plurality of other instructions of different instruction types.
18. The instruction processing apparatus as in claim 17 wherein the other instructions are write-back store instructions.
19. The instruction processing apparatus as in claim 11 wherein the buffer within the CPU is a write-back buffer.
20. The instruction processing apparatus as in claim 11 wherein the CPU comprises a plurality of cores, each having CPU-GPU producer-consumer logic for performing the specified operations.
a graphics processor unit (GPU) for processing a set of graphics instructions to render video; and a central processing unit (CPU) comprising: at least one core and a cache shared by both the core and the GPU; and the core comprising CPU-GPU producer-consumer logic configured to perform the operations of: executing a first instruction, the first instruction being a single instruction, wherein the first instruction comprises a MovNonAllocate store instruction, the executing comprising: writing data to a buffer within the core of the CPU until a designated amount of data has been written, wherein the buffer combines multiple stores until the designated amount of data has been written, and upon detecting that the designated amount of data has been written, responsively generating an eviction cycle, the eviction cycle causing the data to be transferred from the buffer to the cache shared by both the core and the GPU, wherein the cache is a level 3 cache; and setting an indication to indicate to the GPU that data is available in the cache;
wherein upon the GPU detecting the indication, the data is provided to the GPU from the cache upon receipt of a read signal from the GPU.
5530933 June 25, 1996 Frink et al.
5623628 April 22, 1997 Brayton et al.
5630075 May 13, 1997 Joshi et al.
6052769 April 18, 2000 Huff
6513099 January 28, 2003 Smith et al.
6546462 April 8, 2003 Palanca
6598128 July 22, 2003 Yoshioka et al.
7120755 October 10, 2006 Jamil et al.
7577792 August 18, 2009 Hady et al.
8141068 March 20, 2012 Thompson
20030108053 June 12, 2003 Inaba
20030126365 July 3, 2003 Jamil et al.
20030126366 July 3, 2003 Jourdan
20050135402 June 23, 2005 Kawaguchi et al.
20060004966 January 5, 2006 Gonion
20060004996 January 5, 2006 Gonion
20060112226 May 25, 2006 Hady et al.
20060112227 May 25, 2006 Hady et al.
20060143406 June 29, 2006 Chrysos et al.
20060150010 July 6, 2006 Stiffler et al.
20080270708 October 30, 2008 Warner et al.
20090077320 March 19, 2009 Hoover et al.
20090150642 June 11, 2009 Heil
20090282221 November 12, 2009 Heil et al.
20100070730 March 18, 2010 Pop et al.
20100131720 May 27, 2010 Harper, III et al.
20100138607 June 3, 2010 Hughes et al.
20100138610 June 3, 2010 Rappoport et al.
20100220102 September 2, 2010 Wyatt et al.
20120331307 December 27, 2012 Fernandez Gutierrez
1425154 June 2003 CN
1624673 June 2005 CN
1783033 June 2006 CN
19506734 September 1995 DE
200734879 September 2007 TW
200933524 August 2009 TW
201020965 June 2010 TW
201145172 December 2011 TW
M417730 December 2011 TW
PCT/US2011/066674 Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority, or the Declaration, dated Aug. 30, 2012, 8 pages.
International Preliminary Report on Patentability for Application No. PCT/US2011/066630, dated Jul. 3, 2014, 8 pages.
Taiwan Office Action and Taiwan Search Report for Taiwan Application No. 101142183, dated Dec. 22, 2014, 13 pages.
International Search Report and Written Opinion for Application No. PCT/US2011/066630, dated Sep. 19, 2012, 10 pages.
Allowance Decision of Examination for Taiwan Application No. 101142183, dated Oct. 7, 2015, 2 pages.
Allowance Decision of Examination and Taiwan Search Report for Taiwan Application No. 101146624, dated Sep. 26, 2014, 4 pages.
International Preliminary Report on Patentability for International Application No. PCT/US2011/066674, dated Jul. 3, 2014, 5 pages.
Office Action for counterpart U.S. Appl. No. 13/994,724, dated Feb. 12, 2015, 20 pages.
Final Office Action for counterpart U.S. Appl. No. 13/994,724, dated Aug. 28, 2015, 19 pages.
Office Action for counterpart U.S. Appl. No. 13/994,724, dated Feb. 25, 2016, 24 pages.
First China Office Action for foreign counterpart China Application No. 201180075740.6, dated Jan. 28, 2016, 12 pages.
First China Office Action for foreign counterpart China Application No. 201180075875.2, dated Feb. 2, 2016, 11 pages.
Second China Office Action for foreign counterpart China Application No. 201180075875.2, dated Sep. 30, 2016, 9 pages.
Final Office Action for counterpart U.S. Appl. No. 13/994,724, dated Aug. 17, 2016, 20 pages.
Decision on Rejection for foreign counterpart China Application No. 201180075740.6, dated Dec. 27, 2016, 18 pages.
Non-Final Office Action for U.S. Appl. No. 13/994,724, dated Feb. 27, 2017, 21 pages.
Second China Office Action for foreign counterpart China Application No. 201180075740.6, dated Sep. 27, 2016, 30 pages.
Final Office Action for U.S. Appl. No. 13/994,724, dated Jul. 13, 2017, 26 pages.
Third China Office Action for foreign counterpart China Application No. 201180075875.2, dated Jun. 2, 2017, 29 pages.
Patent Publication Number: 20140192069
Inventors: Shlomo Raikin (Ofer), Raanan Sade (Kibutz Sanid), Robert Valentine (Kiryat Tivon), Julius Yuli Mandelblat (Haifa), Ron Shalev (Ceaseria), Larisa Novakovsky (Haifa)
Primary Examiner: Saptarshi Mazumder
Application Number: 13/994,122
Current U.S. Class: Coherency (711/141)
International Classification: G06F 13/38 (20060101); G06T 1/20 (20060101); G06F 12/0811 (20160101); G06F 9/30 (20180101); G06F 9/38 (20180101); G06F 13/16 (20060101); G06T 1/60 (20060101); G09G 5/00 (20060101); G06F 12/0866 (20160101);