Brevet FR3042049A1 — publié le 20170407

Data bus width matching for "in memory processing" "in memory processing" consists in the integration of processors into memory chips, in order to perform calculations on behalf of the main processor. To allow in memory processing, memory chips, named MemC (Memory Chip) in this document, comprise one or several processing units, referred to as DPU (Data Processing Unit) in this document. The main processor generally consists in one SOC (System On Chip) comprising one or several general-purpose processing cores. This SOC is referred to as the HCPU (Host Central Processing Unit) in this document. The HCPU is connected to the memory chips through a memory bus, which comprises a command bus, an address bus and a data bus. The memory accessible to a DPU Integrated into a given MemC is either the totality of the memory contained by this MemC, either a subpart of it: only the HCPU being able to access the totality of the memory of the memory system. The memory accessible to a given DPU is named the DPU_MEM of this given DPU. Classically "in memory processing" requires the DPU_MEM be seen similarly by the HCPU and the DPU. For example a 64-bit word written by the HCPU is seen identically by the DPU, and vice versa. This identical memory visibility is insured by the fact that MemCs comprising DPUs have a data port width equal to the HCPU data bus width. MemCs comprising no DPUs may use narrower data ports and be arranged in a data parallel fashion to meet the HCPU data bus width. The next page figure illustrates such a system. The HCPU data bus is sliced in eight 8-bit wide parts, the data bus connection being as follow: - MemC DPU is connected to the whole data bus[63:0], - MemC 0 is connected to data bus[7:0], - MemC 1 is connected to data bus[15:8], - MemC 2 is connected to data bus[23:16], - MemC 3 is connected to data bus[31:24], - MemC 4 is connected to data bus[39:32], - MemC 5 is connected to data bus[47:40], - MemC 6 is connected to data bus[55:48], - MemC 7 is connected to data bus[63:56]. In this configuration MemC DPU integrates a DPU, the MemC 0-7 don't. The command and address busses (represented by the same green arrows for drawing convenience) are connected in their entirety to each MemC. 1 3042049 Only the MemC integrating DPUs need to have a data port width matching the HCPU bus width. The HCPU uses generally a wide data bus. A data bus width of 64-bit, as used in the DDR3 and DDR4 protocols, is extremely common. To the contrary the data port width used by standard memory chips (without DPU integrated) is typically 4-bit, 8-bit or 16-bit. The reasons why memory chips with narrow data ports are so common are: - chips packages with narrow data ports have less pins, thus are more economical, - narrow data ports enable to build big capacity memory system easily, without requiring extra buffer chips, the use of which increasing cost, power consumption, latency, and decreasing operational frequency thus bandwidth. HCPU address and command bus data bus[63:0] 11111111 2 3042049 Classical "in memory processing" requires data port width matching Suppose that in the figure above, the MemC 0 through MemC7 chips have an integrated DPU, these MemC 0-7 being mapped starting at the address 0x04000000. Suppose then that the HCPU writes a 64-bit word, named counter, into the main memory, at the address 0x04000000, and then want a DPU of MemC 0 to increment counter. Then this DPU would see only the [7:0] fraction of this 64-bits word and thus would be unable to perform the intended calculation, which involves the entire 64-bit word. In fact none of the DPUs present into the Memc 0-7 would be capable of incrementing counter. The only solution would be that the HCPU writes counter at an address corresponding to MemC DPU, and lets one of the DPU integrated into MemC DPU do the job. Object of the invention The invention provides a mean to use memory chips, with integrated DPUs, whose data ports are narrower than the width of the HCPU data bus. The explanations are based onto a system where the HCPU data bus width is 64-bit and where the data port width of the MemC is 8-bit (one byte), this corresponding to a fairly frequent case. Naturally the invention can be used on systems with different HCPU data bus width and different MemC data port width. Definition and use of additional address spaces The memory system is addressed through the physical address space, noted PGAS (Physical Global Address Space) in this document. For the purpose of the invention, the HCPU address space, noted HGAS (HCPU Global Address Space) is defined. In a system without the invention HGAS and PGAS would be identical. In a system with the invention HGAS is defined differently than PGAS: inside HGAS an address region, named DGAS (DPU Global Address Space) is defined and: - when an HGAS address is outside the DGAS region, the corresponding PGAS address is given directly by the value of this HGAS address. - when an HGAS address is inside the DGAS region, the corresponding PGAS address is given by a function named DGAS2PGAS This is illustrated by the following figure: 3 3042049 Each DPUs sees its accessible memory through a local address space named DLAS (DPU Local Address Space), thus there are as many DLAS than DPUs. The DGAS address space maps all the DLAS. The HCPU uses the DGAS address space to access the memory of any of the DPUs (see the figure next page). The following example will be used in the remaining explanations: - there are 8 MemC, numbered from 0 through 7, - each MemC n comprises 8 MB of memory and one DPU, this one being numbered n, - the whole 64 MB of memory is mapped into the PGAS from 0 through 64 MB-1, - each DPU sees its memory from its DLAS address 0 through 8 MB-1, - DGAS is organized as follow: o the DLAS of the DPU 0 is mapped from 0 through 8 MB-1, o the DLAS of the DPU 1 is mapped from 8 MB through 16 MB-1, o the DLAS of the DPU 2 is mapped from 16 MB through 24 MB-1, o the DLAS of the DPU 3 is mapped from 24 MB through 32 MB-1, o the DLAS of the DPU 4 is mapped from 32 MB through 40 MB-1, o the DLAS of the DPU 5 is mapped from 40 MB through 48 MB-1, o the DLAS of the DPU 6 is mapped from 48 MB through 56 MB-1, o the DLAS of the DPU 7 is mapped from 56 MB through 64 MB-1. 4 3042049 The memory system knows natively only the PGAS address space. Thus the invention provides a conversion function, DGAS2PGAS( DGAS address), returning for a given DGAS byte address the corresponding PGAS byte address. The DGAS parameter is in fact the compression of two parameters: the DLAS address and the DPU global index. It is possible to avoid defining DGAS by using a function with two input parameters, the DLAS2PGAS( DLAS address, DPU global index) function. To using an address conversion function with two input parameters is only a variant of the invention. The value DGAS2PGAS( DGAS_address+1 ) is different from the value DGAS2PGAS( DGAS_address ) +1, thus the HCPU can only perform byte sized access into DGAS. 3042049 For example to write counter at the DGAS address 0, where it is visible for example to a DPU of MemC 0, the HCPU performs the following byte writes: - counter[ 7: 0] at DGAS2PGAS( 0 ), - counter[15: 8] at DGAS2PGAS( 1), - counter[23:16] at DGAS2PGAS( 2 ), - counter[31:24] at DGAS2PGAS( 3 ), - counter[39:32] at DGAS2PGAS( 4 ), - counter[47:40] at DGAS2PGAS( 5 ), - counter[55:48] at DGAS2PGAS( 6 ), - counter[63:56] at DGAS2PGAS( 7 ). This way the counter 64-bit value is entirely accessible linearly by the concerned DPU, and this one can load this value in only one 64-bit load instruction, perform the increment, and save it through a 64-bit store instruction. The HCPU will perform the following byte reads to get the incremented value: - new counter[ 7: 0] = the byte at DGAS2PGAS( 0 ), - new counter[15: 8] = the byte at DGAS2PGAS( 1), - new counter[23:16] = the byte at DGAS2PGAS( 2 ), - new counter[31:24] = the byte at DGAS2PGAS( 3 ), - new counter[39:32] = the byte at DGAS2PGAS( 4 ), - new counter[47:40] = the byte at DGAS2PGAS( 5 ), - new counter[55:48] = the byte at DGAS2PGAS( 6 ), - new counter[63:56] = the byte at DGAS2PGAS( 7 ). For the considered example, the DGAS2PGAS function is just the following address bit permutation: - PGAS address [25:0] = { DGAS address[22:0], DGAS address[25:23] } Building a complex system from the simple one Various DPUs organizations are possible. The following explanations show that complex DPUs organizations can be derived from the simplest one. Memory Rank The row of MemC (as seen in the example) is called a rank. When the memory capacity is important, several ranks of MemC can be connected to the data bus, as shown in the figure below: 6 3042049 A rank is composed of identical memory chips. While unusual, different ranks can be made with different memory chips, which can differ both in capacity and in data port width. When several ranks are present, the invention is applied to each rank independently, DGAS being defined in order to comprise a mapping region for each rank. MemC with multiple DPUs When a MemC chip comprises several DPUs then there are three possibilities: 1) All of the DPU of the given MemC are capable of accessing the whole memory of this given MemC as shown in the figure below 7 3042049 In this case the invention will be deployed similarly into each DPU: the DPUs of a given MemC using the same DLAS. 2) The MemC memory is divided in region and a given DPU of the considered MemC can only access one region as shown in the figure below: virtual rank 0 virtual rank 0 virtual rank I virtual rani (y,x) is not the only one which can be used. For example a permutation (x,y) ->(y,-x) could be used instead, the consequence being that the memory chips index are inverted, which still result in a perfectly usable system (this is equivalent to renumbering the memory chips). The invention obviously applies to all similar permutations, which differs only by the fact that they change the way the MemC are numbered. Cache line byte permutation on rectangular burst A rectangular burst is a burst which isn't square. The DDR2 protocol uses rectangular bursts as it moves 8 bytes over 4 accesses. In the DDR2 context, the cache line has a size of 32-byte, and when filled with a burst transaction, and seen as an 8x4 array, it contains: B00 B01 B02 B03 B04 B05 B06 B07 B08 B09 B10 B11 B12 B13 B14 B15 B16 B17 B18 B19 B20 B21 B22 B23 B24 B25 B26 B27 B28 B29 B30 B31 Applying the (x,y) -> (y,x) transformation, and seen the 32-byte cache line as a 4 x 8 array, then we get: B00 B08 B16 B24 B01 B09 B17 B25 B02 B10 B18 B26 B03 B11 B19 B27 B04 B12 B20 B28 B05 B13 B21 B29 11 3042049 B06 B14 B22 B30 B07 B15 B23 B31 The initial permutation was: PGAS address[25:0] = {DGAS address[22:0], DGAS address[25:23]} Note: this initial address permutation is the same than for the DDR3/DDR4 example, since this initial permutation depends only of the number of chips in parallel (8 for both the DDR2 and DDR3/DDR4 example). The new permutation is: PGAS address[25:0] = { DGAS address[22:2], DGAS address[25:23] , DGAS address[1:0] } The HCPU can perform correctly - 16-bit aligned, 16-bit access in DGAS, - 32-bit aligned, 32-bit access in DGAS. while 64-bit access in DGAS doesn't work because the DGAS linearity is only 4-byte large (only the 2-lsb of DGAS are equal to the 2-lsb of PGAS). I n the DDR2 example, the width (8) of the rectangular burst was larger than its depth (4), but the invention can be applied as straightforwardly when the width is smaller than the depth. Software Cache line byte permutation On a high performance, out of order processor, like a modern x86, the permutation by software of a 64-byte cache line could take the following number of cycles: - 8 x load 64-bit : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes and 8 x shifts instructions : 4 cycles - 8 x store bytes : 4 cycles The permutation would take a total of 36 cycles, unless the permutation code was not already present into the instruction 1,1 cache. Note: when working in cache, the considered processor is able to perform two load/store instructions per cycle simultaneously with 2 shift instructions. This number is to be compared with the cost of a random (closed page) DRAM access, which is over 200 cycles. This huge number is mainly due to the memory access has to 12 3042049 cross three levels of caches, be transported through a ring bus, before reaching the DRAM controller (and for reads, the data has to make the same path backward). The MemC memory modification by the DPU activity is not snooped by the cache system. Thus when a data is shared by the HCPU and the DPU, the data cache hierarchy has to be bypassed very frequently (either by using non cacheable mapping, either by using cache line flush and/or invalidation instructions). Thus the cost of a software permutation is only a fraction of the cost of just accessing the data. Hardware Cache line byte permutation The HCPU can comprise one or several permutation units. Since the permutation operation cannot be pipelined (the whole 8x8 array has to be charged before the permutation start), to have several permutation units may be necessary for performance reasons. A permutation unit can be implemented notably by: - the use of dual ported registers, with orthogonal write bus and read bus, - the use of single ported memory, whose entries are shift registers orthogonal to the memory bus, - the use of an 2D array of shift registers, with shift possible in one dimension, then in the other one dimension. In the context of the invention, others permutations mechanisms are possible, only the most obvious ones being described. Permanent Cache line byte permutation Since for the HCPU, when accessing regular memory (without DPU integrated) the orientation in which the cache line is written in memory is of no consequence, as long it stay the same over time, and if the connected memory chips use all the same data port width, then the cache line permutation can be done permanently. It fact this now correspond only in an alteration of the way the cache lines are loaded from, or written to, the DDR2/3/4 bus. Obviously this simplification would remove any need for cache line permutation configuration registers, but with the constraint of using MemC with a pre-determined data port width. The use of a permanent cache line permutation doesn't remove the need for address conversion. Considering the DDR3/DDR4 example: - when reading from the DGAS address space, the 64-byte cache line contains 8 x 64-bit words, each of these words belonging to a different DLAS, and thus these 8 words are not mapped contiguously in DGAS (but spaced by 8 MB in the example). - When reading from the HGAS address space, a 64-byte cache line contains 8 x 64-bit words, and the 8 words are mapped contiguously in HGAS. 13 3042049 The figure at the next page illustrates the transformation flow: the cache line permutation is always performed but the address conversion is done only for DGAS. 64-byte cache line, filled 64-byte cache line, filled from DGAS from HGAS 14 3042049 In the figure above the arrows use the following colour conventions: - orange HGAS address, maybe a DGAS address too. - red DGAS address - green PGAS address - blue data 3042049 Atomicity problems related to software only solution At the HCPU level, 16-bit write, 32-bit write and 64-bit writes, when aligned onto their own size, are atomic in the sense that either ail the concerned bytes are written, either none of them is. To write byte per byte a 16-bit, 32-bit or 64-bit value breaks this atomicity. This break in atomicity exists obviously when the HCPU write byte per byte using the DGAS2PGAS conversion routine, but the cache line permutation by software, which use byte stores to write its result is concerned too. If a control register is mapped into DGAS, and if this control register is wider than 1 byte then there is the risk of having this control register partially written, because of the lack of write atomicity (since the software cannot forbid a cache line to be written back to the memory at any time the hardware choose). The solution is: - to use a colour scheme, where a part of the command register must change at every new command: the command has a colour, and a new command is indicated by the colour being changed with respect to the colour of the previous command (the colour can be reduced to a bit of the command register). - to write the byte holding the command colour (or concluding the colour change) the last. This way the new command is taken into account only when ail its bytes have been written. ECC (Error Correcting Code) With in memory processing, ECC can no longer be implemented classically, since a memory chip may have its content changed by one of its internai DPUs. Consequently each MemC has to handle ECC internally. Considering the example where the MemC have an 8-bit wide data port, a solution is to enlarge the data port to 9-bit, and to enlarge the MemC memory array capacity accordingly. Considering the additional bit transmitted by a MemC, the MemC transmits/receives one additional byte during a complete burst transaction, since an additional bit is transmitted/received for each of the 8 burst accesses of the burst transaction. 16 3042049 Thus the ECC calculation is done "vertically" instead of being done "horizontally", as shown by the figure below: horizontal ECC ffl 9 x 8-bits x 8 accesses vertical ECC 8 x 9-bits x 8 accesses 17 3042049 Instead of widening the MemC data port, another possibility would be to extend the burst length from 8 to 9, in order to transfer the ECC code as an additional burst access in the burst transaction, as shown in the figure below: horizontal ECC 9 x 8-bits x 8 accesses vertical ECC 8 x 8-bits x 9 accesses With this vertical ECC check and generation, the HCPU memory controller needs now to have 8 ECC units working in parallel, but by current HCPU complexity standard, this isn't a big deal. Obviously each MemC integrating a DPU need to have an ECC generation unit, as the integrated DPUs can modify directly the MemC memory, and the ECC code has to be regenerated according the new memory value. 18 3042049 Near memory processing Near memory processing consists, instead of having the DPUs integrated into the memory chip, to have the DPUs integrated into additional chips placed between the HCPU and the memory chip, as shown by the figure below: The totality of the invention applies here, similarly, by considering the grey box, labelled composite MemC 0, and comprising two chips. REVENDICATIONS1) a computer system where - a host processor is connected to one or several memory chips, these memory chips integrating one or several data processors, the connection between these memory chips and the host processor being done by a data bus wider than the data port of these memory chips, these memory chips being connected to the data bus in a data parallel fashion to accommodate the data bus width. - a linear, or linear by segment, address space, is defined, this address space enabling to access linearly, or linearly by segment, the memory of each memory chip, - an address conversion function between this new address space and the original address space of the host processor, enables the host processor to access this new address space. 2) The system of claim 1 where the address conversion function is implemented directly in hardware, 3) The system of claim 2, where the address conversion hardware function is configurable, and where this function changes according to configurable memory segments. 4) The system of claims 1-3 where - the new address space comprises itself sub-address spaces, these sub-address spaces being used by the data processors integrated into the memory chips, providing them a linear, or linear by segment, access to their accessible memories, - these sub-address spaces are linearly, or linearly by segment, mapped into the new address space. 5) The system of claims 1-4 where - the host processor has a data cache, or an universal cache equally acting as a data cache, - a cache line permutation is done in order to change the address conversion function, and make this later linear by segment, enabling the host processor to correctly access the memory accessible to the data processor integrated into the memory chips, when the host processor uses load, store, or any instruction accessing the memory, whose memory access width is wider than the data port of the concerned memory chips. 6) The system of claims 5 where the cache line permutation is done entirely by software. 7) the system of claim 5 where the cache line permutation is done entirely by hardware. 8) the system of claim 5 where the cache line permutation is partially done by software, partially done by hardware. 20 3042049 9) the system of claim 5 where the burst data pattern correspond natively to the cache line permutation, this being equivalent to having always a cache line permutation when the cache line is loaded from, or stored to, memory. 10) the system of claim 5, 7 and 8, where - several types of cache line permutations are supported, - configuration means enable to specify address segments, and which kind of cache line permutation has to be done when an access is inside one of these segments. 11) the system of claims 1-10 where - the memory chips integrate one or several command registers wider than the memory chip data port, - a command colour scheme is used, a change in command colour being requested for a new command to be valid in the concerned command register. 12) the system of claims 1-11 where the host processor is an system on chip comprising several processor cores. 13) the system of claims 1-12 where the host processor is a system on chip comprising several memory buses. 14) a larger system comprising several system, as defined by the claims 1-13, connected together and belonging to a same cache coherency domain, in order to build a multisocket server. 15) a System of claims 1-14 where each given memory chips is enlarged, both in memory capacity and in data port width, in order to allow the transfer, during the totality of a burst transaction, of an ECC code covering the totality of the non enlarged data transferred during this burst transaction from, or to, this given memory chip. 16) a System of claims 1-14 where - each given memory chips is enlarged in memory capacity in order to be able to store ECC, - burst transactions are lengthened allowing the transfer of an ECC code during a burst transaction, - such that the transferred ECC code protect the totality of the non ECC data transferred, from, or to, this given memory chip during a burst transaction. 17) The system of claims 1-16 where the data processor is not integrated into the memory chips, but into one or several additional circuits, placed between the host processor and the memory chips. 18) A system of claims 1-8/10-17 where the address conversion and/or cache line permutation is/are partially or entirely done inside one or several additional circuits placed between the host processor and the memory chips. 19) A computing device comprising: a first processor; 21 3042049 a plurality of memory devices, at least a first of the memory devices integrating one or more further processors; a data bus coupling the first processor to each of the memory devices, wherein each of the memory devices has a data port m bits wide and the data bus has a width of n bits, where n is larger than m, the first processor being adapted to provide a first address for an n-bit data value to be read or written via the data bus, wherein the first processor and/or a further circuit is adapted read or write the n-bit data value to or from the first memory device by: - converting the first address into a plurality of addresses corresponding to m-bit memory locations in the first memory device, at least two of the plurality of addresses being contiguous addresses; and - performing the read or write operation of the n-bit data value to the first memory device over a plurality memory access operations. 22