Accessing memory

A disclosed example method involves performing simultaneous data accesses on at least first and second independently selectable logical sub-ranks to access first data via a wide internal data bus in a memory device. The memory device includes a translation buffer chip, memory chips in independently selectable logical sub-ranks, a narrow external data bus to connect the translation buffer chip to a memory controller, and the wide internal data bus between the translation buffer chip and the memory chips. A data access is performed on only the first independently selectable logical sub-rank to access second data via the wide internal data bus. The example method also involves locating a first portion of the first data, a second portion of the first data, and the second data on the narrow external data bus during separate data transfers.

BACKGROUND

Increases in device scaling and emerging chip-multi processor (CMP) architectures demand greater throughput, power consumption, and reliability from memory systems. Newer generations of dynamic random access memory (DRAM) are designed to provide higher throughput by employing n-bit prefetch and burst access capabilities combined with high-speed signaling techniques. As DRAM channel frequencies increase, adding more ranks or modules deteriorates signal integrity, which limits total memory capacity. A technique for overcoming slow memory access times involves using bank-level parallelism in which multiple memory accesses are issued to different banks of a DRAM to hide DRAM latency. As DRAM dock frequencies increase, switching ranks results in idle cycles on data buses, which introduces time delays between data outputs. This leads to undesirable bus utilization performance of DRAM data buses.

DETAILED DESCRIPTION

Example methods, apparatus, and articles of manufacture disclosed herein may be used to access memories. The disclosed examples may be used to implement a wide internal data bus in a dynamic random access memory (DRAM) module (or on a main printed circuit board (PCB) having DRAM chips) to access memory chips or logical memory ranks of the DRAM module, and a relatively narrower external data bus for exchanging data between the DRAM module and external devices (e.g., memory controllers). The disclosed examples are useful to increase DRAM module capacities while improving performance, power, and reliability. In the disclosed examples, the wide internal data bus is operable at a relatively slower speed than the narrower external data bus, enabling the use of low-power, low-frequency, and low cost DRAM chips on a memory module while supporting external data access speeds of high-performance DRAM interfaces for external devices in communication with the memory module. To support such high-performance DRAM interfaces using lower-performance DRAM chips, examples disclosed herein use memory interface translation techniques to enable retrieving data from the DRAM chips on the wide internal data bus using low-frequency interface standards and supplying that same data on the narrower external data bus using higher-frequency memory interface standards. Although some specific example memory interface standards are described herein for internal and external data buses, the disclosed examples are not limited for use with such specific memory interface standards. Instead, the disclosed examples may be adapted for use with other memory interface standards operating at different frequencies and/or offering different data access features.

FIG. 1is an example memory module100having a wide internal data bus102to access multiple physical memory ranks104, a relatively narrower external data bus106, and a translation buffer chip108. In the illustrated example, the narrow external data bus106is (W) bits wide and operates at a frequency of (f) Hertz (Hz), providing data access speeds of Wf bits/sec for interfacing with the memory module100. The wide internal data bus102is (N)×(W) bits wide and operates at a frequency of (f)/(N) Hz, where (N) is the quantity of the physical memory ranks104per logical rank located on the memory module100. In this manner, the wide internal data bus102provides the same bandwidth as the narrow external data bus106. By using lower frequencies on the wide internal data bus102, the DRAM chips in the physical memory ranks104can be low-power, low-frequency, and low-cost memory chips, while the memory module100can operate as a high-performance memory with high-speed access speeds at the narrow external data bus106. In some examples, the physical memory ranks104may form a logical memory rank, and other physical memory ranks (not shown) of the memory module100may form one or more other logical memory ranks.

In the illustrated example ofFIG. 1, internal narrower data buses110form different portions of the wide internal data bus102such that the width (e.g., bit length) of the wide internal data bus102is equal to the sum of the widths of all the internal narrower data buses110. Each of the internal narrower data buses110of the illustrated example is the width of the narrow external data bus106. Accessing data on the wide internal data bus102involves fetching data on one or more of the multiple internal narrower data buses110simultaneously from corresponding ones of the physical memory ranks104. When data is fetched on the wide internal data bus102from four internal narrower data buses110simultaneously, the data is accessed on the narrow external data bus106as four consecutive data output cycles because the amount of data fetched on the wide internal data bus102is four times the width of the narrow external data bus106. In such examples, the wide internal data bus102can operate four times slower than the narrow external data bus106.

In the illustrated example, a physical memory rank (e.g., one of the physical memory ranks104) is a memory area that is accessed using one of the internal narrower data buses110. A single physical memory rank104may be formed by one or more memory chips. For example, if each of the internal narrower data buses110is thirty-two bits wide, each physical memory rank104can be a single 32-bit wide memory chip, two 16-bit wide memory chips, or four 8-bit wide memory chips.

The translation buffer chip108of the illustrated example translates data exchanges between the wide internal data bus102and the narrow external data bus106so that slower data accesses on the wide internal data bus102can be used to provide high-speed data accesses on the narrow external data bus106. In the illustrated example, the translation buffer chip108is in communication with an example memory controller112. In examples disclosed herein, the memory controller112may be configured to interface with the memory module100using a high-performance DRAM interface (e.g., dual data rate, version 3, (DDR3) DRAM operating at 1600 MHz) even though the physical memory ranks104are implemented using low-power chips having low-speed interfaces (e.g., mobile DRAM such as low-power DDR2 (LPDDR2) DRAM operating at 400 MHz).

In the illustrated example, the translation buffer chip108and memory chips forming the physical memory ranks104are located on a DRAM dual inline memory module (DIMM). In other examples, the translation buffer chip108and the memory chips forming the physical memory ranks104may be arranged in a three-dimensional (3D) stack chip, or may be arranged on a main processor board.

FIG. 2shows internal address buses202a-b(iABUS0and iABUS1) and internal data buses204a-b(iDBUS A and iDBUS B) connected to logical memory ranks of an example memory module200. The example memory module200is configured using an N2 architecture, meaning that a logical memory rank includes two physical memory ranks (i.e., N=2 physical ranks). In the illustrated example ofFIG. 2, a logical rank206a(logical rank 0) includes two physical ranks205a(physical rank A) and205b(physical rank B), and a logical rank206b(logical rank 1) includes two physical ranks207a(physical rank A) and207b(physical rank B). The physical ranks A205aand207ashare the internal data bus204a(iDBUSA), and the physical ranks B205band07bshare the internal data bus204b(iDBUSB). The memory module200of the illustrated example includes a translation buffer208in communication with the logical ranks206a-bthrough the internal address buses202a-band the internal data buses204a-b. The translation buffer208communicatively couples the memory module200with an example memory controller210through an external address bus212and a narrow external data bus214.

The internal address bus202a(iABUS0) of the illustrated example controls the first logical rank206a(logical rank 0) independent of the second logical rank206b(logical rank 1), and the internal address bus202b(iABUS1) of the illustrated example controls the second logical rank206b(logical rank 1) independent of the first logical rank206a(logical rank 0). This provides a higher internal address bus bandwidth by being able to control the logical ranks206a-bindependent of one another, and also reduces electrical load on the internal address buses202a-b.

In the Illustrated example, the internal data buses204a-bare combined to form a wide internal data bus216. The width of each internal data bus204a-bis equal to the width of the narrow external data bus214. As such, the width of the wide internal data bus216is twice the width of the narrow external data bus214. In examples having more physical ranks per logical rank, the width of the wide internal data bus is more than twice the width of the narrow external data bus214. In the illustrated example, a single data access on the wide internal data bus216from the logical ranks206a-bsimultaneously locates a first portion of the data on the internal data bus204aand a second portion of the data on the internal data bus204b. The single data access on the wide internal data bus216involves two data accesses on the narrow external data bus214. In this manner, the logical ranks206a-band the wide internal data bus216may operate at half the frequency of the narrow external data bus214. Memory accesses between the memory controller210and the memory module200can be performed using a relatively higher performance memory standard (e.g., the narrow external data bus214at 1600 MHz, and the external address bus212at 800 MHz), while memory accesses internal to the memory module200can be performed using a relatively lower performance memory standard (e.g., the internal data buses204a-bat 800 MHz, and the internal address buses202a-bat 400 MHz). This enables constructing the memory module200using low-frequency, low-power, low-cost memory, while providing a high-performance memory interface to the memory module200.

In the illustrated example, the internal data buses204a-b(iDBUS A and iDBUS B) ofFIG. 2may be used to implement the internal narrow data buses110ofFIG. 1, one or both of the logical ranks206a-bofFIG. 2may be used to implement one or more logical ranks that include the physical ranks104ofFIG. 1and/or other physical ranks not shown inFIG. 1, the translation buffer208ofFIG. 2may be used to implement the translation buffer chip108ofFIG. 1, the wide internal data bus216ofFIG. 2may be used to implement the wide internal data bus102ofFIG. 1, the narrow external data bus214ofFIG. 2may be used to implement the narrow external data bus106ofFIG. 1, and the memory controller210ofFIG. 2may be used to implement the memory controller112ofFIG. 1.

Although the example memory module200is shown as an N2 architecture, configurations with more physical ranks per logical rank may be implemented using additional separate internal address and data buses. For example, an N4 architecture may be implemented using four internal address buses and four internal data buses in which the internal memory chips operate at one-fourth the frequency of the external data bus (e.g., the narrow external data bus214). An N8 architecture may be implemented using eight internal address buses and eight internal data buses in which the internal memory chips operate at one-eighth the frequency of the external data bus ((e.g., the narrow external data bus214).

FIG. 3is an example memory module300having logical memory ranks302a-b(logical rank 0 and logical rank 1) with independently selectable logical sub-ranks304a-b(logical sub-ranks 0 and 1 of logical rank 0) and independently selectable logical sub-ranks306a-b(logical sub-ranks 0 and 1 of logical rank 1). Internal data buses308a(iDBUS A),308b(iDBUS B),308c(iDBUS C), and308d(IDBUS D) are shown connected to the logical ranks302a-band sub-ranks304a-band306a-b. In the illustrated example, each of the internal data buses308a-dis a portion of a wide internal data bus (e.g., the wide internal data bus102ofFIG. 1) such that the width of the wide internal data bus is equal to the sum of widths of all the internal data buses308a-d. Although not shown, the example memory module300also includes four internal address buses (iABUSes) routed to each logical sub-rank304a-band306a-b. The internal data buses308a-band internal address buses (not shown) are connected between the logical ranks302a-band a translation buffer (e.g., the translation buffer chip108ofFIG. 1). The memory module300of the illustrated example may be used to implement the memory module100ofFIG. 1such that one or more of the logical ranks302a-bofFIG. 3may be used to implement one or more logical ranks that include one or more of the physical ranks104ofFIG. 1and/or other physical ranks not shown inFIG. 1, and the internal data buses308a-dofFIG. 3may be used to implement the internal data buses110ofFIG. 1.

The independent selectability of the logical sub-ranks304a-band306a-bdecreases the access granularity of the wide internal data bus formed by the internal data buses308a-dand decreases activate/precharge power. For example, without increasing access granularity as enabled by the memory module300ofFIG. 3, memory access requests to N4 architectures are served using 4×64-bit words×burst 8 words=128 bytes (B) (this number increases for N8 architectures), and an activate command fetches four times more bits to row buffers. However, unless a memory controller (e.g., the memory controller112ofFIG. 1) actually accesses all of the activated bits, the memory module300has wasted power and energy in accessing the activated but unused bits.

To avoid wasting of power and energy due to accessed but unused bits, the independent selectability of the logical sub-ranks304a-band306a-bin the memory module300of the illustrated example enables accessing only a part of the larger logical ranks302a-bso that only the portion of bits that are desired from a row buffer are retrieved from the logical ranks302a-b. Thus, in some examples, the memory module300may be accessed to retrieve a large width of data from a large row-buffer as shown inFIG. 3as an access to both logical sub-ranks306a-bof the second logical rank302b(i.e., an access to all of the memory chips in the second logical rank302b). In such examples, different portions of the data are simultaneously located on different ones of the internal data buses308a-d. In other examples, the memory module300may be accessed to retrieve a smaller width of data from a small row-buffer as shown inFIG. 3as an access to only the first logical sub-rank304aof the first logical rank302a(i.e., an access to only the memory chip(s) in the first logical sub-rank304aof the first logical rank302a) without accessing the second logical sub-rank304bof the first logical rank302a. In such examples, different portions of the data are simultaneously located on the internal data buses308a-bwithout simultaneously locating any data on the internal data buses308c-dduring the same data access. In the illustrated example, a small row-buffer may also be accessed by accessing only the first logical sub-rank306aof the second logical rank302bwithout accessing the second logical sub-rank306bof the second logical rank302b. Using this logical sub-rank selectability, the active width of the wide internal data bus of the memory module300is dynamic so that in some accesses the wide internal data bus has an active width equal to a small row-buffer of a single logical sub-rank (e.g., a single one of the logical sub-ranks304a-band306a-b), and in some accesses the wide internal data bus has a relatively larger active width equal to a large-row buffer of an entire logical rank (e.g., the entire logical rank302aor302b).

Although the memory module300is shown as an N4 architecture memory, architectures with higher physical ranks per logical rank (e.g., N8, N16, etc.) may be similarly implemented having independently selectable logical sub-ranks. In such architectures, more width-size options of the wide internal data buses may be selected. For example, a small row-buffer access may access a single logical sub-rank, a medium row-buffer access may access two logical sub-ranks, and a large row-buffer access may access four logical sub-ranks.

FIG. 4is an example translation buffer400that may be used with the example memory modules100and200ofFIGS. 1 and 2to exchange data between an internal interface401having a wide internal data bus410(e.g., the wide internal data bus102ofFIG. 1 or 216ofFIG. 2) and an external interface402having a narrow external data bus414(e.g., the narrow external data bus106ofFIG. 1 or 214ofFIG. 2). The translation buffer400of the illustrated example may be used to implement the translation buffer chip108ofFIG. 1and/or the translation buffer208ofFIG. 2. In the illustrated example, the translation buffer400is located between a memory controller (MC)403and memory chips404. In some examples, the translation buffer400and the memory chips404are located on a memory module or memory device such as a DIMM or a 3D chip stack. In other examples, the translation buffer400and the memory chips404are co-located on a processor main board with the memory controller403. The memory controller403may be the memory controller112ofFIG. 1 and/or 210ofFIG. 2, and the memory chips404may implement the logical ranks104ofFIG. 1 and/or 206a-bofFIG. 2.

The example translation buffer400is shown having an N2 architecture for which the internal interface401has a first internal address bus406a(iABUS0) for a first logical memory rank (e.g., the first logical memory rank206aofFIG. 2), a second internal address bus406b(iABUS1) for a second logical memory rank (e.g., the second logical memory rank206bofFIG. 2), a first internal data bus408a(iDBUSA) for accessing the first physical memory ranks of logical ranks, and a second internal data bus408b(iDBUSB) for accessing the second physical memory ranks of logical ranks. The internal data buses408a-bare used in combination to form respective portions of the wide internal data bus410(e.g., similar or identical to the wide internal data bus102ofFIG. 1and/or the wide internal data bus216ofFIG. 2). The external interface402of the example translation buffer400includes an external address bus412(ABUS) and a narrow external data bus414(DBUS) (e.g., similar or identical to the wide external data bus106ofFIG. 1 and/or 214ofFIG. 2). The internal address buses406a-band the internal data buses408a-bare provided to communicate with the memory chips404, and the external address bus412and the narrow external data bus414are provided to communicate with the memory controller403.

In the N2 architecture ofFIG. 4, the narrow external data bus414operates at a frequency (f), the external address bus412operates at one-half of the frequency (f/2), the internal address buses406a-boperate at one-quarter of the frequency (f/4), and the wide internal data bus410operates at one-half of the frequency (f/2). As such, the narrow external data bus414is relatively faster (twice as fast in the illustrated example) than the wide internal data bus410.

To split the external address bus412into the two internal address buses406a-b, the translation buffer400is provided with a data latch (e.g., flip-flops)418and a translator420corresponding to the first internal address bus406a, and a data latch (e.g., flip-flops)422and a translator424corresponding to the second internal address bus406b. In the illustrated example, the external address bus412provides the inputs to the data latches418and422, and the translators420and424provide outputs to the internal address buses406a-b. The translators420and424of the illustrated example provide address command translation logic to convert addresses and command line signals received from the memory controller403on the external address bus412into corresponding addresses and command line signals for the internal address buses406a-bto access corresponding logical ranks (e.g., the logical memory ranks206a-bofFIG. 2and/or logical rank(s) that include(s) the physical ranks104ofFIG. 1) in the memory chips404.

To interface the narrow external data bus414with the wide internal data bus410, the translation buffer400is provided with a data input latch (e.g., flip-flops)426to transfer input data from the narrow external data bus414to the first internal data bus408a, a data input latch (e.g., flip-flops)428to transfer input data from the narrow external data bus414to the second internal data bus408b, a data output latch (e.g., flip-flops)432to output data from the internal data bus408ato the narrow external data bus414, and a data output latch (e.g., flip-flops)434to output data from the internal data bus408bto the narrow external data bus414.

Each of the first and second internal data buses408a-bare the same data width as the narrow external data bus414of the illustrated example. The data input latches426and428coordinate which data from the narrow external data bus414should be output to which of the internal data buses408a. In the illustrated example, the translators420and424analyze addresses and control signals from the external address bus412and control the latches426and428to latch corresponding data words received from the memory controller403on the narrow external data bus414. For example, the translators420and424may determine that data on the narrow external data bus414is to be latched in the latch426four outputting on the first internal data bus408aof the wide internal data bus410, and that data appearing immediately next on the narrow external data bus414is to be latched in the latch428for outputting on the second internal data bus408bof the wide internal data bus410. When the data is latched in corresponding ones of the latches426and428, the translators420and424may cause the latches426and428to output their respective data on corresponding ones of the internal data buses408a-bof the wide internal data bus410. In this manner, the translators420and424can control the latches426and428to write data to corresponding logical memory ranks (e.g., the logical memory ranks206a-bofFIG. 2and/or logical rank(s) that include(s) the physical ranks104ofFIG. 1).

In the illustrated example, a multiplexer436is coupled between the data output latches432and434and the narrow external data bus414to multiplex data from the wide internal data bus410to the narrow external data bus414. Because the wide internal data bus410is twice the width of the narrow external data bus414in the illustrated example, data read from the memory chips404on the wide internal data bus410during a single access is output by the multiplexer436on the narrow external data bus414as two data accesses. In this manner, the wide internal data bus410can operate half as fast as the narrow external data bus414. The translators420and424of the illustrated example analyze address and control signals on the external address bus412to control when the latches432and434are to latch data from the wide internal data bus410and how the multiplexer436arranges data from the latches432and434for outputting on the narrow external data bus414.

Although the translation buffer400is shown for use with an N2 architecture memory module, the translation buffer400can be extended for use with larger N-based architectures (e.g., N8, N16, etc.) by adding additional data input and data output latches similar to the latches426,428,432, and434, and corresponding internal data buses similar to the internal data buses408aand408bto form additional portions of the wide internal data bus410.

The latches418,422,426,428,432, and434, the translators420and424, and the multiplexer436of the translation buffer400enable the external address bus412and the narrow external data bus414to operate using a memory interface standard between the translation buffer400and the memory controller403that is different from another memory interface standard used for the internal address buses406a-band the wide internal data bus410between the translation buffer400and the memory chips404. The translators420and424of the illustrated example generate internal data access timings useable to perform data access on the wide internal data bus410between the translation buffer400and the memory chips404, and external data access timings useable to perform data accesses on the narrow external data bus414between the memory controller403and the translation buffer400.

Table 1 below shows example relatively fast memory interface standards for the external interface402, and slower memory interface standards for the internal interface401based on different N-based architectures. As such, the external data access timings of Table 1 correspond to relatively faster memory access standards for operating the external interface402at relatively higher frequencies, and internal data access timings correspond to relatively slower memory access standards for operating the internal interface401at relatively slower frequencies. The memory interface standards shown in Table 1 are examples only, and examples disclosed herein are not limited to such memory interface standards and/or operating frequencies.

The example memory interface standards of Table 1 use different data access timings for the internal interface401and the external interface402of the translation buffer400. As shown in Table 1, in some examples, the external interface402can operate with external data access timings of a graphics double data rate (GDDR) DRAM (e.g., a GDDR, version five, (GDDR5) DRAM operating at four gigahertz (4 GHz)), while the internal interface401operates with internal data access timings for the internal interface401corresponding to one of a double data rate (DDR) DRAM (e.g., a DDR, version three, (DDR3) DRAM operating at one gigahertz (1 GHz)) in, for example, an N4 architecture or a low power double data rate (LPDDR) DRAM (e.g., a LPDDR, version two, (LPDDR2) DRAM operating at five hundred megahertz (500 MHz)) in, for example, an N8 architecture. As also shown in Table 1, in some examples, the external interface402can operate with external data access timings of a double data rate (DDR) DRAM (e.g., a DDR, version three, (DDR3) DRAM operating at one thousand six hundred megahertz (1.6 GHz)), and the internal interface401can operate with internal data access timings corresponding to one of a double data rate (DDR) DRAM (e.g., a DDR, version three, (DDR3) DRAM operating at eight hundred megahertz (800 MHz)); a low power double data rate (LPDDR) (e.g., a LPDDR, version two, (LPDDR2) DRAM operating at four hundred megahertz (400 MHz)); or a non-volatile memory such as, for example, a phase-change random access memory (PCRAM) (e.g., operating at four hundred megahertz (400 MHz)), a spin-torque transfer random access memory (STTRAM), or a memristor memory. As also shown in Table 1, in some examples, the external interface402can operate with external data access timings of a low power double data rate, version two, (LPDDR2) DRAM (e.g., operating at eight hundred megahertz (800 MHz)), and the internal interface401can operate at internal data access timings corresponding to a lower power double data rate (LPDDR) DRAM (e.g., operating at two hundred megahertz (200 MHz)). As also shown in Table 1, in some examples, the external interface402can operate with external data access timings of a custom interface (e.g., a non-industry-standard memory interface) operating at a first frequency (e.g., twenty gigahertz (20 GHz)), and the internal interface401can operate with internal data access timings corresponding to an industry-standard memory interface such as an extreme data rate (XDR) dynamic random access memory operating at a second frequency (e.g., five gigahertz (5 GHz)) that is relatively slower than the first frequency of the external interface402.

Different memory interface standards have different address/command formats. For example, some standards use 1-cycle command transfers, while other standards use 2-cycle command transfers. The translators420and424enable a seamless interface between different memory technologies without needing to re-design or change memory controllers. In some examples, different translation buffers like the translation buffer400ofFIG. 4enable implementing a high-performance memory system (e.g., a GDDR5 memory system) using different memory modules (e.g., the memory modules100,200, and300ofFIGS. 1-3) having different lower-performance memory chips that operate at different memory interface standards. For example, a translation buffer of a first memory module may translate data exchanges between LPDDR2 memory chips and the GDDR5 interface of the high-performance memory system, while another translation buffer of a second memory module may translate data exchanges between DDR3 memory chips and the GDDR5 interface of the high-performance memory system.

In the illustrated example ofFIG. 4, transferring a data block on the wide internal data bus410and the narrow external data bus414uses the same amount of time, because although the wide internal data bus410is wider and retrieves more data per data access, it operates at a lower frequency than the narrow external data bus414. The higher operating frequency of the narrow external data bus414enables it to sequentially output a first portion of data from the first internal data bus408aof the wide internal data bus410and a second portion of the data from the second internal data bus408bof the wide internal data bus410in the same amount of time used by the wide internal data bus410to retrieve the data from the memory chips404. To illustrate this point, an example timing diagram500ofFIG. 5shows example timings of data transfers between the internal interface401(FIG. 4) and the external interface402(FIG. 4) using the example translation buffer400ofFIG. 4.

In the illustrated example ofFIG. 5, the translation buffer400receives a burst 16 read request502on the external address bus412(ABUS). The translation buffer400converts the burst 16 read request502into two burst 8 read requests504for outputting on the internal address buses406aand406bto the memory chips404. In the illustrated example, the memory chips404return a first data block of 64 bytes (B)506on the first internal data bus408acorresponding to the burst 8 read request504on the first internal address bus406a, and simultaneously return a second data block of 64 bytes (B)508on the second internal data bus408bcorresponding to the burst 8 read request504on the second internal address bus406b. For example, if the memory chips404are implemented using the logical memory ranks206aand206bofFIG. 2, the first logical rank206areturns the first data block of 64 bytes on the first internal data bus408a, and the second logical rank206breturns the second data block of 64 bytes on the second internal data bus408b. In combination, the first and second data blocks506and508form respective portions of the total amount of data retrieved simultaneously on the wide internal data bus410ofFIG. 4. As shown inFIG. 5, the translation buffer400generates a data output510of the first and second data blocks506and508on the narrow external data bus414the uses the same amount of time used to retrieve the first and second data blocks506on the wide internal data bus410.

FIG. 6is another example translation buffer600that may be used with the example memory modules100and300ofFIGS. 1 and 3to enable data transfers with independently selectable logical sub-rank memory module configurations. Portions of the example translation buffer600are similar to the example translation buffer400ofFIG. 4, and like reference numerals inFIG. 6refer to the same or similar elements ofFIG. 4. The example translation buffer600differs from the example translation buffer400in that it includes example input synchronization queues602and604, and example output synchronization queues606and608, which replace the latches426,428,432, and434ofFIG. 4. Unlike the latches426,428,432, and434ofFIG. 4, the synchronization queues602,604,606, and608are used in the translation buffer600to better handle transfers of data blocks when there are different timings between different data blocks. Such different timings can occur in an independently selectable logical sub-rank memory module such as the example memory module300ofFIG. 3.

In the illustrated example, the internal data bus408amay be coupled to the internal data bus308a(FIG. 3), and the internal data bus408bmay be coupled to the internal data bus308b(FIG. 3). Although not shown inFIG. 6, two additional input synchronization queues (similar to the input synchronization queues602and604), two additional output synchronization queues (similar to the output synchronization queues606and608), and two additional corresponding internal data buses may be added to the translation buffer600to interface with the internal data buses308cand308dofFIG. 3. In this manner, the translation buffer600is useable with the N4 architecture memory module300ofFIG. 3. Support for higher N-based architectures (e.g., N8, N16, etc.) may also be provided by adding additional synchronization queues and data input buffers. In addition, the translation buffer600may be used to translate between the different example memory interface standards shown in Table 1 above, or any other memory interface standards.

The synchronization queues602,604,606, and608synchronize input data and output data that is subjected to different data access timings in the memory chips404due to different selections of logical sub-ranks (e.g., the logical sub-ranks304a-band306a-bofFIG. 3). That is, data access timings are typically different when a single logical sub-rank is selected for a data access to the memory chips404compared to when two or more logical sub-ranks are selected for a data access to the memory chips404. Although independently selecting different logical sub-ranks introduces different data access timings between data transfers, such independent selection is useful to reduce wasted bit retrievals. That is, when multiple logical sub-ranks are automatically accessed for each data access, bits retrieved from some logical sub-ranks may not be needed but are retrieved anyway as being part of the same accessed logical rank containing desired data. Such unneeded bits are discarded after retrieval. By independently selecting only logical sub-ranks having desired data, unneeded bits are not retrieved and, thus, data access efficiency is increased.

The synchronization queues602,604,606, and608substantially decrease or eliminate the differences in access timings from propagating through from the wide internal data bus410to the narrow external data bus414by queuing and synchronizing data between the data buses410and414before completing the data transfers from the wide internal data bus410to the narrow external data bus414or from the narrow external data bus414to the wide internal data bus410. An example timing diagram700ofFIG. 7shows data transfer timings between internal data buses308a-c(FIG. 3) in communication with the wide internal data bus410ofFIG. 6, and the narrow external data bus414ofFIG. 6. In the illustrated example, the translation buffer600receives a burst 16 read request702on the external address bus412, and the translators420and424convert the burst 16 read request702into two burst 8 read requests704for outputting on the internal address buses406a-b. The selected logical sub-rank304aor306aofFIG. 3responds by transferring two burst 8 data blocks706aand706bof 64 bytes each on the internal data buses308aand308b. The output synchronization queues606and608ofFIG. 6simultaneously receive and buffer the data blocks706a-buntil it is time to output the data on the narrow external data bus414. For example, the output synchronization queue606buffers the first data block706a, and the output synchronization queue608buffers the second data block706b. Subsequently, the translation buffer600receives another burst 16 read request708, and the translators420and424convert the burst 16 read request708into two burst 8 read requests710for outputting on the internal address buses406a-b. The subsequently selected logical sub-rank304bor306bresponds by transferring two burst 8 data blocks712aand712bof 64 bytes each on the internal data buses308cand308d. Output synchronization queues (not shown inFIG. 6) that correspond to the internal data buses308cand308dsimultaneously receive and buffer the data blocks712a-buntil it is time to output the data on the narrow external data bus414. As shown inFIG. 7, while the data blocks706a-band712a-bare being received at the translation buffer600and queued in their respective synchronization queues, the multiplexer436begins outputting the data blocks706a-bon the narrow external data bus414as a burst 16 data transfer714by locating the first data block706aon the narrow external data bus414at a first time and subsequently locating the second data block706bon the narrow external data bus414at a second time different from the first time. Similarly, while synchronization queues continue to buffer the data blocks712a-bin the translation buffer600, and when output of the burst 16 data block714is complete, the multiplexer436outputs the data blocks712a-bfrom their respective synchronization queues on the narrow external data bus414as a burst 16 data block716in a similar manner without any delay between the burst 16 data block714and the burst 16 data block716.

While example manners of implementing the translation buffers400and600have been illustrated inFIGS. 4 and 6, one or more of the elements, processes and/or devices illustrated inFIGS. 4 and/or 6may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example translators420and424and/or, more generally, the example translation buffers400and600ofFIGS. 4 and 6may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example translators420and424and/or, more generally, the example translation buffers400and600could be implemented by one or more circuit(s), programmable processor(s), application specific integrated circuit(s) (ASIC(s)), programmable logic device(s) (PLD(s)) and/or field programmable logic device(s) (FPLD(s)), etc. When any of the apparatus or system claims of this patent are read to cover a purely software and/or firmware implementation, at least one of the example translators420or424is hereby expressly defined to include a tangible computer readable storage medium such as a solid state memory storing the software and/or firmware. Further still, the example translation buffers400and600ofFIGS. 4 and 6may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated inFIGS. 4 and 6, and/or may include more than one of any or all of the illustrated elements, processes and devices.

FIG. 8is an example timing diagram800for an N2 architecture showing time delay bubbles802a-bthat occur on a wide internal data bus (e.g., the wide internal data buses102ofFIG. 1, 216ofFIG. 2, and 410ofFIGS. 4 and 6) of a memory module (e.g., the memory modules100,200, and300ofFIGS. 1-3) during burst accesses, but do not propagate to an external data bus (e.g., the external data buses106ofFIG. 1, 214ofFIG. 2, and 414ofFIGS. 4 and 6) of the memory module. In the illustrated example, the time delay bubbles802a-boccur during a burst chop 4 (BC4) transfer mode in the DDR3 memory interface standard. The BC4 transfer mode enables transferring data using burst 4 accesses. However, this makes the DRAM chips inaccessible for a certain amount of time, which in the illustrated example is the same amount of time required to transfer a burst 4 data block. In the illustrated example, the time delay bubble802aoccurs between retrievals of first data804and second data806from a first logical memory rank (e.g., the logical rank206aofFIG. 2 and/or 302ofFIG. 3), and the time delay bubble802aoccurs between retrievals of third data808and fourth data810from a second logical memory rank (e.g., the logical ranks206bofFIG. 2 and/or 302bofFIG. 3). In some examples, the first data804and the third data808may be retrieved from the physical rank A205aofFIG. 2, and the second data806and the fourth data810may be retrieved from the physical rank B205bofFIG. 2. In the illustrated example ofFIG. 8, the time delay bubbles802a-blead to undesirable bus utilization performance. However, using examples disclosed herein, a translation buffer such as the translation buffers400and600ofFIGS. 4 and 6can be used to prevent propagating the time delay bubbles802a-bfrom a wide internal data bus804to a narrow external data bus806.

In the illustrated example ofFIG. 8, when the sum of a time delay bubble802aand the time to perform a BC4 transfer on the wide internal data bus804matches the external transfer time for a corresponding burst 8 transfer on the narrow external data bus806, examples disclosed herein may be used to hide or prevent propagating the time delay bubbles802a-bto the narrow external data bus806by buffering output data812and814in data output synchronization queues (e.g., the synchronization queues602,604,606,608ofFIG. 6) as described above in connection withFIG. 6, and/or by latching the output data812and814in data output latches as described above in connection withFIG. 4. In this matter, the output data812(which includes the data804and808) is output on narrow external data bus806immediately followed by the output data814(which includes the data806and810) without any time delay between the output data812and the output data814, as shown in the illustrated example ofFIG. 8.

In the illustrated example, the bubble penalty is tBURST/2 when accessing the same logical memory rank, but it is tRTRS when switching between logical memory ranks every BC4 access. Typically, tBURST/2 is four DRAM clock cycles in DDR3, while tRTRS is two DRAM clock cycles in some systems (e.g., tRTRS is not a fixed parameter and can be different in different designs). Thus, if the frequency (f) (e.g., DRAM frequency) of the wide internal bus is greater than or equal to (4+2)/8×f in a memory module with internal BC4 access capabilities, the memory module provides external burst 8 access capabilities, and tRTRS is 2 cycles for the internal data bus.

FIG. 9is an example storage configuration for storing data902a-band error correcting codes (ECCs)904in a memory module (e.g., the memory modules100,200, and300ofFIGS. 1-3).FIG. 10is an example data transfer configuration1000to transfer the data902a-band the corresponding ECCs904between the translation buffers400and600ofFIGS. 4 and 6and a memory controller (e.g., the memory controllers112ofFIG. 1, 210ofFIG. 2, or403ofFIGS. 4 and 6).

The example configurations ofFIGS. 9 and 10may be used to implement chipkill-correct, which is a memory protection mechanism that can tolerate a chip failure. Typically, chipkill-correct requires 128-bit wide channels. Therefore, traditional memory controllers use two 64-bit channels in lock-step mode.

The wide internal data bus architectures disclosed herein enable chipkill-correct level memory protection with a single DIMM including a narrow external data bus (e.g., the narrow external data buses106ofFIG. 1, 214ofFIG. 2, and 414ofFIGS. 4 and 6) having a width of 64 bits. For example, an N2 architecture stores the data902a-band ECCs904as shown inFIG. 9. The illustrated example ofFIGS. 9 and 10assume internal burst 4 access capabilities and 8-bit DRAM chips, for which the notation Dx represents 8-bit data, and the notation Ey represents the ECCs904. In some examples, the data902amay be stored in a separate logical memory rank from the data902b, and the ECCs904may be stored in yet another separate logical memory rank.

In the illustrated example ofFIG. 10, the translation buffer (e.g., the translation buffer400ofFIG. 4 and/or 600ofFIG. 6) is configured to be aware of internal DRAM and channel structures of example memory modules as disclosed herein. The translation buffer can then re-organize the data blocks from the internal channels as shown inFIG. 9to the data layouts as shown inFIG. 10to apply chipkill-correct level ECC.

In some examples, the first translator420(FIGS. 4 and 6) causes retrieval of a first chipkill ECC (e.g., E0of the ECCs904) corresponding to a first data block906of the data902a, the second translator424(FIGS. 4 and 6) causes retrieval of a second chipkill ECC (e.g., E1of the ECCs904) corresponding to a second data block908of the data902b. The first output data synchronization queue606(FIG. 6) stores the retrieved first data block906with the first chipkill ECC (E0) (denoted by reference numeral1002inFIG. 10) to output the first data block906and the first chipkill ECC (E0) simultaneously on the narrow external data bus414at a first time. In addition, the second output data synchronization queue608(FIG. 6) stores the second data block908with the second chipkill ECC (E1) (denoted by reference numeral1004inFIG. 10) to output the second data block908and the second chipkill ECC (E1) simultaneously on the narrow external data bus414at a second time different from the first time at which the first data block906and the first chipkill ECC (E0) were output.

FIG. 11is a flow diagram representative of instructions and/or actions which may be performed to access data in memory modules having a wide internal data bus and a narrow external data bus using examples disclosed herein. The flow diagram ofFIG. 11is representative of operations to implement the translation buffers108(FIG. 1),208(FIG. 2), and/or400(FIGS. 4 and 6). In this example, the operations are performed by one or more circuits (e.g. logic and/or analog circuits). In some examples, some of the operations are performed based on computer readable instructions executed by the translators420and424ofFIGS. 4 and 6. Such computer readable instructions may be embodied in firmware or software stored on a tangible computer readable storage medium such as a flash, read only memory (ROM), or DRAM, but such some operations and/or parts thereof could alternatively be executed by a device other than the translators420and/or424and/or embodied in dedicated hardware. Further, although the example operations are described with reference to the flow diagram illustrated inFIG. 11, many other methods of implementing the example translation buffers108,208, and/or400may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.

As mentioned above, example operations ofFIG. 11may be implemented using coded instructions (e.g., computer readable instructions) stored on a tangible computer readable medium such as a flash memory, a read-only memory (ROM), a cache, a random-access memory (RAM) and/or any other storage media in which information is stored for any duration (e.g., for extended time periods, permanently, brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term tangible computer readable medium is expressly defined to include any type of computer readable storage and to exclude propagating signals. Additionally or alternatively, example operations ofFIG. 11may be implemented using coded instructions (e.g., computer readable instructions) stored on a non-transitory computer readable medium such as a flash memory, a read-only memory, a cache, a random-access memory and/or any other storage media in which information is stored for any duration (e.g., for extended time periods, permanently, brief instances, for temporarily buffering, and/or for caching of the information). As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable medium and to exclude propagating signals. As used herein, when the phrase “at least” is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term “comprising” is open ended. Thus, a claim using “at least” as the transition term in its preamble may include elements in addition to those expressly recited in the claim.

InFIG. 11, initially, the translator(s)420and/or424(FIGS. 4 and 6) select(s) logical sub-ranks (block1102). For example, the translator(s)420and/or424may output an address on the internal address bus(es)406aand/or406b(FIGS. 4 and 6) to select or activate the logical sub-ranks304a-bindependent and mutually exclusive of the logical sub-ranks306a-bofFIG. 3based on an address received at the external address bus412(FIGS. 4 and 6). The translation buffer400(or600) simultaneously access first data in the selected logical sub-ranks304a-bon the wide internal data bus410(FIGS. 4 and 6) (block1104). The multiplexer436(FIGS. 4 and 6) locates a first portion of the accessed first data on the narrow external data bus414(FIGS. 4 and 6) at a first time (T1) (block1106). The multiplexer436locates a second portion of the accessed first data on the narrow external data bus414(FIGS. 4 and 6) at a second time (T2) (block1108). In the illustrated example, the first time (T1) is temporally separate from the second time (T2) (but may occur consecutively without an intervening time delay), the width of the first portion of the first data equals the width of the narrow external data bus414, and the width of the second portion of the first data also equals the width of the narrow external data bus414.

The translator(s)420and/or424(FIGS. 4 and 6) select(s) a subset of the logical sub-ranks304a-b(block1110). For example, the translator(s)420and/or424may output an address on the internal address bus406ato select or activate the logical sub-rank304aindependent and mutually exclusive of the logical sub-ranks304band306a-bbased on another address received at the external address bus412. In examples in which the four logical sub-ranks304a-band306a-bor more are selected at block1102, less than all of those logical sub-ranks are selected at block1110. The translation buffer400(or600) access second data in the selected logical sub-rank304aon the wide internal data bus410(block1112). In the illustrated example, the second data is accessed on a smaller portion (e.g., the first internal data bus408aofFIGS. 4 and 6) of the wide internal data bus410at block1112than the larger portion (e.g., the first and second internal data buses408a-b) used to access the first data at block1104. In the illustrated example, the multiplexer436(FIGS. 4 and 6) locates the entire second data on the narrow external data bus414(FIGS. 4 and 6) at a third time (T3) (block1114). In the illustrated example, the second data is the same width as the width of the narrow external data bus414. Therefore, only a single data output is needed to transfer the second data on the narrow external data bus414, whereas two data output cycles were used at blocks1106and1108to output the first data on the narrow external data bus414. In addition, the third time (T3) is temporally separate from the first time (T1) and the second time (T2), but the third time (T3) may occur consecutively following the second time (T2) without an intervening time delay. The example method ofFIG. 11then ends.

The method ofFIG. 11may be used for write accesses and/or read accesses to a memory module. The operations ofFIG. 11are arranged for use in read accesses. For use in write accesses, blocks1106and1108may be performed before blocks1102and1104, and block1114may be performed before blocks1110and1112.

Although the above discloses example methods, apparatus, and articles of manufacture including, among other components, software executed on hardware, it should be noted that such methods, apparatus, and articles of manufacture are merely illustrative and should not be considered as limiting. For example, it is contemplated that any or all of these hardware and software components could be embodied exclusively in hardware, exclusively in software, exclusively in firmware, or in any combination of hardware, software, and/or firmware. Accordingly, while the above describes example methods, apparatus, and articles of manufacture, the examples provided are not the only way to implement such methods, apparatus, and articles of manufacture. Thus, although certain methods, apparatus, and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. To the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the claims either literally or under the doctrine of equivalents.