Abstract:
One embodiment provides a memory system including first dynamic random access memories and a first memory buffer. The first memory buffer is configured to receive southbound data at a first data rate and provide northbound data at a second data rate. The first memory buffer is also configured to read data from the first dynamic random access memories at a third data rate, wherein the first memory buffer is configured to decouple the third data rate from the first data rate and the second data rate.

Description:
BACKGROUND 
       [0001]    Typically, a computer system includes a number of integrated circuits that communicate with one another to perform system applications. Often, the computer system includes one or more host controllers and one or more electronic subsystem assemblies, such as memory modules, a graphics card, an audio card, a facsimile card, and a modem card. 
         [0002]    The memory modules can be dual in-line memory modules (DIMMs) that include random access memory (RAM) chips, such as dynamic RAM (DRAM) chips. The DRAM can be any suitable type of DRAM including double data rate DRAM (DDR-DRAM) and double data rate synchronous DRAM (DDR-SDRAM). Also, the DRAM can be any suitable generation, such as first, second, and third generation DDR-SDRAM. 
         [0003]    To perform system functions, the host controller(s) and subsystem assemblies communicate via communication links, such as serial communication links and parallel communications links. Serial communication links include links that implement the fully buffered DIMM (FB-DIMM) advanced memory buffer (AMB) standard or any other suitable serial communications link interface. 
         [0004]    An AMB chip is a key device in an FB-DIMM. An AMB has two serial links, one for upstream traffic and the other for downstream traffic, and a memory bus to on-board memory, such as DRAM on the FB-DIMM. Serial data from a host controller or AMB sent through the downstream serial link (southbound) is temporarily buffered in an AMB and can then be sent to memory on the FB-DIMM. The southbound data contains the address, data, and command information given to the FB-DIMM, converted in the AMB, and sent to the memory bus. The AMB writes in and reads out data from the memory as instructed by the host controller. The read data is converted to serial data and sent back to the host controller on the upstream serial link (northbound). 
         [0005]    An AMB also performs as a repeater between FB-DIMMs on the same memory channel. The AMB transfers information from a primary southbound link connected to the host controller or an upper AMB to a lower AMB in the next FB-DIMM via a secondary southbound link. The AMB receives information in the lower FB-DIMM from a secondary northbound link, and after merging the information with information of its own, sends it to the upper AMB or host controller via a primary northbound link. This forms a daisy-chain among FB-DIMMs. A key attribute of the FB-DIMM architecture is the high-speed, serial, point-to-point connection between the host controller and FB-DIMMs on the memory channel. 
         [0006]    Typically, in an FB-DIMM system, the controller and the AMBs send southbound data at one data rate and receive northbound data at double the southbound data rate. This leads to a 1:2 write to read ratio, which reflects statistics in typical memory access patterns. On the FB-DIMM, the AMB is coupled to DRAMs via a standard DRAM interface. The DRAM interface consists of a stub-bus for the commands, addresses, and control signals and point-to-point or point-to-multiple points for data. 
         [0007]    Data rates of the controller and the AMBs are coupled to the DRAM data rates. The northbound data rates are matched to the data rate of the DRAM interface. Further increases in data bandwidth can be obtained by increasing the bandwidth of all connections by the same amount. In this architecture, a faster controller and faster AMBs does not lead to a higher bandwidth, unless a higher bandwidth DRAM is available. 
         [0008]    Although this architecture allocates enough bandwidth for sending up to three commands in each southbound frame, due to bandwidth matching of the northbound data rate and the DRAM interface it is only possible to read one FB-DIMM at a time. In systems having many FB-DIMMs per memory channel, several FB-DIMMs remain idle or in the best case only receive southbound data. 
         [0009]    For these and other reasons there is a need for the present invention. 
       SUMMARY 
       [0010]    The present disclosure describes a memory system that includes one or more memory buffers configured to decouple the memory data rate from the southbound data rate and the northbound data rate. One embodiment provides a memory system including first dynamic random access memories and a first memory buffer. The first memory buffer is configured to receive southbound data at a first data rate and provide northbound data at a second data rate. The first memory buffer is also configured to read data from the first dynamic random access memories at a third data rate, wherein the first memory buffer is configured to decouple the third data rate from the first data rate and the second data rate. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0011]    The accompanying drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification. The drawings illustrate the embodiments of the present invention and together with the description serve to explain the principles of the invention. Other embodiments of the present invention and many of the intended advantages of the present invention will be readily appreciated, as they become better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts. 
           [0012]      FIG. 1  is a block diagram illustrating one embodiment of an electrical system according to the present invention. 
           [0013]      FIG. 2  is a diagram illustrating one embodiment of an advanced memory buffer. 
           [0014]      FIG. 3  is a timing diagram illustrating the operation of one embodiment of an electrical system. 
           [0015]      FIG. 4  is a timing diagram illustrating the operation of another embodiment of an electrical system. 
       
    
    
     DETAILED DESCRIPTION 
       [0016]    In the following Detailed Description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. In this regard, directional terminology, such as “top,” “bottom,” “front,” “back,” “leading,” “trailing,” etc., is used with reference to the orientation of the Figure(s) being described. Because components of embodiments of the present invention can be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims. 
         [0017]      FIG. 1  is a block diagram illustrating one embodiment of an electrical system  20  according to the present invention. Electrical system  20  includes a host controller  22  and interleaved FB-DIMMs  24   a - 24   n . Host controller  22  controls FB-DIMMs  24   a - 24   n  to provide system memory functions. FB-DIMMs  24   a - 24   n  are one type of subsystem assembly. In other embodiments, electrical system  20  includes host controller  22  and any other suitable subsystem assembly, such as a graphics card, an audio card, a facsimile card, or a modem card, and host controller  22  controls the subsystem assembly to provide corresponding system functions. 
         [0018]    FB-DIMMs  24   a - 24   n  are daisy-chained together and coupled to host controller  22  via memory channel  26 . FB-DIMMs  24   a - 24   n  receive southbound data at a southbound data rate via memory channel  26  and FB-DIMMs  24   a - 24   n  provide northbound data at a northbound data rate via memory channel  26 . Each of the FB-DIMMs  24   a - 24   n  communicates with on-board memory at a memory data rate that is decoupled from the southbound data rate and the northbound data rate. Also, the memory data rate on one of the FBDIMMs  24   a - 24   n  can be different than the memory data rate on any of the other FB-DIMMs  24   a - 24   n . In one embodiment, FB-DIMMs  24   a - 24   n  interleave data in the northbound data at the northbound data rate. In one embodiment, the southbound data rate is different than the northbound data rate and each of the memory data rates is the same or different than the southbound data rate and the northbound data rate. 
         [0019]    As used herein, the term “electrically coupled” is not meant to mean that the elements must be directly coupled together and intervening elements may be provided between the “electrically coupled” elements. 
         [0020]    FB-DIMMs  24   a - 24   n  are electrically coupled to host controller  22  via memory channel  26 , which includes southbound data paths  28   a - 28   n  and northbound data paths  30   a - 30   n . Host controller  22  is electrically coupled to FB-DIMM 1  at  24   a  via southbound data path  28   a  and northbound data path  30   a . FB-DIMM 1  at  24   a  is electrically coupled to FB-DIMM 2  at  24   b  via southbound data path  28   b  and northbound data path  30   b . FB-DIMM 2  at  24   b  is electrically coupled to the next FB-DIMM via southbound data path  28   c  and northbound data path  30   c  and so on, up to the previous FB-DIMM being electrically coupled to FB-DIMMn at  24   n  via southbound data path  28   n  and northbound data path  30   n.    
         [0021]    FB-DIMM 1  at  24   a  includes AMB 1  at  32   a  and DRAMs at  34   a . FB-DIMM 2  at  24   b  includes AMB 2  at  32   b  and DRAMs at  34   b  and so on, up to FB-DIMMn at  24   n  that includes AMBn at  32   n  and DRAMs at  34   n . DRAMs  34   a - 34   b  can be any suitable speed and/or type of DRAM including DDR-SDRAM. Also, DRAMs  34   a - 34   b  can be any suitable generation, such as first, second, and third generation DDR-SDRAM. In one embodiment, DRAMs at  34   a  are one speed of DRAM, DRAMs at  34   b  are another speed of DRAM, and DRAMs at  34   n  are a third speed of DRAM. In one embodiment, each of the FB-DIMMs at  24   a - 24   n  includes 18 DDR-SDRAM circuits. In one embodiment, each of the FB-DIMMs at  24   a - 24   n  includes any suitable number of DDR-SDRAM circuits. 
         [0022]    AMB 1  at  32   a  is electrically coupled to DRAMs at  34   a  via memory paths  36   a  and to host controller  22  via southbound data path  28   a  and northbound data path  30   a . AMB 2  at  32   b  is electrically coupled to DRAMs at  34   b  via memory paths  36   b  and to AMB 1  via southbound data path  28   b  and northbound data path  30   b . Also, AMB 2  at  32   b  is electrically coupled to the next AMB via southbound data path  28   c  and northbound data path  30   c . AMBn at  32   n  is electrically coupled to DRAMs at  34   n  via memory paths  36   n  and to the previous AMB via southbound data path  28   n  and northbound data path  30   n.    
         [0023]    Host controller  22  provides southbound data to FB-DIMM 1  at  24   a  and AMB 1  at  32   a  via southbound data path  28   a . The southbound data includes commands, addresses, and data for controlling FB-DIMMs  24   a - 24   n . The commands include activate, read, and write commands. The addresses include FB-DIMM addresses and DRAM read and write addresses. Data includes write data to be written into the DRAMs  34   a - 34   n . In one embodiment, the commands include a put command for putting read data that was read from one or more DRAMs on one of the FB-DIMMs  24   a - 24   n  into the northbound data. 
         [0024]    Host controller  22  receives northbound data from FB-DIMM 1  at  24   a  and AMB 1  at  32   a  via northbound data path  30   a . The northbound data includes read data from the FB-DIMMs  24   a - 24   n . Read data from multiple FB-DIMMs  24   a - 24   n  can be interleaved in the northbound data. 
         [0025]    AMB 1  at  32   a  receives southbound data from host controller  22  at the southbound data rate. AMB 1  at  32   a  temporarily buffers the received southbound data. If AMB 1  at  32   a  detects a command for FB-DIMM 1  at  24   a  in the buffered southbound data, AMB 1  at  32   a  provides the command to on-board DRAMs  34   a  via memory paths  36   a . AMB 1  at  32   a  writes data into and reads data out of addressed DRAMs  34   a . The data written into and read out of the DRAMs  34   a  is communicated at the memory data rate of the DRAMs  34   a  over memory paths  36   a . AMB 1  at  32   a  temporarily buffers read data and then provides the read data in northbound data to host controller  22  at the northbound data rate. AMB 1  at  32   a  decouples the memory data rate from the southbound data rate and the northbound data rate. In one embodiment, AMB 1  at  32   a  provides the read data in northbound data to host controller  22  at the northbound data rate in response to a put command from host controller  22 . 
         [0026]    If the buffered southbound data is not addressed to FB-DIMM 1  at  24   a , AMB 1  at  32   a  provides the buffered southbound data to FB-DIMM 2  at  24   b  and AMB 2  at  32   b  via southbound data path  28   b . AMB 1  at  32   a  receives northbound data from FB-DIMM 2  at  24   b  and AMB 2  at  32   b  via northbound data path  30   b . The northbound data includes read data from the FB-DIMMs  24   b - 24   n.    
         [0027]    AMB 2  at  32   b  receives southbound data from AMB 1  at  32   a  at the southbound data rate. AMB 2  at  32   b  temporarily buffers the received southbound data. If AMB 2  at  32   b  detects a command for FB-DIMM 2  at  24   b  in the buffered southbound data, AMB 2  at  32   b  provides the command to on-board DRAMs  34   b  via memory paths  36   b . AMB 2  at  32   b  writes data into and reads data out of addressed DRAMs  34   b . The data written into and read out of the DRAMs  34   b  is communicated at the memory data rate of the DRAMs  34   b  over memory paths  36   b . AMB 2  at  32   b  temporarily buffers read data and then provides the read data in northbound data to host controller  22  at the northbound data rate. AMB 2  at  32   b  decouples the memory data rate from the southbound data rate and the northbound data rate. In one embodiment, AMB 2  at  32   b  provides the read data in northbound data to host controller  22  at the northbound data rate in response to a put command from host controller  22 . 
         [0028]    If the buffered southbound data is not addressed to FB-DIMM 2  at  24   b , AMB 2  at  32   b  provides the buffered southbound data to the next FB-DIMM and AMB via southbound data path  28   c . AMB 2  at  32   b  receives northbound data from the next FB-DIMM and AMB via northbound data path  30   c . The northbound data includes read data from the FB-DIMMs  24   c - 24   n . This forms a daisy-chain among FB-DIMMs  24   a - 24   n  up to FB-DIMMn at  24   n.    
         [0029]    FB-DIMMn at  24   n  and AMBn at  32   n  receives southbound data from the previous AMB at the southbound data rate via southbound data path  28   n . AMBn at  32   n  temporarily buffers the received southbound data. If AMBn at  32   n  detects a command for FB-DIMMn at  24   n  in the buffered southbound data, AMBn at  32   n  provides the command to on-board DRAMs  34   n  via memory paths  36   n . AMBn at  32   n  writes data into and reads data out of addressed DRAMs  34   n . The data written into and read out of the DRAMs  34   n  is communicated at the memory data rate of the DRAMs  34   n  over memory paths  36   n . AMBn at  32   n  temporarily buffers read data and then provides the read data in northbound data to host controller  22  at the northbound data rate. AMBn at  32   n  decouples the memory data rate from the southbound data rate and the northbound data rate. In one embodiment, AMBn at  32   n  provides the read data in northbound data to host controller  22  at the northbound data rate in response to a put command from host controller  22 . 
         [0030]      FIG. 2  is a diagram illustrating one embodiment of AMB 1  at  32   a . In one embodiment, each of the other AMBs  32   b - 32   n  is similar to AMB 1  at  32   a . In other embodiments, each of the other AMBs  32   b - 32   n  can be any suitable type or types of AMB. 
         [0031]    AMB 1  at  32   a  includes a southbound input circuit  50 , a southbound (control circuit) input buffer  52 , a southbound re-synchronization circuit  54 , a southbound output circuit  56 , an input first-in-first-out (FIFO)  58 , and a DRAM interface circuit  60 . Southbound input circuit  50  is electrically coupled to southbound input buffer  52  via buffer input path  62 . Southbound input buffer  52  is electrically coupled to southbound re-synchronization circuit  54  via buffer output path  64  and to input FIFO  58  via FIFO input path  66 . Southbound re-synchronization circuit  54  is electrically coupled to southbound output circuit  56  via re-synchronized data path  68 . Input FIFO  58  is electrically coupled to DRAM interface circuit  60  via FIFO output path  70 . 
         [0032]    AMB 1  at  32   a  and southbound input circuit  50  receive input southbound data SBDIN at  28   a  at the southbound data rate via southbound data path  28   a . The input southbound data SBDIN at  28   a  includes the commands, addresses, and data for controlling FB-DIMMs  24   a - 24   n . The commands include activate, read, write, and put commands. 
         [0033]    Southbound input circuit  50  provides the input southbound data SBDIN at  28   a  to southbound input buffer  52  via buffer input path  62 . Southbound input buffer  52  temporarily buffers the received input southbound data SBDIN. If AMB 1  at  32   a  and southbound input buffer  52  detect a command for FB-DIMM 1  at  24   a  in the input southbound data SBDIN, southbound input buffer  52  provides the command, corresponding addresses and, if applicable, write data to input FIFO  58  via FIFO input path  66 . Input FIFO  58  provides the command, corresponding addresses and, if applicable, write data to DRAM interface circuit  60  via FIFO output path  70 . DRAM interface circuit  60  provides the command, corresponding addresses and, if applicable, write data to on-board DRAMs  34   a  via memory paths  36   a . The on-board DRAMs  34   a  respond to the received command, such as by writing the write data into DRAMs  34   a  or reading out read data from DRAMs  34   a . The data written into and read out of DRAMs  34   a  is communicated at the memory data rate of DRAMs  34   a  over memory paths  36   a . The memory data rate is decoupled from the southbound data rate via circuits such as southbound input buffer  52 , input FIFO  58 , and DRAM interface circuit  60 . 
         [0034]    If AMB 1  at  32   a  and southbound input buffer  52  do not detect a command, address or data for FB-DIMM 1  at  24   a  in the input southbound data SBDIN, southbound input buffer  52  provides the input southbound data to southbound re-synchronization circuit  54  via buffer output path  64 . Southbound re-synchronization circuit  54  re-synchronizes the input southbound data to the southbound data and southbound data rate and provides the re-synchronized southbound data to southbound output circuit  56  via re-synchronized data path  68 . Southbound output circuit  56  provides the re-synchronized southbound data as output southbound data SBDOUT at  28   b  via southbound data path  28   b.    
         [0035]    AMB 1  at  32   a  includes a read data FIFO  72 , a northbound input circuit  74 , a northbound re-synchronization circuit  76 , a framing circuit  78 , and a northbound output circuit  80 . DRAM interface circuit  60  is electrically coupled to read data FIFO  72  via read data path  82 . Northbound input circuit  74  is electrically coupled to northbound re-synchronization circuit  76  via northbound data input path  84 . Framing circuit  78  is electrically coupled to read data FIFO  72  via FIFO output path  86 , to northbound re-synchronization circuit  76  via re-synchronized data path  88 , and to southbound input buffer  52  via put command path  90 . Also, framing circuit  78  is electrically coupled to northbound output circuit  80  via output data path  92 . 
         [0036]    DRAM interface  60  receives data read from DRAMs  34   a  at the memory data rate via memory paths  36   a . DRAM interface  60  provides the read data to read data FIFO  72  via read data path  82 . Read data FIFO  72  buffers the read data. 
         [0037]    Northbound input circuit  74  receives input northbound data NBDIN at  30   b  via northbound data path  30   b . The input northbound data NBDIN at  30   b  includes read data from FB-DIMMs  24   b - 24   n . Northbound input circuit  74  provides the received input northbound data NBDIN at  30   b  to northbound re-synchronization circuit  76  via northbound data input path  84 . Northbound re-synchronization circuit  76  resynchronizes the input northbound data NBDIN to northbound data and the northbound data rate. Framing circuit  78  receives the re-synchronized northbound data from northbound re-synchronization circuit  76  via re-synchronized data path  88 . Framing circuit  78  provides the re-synchronized northbound data to northbound output circuit  80  via output data path  92 . Northbound output circuit  80  provides re-synchronized northbound data as output northbound data NBDOUT at  30   a  at the northbound data rate via northbound data path  30   a.    
         [0038]    Framing circuit  78  receives a put command from southbound input buffer  52  via put command path  90  and read data FIFO  72  provides the read data to framing circuit  78  via FIFO output path  86 . In response to the put command, framing circuit  78  inserts the read data into the northbound data that is provided to northbound output circuit  80  via output data path  92 . Northbound output circuit  80  provides the northbound data as output northbound data NBDOUT at  30   a  at the northbound data rate via northbound data path  30   a . The memory data rate is decoupled from the northbound data rate via circuits, such as DRAM interface  60 , read data FIFO  72 , and framing circuit  78 . AMB 1  at  32   a  decouples the memory data rate from the southbound data rate and the northbound data rate. In one embodiment, framing circuit  78  does not receive put commands and framing circuit  78  inserts the read data into the northbound data at the northbound data rate in response to the previous read command. 
         [0039]      FIG. 3  is a timing diagram illustrating the operation of one embodiment of electrical system  20 . Host controller  22  provides commands in southbound data at  100  to FB-DIMMs at  24   a - 24   n  via southbound data paths  28   a - 28   n . FB-DIMM 1  at  24   a  receives the southbound commands and provides FB-DIMM 1  commands at  102  to DRAMs  34   a . The DRAMs  34   a  provide FB-DIMM 1  data at  104 . FB-DIMM 2  at  24   b  receives the southbound commands and provides FB-DIMM 2  commands at  106  to DRAMs  34   b . The DRAMs  34   b  provide FB-DIMM 2  data at  108 . FB-DIMM 1  at  24   a  and FB-DIMM 2  at  24   b  provide data in northbound data at  110 , which is transmitted back to host controller  22  via northbound data paths  30   a - 30   n.    
         [0040]    In this example, the memory data rate between each of the AMBs  32   a - 32   n  and corresponding DRAMs  34   a - 34   n  is equal to each of the other memory data rates between the other AMBs  32   a - 32   n  and DRAMs  34   a - 34   n . Also, the memory data rate is one half the northbound data rate. Since, accessing DRAMs  34   a - 34   n  at the memory data rate is independent of or decoupled from the northbound data rate, DRAMs  34   a - 34   n  on different FB-DIMMs  24   a - 24   n  can be accessed in parallel. Host controller  22  provides up to three commands in each of the southbound FB-DIMM frames  112 . FB-DIMMs  24   a - 24   n  provide one command in each of the DRAM clock periods  114 . 
         [0041]    At  116 , host controller  22  provides an activate command for FB-DIMM 1  at  24   a  in southbound data at  100 . FB-DIMM 1  at  24   a  receives the activate command and at  118  provides an activate command to DRAMs  34   a . This activates addressed DRAMs  34   a . At  120 , host controller  22  provides an activate command for FB-DIMM 2  at  24   b  in southbound data at  100 . FB-DIMM 2  at  24   b  receives the activate command and at  120  provides an activate command to DRAMs  34   b . This activates addressed DRAMs  34   b.    
         [0042]    At  124 , host controller  22  provides a read command for FB-DIMM 1  at  24   a  in southbound data at  100 . FB-DIMM 1  at  24   a  receives the read command and at  126  provides a first read command to DRAMs  34   a  and at  128  provides a second read command to DRAMs  34   a . After read latency period  130 , DRAMs  34   a  provide four blocks of data in four frames at  132  in response to the first read command. Also, after a read latency period DRAMs  34   a  provide four blocks of data in four frames at  134  in response to the second read command. 
         [0043]    At  136 , host controller  22  provides a read command for FB-DIMM 2  at  24   b  in southbound data at  100 . FB-DIMM 2  at  24   b  receives the read command and at  138  provides a first read command to DRAMs  34   b  and at  140  provides a second read command to DRAMs  34   b . After a read latency period, DRAMs  34   b  provide four blocks of data in four frames at  142  in response to the first read command. Also, after a read latency period DRAMs  34   b  provide four blocks of data in four frames at  144  in response to the second read command. 
         [0044]    At  146 , host controller  22  provides a put command for FB-DIMM 1  at  24   a  in southbound data at  100 . FB-DIMM 1  at  24   a  receives the put command and inserts the four blocks of data at  132  into two frames at  148  of northbound data  110 . At  150 , host controller  22  provides a put command for FB-DIMM 2  at  24   b  in southbound data at  100 . FB-DIMM 2  at  24   b  receives the put command and inserts the four blocks of data at  142  into two frames at  152  of northbound data  110 . At  154 , host controller  22  provides a put command for FB-DIMM 1  at  24   a  in southbound data at  100 . FB-DIMM 1  at  24   a  receives the put command and inserts the four blocks of data at  134  into two frames at  156  of northbound data  110 . At  158 , host controller  22  provides a put command for FB-DIMM 2  at  24   b  in southbound data at  100 . FB-DIMM 2  at  24   b  receives the put command and inserts the four blocks of data at  144  into two frames at  160  of northbound data  110 . 
         [0045]    Host controller  22  and FB-DIMMs  24   a  and  24   b  interleave the read data from FB-DIMMs  24   a  and  24   b  in northbound data at  110 . In one embodiment of electrical system  20  with uniform memory data rates, the AMBs  32   a - 32   n  are programmed to insert the read data in northbound data traffic after a period of time in response to the read command from host controller  22 , and without receiving a put command. 
         [0046]    The degrees of freedom provided by electrical system  20  can be used to reduce latency times. Also, the degrees of freedom provided by electrical system  20  can be used to save power in systems where the northbound data rate is matched to a high data rate. 
         [0047]      FIG. 4  is a timing diagram illustrating the operation of one embodiment of electrical system  20 . Host controller  22  provides commands in southbound data at  200  to FB-DIMMs at  24   a - 24   n  via southbound data paths  28   a - 28   n . FB-DIMM 1  at  24   a  receives the southbound commands and provides FB-DIMM 1  commands at  202  to DRAMs  34   a . The DRAMs  34   a  provide FB-DIMM 1  data at  204 . FB-DIMM 2  at  24   b  receives the southbound commands and provides FB-DIMM 2  commands at  206  to DRAMs  34   b . The DRAMs  34   b  provide FB-DIMM 2  data at  208 . FB-DIMMn at  24   n  receives the southbound commands and provides FB-DIMMn commands at  210  to DRAMs  34   n . The DRAMs  34   b  provide FB-DIMMn data at  212 . FB-DIMM 1  at  24   a , FB-DIMM 2  at  24   b , and FB-DIMMn at  24   n  provide data in northbound data at  214 , which is transmitted back to host controller  22  via northbound data paths  30   a - 30   n.    
         [0048]    In this example, the memory data rate between each of the AMBs  32   a - 32   n  and corresponding DRAMs  34   a - 34   n  is different from each of the other memory data rates between the other AMBs  32   a - 32   n  and DRAMs  34   a - 34   n . Since, accessing DRAMs  34   a - 34   n  at the memory data rate is independent of or decoupled from the northbound data rate, DRAMs  34   a - 34   n  on different FB-DIMMs  24   a - 24   n  can be accessed in parallel. Host controller  22  provides up to three commands in each of the southbound FB-DIMM frames  216 . FB-DIMM 1  at  24   a  provides one command in each of the DRAM clock periods  218 . FB-DIMM 2  at  24   b  provides one command in each of the DRAM clock periods  220 . FB-DIMMn at  24   n  provides one command in each of the DRAM clock periods  222 . 
         [0049]    At  224 , host controller  22  provides an activate command for FB-DIMM 1  at  24   a  and an activate command for FB-DIMM 2  at  24   b  in southbound data at  200 . At  226 , host controller  22  provides an activate command for FB-DIMMn at  24   n  in southbound data at  200 . FB-DIMM 1  at  24   a  receives the activate command and at  228  provides an activate command to DRAMs  34   a , which activates addressed DRAMs  34   a . FB-DIMM 2  at  24   b  receives the activate command and at  230  provides an activate command to DRAMs  34   b , which activates addressed DRAMs  34   b . FB-DIMMn at  24   n  receives the activate command and at  232  provides an activate command to DRAMs  34   n , which activates addressed DRAMs  34   n.    
         [0050]    At  234 , host controller  22  provides a read command for FB-DIMM 1  at  24   a  and a read command for FB-DIMM 2  at  24   b  in southbound data at  200 . At  236 , host controller  22  provides a read command for FB-DIMMn at  24   n  in southbound data at  200 . FB-DIMM 1  at  24   a  receives the read command and at  238  provides a read command to DRAMs  34   a . After a read latency period, DRAMs  34   a  provide four blocks of data at  240  in response to the read command. FB-DIMM 2  at  24   b  receives the read command and at  242  provides a read command to DRAMs  34   b . After a read latency period, DRAMs  34   b  provide four blocks of data at  244  in response to the read command. FB-DIMMn at  24   n  receives the read command and at  246  provides a read command to DRAMs  34   n . After a read latency period, DRAMs  34   n  provide four blocks of data at  248  in response to the read command. 
         [0051]    DRAMs  34   n  provide the four blocks of data at  248  at a higher memory data rate than the memory data rates that DRAMs  34   a  provide the four blocks of data at  240  and DRAMS  34   b  provide the four blocks of data at  244 . DRAMs  34   b  provide the four blocks of data at  244  at a higher memory data rate than the memory data rate that DRAMs  34   a  provide the four blocks of data at  240 . The four blocks of data at  248  are available before the four blocks of data at  244 , which are available before the four blocks of data at  240 . 
         [0052]    At  250 , host controller  22  provides a put command for FB-DIMMn at  24   n  in southbound data at  200 . FB-DIMMn at  24   n  receives the put command and inserts the four blocks of data at  248  into two frames at  252  of northbound data  214 . At  254 , host controller  22  provides a put command for FB-DIMM 2  at  24   b  in southbound data at  200 . FB-DIMM 2  at  24   b  receives the put command and inserts the four blocks of data at  244  into two frames at  256  of northbound data  214 . At  258 , host controller  22  provides a put command for FB-DIMM 1  at  24   a  in southbound data at  200 . FB-DIMM 1  at  24   a  receives the put command and inserts the four blocks of data at  240  into two frames at  260  of northbound data  214 . 
         [0053]    In this example, electrical system  20  operates with DRAMs  34   a - 34   n  that do not have uniform speed grades. Electrical systems without uniform speed grades in DRAMs  34   a - 34   n  allow a more complex trade off between power consumption, DRAM capacity, DRAM speed grades, and system cost. Also, DRAMs with lower speed grades are usually available sooner than DRAMs with higher speed grades and systems that are already built with lower speed grades can be expanded in capacity and performance improved via higher speed grade DRAMs. 
         [0054]    Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein. Therefore, it is intended that this invention be limited only by the claims and the equivalents thereof.