Abstract:
A computer program product and a hub device for deriving clocks in a memory system are provided. The computer program product includes a storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for facilitating a method. The method includes receiving a reference oscillator clock at the hub device. The hub device is in communication with a controller channel via a controller interface and in communication with a memory device via a memory interface. A base clock operating at a base clock frequency is derived from the reference oscillator clock. A memory interface clock is derived by multiplying the base clock by a memory multiplier. A controller interface clock is derived by multiplying the base clock by a controller multiplier. The memory interface clock is applied to the memory interface and the controller interface clock is applied to the controller interface.

Description:
CROSS-REFERENCE TO RELATED APPLICATION 
       [0001]    This application is a continuation of U.S. patent application Ser. No. 11/263,344, filed Oct. 31, 2005, the disclosure of which is incorporated by reference herein in its entirety. 
     
    
     BACKGROUND 
       [0002]    This invention relates to memory systems comprised of hub devices connected to a memory controller by a daisy chained controller channel. The hub devices are attached to or reside upon memory modules that contain memory devices. More particularly, this invention relates to allowing the memory devices on the same controller channel to operate at varying frequencies. 
         [0003]    Most high performance computing main memory systems use multiple memory modules with multiple memory devices connected to a controller by one or more controller channels. All memory modules connected to the same controller channel operate at the same controller frequency and all of their memory devices operate at the same frequency. The ratio of the controller channel frequency to the memory device clock frequency is typically a fixed integer. These restrictions limit the memory device operating frequencies when mixed within a channel. Due to the fixed ratio of channel frequency to memory device frequency, channels that are not able to attain the highest data rate will operate with a decrease in both channel and memory device frequency. These typical main memory systems must operate no faster than the slowest memory module on the channel. When a channel is populated with a memory module that is slower than the others, the entire channel, and perhaps the entire memory system, must slow down to accommodate the capabilities of the slow memory module. 
         [0004]    The reductions in memory system operating frequency result in a corresponding reduction in computer system main memory performance. What is needed is a memory system that operates its controller channel at the highest supported rate while operating all memory devices on the memory modules at their highest supported rates. This capability would maximize the performance of the main memory system. 
       SUMMARY 
       [0005]    Exemplary embodiments include a computer program product for deriving clocks in a memory system. The computer program product includes a storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for facilitating a method. The method includes receiving a reference oscillator clock at a hub device. The hub device is in communication with a controller channel via a controller interface and in communication with a memory device via a memory interface. A base clock operating at a base clock frequency is derived from the reference oscillator clock. A memory interface clock is derived by multiplying the base clock by a memory multiplier. A controller interface clock is derived by multiplying the base clock by a controller multiplier. The memory interface clock is applied to the memory interface and the controller interface clock is applied to the controller interface. 
         [0006]    Additional exemplary embodiments include a hub device in a memory system. The hub device includes a memory interface, a controller and a clock derivation mechanism. The memory interface is utilized for transmitting and receiving data from a memory device located on a memory module. The transmitting and receiving occur in response to a memory interface clock operating at a memory module clock frequency. The controller interface is utilized for transmitting and receiving data from a controller channel in response to a controller interface clock operating at a controller channel clock frequency. The clock derivation mechanism facilitates: receiving a reference oscillator clock; deriving a base clock operating at a base clock frequency from the reference oscillator clock; deriving the memory interface clock by multiplying the base clock by a memory multiplier; deriving the controller interface clock by multiplying the base clock by a controller multiplier; applying the memory interface clock to the memory interface; and applying the controller interface clock to the controller interface. 
         [0007]    Further exemplary embodiments include a memory system. The memory system includes a controller, a controller channel in communication with the controller, one or more memory modules and one or more hub devices. The memory modules each include one or more memory devices. The hub devices buffer addresses, commands and data. Each hub device is in communication with one or more of the memory modules and in communication with the controller via the controller channel. Each of the hub devices are independently configured with a controller channel operating frequency and a memory device operating frequency suing multiples of a base clock derived from a reference oscillator clock. The controller channel operating frequency is utilized for communicating with the controller channel. The memory device operating frequency is utilized for communicating with the memory devices. 
     
    
     
       BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
         [0008]    Referring now to the drawings wherein like elements are numbered alike in the several FIGURES: 
           [0009]      FIG. 1  depicts an exemplary memory system with multiple levels of daisy chained memory modules with point-to-point connections; 
           [0010]      FIG. 2  depicts an exemplary memory system with hub devices that are connected to memory modules and to a controller channel by a daisy chained channel; 
           [0011]      FIG. 3  depicts an exemplary hub device using m:n clocking with a forwarded controller interface bus clock reference; 
           [0012]      FIG. 4  depicts an exemplary hub device using m:n clocking with a separately distributed clock reference; 
           [0013]      FIG. 5  depicts an exemplary memory system controller channel with a controller interface forwarded reference clock and independent memory device frequencies using m:n clocking; 
           [0014]      FIG. 6  depicts an exemplary memory system controller channel with a separately distributed reference clock and independent memory device frequencies using m:n clocking; and 
           [0015]      FIG. 7  is a table of sample controller and memory interface data rates with m:n ratios that may be implemented by exemplary embodiments. 
       
    
    
     DETAILED DESCRIPTION  
       [0016]    Exemplary embodiments pertain to computer memory systems constructed of memory modules interconnected by a controller channel originating from a controller. The memory modules are attached to hub logic devices that are further attached to memory devices on the memory modules. The memory controller channel operates at a common clock frequency. Each memory module receives a common reference oscillator frequency, either by a forwarded controller interface bus clock on the controller channel or by separate reference oscillator input signal. The hub devices are uniquely configured to operate their attached memory devices at operating frequencies that may be non-integer multiples of the reference oscillator frequency. This enables memory modules of varying memory device speed grades to be operated at independent frequencies while residing on a memory controller channel that operates at a common clock frequency. 
         [0017]    Exemplary embodiments include memory systems constructed of one or more memory modules  110  that are connected to a memory controller  102  by a daisy chained controller channel  114  as depicted in  FIG. 1 . The memory modules  110  contain both a hub device  112  that buffers commands, address and data signals to and from the controller memory channel  114  as well as one or more memory devices  108  connected to the hub device  112 . The downstream portion of the controller channel  114  transmits write data and memory operation commands to the hub devices  112 . The upstream portion of the controller channel  114  returns requested read data to the controller  102 . In exemplary embodiments, each of the hub devices  112  may be independently configured with a controller channel operating frequency and a memory device operating frequency to allow the controller channel  114  to be operating at one frequency and the memory devices  108  to be operated at a different frequency. In addition, each memory module  110  in the memory system and its associated memory devices  108  may be operating at different operating speeds, or frequencies. 
         [0018]      FIG. 2  depicts an alternate exemplary embodiment that includes a memory system constructed of one or more memory modules  110  connected to hub devices  112  that are further connected to a memory controller  102  by a daisy chained controller channel  114 . In this embodiment, the hub device  112  is not located on the memory module  110 ; instead the hub device  112  is in communication with the memory module  110 . The controller channel  114  may be constructed using multi-drop connections to the hub devices  112  or by using point-to-point connections. As depicted in  FIG. 2 , the memory modules  110  may be in communication with the hub devices  112  via multi-drop connections and/or point-to-point connections. Other hardware configurations are possible, for example exemplary embodiments may utilize only a single level of daisy chained hub devices  112  and/or memory modules  110 . 
         [0019]      FIG. 3  depicts an exemplary hub device  112  using m:n clocking with a forwarded controller interface bus clock reference  322  as the reference oscillator clock. The hub device  112  includes a clock domain crossing function  304 , a memory interface  302 , a controller interface  306 , and a phased lock loop (PLL)  308  (also referred to herein as a clock derivation mechanism because it may be implemented in other manners including software and/or hardware). The memory interface  302  sends data to and receives data from memory devices  108  on the memory module  110  via a mem_data bus  310  operating at ‘2*Y’ Mbps and clocked by a memory_clock  312  with a frequency of ‘Y’ MHz. The controller interface  306  communicates with downstream memory modules  110  via a downstream_drv  314  (to drive data and commands downstream) and a Downstream_rcv  316  (to receive data). In addition, the controller interface  306  communicates with upstream memory modules  110  or the controller  102  (if there are no upstream memory modules  110 )  110  via an upstream_rcv  318  (to receive data and commands) and an upstream_drv  320  (to drive data and commands upstream). 
         [0020]    Exemplary embodiments of the present invention use two configurable integer ratios, named ‘m’ and ‘n’, within the hub device  112  to allow each memory module  110  within the controller channel  114  to operate at a common channel frequency (also referred to herein as a controller channel clock frequency) but with a unique memory device frequency (also referred to herein as a memory module clock frequency). ‘m’, a controller multiplier, is defined as the ratio of controller channel frequency, ‘X’ to a small, fixed, base clock frequency such as, but not limited to 133 MHz, 100 MHz, 66 MHz, etc. Hub devices  112  that use the clock forwarded on the controller channel  114  as their internal reference clock will divide the frequency of the forwarded controller interface bus clock reference  322  by ‘m’ to create, for example, a 133 MHz base clock. If the intended controller interface frequency is not evenly divisible by the base clock frequency, then the controller interface frequency is derived by rounding down to the next integer multiple of the frequency of the base clock (‘b’). This base clock will be used as the reference oscillator clock and input to a PLL  308  where it will be multiplied by ‘m’ to produce a cleaned up and distributed version of the controller interface clock. ‘n’, the memory multiplier, is defined as the ratio of the memory device clock frequency to the base frequency (e.g., 133 MHz). Hub devices  112  multiply the 133 MHz base clock by ‘n’ in their PLL  308  to produce the cleaned up memory interface clock running at ‘Y’ MHz. The resulting controller channel frequency to memory device operating frequency ratio is ‘m:n’. 
         [0021]    Because the ratio of controller interface to memory interface operating frequency is known by the hub device  112 , a simplified clock domain crossing function  304  is employed in the hub device  112  to transfer controller interface information to and from the memory interface  302 . If the controller interface  306  and/or memory interface  302  operate using double data rate (DDR) clocking, the data rates (in Mbps) will be twice the respective interface clock frequency, (i.e., 2X and/or 2Y). If DDR is used on both interfaces, the ratio of the data rates will also be ‘m:n’. 
         [0022]      FIG. 4  depicts an exemplary hub device using m:n clocking with a separately distributed reference clock  402  input to the PLL  308  as the reference oscillator clock. Main memory systems that use a separately distributed reference clock  402  can also use ‘m:n’ clocking. In this case, the frequency of the incoming reference clock  402  must be an integer multiple of the frequency of the base clock (e.g., 133 MHz). The reference clock  402  operating at a frequency of ‘W’ MHz is divided by an integer ‘L’ to produce the 133 MHz base clock that is used as the input clock to the multipliers in the PLL  308 . If the separately distributed reference clock  402  has a frequency that is equal to 133 MHz, then ‘L’ is simply one. The PLL  308  multiplies the base clock by ‘m’ to produce the cleaned up controller interface clock whose frequency is ‘X’. The PLL  308  also multiplies the base clock by ‘n’ to produce the memory interface clock whose frequency is ‘Y’. A simplified clock domain crossing function  304  is used to transfer information between the logic in the controller interface  306  and the memory interface  302 . 
         [0023]      FIG. 5  depicts an exemplary memory system controller channel  114  with a controller interface forwarded reference clock  322  and independent memory device frequencies using m:n clocking. Memory systems that use ‘m:n’ clocking are able to operate their memory modules  110  at uniquely configured memory interface frequencies equal to the highest frequency supported by their memory devices  108 .  FIG. 5  shows a single channel of a memory system in which the memory module labeled DIMM  0   502  is configured to operate its memory devices  108  at the ‘Y0’ frequency while the memory module labeled DIMM  1   504  is configured to operate its memory devices  108  at the ‘Y1’ frequency. Both DIMM  0   502  and DIMM  1   504  operate at a common, ‘X’ controller interface frequency.  FIG. 6  depicts an exemplary memory system channel with a separately distributed reference clock  402  and independent memory device frequencies using m:n clocking to maximize frequencies and performance. 
         [0024]    If the memory channel frequency, ‘X’ is limited by its electrical and/or timing requirements in a particular system, the memory device frequencies can still be maximized through the use of m:n clocking. This maximization of operating frequencies results in an optimization of memory channel, and therefore computer system, performance. 
         [0025]    When configuring a memory system for optimum performance using m:n clocking, users should first evaluate the highest supported controller channel frequency. This is rounded down to the next integer multiple of the base clock frequency, (e.g., 133 MHz) and yields ‘X’. ‘X’ is divided by the base clock frequency to determine ‘m’ for all hub devices  112  in the controller channel  114 . For each memory module  110  in the controller channel  114 , users should evaluate the highest supported memory device operating frequency. This will be a function of hub device  112  and memory device  108  specifications along with the results of electrical analysis of the memory interface  302  on the memory module  110  itself. This maximum operating frequency should be rounded down to the next integer multiple of the base clock frequency, yielding ‘Y’ for that memory module  110 . ‘Y’ is divided by the base clock frequency to determine ‘n’ for that particular memory module  110  and/or hub device  112 . 
         [0026]      FIG. 7  is a table of sample controller and memory interface data rates with m:n ratios that may be implemented by exemplary embodiments. Memory systems using m:n clocking are highly flexible and can be greatly optimized. The following table shows various m and n values, data rates and m:n ratios for a base clock frequency of 133 MHz. Some interesting integer m:n ratios are highlighted with a ‘*’ to illustrate settings that can be used to recreate the more typical, fixed data rate ratios at various controller channel and memory device operating frequencies. 
         [0027]    Exemplary embodiments may be utilized to maximize the performance of a memory system by operating the controller channel at its highest supported rate while at the same time operating all memory devices on the memory modules at their highest supported frequencies. The frequencies of the memory devices on each memory module connected to the controller channel can be different for each memory module, allowing memory devices of varying speeds to be optimized on the same controller channel. 
         [0028]    As described above, the embodiments of the invention may be embodied in the form of computer-implemented processes and apparatuses for practicing those processes. Embodiments of the invention may also be embodied in the form of computer program code containing instructions embodied in tangible media, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. The present invention can also be embodied in the form of computer program code, for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. When implemented on a general-purpose microprocessor, the computer program code segments configure the microprocessor to create specific logic circuits. 
         [0029]    While the invention has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this invention, but that the invention will include all embodiments falling within the scope of the appended claims. Moreover, the use of the terms first, second, etc. do not denote any order or importance, but rather the terms first, second, etc. are used to distinguish one element from another.