Abstract:
A read writeable random accessible non-volatile memory module includes a printed circuit board with an edge connector that can be plugged into a socket of a printed circuit board. The read writeable random accessible non-volatile memory modules further include a plurality of read writable non-volatile memory devices.

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
       [0001]    This non-provisional United States (U.S.) patent application is a continuation application and claims the benefit of U.S. patent application Ser. No. 13/747,424 entitled NON-VOLATILE TYPE MEMORY MODULES FOR MAIN MEMORY filed on Jan. 22, 2013 by inventors Vijay Karamcheti et al., now allowed. 
         [0002]    U.S. patent application Ser. No. 13/747,424 is a divisional application and claims the benefit of U.S. patent application Ser. No. 12/831,206 entitled SYSTEMS AND APPARATUS FOR MAIN MEMORY filed on Jul. 6, 2010 by inventors Vijay Karamcheti et al., now issued as U.S. Pat. No. 8,364,867. U.S. patent application Ser. No. 13/747,424 is also a continuation application and claims the benefit of U.S. patent application Ser. No. 12/832,409 entitled METHODS FOR MAIN MEMORY WITH NON-VOLATILE TYPE MEMORY MODULES filed on Jul. 8, 2010 by inventors Vijay Karamcheti et al., now issued as U.S. Pat. No. 8,380,898. 
         [0003]    U.S. patent application Ser. No. 12/831,206 is a divisional application and claims the benefit of U.S. patent application Ser. No. 11/848,013 entitled SYSTEMS AND APPARATUS FOR MAIN MEMORY WITH NON-VOLATILE TYPE MEMORY MODULES, AND RELATED TECHNOLOGIES filed on Aug. 30, 2007 by inventors Vijay Karamcheti et al., now issued as U.S. Pat. No. 7,761,624. U.S. patent application Ser. No. 12/832,409 is a divisional application and claims the benefit of U.S. patent application Ser. No. 11/848,040 entitled METHODS FOR MAIN MEMORY WITH NON-VOLATILE TYPE MEMORY MODULES, AND RELATED TECHNOLOGIES filed on Aug. 30, 2007 by inventors Vijay Karamcheti et al., now issued as U.S. Pat. No. 7,761,625. 
         [0004]    U.S. patent application Ser. Nos. 11/848,013 and 11/848,040 claim the benefit of U.S. Provisional Patent Application No. 60/827,421 entitled SUBSTITUTION OF A PROCESSOR WITH A BUILT IN DRAM MEMORY CONTROLLER BY A NON-DRAM MEMORY CONTROLLER TO CONTROL ACCESS TO NON-DRAM TYPE MEMORY MODULES filed on Sep. 28, 2006 by inventor Kumar Ganapathy et al. 
     
    
     FIELD 
       [0005]    The document generally relates to memory controllers and memory modules. 
       BACKGROUND 
       [0006]    Some computing systems use dynamic random access memory (DRAM) integrated circuits in their main memory. DRAM integrated circuits (ICs) retain information by storing a certain amount of charge on a capacitor in each memory cell to store a logical one or alternatively, a logical zero. Over time, and because of read operations, the stored charge on the capacitor dissipates, in a process often referred to as leaking off. To preserve the stored charge on a DRAM capacitor, and thus maintain the ability of the DRAM to maintain its memory contents, the stored charge in the memory cell may be increased through refresh cycles, which sometimes are performed periodically. 
     
    
     
       BRIEF DESCRIPTIONS OF THE DRAWINGS 
         [0007]      FIG. 1  is a functional block diagram of a computer system with only DRAM DIMMS. 
           [0008]      FIG. 2A  is a functional block diagram of the computer system of  FIG. 1  upgraded with a memory controller to control non-DRAM memory DIMMS. 
           [0009]      FIG. 2B  is a functional block diagram of the computer system of  FIG. 1  upgraded with dual memory controllers to control both DRAM memory DIMMS and non-DRAM memory DIMMS. 
           [0010]      FIG. 3  is a functional block diagram of a non-DRAM memory module. 
           [0011]      FIG. 4  is a functional block diagram of an internet server coupled to the internet. 
           [0012]      FIG. 5  is a flow chart of a method for upgrading a computing system. 
           [0013]      FIGS. 6A-6B  are functional block diagrams of implementations of a buffer IC. 
           [0014]      FIG. 7  is a flow chart of a method for accessing main memory including a pluggable non-volatile memory module. 
       
    
    
     DETAILED DESCRIPTION 
       [0015]    In the following detailed description, numerous examples of specific implementations are set forth. However, implementations may include configurations that include less than all of or alternatives for the detailed features and combinations set forth in these examples. 
         [0016]    For similar memory capacity, dynamic random access memory (DRAM) integrated circuits (ICs) typically consume more power than non-volatile memory integrated circuits, particularly when data is read. Non-volatile memory integrated circuits typically do not require refresh cycles and thus conserve power. To reduce power consumption in system applications with a main memory, a non-volatile memory integrated circuit may be used in place of or as a supplement to a DRAM integrated circuit. 
         [0017]    Typically, a write access to non-volatile memory integrated circuits takes more time than a write access to DRAM integrated circuits. Some types of non-volatile memory integrated circuits, such as NOR FLASH EEPROM integrated circuits, may be configured with improved read access times (e.g., twice that of DRAM integrated circuits). In order to address differences between read and write performance, a data communication protocol may be used that accesses the non-volatile memory modules in a different manner than DRAM memory modules. 
         [0018]    The following paragraphs describe how a non-DRAM memory controller and non-volatile memory modules may be introduced into or integrated by a computer system. 
         [0019]    Referring now to  FIG. 1 , a functional block diagram of a computer system is illustrated with dynamic random access memory (DRAM) type of dual in-line memory modules (DIMMS). The computer system includes a multi-processor motherboard  100 . Mounted to the mother board  100  are a plurality of processor sockets  112 A- 112 N. Additionally mounted to the mother board  100  are dual in-line memory module (DIMM) sockets  115 A- 115 N in each of a plurality of memory channels  113 A- 113 N. The plurality of memory channels  113 A- 113 N are respectively coupled to each processor socket  112 A- 112 N as illustrated via groups of printed circuit board traces  125 A- 125 N. 
         [0020]    One or more processors  111 A- 111 N including built in DRAM type memory controllers  121 A- 121 N may or may not be plugged into the processor sockets  112 A- 112 N in any given system. For example, processor socket  112 B may be vacant without any processor plugged therein. 
         [0021]    Each processor socket  112 A- 112 N has one or more connections to the interconnect fabric  110  that includes printed circuit board trace groups  116 A- 116 N between the processor sockets  112 A- 112 N and the interconnect fabric (which may or may not include additional integrated circuits) but which also connects to the input/output (I/O) circuitry  118 . Groups of printed circuit board traces  125 A- 125 N in each memory channel  113 A- 113 N are coupled between the memory module sockets  115 A- 115 N and the processor sockets  112 A- 112 N. 
         [0022]    A packaged processor  111 A- 111 N includes one or more processing core elements (or execution units)  131  and one or more DRAM type memory controllers  121 A- 121 N. The packaged processor  111 A- 111 N may be plugged into any of the processor sockets  112 A- 112 N. The memory controller  121 A may furnish data to the processing core elements in the packaged processor  111 A, for example, from some DRAM DIMM  114 A- 114 N over one of the groups of printed circuit board traces  125 A- 125 N coupled to socket  112 A and (through the interconnect fabric  110 ) to other processors  111 B- 111 N in their respective sockets  112 B- 112 N. That is, the main memory formed by the plurality of memory channels  113 A- 113 N coupled to each processor  111 A- 111 N is a shared main memory  150  that is shared amongst the processors  111 A- 111 N which are plugged into the processor sockets  112 A- 112 N. 
         [0023]    The DIMM sockets  115 A- 115 N couple to a processor socket  112 A- 112 N through groups of PCB traces  125 A- 125 N. If a processor socket is vacant, the DRAM DIMMS  114 A- 114 N are probably not plugged into DIMM sockets  115 A- 115 N of the one or more memory channels coupled to the vacant processor socket. That is, the DIMM sockets  115 A- 115 N are likely to be vacant if the processor socket  112  to which they couple is vacant. 
         [0024]    As discussed previously, there are groups of printed circuit board traces  125 A- 125 N in each memory channel  113 A- 113 N that are coupled between the memory module sockets  115 A- 115 N and the processor sockets  112 A- 112 N. With a processor  111 A plugged into the corresponding processor socket  112 A and the DRAM memory modules  114 A- 114 N plugged into the memory module sockets  115 A- 115 N, the groups of printed circuit board traces  125 A- 125 N interconnect the processor  111 A with the memory modules  114 A- 114 N. Some of the groups of printed circuit board (PCB) traces  125 A- 125 N between the processor socket and the memory module sockets are shared amongst all of the memory modules sockets in that channel. Some of the groups of printed circuit board traces  125 A- 125 N between the processor socket  112 A and the memory module sockets  115 A- 115 N are not shared amongst all. There may be one or more printed board traces in the groups of printed circuit board traces  125 A- 125 N that are uniquely routed between the processor socket and the memory module sockets  115 A- 115 N. For example, a printed circuit board trace may be dedicated to providing a route between the processor socket  112  and the first memory module socket  115 A, without being routed to the other memory module sockets  115 B- 115 N in the memory channel. 
         [0025]    The DRAM DIMMs  114 A- 114 N plugged into the memory module sockets  115 A- 115 N are printed circuit boards including a plurality of DRAM type memory integrated circuit chips mounted to the printed circuit board. The entirety or a subset of the plurality of DRAM type memory integrated circuit chips on a DIMM are accessed in parallel by the memory controller to read data from or write data to the memory. 
         [0026]    Referring now to  FIG. 2A , a functional block diagram of the computer system of  FIG. 1  is illustrated as having been upgraded with a memory controller to control non-DRAM memory DIMMS, such as non-volatile memory modules. These non-DRAM type memory modules may help increase the memory capacity and/or reduce the power consumption of the system. 
         [0027]    As discussed previously, one or more processor sockets  112 A- 112 N on a mother board may be vacant. The vacancy in the processor socket may be from a user pulling out the processor from that socket. That is, the processor is unplugged by a user to generate the vacant processor socket. Alternatively, a processor may have not been plugged into the processor socket—it was originally vacant. Moreover, if the memory channels to be upgraded are not vacant of DRAM type memory modules, a user may unplug the DRAM-type memory modules to make all the memory module sockets in a memory channel available for non-DRAM type memory modules. 
         [0028]    In  FIG. 2A , the upgraded mother board  200  is illustrated. The upgraded mother board  200  has had one or more non-DRAM type memory controllers  212  plugged into a respective one or more processor sockets  112 . In  FIG. 2A , the non-DRAM type memory controller  212  is plugged into a previously vacant processor socket  112 B so that the one or more memory channels  213 A- 213 N coupled thereto can be used with memory modules having different types of memory integrated circuits other than DRAM integrated circuits to upgrade the shared main memory  150 ′. 
         [0029]    The one or more memory channels  213 A- 213 N are the memory channels used by the non-DRAM memory controller  212  to communicate to the non-DRAM memory modules  214 A- 214 N. But for the non-DRAM memory modules  214 A- 214 N, the structure of the one or more memory channels  213 A- 213 N is substantially similar to the structure of the memory channels  113 A- 113 N using the same groups of printed circuit board traces  125 A- 125 N and sockets  115 A- 115 N as before. 
         [0030]    Each of the one or more memory channels  213 A- 213 N includes a plurality of memory module sockets  115 A- 115 N with non-DRAM memory modules  214 A- 214 N plugged into the plurality of memory module sockets  115 A- 115 N. The groups of printed circuit board traces  125 A- 125 N in each of the one or more memory channel  213 A- 213 N are coupled between the memory module sockets  115 A- 115 N and the processor socket  112 B. 
         [0031]    While the structure of the groups of PCB traces (also referred to as “interconnects” herein)  125 A- 125 N in each upgraded memory channel  213 A- 213 N are the same, the signals propagating over one or more traces of the groups of PCB traces  125 A- 125 N may differ to control the non-DRAM type memory modules  214 A- 214 N. That is, the meaning of some signal lines in the pre-existing interconnections (e.g., groups of PCB traces  125 A- 125 N) between the processor socket  112 B and the memory module sockets  115 A- 115 N in each upgraded memory channel  213 A- 213 N may be changed to appropriately control the non-DRAM type memory modules  214 A- 214 N. 
         [0032]    A data strobe signal line used to access DRAM memory modules may change to be a feedback status control signal line that can be communicated from a non-volatile memory module to the memory controller to alleviate the non-deterministic nature of the erase and write operations in the non-volatile memory modules. With a feedback status control signal, the memory controller can avoid constantly polling the non-volatile memory module as to when an erase or write operation is completed. 
         [0033]    For example, data strobe signals DQS 13 , DQS 14 , DQS 15 , DQS 16  respectively change to status signals RY/BY_N_R 1 D 0 , RY/BY_N_R 1 D 1 , RY/BY_N_R 1 D 2 , RY/BY_N_R 1 D 3  when a non-volatile memory module is being accessed within a memory module socket of a memory channel. The data strobe signals DQS 13 , DQS 14 , DQS 15 , DQS 16  are used to clock data out each memory module in a DRAM memory channel. The RY/BY_N_R 1 D 0 , RY/BY_N_R 1 D 1 , RY/BY_N_R 1 D 2 , RY/BY_N_R 1 D 3  signals are status signals for rank one memory of each of four DIMM modules/sockets that are in the memory channel. These status signals are fed back and coupled to the heterogeneous memory controller to more efficiently access the non-volatile memory module. Each status signal indicates whether or not a rank of memory in a memory module is busy or ready for another access to alleviate the non-deterministic nature of erase and write operations to non-volatile memory modules. 
         [0034]    While one or more memory channels  213 A- 213 N are upgraded to use non-DRAM type memory modules  214 A- 214 N and the associated processor socket  112 B is filled by a non-DRAM memory controller  212 , the structure of the mother board  200  is similar to the structure of mother board  100 . The prior discussion of elements of the mother board  100  having the same reference numbers on mother board  200  are incorporated here by reference for reasons of brevity. 
         [0035]      FIG. 2B  is a functional block diagram of an upgraded computer system with a dual memory controller  212 ′ plugged into a processor socket  112 B of the mother board  200 ′. The dual memory controller  212 ′ includes a non-DRAM memory controller  212  and a DRAM memory controller  121  co-packaged together to respectively control access to non-DRAM memory DIMMS  214 A- 214 N plugged into sockets  115 A- 115 N of the memory channel  213 N and DRAM memory DIMMS  114 A- 114 N plugged into sockets  115 A- 115 N of the memory channel  113 N. The dual memory controller  212 ′ plugs into the processor socket  112 B and couples to sockets  115 A- 115 N in the memory channel  213 N by printed circuit board traces  125 A and to sockets  115 A- 115 N in the memory channel  113 N by printed circuit board traces  125 N. 
         [0036]      FIG. 2B  further illustrates a functional block diagram of an upgraded computer system with a processor  211  having an execution unit (EU)  131 , integrated DRAM memory controller (IMC)  121 ′, and integrated non-DRAM memory controller (IMC)  222 ′. The processor  211  is plugged into a processor socket  112 N of the mother board  200 ′ and coupled to sockets  115 A- 115 N in the memory channel  113 N by printed circuit board traces  125 A and to sockets  115 A- 115 N in the memory channel  213 N by printed circuit board traces  125 N. The integrated non-DRAM memory controller  222 ′ controls access to non-DRAM memory DIMMS  214 A- 214 N plugged into sockets  115 A- 115 N in the memory channel  213 N. In one implementation, the integrated non-DRAM memory controller  222 ′ is a non-volatile memory controller and the non-DRAM memory DIMMS  214 A- 214 N are non-volatile memory DIMMS. The integrated DRAM memory controller  121 ′ controls access to DRAM memory DIMMS  114 A- 114 N plugged into sockets  115 A- 115 N in the memory channel  113 N. 
         [0037]    Referring now to  FIG. 5 , a flow chart of a method for upgrading a computing system is illustrated. 
         [0038]    At block  502 , a non-DRAM memory controller is plugged into a processor socket normally reserved for a processor. If a processor was plugged into the processor socket, the processor may be removed prior to plugging in the memory controller. The memory controller plugged into the processor socket is used to control read and write accesses to non-DRAM memory modules (e.g., memory modules of a type other than DRAM memory modules) in the computing system. In one configuration, the non-DRAM memory modules is non-volatile memory modules. 
         [0039]    At block  504 , a plurality of non-DRAM memory modules are plugged into memory sockets normally reserved for DRAM memory modules. If DRAM memory modules were plugged into these memory sockets, they would be removed prior to plugging in the plurality of non-DRAM memory modules into the memory sockets. The memory sockets are coupled to the processor socket by pre-existing groups of printed circuit board traces so that the memory controller plugged into the processor socket can control read and write accesses to non-DRAM memory modules in the computing system. 
         [0040]    In one configuration, the non-DRAM memory modules are non-volatile memory modules (e.g., memory modules of a type other than volatile memory modules). For instance, in one particular example, the non-volatile memory mounted to the non-volatile memory module is a NOR flash electrically erasable programmable read only memory (EEPROM). 
         [0041]    At block  506 , the non-DRAM memory modules are accessed via the memory controller in the processor socket by using a data communication protocol to access non-DRAM memory modules. The data communication protocol to access non-DRAM memory modules may be specific to the type of non-DRAM memory module plugged into the memory module sockets and may differ from the data communication protocol used to access DRAM memory modules. The data communication protocol to access non-DRAM memory modules is communicated over the groups of pre-existing printed circuit board traces and through the sockets normally used to access DRAM type memory modules. In one configuration, the non-DRAM memory modules are non-volatile types of memory modules. For example, data strobe signals may change to status signals when a non-volatile memory module is being accessed within a memory module socket of a memory channel. 
         [0042]    Referring now to  FIG. 3 , a functional block diagram of a non-DRAM memory module  214  is illustrated. The non-DRAM memory module  214  may be plugged into the memory module sockets  115 A- 115 N of the one or more upgraded memory channels  213 A- 213 N. 
         [0043]    In one configuration, the non-DRAM memory module  214  is a non-volatile memory module. In this case, the non-DRAM memory controller  212  is a non-volatile memory controller. In particular, for example, the non-volatile memory module may include at least one NOR-gate flash electrically erasable programmable read only memory (EEPROM) integrated circuit. 
         [0044]    The non-DRAM memory module  214  includes a printed circuit board  300  having pads of edge connectors  301  (one on each side for a DIMM) formed thereon, a plurality of non-DRAM memory chips  302 A- 302 N, and a plurality of support chips  303 A- 303 N. In another configuration, the plurality of support chips may be co-packaged with some of the non-DRAM memory chips  302 A- 302 N into one IC package. The memory module  214  further includes a plurality of interconnects (e.g., package interconnects or printed circuit board traces)  304 A- 304 N and  306 A- 306 L formed on the PCB  300 , providing a coupling between the non-DRAM memory chips  302 A- 302 N and the support chips  303 A- 303 N, and between the support chips  303 A- 303 N and the pads of the edge connectors  301 . 
         [0045]    In one configuration, the memory module  214  is a dual in-line memory module (DIMM) and the printed circuit board (PCB)  300  is a DIMM PCB. The non-DRAM memory chips  302 A- 302 N may be NOR FLASH EEPROM integrated circuit chips or some other kind of non-DRAM memory integrated circuit chips, such as non-volatile memory integrated circuit chips. 
         [0046]    The plurality of support chips  303 A- 303 N may be used to buffer addresses, and/or multiplex and de-multiplex data to and from the non-DRAM memory chips  302 A- 302 N. The plurality of support chips  303 A- 303 N may also be referred to herein as a plurality of buffer integrated circuits  303 . The plurality of support chips  303 A- 303 N may be co-packaged with some of the non-DRAM memory chips  302 A- 302 N. 
         [0047]    Referring now to  FIG. 6A  in accordance with one configuration, each of the plurality of buffer integrated circuits  303  includes a many-to-one bus multiplexer  602  and a one-to-many bus demultiplexer  604 . The many-to-one bus multiplexer  602  is used to write data onto a data bus at the edge connection  301 . The one-to-many bus demultiplexer  604  is used to read data from the data bus at the edge connection  301  onto one of many data buses  304 A- 304 N coupled to the memory integrated circuits  302 A- 302 N. 
         [0048]    Referring now to  FIG. 6B , in accordance with another configuration, each of the plurality of buffer integrated circuits  303 ′ instead includes a cross-bar switch  606  coupled between the plurality of data busses  304 A- 304 N connected to the memory integrated circuits  302 A- 302 N and a data bus at the edge connection  301 . The cross bar switch  606  is used to write data onto the data bus at the edge connection  301  from the memory integrated circuits  302 A- 302 N. The cross bar switch  606  is used further to read data from the data bus at the edge connection  301  and couple the data onto one of data buses  304 A- 304 N connected to the memory integrated circuits  302 A- 302 N. 
         [0049]    Referring now to  FIG. 4 , a block diagram of an internet server  400  and a remote client  401  coupled to the internet  402  is illustrated. The internet server  400  includes the motherboard  200  that has been upgraded to include non-volatile memory modules plugged into the memory module sockets of one or more memory channels. 
         [0050]    An example of the use of non-volatile memory modules in main memory is now described. The remote client  401  executes a search query  410  against a search engine running on the internet server  400  to search for data. In this case, the main memory  412  on the mother board  200  may be more often read that it is written. Non-volatile memory modules may be plugged into one or more sockets of one or more memory channels. With the mother board  200  upgraded to include non-volatile memory modules in its main memory  412 , power is conserved over that of a main memory solely having DRAM memory modules. 
         [0051]    Referring now to  FIG. 7 , a method in a server with a main memory including a pluggable non-volatile memory module is illustrated. 
         [0052]    At block  702 , a software application is executed with a processor. 
         [0053]    At block  704 , the main memory of the server is randomly accessed by the software application. As previously mentioned, the main memory includes a pluggable non-volatile memory module. The pluggable non-volatile memory module may have a read access time substantially similar to a DRAM memory module (e.g., approximately twice the read access time of a DRAM memory module). However, the write access time of the pluggable non-volatile memory module may differ from the write access time of a DRAM memory module. 
         [0054]    At block  706 , information is written into the pluggable non-volatile memory module. The information that is written into the pluggable non-volatile memory module may include data and/or code. Writing information into the pluggable non-volatile memory module may be in response to executing the software application. In one configuration, the software application writes the information into the memory module. 
         [0055]    In this configuration, the software application may be a search engine to search for data on a server, such as the internet server previously mentioned. 
         [0056]    When implemented in software, the memory controller may include code segments configured to perform the necessary tasks. The program or code segments can be stored in a processor readable medium or transmitted by a computer data signal embodied in a carrier wave over a transmission medium or communication link. The “processor readable medium” may include any medium that can store or transfer information. Examples of the processor readable medium include an electronic circuit, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, a fiber optic medium, a radio frequency (RF) link, etc. The computer data signal may include any signal that can propagate over a transmission medium such as electronic network channels, optical fibers, air, electromagnetic, RF links, etc. The code segments may be downloaded via computer networks such as the Internet, Intranet, etc. 
         [0057]    While certain configurations are described and shown in the accompanying drawings, it is to be understood that such configurations are merely illustrative of and not restrictive of the scope of the disclosure. Other implementations are within the scope of the following claims. For example, the memory modules and the memory sockets have been described as being dual in-line memory modules (DIMM) and DIMM sockets. However, the memory modules and memory sockets may have other types of form factors such as single in-line memory modules (SIMM), for example.