VIRTUAL GROUPING OF MEMORY

The present disclosure includes identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determining a memory segment size from the capacities of the memory modules; identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

TECHNICAL FIELD

This disclosure relates to virtual memory grouping. In particular, it relates to virtual memory grouping across memory channels.

BACKGROUND

Main memory in a system can be setup in various physical configurations to optimize power consumption and performance. Memory may be mirrored for redundancy to mitigate DIMM failures. Fully mirrored memory, while being fully redundant, would cut the capacity of the memory in half vs. non-mirrored memory. However, memory may be selectively mirrored so that a portion of the memory is mirrored, such as that containing information critical to program operation, and the remainder of the memory is non-mirrored.

SUMMARY

The present disclosure includes a method, comprising: identifying, in a memory system, a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determining a memory segment size from the capacities of the memory modules; identifying a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and creating a virtual group that includes the first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

The present disclosure further includes a method, comprising identifying a capacity for each of a plurality of memory modules for a first memory channel and a second memory channel in a memory system; determining a first portion of memory modules and a second portion of memory modules from the capacities of the memory modules for each of the first and second memory channels; creating a first virtual group that includes the first portion of the memory modules from the first memory channel and the first portion of memory modules from the second memory channel; and creating a second virtual group that includes the second portion of the memory modules from the first memory channel and the second portion of memory modules from the second memory channel.

The present disclosure further includes a computer system, comprising service processor firmware configured to identify a capacity for each of a plurality of memory modules for a first memory channel having a first amount of memory and a second memory channel having a second amount of memory; determine a memory segment size from the capacities of the memory modules; identify a first memory segment of the memory segment size for the first memory channel and a second memory segment of the memory segment size for the second memory channel; and create a virtual group that includes first memory segment and the second memory segment and that uses less than the entire first amount of memory from the first memory channel.

DETAILED DESCRIPTION

Memory modules in a memory system may be organized into groups along memory channels behind memory controllers and memory buffers to match the internal throughput of the system bus of a computer system. For example, a memory controller may have multiple memory buffers, each memory buffer having multiple memory channels leading to one or more DIMMs. Memory modules may be populated into memory channels in groups of similar sizes of memory to enable memory minoring and memory interleaving, while unequal sizes of memory may be populated to form different groups. An operating system or hypervisor may allocate memory for an application as a physical group across multiple memory modules so that those memory modules may be accessed at the same time. The memory map for a particular physical group will likely be a single contiguous space.

Accessing memory according to the physical organization of the memory behind the memory controller may compromise memory throughput and affinity. Access to the memory groups may be limited by the physical characteristics of the memory, as the grouping is performed along physical boundaries. Mirrored data may be allocated to physical memory groups of equal sizes behind a memory buffer, which may reduce the memory flexibility and utilization of the system. Memory grouped according to a particular bandwidth may not be optimized for the way in which the memory is actually utilized by applications.

According to embodiments of the disclosure, memory may be virtually grouped and accessed to improve memory throughput and affinity. Memory segments of an identified memory size may be grouped virtually according to considerations such as the minimum memory module capacity, application priority, criticality, or memory access characteristics, and performance data for various memory module configurations. Virtual groups accessed by multiple memory controllers may scale the throughput high, as a greater number of memory modules may be accessed in parallel. Virtual groups of equal memory segments may be created from the same physical memory group or from physical memory groups of different sizes and behind multiple memory controllers. Memory presented as and allocated through virtual groups may be accessed symmetrically and diffusely through operations such as memory interleaving.

Performance and configuration information collected from memory in the virtual groups may be utilized by systems and users to improve physical memory access and application performance. The performance of memory in various virtual configurations may be used to create improved physical groupings of the memory modules through memory configuration and population rules. Data may be allocated to a particular memory group based on the priority of the application's data or the memory access characteristics of the application. Virtual groups may also support selective memory mirroring by identifying data to be mirrored and configuring virtual memory groups accordingly.

Method

FIG. 1is a flowchart of a method for creating virtual groups of memory, according to embodiments of the disclosure. The memory capacities of one or more memory modules in the memory system may be determined, as in100. Memory from the memory modules may be grouped into virtual groups, as in110. Once the virtual groups are created, memory from the virtual groups may be allocated to applications, as in120. The memory modules having memory in a virtual memory group may be monitored for performance, power consumption, and other usage statistics, as in130.

To determine memory capacities of the memory modules, as in100, the system may perform an initial program load, as in101. Once the system has been initialized, vital product data for the memory modules may be collected, as in102. Vital product data may include hardware information, such as serial number and capacity. The vital product data may be determined for the system or collected from a vital product data storage location on the system. The collection of vital product data may be performed by a system such as the host system of the memory system, a remote system, or a field replaceable unit.

Memory from the memory modules may be grouped into virtual memory groups, as in110. The capacities of the memory modules may be scanned and evaluated from the vital product data of the memory modules, as in111. A memory segment size (or portion) may be determined from the scanned memory module capacities, as in112, according to the memory access characteristics of the memory module. In certain embodiments, the memory segment size may be the smallest independently operated memory unit for one or more physical memory groups, such as the smallest memory module size or a rank size in a memory module. The memory segment size may be evaluated for memory modules across two or more memory channels. For example, if a first memory channel has two 1 GB DIMMs, a second memory channel has four 1 GB DIMMs, the memory segment size may be the smallest memory module size in common between the two memory channels. In this example, 1 GB may be the memory segment size; alternately, 2 GB may be the memory segment size, as both groups have at least two 1 GB DIMMs.

Memory segments of the memory segment size may be identified for memory modules in the memory system, as in113, and used to create a virtual memory group, as in114. In the example above, the two 1 GB DIMMs from the first channel and two 1 GB DIMMs from the second channel may be grouped into a first group. If a third memory channel has two 1 GB DIMMs, the remaining two 1 GB DIMMs from the second memory channel may be grouped with the two 1 GB DIMMs from the third memory channel.

The virtual groups may contain memory from memory channels that is less than the total memory capacity of the memory channels. In some embodiments, a memory buffer or controller having two memory channels may have memory grouped into two or more virtual groups, each group containing memory from both memory channels. In other embodiments, a virtual group may have memory from memory channels of different memory capacity connected to two different controllers. In other embodiments, a virtual group may have memory from memory channels having different memory capacities.

Creation of virtual groups may be informed by performance information of the memory modules, as in131. For example, if an executing processor in a distributed shared memory system has greater processor affinity to a certain group of DIMMs, those DIMMs may be used for the virtual group assigned to an application executed on that processor.

Once the virtual groups are created, memory from the virtual groups may be allocated to applications, as in120. The application's priority, criticality, or memory access characteristics may inform the operating system or hypervisor as to which virtual group may be allocated to the application, as in121. For example, if an application is known to have fairly linear memory access or is non-critical, a virtual group providing deeper memory access or limited to fewer memory modules may be used, so that memory for applications with more diffuse memory access or requiring greater fault tolerance may use virtual groups having a greater usage of memory modules. The virtual group may be targeted by or allocated to the application, as in122. The application's performance may be monitored to determine the appropriateness of the virtual group for that application.

The memory modules in a virtual group may be monitored for performance and power consumption, as in130. Information of the memory modules, such as performance and power consumption, may be collected, as in131. The information may be used as feedback to determine memory module configuration options, as in132. These memory module configuration options may be used to physically configure or populate the memory modules so as to be improve memory access, such as by increasing memory affinity or optimizing throughput. These configuration options may be used as DIMM plug rules when the DIMMs are off-line.

System

FIG. 2Ais a diagram of a memory system, according to embodiments of the disclosure. A processor201may be connected to one or more memory controllers202. The memory controller202may be part of the processor itself, as shown inFIG. 2, or on a separate chip. In this embodiment, the processor contains a memory controller A202A and a memory controller B202B.

Each memory controller may support one or more memory buffers203. The memory buffer203may be part of the memory controller202or on a separate chip. Shown here, memory controller A202A is connected to memory buffer A203A, and memory controller B202B is connected to memory buffer B203B. Each memory controller202or memory buffer203may be connected to one or more memory modules204through one or more memory channels209. Shown here, memory buffer A203A is connected to memory module A204A and memory module C204C through memory channel A209A, and memory buffer B203B is connected to DIMM B204B through memory channel B. The performance of the memory modules204and the processor201may be monitored by performance monitor logic207.

The memory system may contain a service processor205. The service processor205may be a hardware or firmware unit configured for virtual memory management. The service processor205may be configured to create virtual groups206of memory from the memory modules204of different memory channels209. The service processor205may receive feedback from the performance monitor logic207as to performance of memory modules204in the virtual groups206and applications assigned to certain virtual groups206. This performance information may be used by a hypervisor208to assign applications to virtual groups206or to provide configuration options for physical configuration of the DIMMs204.

FIG. 2Bis a diagram of memory allocation in a memory system, according to embodiments of the disclosure. Service processor firmware214may create virtual memory groups from memory215. The memory215may include memory modules from across a memory system, including memory modules from different physical groups behind different processors, memory controllers, and memory buffers. An operating system212, hypervisor213, or application211may allocate memory215from the memory groups to the application211. The service processor firmware214may present the memory215to the hypervisor213or operating system212as virtual groups.

Example

FIG. 3is an example diagram of a memory system for use in the examples diagrams ofFIGS. 4,5A,5B, and5C, according to embodiments of the disclosure. In this example, a processor300has eight memory controllers, each connected to multiple DIMMs through one or more memory channels (and optionally one or more memory buffers, not shown). The memory contained behind each memory controller may be referred to as the memory controller segment (MCS) for the memory controller. Memory controller 1301controls DIMM 1A311A and DIMM 1B311B as MCS 1; memory controller 2302controls DIMM 2A312A and DIMM 2B312B as MCS 2; memory controller 3303controls DIMM 3A313A and DIMM 3B313B as MCS 3; memory controller 4304controls DIMM 4A314A and DIMM 4B314B as MCS 4; memory controller 5305controls DIMM 5A315A, DIMM 5B315B, DIMM 5D315C, and DIMM 5D315D as MCS 5; memory controller 6306controls DIMM 6A316A, DIMM 6B316B, DIMM 6C316C, and DIMM 6D316D as MCS 6; memory controller 7307controls DIMM 7A317A, DIMM 7B317B, DIMM 7C317C, DIMM 7D317D, DIMM 7E317E, DIMM 7F317F, DIMM 7G317G, and DIMM 7H317H as MCS 7; and memory controller 8308controls DIMM 8A318A, DIMM 8B318B, DIMM 8C318C, DIMM 8D318D, DIMM 8E318E, DIMM 8F318F, DIMM 8G318G, and DIMM 8H318H as MCS 8. For this example, each DIMM may be equal to 1 gigabyte (GB) of memory for simplicity of explanation.

FIG. 4is an example diagram of physical and virtual groupings of memory segments for the system ofFIG. 3, according to embodiments of the disclosure. For a physical grouping, memory from a memory controller segment (MCS) may be grouped and accessed with memory from other memory controller segments having the same size and configuration to form physical groups for memory access. In this example, three physical groups may be formed. Group 1 may contain 8 GB of total memory, comprising the 2 GB of memory behind each of the four memory controllers 1-4. Group 2 may contain 8 GB of total memory, comprising the 4 GB of memory behind memory controllers 5 and 6. Group 3 may contain 16 GB of total memory behind memory controllers 7 and 8.

When the memory is grouped virtually, memory access may not be constrained to physical memory groups or memory controller segments, and a greater number of memory access configurations may be created. By virtually grouping smaller, equal segments of memory across memory channels, a greater number of memory buffers and controllers may access memory at a time, reducing access time to the memory. In this example, virtual groups having the same size as the physical groups discussed above may be formed, but with different memory access configurations.

For virtual grouping 1, 2 GB of memory from each memory controller 1-8 may be grouped as memory segments into group 3 to form a 16 GB group; 2 GB of memory from each memory controller 5-8 may be grouped into group 2 to form an 8 GB group; and 4 GB of memory from each memory controller 7 and 8 may be grouped into group 1 to form an 8 GB group.

For virtual grouping 2, 2 GB of DIMMs from each memory controller 1-4 may be grouped as memory segments into group 1 to form an 8 GB group; 4 GB of memory from each memory controller 7 and 8 may be grouped into group 2 to form an 8 GB group; and 4 GB of memory from each memory controller 5-8 may be grouped into group 3 to form a 16 GB group.

FIG. 5A,FIG. 5B, andFIG. 5Care diagrams of the physical and virtual groupings of memory fromFIG. 4, according to embodiments of the disclosure.FIG. 5Ais a diagram of memory from memory controller segments in physical groups for the physical grouping, according to embodiments of the disclosure. As discussed above, each physical group contains memory from memory controller segments of the same size and configuration.

FIG. 5Bis a diagram of memory from memory controller segments in virtual groups for the virtual grouping 1, according to embodiments of the disclosure. Memory access to group 3 is spread across 8 memory controllers, which may allow for greater throughput. Processor operations that require accessing multiple segments of information simultaneously, such as parallel processing, or containing critical data may best utilize the memory access characteristics of virtual group 3. Critical process operations requiring similar fast access or high availability but for a smaller 8 GB capacity may best utilize the memory access characteristics of virtual group 2. Non-critical applications or application that may better utilize access to the same DIMM for information, such as deep or linear computing operations, may utilize virtual group 1. The memory segments from each memory controller segment are similarly sized, and may be used for symmetrical processes such as memory mirroring and interleaving.

For memory access administered with a lower overhead or for fewer critical or parallel access operations, a different virtual grouping configuration may be used.FIG. 5Cis a diagram of memory from memory controller segments in virtual groups for the virtual grouping 2, according to embodiments of the disclosure. Applications utilizing faster memory access for parallel processes or performing more critical operations may be assigned group 1. Applications involving non-critical storage may be assigned group 2. Operations involving linear memory access or minoring, such as database storage, may be assigned group 3, as access is spread across memory controllers. The 16 GB group capacity of group 3 is divided among four memory controllers for redundancy, two of which may contain original data and two of which may contain copy data.