Conditional memory spreading for heterogeneous memory sizes

A processor implemented method for spreading data traffic across memory controllers with respect to conditions is provided. The processor implemented method includes determining whether the memory controllers are balanced. The processor implemented method includes executing a conditional spreading with respect to the conditions when the memory controllers are determined as unbalanced. The processor implemented method includes executing an equal spreading when the memory controllers are determined as balanced.

BACKGROUND

The disclosure relates generally to conditional memory spreading for heterogeneous memory sizes.

In a symmetric multiprocessing computer system with processors and input/output (I/O), cache and memory hierarchy is designed to provide low latency and high bandwidth. When it comes to volatile, dynamic random access memory (DRAM) (e.g. dual in-line memory modules (DIMMs)), there are several factors to consider when designing the cache and memory hierarchy: intrinsic memory latency and performance (e.g. unloaded latency), queuing, and background operations (e.g., refresh/scrubbing/periodics). Queuing can adversely affect memory bandwidth. To minimize queuing, memory spreading can be included in the cache and memory hierarchy design.

In general, the symmetric multiprocessing computer system utilizes internal memory resources by spreading equally data traffic to portions of the internal memory resources. While evenly spreading data traffic across memory controllers often works for uniform DIMM sizes, when mixed DIMM sizes are used, this contemporary equal spreading can cause an unbalanced use of the internal memory resources, as each portion of the internal memory receives the same amount of data traffic regardless of availability and/or capability. The unbalanced use of the internal memory resources often results in uneven and inconsistent workload behaviors of the internal memory resources, along with pathological queuing issues as internal memory resources fill up that degrade memory performance.

SUMMARY

According to one or more embodiments, a processor-implemented method for spreading data traffic across memory controllers with respect to conditions is provided. The processor-implemented method includes determining whether the memory controllers are balanced. The processor-implemented method includes executing a conditional spreading with respect to the conditions when the memory controllers are determined as unbalanced. The processor-implemented method includes executing an equal spreading when the memory controllers are determined as balanced.

According to one or more embodiments, a processor-implemented method can be implemented in a computer program product and/or a system.

DETAILED DESCRIPTION

In view of the above, embodiments disclosed herein may include system, method, and/or computer program product (herein system) that spreads data traffic across memory controllers with respect to one or more conditions (i.e., conditional spreading). By utilizing conditional spreading over contemporary equal spreading, the system enables each memory control unit to receive an amount of use with respect to the one or more conditions, thereby eliminating the unbalanced memory use by the contemporary equal spreading that causes performance degradation as noted herein.

Embodiments described herein are necessarily rooted in a processor and memory of the system to perform proactive operations to overcome problems specifically arising in the realm of memory spreading in a heterogeneous environment (e.g., these problems include the unbalanced memory use, resulting in unwanted performance costs and expenses). Thus, the technical effects and benefits of conditional spreading include higher efficiencies in memory use and a reduction in memory queuing.

Turning now toFIG. 1, a process flow100is generally shown in accordance with an embodiment. The process flow100is an operational example of conditional spreading in the system described herein. The system can be an electronic, computer framework comprising and/or employing any number and combination of computing device and networks utilizing various communication technologies, as described herein. The system can be easily scalable, extensible, and modular, with the ability to change to different services or reconfigure some features independently of others.

The system can comprise a memory. The memory can be any level of a memory hierarchy, such as memory controllers, memory control units, ranks, banks, etc. Herein for ease of explanation, the system is described with reference to memory controllers, but is not limited thereto.FIG. 2depicts a table200providing seven possible memory configurations for the system in accordance with one or more embodiments. The seven possible memory configurations are shown in table200as Memory Configurations1-7, as noted in the first column. Each of these Memory Configurations1-7further provides a memory size with respect to four memory controllers (MC-1, MC-2, MC-3, and MC-4, as noted in the header row columns two through four).

The process flow100begins at decision block110, where the system determines if the memory controllers are balanced. The memory controllers are considered balanced when an available space is the same for each of the memory controllers. If the memory controllers are balanced, the process flow100proceeds to block115(as shown by the YES arrow). At block115, the system spreads data traffic evenly across the memory controllers.

If the memory controllers are unbalanced, the process flow100proceeds to block120(as shown by the NO arrow). For example, utilizing the Memory Configuration1ofFIG. 2, the system can determine that MC-2is providing 64 gigabytes, MC-3is providing 32 gigabytes, and MC-4is providing 32 gigabytes (note that MC-1is not providing memory to Memory Configuration1). Because MC-2is providing twice as much memory as MC-3and MC-4, the memory controllers are unbalanced.

At block120, the system spreads the data traffic conditionally across the memory controllers (e.g., with respect to one or more conditions). The one or more conditions can include spreading memory proportionally to a memory installed (e.g., proportional memory spreading), proportionally to a remaining available memory (e.g., available memory spreading), and/or with job priority consideration (e.g., job memory spreading).

An example of the proportional memory spreading includes when two memory controllers (i.e., first and second) are installed in the system. The first memory control unit is twice the size of the second memory control unit. Then, the first memory control unit proportionally receives twice as much as traffic as the second memory control unit. Turning now toFIG. 3, a table300is shown depicting a comparison between equal spreading and conditional spreading operations in accordance with one or more embodiments. The table300particularly shows the Memory Configuration1ofFIG. 2with respect to equal spreading and conditional spreading operations.

Regarding the equal spreading operation, the data traffic is interleaved to the memory controllers MC-2, MC-3, and MC-4(in such a way that the systems achieves as much spreading as it can while it can). Under the Memory Configuration1ofFIG. 2and using equal spreading memory allocation as shown inFIG. 3, the memory controllers MC-2, MC-3, and MC-4each receive memory segment allocations evenly distributed as shown by assignments of memory segments A, B, C, D, E, F, G, H, and I. Assuming an even data traffic distribution across memory segments, then each MCU receives 33% of the data traffic. However, once the memory controllers MC-3and MC-4are filled, then the memory controller MC-2receives 100% of additional memory segment allocations and also the corresponding data traffic. The data traffic sent to memory controller MC-2then can experience extreme latency.

Regarding the conditional spreading operation, the memory controller MC-2is twice as large (64 gigabytes) as the memory controllers MC-3and MC-4(32 gigabytes each). Given this 2:1 ratio, the system can proportionally spread two jobs to the memory controller MC-2for every one job sent to each of the memory controllers MC-3and MC-4(i.e., proportionally provide data traffic distribution). In turn, the memory controllers MC-2, MC-3, and MC-4consistency fill up at the same rate without a latency issue as seen when the memory controller MC-2receives 100% during equal spreading operation.

An example of the available memory spreading includes when two memory controllers (i.e., first and second) are installed in the system. The first memory control unit is three times the size of the second memory control unit. A third of the first memory control unit is unavailable due to being preassigned. The second memory control unit is available. Then, the first memory control unit proportionally receives twice as much as traffic as the second memory control unit because the available amount of the first memory control unit is twice the size of the available amount of the second memory control unit. The available memory spreading can be considered an embodiment that analyses remaining memory versus total memory. Note that as memory fills up and/or becomes available, the system can reevaluate the proportions to balance the rate of data traffic distribution.

An example of the job memory spreading includes a maximum spreading for high performance partitions and proportional spreading on lower performance partitions. Turning now toFIG. 4, a process flow400is generally shown in accordance with an embodiment. The process flow400is an operational example of job memory spreading in the system described herein. In this example, the system can include memory control units MCU-1, MCU-2, MCU-3, and MCU-4, some or all of which may have installed memory.

The process flow400begins at block401, where a new assignment is received by the system. At decision block405, the system determines if the memory control units MCU-1, MCU-2, MCU-3, and MCU-4have installed memory, and if so, are balanced. If the memory control units MCU-1, MCU-2, MCU-3, and MCU-4which have installed memory are balanced, the process flow400proceeds to decision block410(as shown by the YES arrow). At block410, the system spreads data traffic evenly across the memory control units MCU-1, MCU-2, MCU-3, and MCU-4.

Returning to decision block405, if the memory control units MCU-1, MCU-2, MCU-3, and MCU-4which have installed memory are not balanced, the process flow400proceeds to block415(as shown by the NO arrow). At block415, the system can spread the new assignment based on a priority of the new assignment. In a non-limiting embodiment, the system can utilize a multi-tiered priority structure. For instance, the system can use first, second, and third priorities to determine how to spread the new assignment. When the new assignment is assigned a first or low priority, the process flow400proceeds to block417(as shown by the LOW arrow) so that the new assignment can be spread to the larger partition of the unbalanced memory to use up the excess memory (allowing higher priority assignment to receive more efficient spreading operations). When the new assignments is assigned a third or high priority, the process flow400proceeds to block410(as shown by the HIGH arrow) so that the new assignment can use maximum spreading. When the new assignment is assigned a second or medium priority, the process flow400proceeds to block420(as shown by the MED arrow) so that the new assignment can use memory spreading proportional to available memory.

At block420, the system spreads the data traffic conditionally across the memory controllers (e.g., with respect to available memory and/or ranks). The sub-process of block420is further described with respect to blocks440and445. At dotted-block440, the system divides the memory control units into a largest common increment size. Turning now toFIG. 5, a table500is depicted. The table500shows an example of how memory allocation occurs using the different priorities described inFIG. 4as time progresses for four memory control units MC-1, MC-2, MC-3, and MC-4. Note that the memory control unit MC-1is unavailable, the memory control units MC-2and3are of equal size (e.g., 128), and the memory control unit MC-4is half the size of one of the memory control units MC-2and MC-3.

For the first row in table500labeled as Low Priority, memory segment assignment occurs based on total installed memory on each MC since no memory allocation has occurred yet. All memory segments are allocated to MC-2and MC-3, consistent with block417inFIG. 4. As shown in table500, there is a 50% spreading utilization for these low priority assignments.

The next row in table500labeled AVAIL shows the amount of remaining memory in MC-2, MC-3and MC-4after completion of the low priority workload. The next row in table500then shows a switch to memory allocation for a medium priority workload. Allocation for medium priority workload is based on available memory, consistent with block420inFIG. 4. The largest common increment size is 64 gigabytes on MC-4. MC-2and MC-3have 200 gigabytes available and so can accommodate three 64 gigabyte increments. Memory allocation then proceeds by alternately allocating 3 memory segments to MC-2and MC-3and then allocating a fourth memory segment to MC-4. This allocation results in a 37.5% utilization for MC-2and MC-3, and a 25% allocation to MC-4for medium priority assignments in this example.

The next AVAIL row in table600shows the remaining available memory after completion of this medium priority allocation. A high priority workload then proceeds and allocates memory with maximum spreading consistent with block410inFIG. 4. This allocation results in 33% utilization for MC-2, MC-3and MC-4

The next AVAIL row in table500shows remaining memory after completion of the high priority workload. A new medium priority workload begins, and now the largest common increment size is 20 gigabytes. MC-2and MC-3can now accommodate 4 of these segments, and so memory allocation now proceeds by assigning alternately 4 segments to MC-2and MC-3for every 1 segment assigned to MC-4.

Thus, the system enables higher priority and lower/medium assignments to mix and to more effectively use the available memory with a more optimal performance.

In view of the above, the conditional spreading can include an embodiment where data traffic is spread proportionally to the installed memory (e.g., at whatever level spreading is performed, which in an embodiment is typically performed at a node level). The conditional spreading can include an embodiment where data traffic is spread in a hierarchical manner. An embodiment may have a collection of memory controllers on a node which consists of central processing (CP) chips each including a memory controller. Additionally, the node may include memory DIMMs and a system controller (SC) chip. Another configuration may entail sharing of an SC chip between two clusters of CP chips with their associated memory controllers and memory DIMMS. A drawer may then consist of either multiple nodes or clusters. Memory spreading can then occur first to a drawer, then to a node or cluster, then across a memory controller according to installed memory and/or remaining memory.

Further, the conditional spreading can include an embodiment where the system uses low bandwidth but high capacity memory to fill heterogeneous gaps, and then spreads according to remaining memory. The conditional spreading can include an embodiment where the system treats a system kernel evenly across memory controllers, and then spreads memory segments across remaining memory in the memory controllers proportional to the remaining memory. The conditional spreading can include an embodiment where the system allows for a difference in source vs. destination spreading (e.g., when moving memory segments from one topology to the next, use the destination topology to re-optimize the spreading).

The conditional spreading can include an embodiment where the system keeps an original spreading capability on a move memory operation. The conditional spreading can include an embodiment where the system spreads over a close proximity ratio that considers locality (e.g., memory controllers that are far apart may cause latency and are thus avoided to achieve cluster spreading).

FIG. 6depicts an example of a system600in accordance with one or more embodiments. The system600has one or more central processing units (CPU(s))601a,601b,601c, etc. (collectively or generically referred to as processor(s)601). The processors601, also referred to as processing circuits, are coupled via a system bus602to system memory603and various other components. The system memory603can include a read only memory (ROM)604and a random access memory (RAM)605. Memory controllers may be part of either processors601or system memory603. The ROM604is coupled to the system bus602and may include a basic input/output system (BIOS), which controls certain basic functions of the system600. The RAM is read-write memory coupled to the system bus602for use by the processors601.

FIG. 6further depicts an input/output (I/O) adapter606and a communications adapter607coupled to the system bus602. The I/O adapter606may be a small computer system interface (SCSI) adapter that communicates with a hard disk608and/or any other similar component. The I/O adapter606and the hard disk608are collectively referred to herein as a mass storage610. A software611for execution on the system600may be stored in the mass storage610. The mass storage610is an example of a tangible storage medium readable by the processors601, where the software611is stored as instructions for execution by the processors601to cause the system600to operate, such as is described herein with reference toFIGS. 2 and 4. Examples of computer program product and the execution of such instruction is discussed herein in more detail. Referring again toFIG. 6, a communications adapter607interconnects the system bus602with a network612, which may be an outside network, enabling the system600to communicate with other such systems. A display (e.g., screen, a display monitor)615is connected to the system bus602by a display adapter616, which may include a graphics controller to improve the performance of graphics intensive applications and a video controller. In one embodiment, the adapters606,607, and616may be connected to one or more I/O buses that are connected to the system bus602via an intermediate bus bridge (not shown). Suitable I/O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI). Additional input/output devices are shown as connected to the system bus602via an interface adapter620and the display adapter616. A keyboard, a mouse, a speaker, etc. can be interconnected to the system bus602via the interface adapter620, which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit.

Thus, as configured inFIG. 6, the system600includes processing capability in the form of the processors601, and, storage capability including the system memory603and the mass storage610, input means such as the keyboard and the mouse, and output capability including the speaker and the display615. In one embodiment, a portion of the system memory603and the mass storage610collectively store an operating system, such as the z/OS or AIX operating system from IBM Corporation, to coordinate the functions of the various components shown inFIG. 6.