Information handling system including dynamically merged physical partitions

An information handling system includes instruction processing nodes in respective physical partitions. A communications bus couples two information processing nodes together. Each node includes hardware resources such as CPUs, memories and I/O adapters. Prior to a command to merge the physical partitions, the communication bus exhibits a disabled state such that the two information processing nodes are effectively disconnected. After receiving a command to merge the physical partitions, the system enables the communication bus to effectively hot-plug the two nodes together. A modified master hypervisor in one node stores data structures detailing the hardware resources of the two nodes. The modified master may assign resources from one node to a logical partition in another node.

BACKGROUND

The disclosures herein relate generally to information handling systems, and more specifically, to information handling systems that employ multiple processors.

Modern information handling systems (IHSs) frequently use multiple processors to handle the heavy workloads of today's complex and feature-rich application software. Contemporary IHSs may in fact handle several applications at the same time.

An IHS may include hardware resources such as multiple processors or central processing units (CPUs), multiple memories and multiple I/O adapters. The IHS may employ a hypervisor to allocate these CPU, memory and I/O hardware resources to a number of different logical partitions (LPARs). The hypervisor is a software abstraction layer between the hardware resources and the logical partitions. Each logical partition will execute or run a unique operating system that may only access to the resources that the hypervisor defines for that particular logical partition. Each operating system may execute multiple software applications. In this manner, the modern IHS may handle the heavy workload of several different software applications at the same time.

BRIEF SUMMARY

Accordingly, in one embodiment, a method is disclosed for operating an information handling system (IHS). The method includes providing first and second nodes that are physically partitioned from one another, the first and second nodes being in a pre-merger state. The method also includes configuring the first node to operate in a first predetermined address range while the first node exhibits the pre-merger state. The method further includes configuring the second node to operate in a second predetermined address range that is non-overlapping with respect to the first predetermined address range while the second node exhibits the pre-merger state. The method still further includes activating, in response to a merge command, a communication bus between the first and second nodes to merge the first and second nodes to form a merged physical partition in a post-merger state. The first node may communicate over the communication bus with the second node via the second predetermined address range of the second node. The second node may communicate over the communication bus with the first node via the first predetermined address range of the first node.

In another embodiment, an information handling system (IHS) is disclosed. The IHS includes first and second nodes that are physically partitioned from one another, the first and second nodes being in a pre-merger state, wherein the first node is operative in a first predetermined address range and the second node is operative in a second predetermined address range that is non-overlapping with respect to the first predetermined address range. The IHS also includes a communication bus situated between the first and second nodes, the communication bus being logically disabled during the pre-merger state, the communication bus responding to a merge command to merge the first and second nodes to form a merged physical partition in a post-merger state wherein the first node may communicate over the communication bus with the second node via the second predetermined address range of the second node, and further wherein the second node may communicate over the communication bus with the first node via the first predetermined address range of the first node.

DETAILED DESCRIPTION

FIG. 1shows an information handling system (IHS)100that includes information processing nodes10and20. Nodes10and20are physically separate or physically partitioned from one another.FIG. 1shows node10within a physical partition1and further shows node20within a physical partition2. Nodes10and20include hardware resources such as multiple processors or CPUs, multiple memories and multiple I/O adapters, as described in more detail below. At some point in time that a user or other entity determines, node10of physical partition1and20of physical partition2may merge into a merged partition, as described in more detail below. In the embodiment ofFIG. 1, physical partitions1and2may merge to form the merged partition. A partition may include multiple nodes.

Node10of physical partition1includes a processor group20with processors or CPUs21A,22A,23A and24A. Internal coherency busses CB1, CB2, CB3, CB4, CB5and CB6couple CPUs21A,22A,23A and24A together within processor group20to enable these CPUs or processors to communicate with one another. CPUs21A,22A,23A and24A couple respectively to memories21B,22B,23B and24B. CPUs21A,22A,23A and24A also couple respectively to I/O adapters21C,22C,23C and24C. One or more of the I/O adapters, such as I/O adapter22C, couples to non-volatile storage25and a network adapter30. Network adapter30enables node10to connect by wire or wirelessly to a network and other information handling systems. In one embodiment, a designer may configure processors21A-24A in a symmetric multiprocessor (SMP) arrangement such that each processor or CPU may access any of memories21B-24B and any of I/O adapters21C-24C.

Nonvolatile storage25may provide storage for software such as a hypervisor35, operating systems40and software applications (APPS)45. Within node10of physical partition1, hypervisor35provides a hardware abstraction layer between one or more logical partitions (not shown) and hardware resources such as CPUs21A-24A, memories21B-24B and I/O adapters21C-24C. The same or different operating system40may operate on each logical partition. One or more software applications (APPS)45execute on each operating system40to provide node10with a workload to process.

Node20of partition2includes a processor group50with processors or CPUs51A,52A,53A and54A. Internal coherency busses CB1′, CB2′, CB3′, CB4′, CB5′ and CB6′ couple CPUs51A,52A,53A and24A together within processor group20to enable these CPUs or processors to communicate with one another. CPUs51A,52A,53A and54A couple respectively to memories51B,52B,53B and54B. CPUs51A,52A,53A and54A also couple respectively to I/O adapters51C,52C,53C and54C. One or more of the I/O adapters, such as I/O adapter52C couples to non-volatile storage55and a network adapter60. Network adapter60enables node20to connect by wire or wirelessly to a network and other information handling systems. In one embodiment, the designer may configure processors51A-54A in a symmetric multiprocessor (SMP) arrangement such that each processor may access any of memories51B-54B and any of I/O adapters51C-54C.

In a manner similar to nonvolatile storage25discussed above with respect to node10, the nonvolatile storage55of node2may provide storage for software such as a hypervisor65, operating systems70and software applications (APPS)75. Within node20of physical partition2, hypervisor65provides a hardware abstraction layer between one or more logical partitions (not shown) and hardware resources such as CPUs51A-54A, memories51B-54B and I/O adapters51C-54C. The same or different operating system70may operate on each logical partition. One or more software applications (APPS)75execute on each operating system70to provide node20with a workload to process.

IHS100also includes an inter-node coherency bus80, namely a communication bus that couples between node10of partition1and node20of partition2. Prior to the merger of physical partitions10and20, system100maintains coherency bus80in a disabled state such that partition1and partition2are effectively physically separate partitions. However, when IHS100receives an instruction or command to merge node10of physical partition1and node20of physical partition2, coherency bus80changes to an enabled stated to merge the 2 physical partitions, as described below in more detail.

IHS100also includes a hardware management console (HMC)85which a user may operate to initiate a physical merger operation. HMC85couples to nodes10and20via respective flexible service processors (FSPs)90and95. In one embodiment, a user or other entity may employ HMC85to commence a physical merger operation of nodes10and20without first powering down nodes10and20. In other words, IHS performs this merger operation dynamically without significantly disturbing applications and operating systems that execute within nodes10and20.

FIG. 2shows a representation of the hardware resources and software layers of physical partition1and physical partition2of IHS100prior to a merger of physical partition1and physical partition2. Nodes10and20include hardware resources such as multiple CPUs, multiple memories (MEM) and multiple I/O adapters (I/O) in a manner similar toFIG. 1. However, due to space limitations,FIG. 1does not label these hardware resources with component numbers. IHS100includes a master hypervisor software layer35between the hardware resources of node10and logical partitions (LPARs)201and202. Master hypervisor35includes hardware resource allocation data structures37that store information designating the particular hardware resources of node10(CPUs, memories and I/O) that IHS100allocates to each of logical partitions201and202of physical partition1. An operating system40-1communicates with logical partition201and an operating system40-2communicates with logical partition202. The operating systems on logical partitions201and202may be different operating systems or copies of the same operating system. Multiple applications (APPS)45-1may execute on operating system40-1while multiple applications (APPS)45-2execute on operating system40-2.

IHS100also includes a slave hypervisor software layer65between the hardware resources of node20and logical partitions (LPARs)211and212. Slave hypervisor65includes hardware resource allocation data structures67that store information designating the particular hardware resources of node20(CPUs, memories and I/O) that IHS100allocates to each of logical partitions211and212of physical partition2. An operating system70-1communicates with logical partition211and an operating system70-2communicates with logical partition212. The operating systems on logical partitions201and202may be different operating systems or copies of the same operating system. Multiple applications (APPS)75-1may execute on operating system70-1while multiple applications (APPS)75-2execute on operating system70-2.

The pre-merger state refers to IHS100prior to the merger of node10of physical partition1and node20of physical partition2. The post-merger state refers to IHS100after the merger of node10of physical partition1and node20of physical partition2. During the pre-merger state, IHS100maintains coherency bus80in a disabled mode such that node10of partition1and node20of partition are physically separate. In other words, node10and node20are effectively different computer systems that operate as standalone systems although they may include coherency bus80.

FIG. 3shows an address memory map to demonstrate that IHS100assigns non-overlapping address ranges to the memories of node10and the memories of node20during the pre-merger state. In other words, prior to the merger of node10and node20, the memories of nodes10and20exhibit non-overlapping or disjoint address ranges. More specifically, memories21B,22B,23B and24B of node10exhibit address ranges of 0-1 GB, 1-2 GB, 2-3 GB and 3-4 GB, respectively. Memories51B,52B,53B and54B of node20exhibit address ranges of 4-5 GB, 5-6 GB, 6-7 GB and 7-8 GB, respectively. Stated alternatively, system-wide each memory exhibits a dedicated address range prior to the physical merger of node10of partition1and node20of partition2. To avoid memory conflicts, IHS100employs these same dedicated memory ranges for its respective memories after the merger of node10and20that IHS100employed prior to the merger of nodes10and20. After the physical partition merger, each of memories21B,22B,23B,24B,51B,52B,53B and54B still exhibits its own unique address space as it did prior to the merger. FSP90sets up the CPUs in physical partition1to boot from memory21B,22B,23B and24B. FSP95sets up the CPUs in physical partition2to boot from memory51B,52B,53B and54B.

FIG. 4is a representation of IHS100′, namely IHS100after the merger of node10of physical partition1and node20of physical partition2.FIG. 4depicts a modified master hypervisor35′ between all node hardware resources and multiple logical partitions201,202,211and212. Modified master hypervisor35′ includes combined hardware resource data structures37′ containing not only the hardware resource data structures37of master hypervisor35, but also the hardware resource data structures67of slave hypervisor65. Modified master hypervisor35′ maintains the CPU, memory and I/O allocations that existed prior to the physical merger. In other words, modified master hypervisor35′ allocates or assigns each logical partition to the same hardware resources that the logical partition employed before the physical merger, as described in more detail below. Operating systems40-1,40-2,70-1and70-2are unaware of the physical merger of nodes10and20. Likewise, applications (APPS)45-1,45-2,75-1and75-2are unaware of the physical merger of nodes10and20.

In the course of merging physical partition1and physical partition2, IHS100effectively merges two hypervisors into one controlling hypervisor35′ that controls the hardware resources of the merged physical partition. More specifically, IHS100merges master hypervisor35and slave hypervisor70into an enhanced or modified master hypervisor35′ that controls allocation of hardware resources in the resultant merged partition400of IHS100′. In other words, modified master hypervisor35′ now controls allocation of the hardware resources of both of the previously existing nodes10and20to logical partitions from both nodes, namely logical partitions201,202,211and212.

FIG. 5shows a simplified representation of IHS100prior to the merger of node10of partition1and node20of partition2. In this pre-merger state, nodes10and20run essentially as standalone machines. Coherency bus80exhibits the disabled state prior to physical merger. In other words, with coherency bus80disabled, nodes10and20are separate from one another and operate as separate machines. Prior to the physical merger of nodes10and20, master hypervisor35includes data structures37that contain information designating the particular hardware resources (CPUs, memories and I/O) that IHS100assigns or allocates to each of logical partitions201and202of physical partition1. However, master hypervisor35is unaware of node20and slave hypervisor65. Data structures37may be in the form of a look-up table (TABLE 1 below) that, for each logical partition, specifies a logical partition, one or more CPUs, memories and I/O adapters.

TABLE 1LogicalPartitionCPUsMemoriesI/O20121A, 22A, 23A21B, 22B22C20224A23B, 24B21C, 23C, 24C
FIG. 5shows the hardware resources of node10collectively as hardware resources505. Prior to the merger of nodes10and20, slave hypervisor65includes data structures67that contain information designating the particular hardware resources (CPUs, memories and I/O) that IHS100assigns or allocates to each of logical partitions211and212of physical partition2. Slave hypervisor65is unaware of node10and master hypervisor35. Data structures67may be in the form of a look-up table (TABLE 2 below) that, for each logical partition, specifies one or more CPUs, memories and I/O adapters in physical partition2.

FIG. 6shows a simplified representation of IHS100prior to the merger of node10of partition1and node20of partition2. In this pre-merger state, nodes10and20run essentially as standalone machines. HMC85couples to the hardware resources of node10via flexible service processor (FSP)90. HMC85also couples to the hardware resources of node20via flexible service processor (FSP)95. An Ethernet network605may couple HMC85to FSPs90and95. In one embodiment, FSP90couples to the CPUs of node10via a Joint Test Action Group (JTAG) interface610. FSP95couples to the CPUs of node20via JTAG interface615. Flexible service processors (FSPs)90and95employ firmware layers620and625to control their respective operations.

FSP90instructs node10to perform a boot sequence upon command from HMC85. The boot sequence includes starting clocks, initializing modes, configuring hardware, loading a hypervisor, loading operating systems and starting instruction execution. FSP95likewise instructs node20to perform a boot sequence. In one embodiment, the boot sequence includes setting up the address ranges of memory controllers (MC) respectively associated with each of the memories (MEM) of a node.FIG. 6shows a memory controller (MC) with its respective memory (MEM) as MEM/MC. Referring to nodeFIG. 1, although not specifically shown, IHS100may include a memory controller between each CPU and its CPU's memory. For example, IHS100may include a memory controller between CPU21A and memory21B, a memory controller between CPU22A and memory22B, a memory controller between CPU23A and memory23B, and so forth for the remaining CPUs and memories of IHS100.

Referring to bothFIG. 1andFIG. 6, FSP90configures or instructs the memory controllers in node10to be responsive to requests in the address ranges that the address map ofFIG. 3indicates for the CPUs of node10. For example, the memory controller for the CPU that associates with memory21B responds to the address range 0-1 GB. The memory controller for the CPU that associates with memory22B responds to the address range 1-2 GB. The memory controller for the CPU that associates with memory23B responds to the address range 2-3 GB. The memory controller for the CPU that associates with memory24B responds to the address range 3-4 GB.

FSP95configures or instructs the memory controllers in node20to be responsive to requests in the address ranges that the address map ofFIG. 3indicates for the CPUs of node20. FSP90configures the memory controllers in node20such that the address ranges of the memories/memory controllers of node20do not overlap the address ranges of the memories/memory controllers of node10. For example, the memory controller for the CPU that associates with memory51B responds to the address range 4-5 GB. The memory controller for the CPU that associates with memory52B responds to the address range 5-6 GB. The memory controller for the CPU that associates with memory53B responds to the address range 6-7 GB. The memory controller for the CPU that associates with memory54B responds to the address range 7-8 GB.

FIG. 7is a flowchart that describes one embodiment of the method that IHS100employs to merge a node of one physical partition with a node of another physical partition. A user may start the process of initializing IHS100as a multi-node symmetric multiprocessor (SMP) system by inputting a start initialization command at HMC85, as per block700. HMC85assigns a primary flexible service processor (FSP) to each node, as per block705. Each physical partition employs a respective FSP. In the present example, HMC85assigns FSP90to node10and FSP95to node20.

FSP90boots node10to initialize the CPUs of node10to form a physical partition (PPAR)1that exhibits the configuration shown inFIG. 2, as per block710. Hypervisor35loads and sets up multiple logical partitions (LPARs) such as logical partition201and logical partition202. Hypervisor35assigns hardware resources (CPUs, MEMs and I/O) to logical partitions201and202in response to commands from HMC85. Operating systems load on logical partition201and logical partition202. The data structures37within hypervisor35specify the particular hardware resources of node10assigned to each logical partition201and202. Hypervisor35may create more logical partitions than shown in the example ofFIG. 2. In this manner, IHS100forms physical partition1. In a similar manner, hypervisor65assigns hardware resources (CPUs, MEMs and I/O) to logical partitions211and212in response to commands from HMC85. Operating systems load on logical partition211and logical partition212. The data structures67within hypervisor65specify the particular hardware resources of node20assigned to each logical partition211and212. Hypervisor65may create more logical partitions than shown in the example ofFIG. 2. In this manner, IHS100forms physical partition2. The respective machines that physical partition1and physical partition2form operate as separate, stand-alone machines prior to partition merger. Before partition merger, coherency bus80remains in the disabled state such that physical partition1and physical partition2are effectively physically separate and unconnected. Physical partition1and physical partition2execute respective workloads separately, as per block715. In other words, physical partition (PPAR)1executes one application software workload and physical partition (PPAR)2executes another application software workload with neither partition using the hardware resources of the other prior to partition merger.

Hardware management console (HMC) monitors for a request or command to merge node10of physical partition1and node20of physical partition2, as per decision block720. One reason why a user of HMC85may request a merger of the two nodes is to add hardware resources from one node to a logical partition of the other node. This may enable that logical partition to handle a larger workload. If HMC85does not receive a request to merge physical partitions1and2, then physical partitions1and2continue executing separate workloads, as per block715. However, if HMC85receives a request to merge physical partitions1and2, then master hypervisor35dynamically hot plugs or hot connects node10of physical partition1and node20of physical partition2together via coherency bus80, as per block725. In response to the request to merge, IHS100enables the formerly disabled coherent bus80. This creates a coherent bus connection between the previously separate nodes10and20. A coherent bus is one in which all bus masters snoop the requests of all other bus masters such that data are modified in a coherent fashion between multiple caches of memory in the SMP computer. As discussed above, hypervisor35is a master hypervisor and hypervisor65is a slave hypervisor. The above actions effectively connect coherency bus80between nodes10and20. Both nodes10and20may now observe transactions on coherency bus80. However, both master hypervisor35and slave hypervisor65are still active. The flowchart ofFIG. 8, discussed below, provides more detail with respect to one method for hot plugging node20into node10via coherency bus80. Node10may observe memory transactions of node20via the now enabled coherency bus80. Likewise, node20may observe memory transactions of node10via coherency bus80. No conflicts occur when one node observes the other node's memory transactions because node10and a node20each employ respective non-overlapping memory regions as seen inFIG. 3.

The master hypervisor35of node10and the slave hypervisor65of node20continue to execute or run in their own respective non-overlapping address spaces, as per block730. As shown in the pre-merger address map ofFIG. 3, master hypervisor35executes in the address space of node10, for example in the first 256 KB of memory21B in one embodiment. Master hypervisor35begins communication with slave hypervisor65via a communication region310at a predetermined absolute address location in one of the memories of node10, such as memory21B, as per block735. For example, communication region300may occupy the second 256 KB of memory21B. At this point, HMC85can still communicate with both master hypervisor35and slave hypervisor65. HMC85may query master hypervisor35to determine the address range for region310and then communicate that address range to the slave hypervisor65. In this manner, master hypervisor35of node10and slave hypervisor65of node20both know the location of communication region310.

Master hypervisor35queries slave hypervisor65via communication region310to access and retrieve a shadow copy of the hardware resource allocation data structures67of node2, as per block740. Slave hypervisor65transmits the shadow copy of hardware resource data structures67back to master hypervisor35via coherency bus80. After receiving the shadow copy of hardware resource data structures67, master hypervisor35rebuilds its own data structures37to include not only the existing hardware resource allocations of data structures37of node10but also the hardware resource allocation data structures67from node20, as per block745. The resultant modified master hypervisor35′ includes the hardware resource data structures of node10and the hardware resource data structures of node20.

Slave hypervisor65communicates with its flexible service processor (FSP) to terminate communication with HMC85, as per block750. Modified master hypervisor35′ uses communication region310to communicate with slave hypervisor65to complete the merger. As part of completing the merger, slave hypervisor65momentarily quiesces the CPUs that associate with it, namely CPUs51A,52A,53A and54A, as per block755. Slave hypervisor65hands over ownership of the hardware resources formerly associated with node20of partition2to master hypervisor35of node10of partition1, also as per block755. With the merger complete, modified master hypervisor35′ communicates to HMC85that merger of node10of physical partition1and node20of physical partition2is complete, as per block760. The partition merger ends at end block765. Applications associated with node10and applications formerly associated with node20continue to execute on their respective operating systems after the partition merger. The merged memory space continues to look like the memory map ofFIG. 3, except that slave hypervisor65is effectively no longer present and master hypervisor35becomes modified master hypervisor35′.

In this particular example, prior to the merger, physical partition1included 4 CPUs, 4 memories (MEM) and 4 I/O adapters in node10, as shown inFIG. 2. Master hypervisor35controlled those CPU, memory and I/O hardware resources. Prior to the merger, physical partition2included 4 CPUs, 4 memories (MEM) and 4 I/O adapters in node10, as shown inFIG. 2. Slave hypervisor65controlled those CPU, memory and I/O hardware resources. However, after physical partition merger, IHS100′ forms a merged physical partition400in which modified master hypervisor35′ controls the allocation of all 8 CPU, memory and I/O hardware resources, as shown inFIG. 4. Modified master hypervisor35′ may allocate any of these hardware resources originally from nodes10and20to any of the logical partitions, such as logical partitions201,202,211and212, as well as other logical partitions.

FIG. 8is a high-level flowchart that shows steps of a hot plugging or hot-adding operation wherein IHS100, under the direction of HMC85, hot-plugs or hot-adds node20to node10via coherency bus80. The hot-plug operation starts at block800. A user at hardware management console (HMC)85sends an activate coherency bus command to coherency bus80via a flexible service processor (FSP), such as FSP90for example, as per block805. FSP90performs a continuity test on coherency bus80, as per block810. If the coherency bus80fails the continuity test, then the hot-plug operation terminates. If the coherency bus80passes the continuity test, then the hot plug operation continues. In response to passing the continuity test, FSP90quiesces coherency bus80, as per block815. FSP90then waits and allows pending node10and20operations to complete, as per block820. When pending operations complete, FSP90unquiesces coherency bus80, as per block825. The hot-plug operation ends at end block830.

As seen in the flowchart ofFIG. 7, IHS100may operate in either the pre-merger or the post-merger state. In the pre-merger state, node10of physical partition1and node20of physical partition2are physically separate with each node executing a different application software workload. Even though node10of physical partition1and node20of physical partition2are separate or unmerged, each node may employ a respective memory address range that does not overlap the address range of the other node. In this manner, nodes10and20are ready for a merger operation even if a user never requests a merger operation at decision block720. However, should the user at HMC85request a merger of nodes10and20, IHS100enables coherency bus80to connect nodes10and20. Each node continues using its respective predetermined address range that does not overlap the address range of the other. IHS100′ may thus avoid addressing conflicts in the post-merger state. In one embodiment, the merger of the first and second partitions is dynamic because the user need not power down both of the partitions to conduct the merger operation. The user instead may use HMC85to send an enable command that logically enables coherency bus80to hot plug or hot connect node10of partition1to node20of partition2to form a merged physical partition. The modified master hypervisor35′ of the merged partition400allows a logical partition to access hardware resources from either node10or node20in the merged partition as long as those hardware resources are available.

WhileFIG. 1shows one form of IHS100, IHS100may take many forms such as of a desktop, server, portable, laptop, notebook, or other form factor computer or data processing system. IHS100may take still other form factors such as a gaming device, a personal digital assistant (PDA), a portable telephone device, a communication device or other devices that include a processor and memory.