System and method to reduce service disruption in a shared infrastructure node environment

A method of reducing downtime in a node environment is disclosed. The method includes identifying an originating system board of a plurality of system boards that requires service where the originating system board includes a node operating on a processor. The method further includes identifying a target system board of the plurality of system boards where the target system board includes a target processor. The method further includes transferring operation of the node to the target processor before the originating system board is serviced, and operating the node on the target processor.

TECHNICAL FIELD

This disclosure relates generally to information handling systems and more particularly to information handling systems including nodes configured to share physical components.

BACKGROUND

As space constraints increase, there is pressure to increase the density of information handling systems by housing a greater number of nodes in an information handling system chassis. In an information handling system configured to house rack servers, it is not uncommon for a system board, power supply and other physical components to be associated with each node. To increase the density of an information handling system, however, the information handling system may be configured such to share physical components, such as a system board, a power supply, and/or fans.

SUMMARY

In one embodiment of the present disclosure, a method of reducing downtime in a node environment is disclosed. The method includes identifying an originating system board of a plurality of system boards that requires service where the originating system board includes a node operating on a processor. The method additionally includes identifying a target system board of the plurality of system boards where the target system board includes a target processor. The method further includes transferring operation of the node to the target processor before the originating system board is serviced, and operating the node on the target processor.

In another embodiment of the present disclosure, an information handling system including a plurality of system boards, a plurality of storage devices communicatively coupled to the plurality of system boards, and a controller communicatively coupled to the plurality of system boards and the plurality of storage devices is disclosed. The controller is configured to identify an originating system board of the plurality of system boards that requires service where the originating system board includes a node operating on a processor. The controller is additionally configured to identify a target system board of the plurality of system boards where the target system board includes a target processor. The controller is further configured to transfer operation of the node to the target processor before the originating system board is serviced.

In yet another embodiment of the present disclosure, a non-transitory computer-readable medium including computer-executable instructions encoded in the computer-readable medium is disclosed. The instructions, when executed by a processor, are operable to perform operations including identifying an originating system board of a plurality of system boards that requires service where the originating system board includes a node operating on a processor. The instructions, when executed by a processor, are additionally operable to perform operations including identifying a target system board of the plurality of system boards where the target system board includes a target processor. The instructions, when executed by a processor, are further operable to perform operations including transferring operation of the node to the target processor before the originating system board is serviced.

DETAILED DESCRIPTION

Preferred embodiments and their advantages are best understood by reference toFIGS. 1-4, wherein like numbers are used to indicate like and corresponding parts.

An information handling system may be configured such that multiple nodes share physical components. A node may include one or more processors configured to run an instance of an operating system. A node may further include a memory and a storage resource. Shared components may include, for example, a system board, power supply, and/or other physical components necessary to process, store, and/or communicate data. In an information handling system configured in this manner, servicing one of the shared components may require that multiple nodes be powered down and/or taken off-line for the duration of the service. For example, each system board may include multiple processors, each of which functions as an individual node. The applications running on each node may experience a disruption in service if the system board is powered off or otherwise taken off-line for service. In accordance with the teachings of the present disclosure, the system state, metadata, and/or system image of each node on a system board requiring service may be captured and transferred or migrated to a target node on other system boards before the system board is powered down for service. By transferring or migrating the operation of the nodes, instead of powering down or otherwise taking the nodes off-line, the disruption in service experienced by the applications running on the nodes may be reduced or eliminated.

FIG. 1illustrates an example information handling system100including internal storage, in accordance with the teachings of the present disclosure. The various components of information handling system100may be housed in chassis160, which may fully or partially enclose the components of information handling system100. Chassis160may include system boards120, internal storage140, and a chassis management controller (“CMC”)110.

System boards120may include, for example, one or more processors130. System boards120may also include a voltage regulator (not expressly shown), a network interface card (not expressly shown), memory (not expressly shown), a board management controller (not expressly shown), and/or any other component necessary to permit each system board120to process, communicate, and/or store data. As shown inFIG. 1, each system board120may be communicatively coupled to CMC110and internal storage140. AlthoughFIG. 1depicts three system boards120A-120C, chassis160may include more or less than three system boards120.

Processors130of system boards120may include any system, device, or apparatus operable to interpret and/or execute program instructions and/or process data, and may include, without limitation, a microprocessor, microcontroller, digital signal processor (DSP), application specific integrated circuit (ASIC), or any other digital or analog circuitry configured to interpret and/or execute program instructions and/or process data. In some embodiments, processors130may interpret and/or execute program instructions and/or process data stored in internal storage140and/or another component of information handling system100. In some embodiments, processors130may include ARM architecture processors. In some embodiments, each processor130of a system board120may function as an individual node, and may be configured to run an instance of an operating system and/or additional applications. In other embodiments, a plurality of processors may be configured to function as an individual node, and may be configured to run an instance of an operating system and/or additional applications. An operating system image for each node may be stored on a storage resource150of internal storage140. An operating system image may include all the data necessary to operate a particular node. For example, an operating system image may include the operating system and applications running on the node, as well as configuration and files for the operating system and applications running on the node. AlthoughFIG. 1depicts six processors130per system board120, a system board may include more than or less than six processors.

Internal storage140may include any instrumentality or aggregation of instrumentalities that may retain data and/or instructions for a period of time. For example, internal storage140may include storage resources150, which may include random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), a Personal Computer Memory Card International Association (PCMCIA) card, flash memory, solid state disks, hard disk drives, magnetic tape libraries, optical disk drives, magneto-optical disk drives, compact disk drives, compact disk arrays, disk array controllers, and/or any suitable selection or array of volatile or non-volatile memory operable to store data. In some embodiments, one or more of storage resources150may be allocated to a particular system board120. For example, storage resources150A-150D may be allocated to system board120A, storage resources150E-150H may be allocated to system board120B, and storage resources150I-150L may be allocated to system board120C. In such a configuration, the system images for the nodes operating on a particular system board120may be stored on the storage resources150allocated to that system board120. In other embodiments storage resources150A-150L may be shared among system boards120A-120C. In such a configuration, the system images for the nodes operating on system boards120A-120C may be distributed among storage resources150A-150L.

As shown inFIG. 1, CMC110may be communicatively coupled to system boards120and internal storage140. CMC110may provide a user interface that permits the user to configure and control system boards120. For example, the user interface may enable a user to schedule service for a particular system board120. Service may be scheduled, for example, on a routine basis or in response to an error or fault in the operation of the system board. When a particular one of system boards120requires service, CMC110may either power down nodes operating on the system board120requiring service (also referred to as “the originating system board”) or transfer the operation of the nodes to unused processors130(also referred to as “target processors”) on another system board120(also referred to as “the target system board”). In some embodiments, the operation of each node operating on an originating system board may be transferred before the originating system board is powered down for service. In other embodiments, the operation of some, but not all, of the nodes operating on an originating system board may be powered down for service, while others may be transferred to a target system board. For example, a distributed application may operate on several nodes spread across several of system boards120. When one of the nodes fails, the remaining nodes may be configured to take over the operations performed by the failed node. As such, the nodes on which a distributed application is operating may be powered down instead of being transferred before the originating system board is powered down for service.

Consider for example, that system board120A requires service. CMC110may determine which of the nodes operating on processors130A-130F of system board120A may be powered down and which may be transferred to system board120B or system board120C. For example, a distributed application may be operating on processors130A,130G, and130M of system boards120A,120B, and120C, respectively. The distributed application may be configured such that processors130G and130M assume the operations performed by processor130A in the event processor130A experiences a failure. As such, CMC110may identify the node operating on processor130A as a node that may be powered down. CMC110may identify the nodes operating on the remaining processors130B-130F of system board120A for transfer to system boards120B and/or120C.

Once CMC110has identified the nodes of system board120A that will be transferred, CMC110may identify target processors130on system boards120B and/or120C to which the nodes may be transferred. The nodes identified for transfer need not be transferred to a single system board120; instead, they may be distributed between system boards120B and120C. In the example, discussed above, CMC110identified the nodes operating on processors130B-130F for transfer; thus, CMC110may identify five unused processors on system boards120B and/or120C that may serve as target processors. Consider, for example, that CMC110determines that processors130H and130K on system board120B, as well as processors130M,130N, and130Q on system board120C, are unused. Processors130H,130K,130M,130N, and130Q may thus be identified by CMC110as target processors for the nodes operating on processors130B-130F.

CMC110may capture the system state of the nodes identified for transfer by, for example, freezing all processes running on the nodes and saving, for each node, a snapshot of those processes to memory of the CMC110. The system state may then be transferred by CMC110to target processors130on system boards120B and/or120C. In some embodiments, CMC110may also transfer the system image of the nodes identified for transfer to target processors130. For example, as discussed above, one or more of storage resources150may be allocated to a particular one of system boards120. Where storage resources are allocated in this manner, CMC110may retrieve the system images of the nodes identified for transfer from storage resources150allocated to the originating system board120because target processors130may not otherwise have access to the system images. CMC110may then transfer the retrieved system images to target processors130. In other embodiments, storage resources150may be shared among system boards120A-120C. Where there is shared storage, each of processors130on system boards120may have access to the particular one of storage resources150on which the system image for a particular node is stored. Thus, the system image may be accessed and transferred directly by target processor130.

Target processors130may be initialized in a pre-execution environment (PXE). The system state and system image of the nodes transferred from system board120A may be loaded on target processors130of one or more of system boards120B or120C, thereby enabling the transferred nodes to operate on target processors130. In the example, discussed above, CMC110identified the nodes operating on processors130B-130F for transfer; thus, they system state and system image for the nodes operating on processors130B-130F may be loaded on target processors130. By transferring the nodes operating on processors130B-130F instead of powering down or otherwise taking the nodes off-line, the nodes may operate on target processors130and the disruption in service to the applications running on these nodes may be reduced or eliminated.

FIG. 2illustrates an example information handling system200including external storage, in accordance with the teachings of the present disclosure. Information handling system200may include CMC110, system boards120, external storage240, and network270. CMC110and system boards120may be housed in chassis260, which may fully or partially enclose the components of information handling system200. AlthoughFIG. 2depicts three system boards120A-120C, chassis260may include more or less than three system boards120.

External storage240, like internal storage140, may include any instrumentality or aggregation of instrumentalities that may retain data and/or instructions for a period of time. For example, external storage240may include storage resources150, which are discussed in detail in conjunction withFIG. 1. As discussed above, in some embodiments, one or more of storage resources150may be allocated to a particular system board120. In such a configuration, the system images for the nodes operating on a particular system board120may be stored on the storage resources150allocated to that system board120. In other embodiments, storage resources150may be shared among system boards120A-120C. In such a configuration, the system images for the nodes operating on system boards120A-120C may be distributed among storage resources150A-150F.

External storage240may be communicatively coupled to system boards120and CMC110via network270. Network270may be a network and/or fabric configured to communicatively couple system boards120, CMC110, external storage240, and/or any element associated with information handling system200. Network270may be implemented as, or may be a part of, a storage area network (SAN), personal area network (PAN), local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wireless local area network (WLAN), a virtual private network (VPN), an intranet, the Internet or any other appropriate architecture or system that facilitates the communication of signals, data and/or messages (generally referred to as data). Network270may transmit data using any storage and/or communication protocol, including without limitation, Fibre Channel, Frame Relay, Asynchronous Transfer Mode (ATM), Internet protocol (IP), other packet-based protocol, small computer system interface (SCSI), Internet SCSI (iSCSI), advanced technology attachment (ATA), serial ATA (SATA), advanced technology attachment packet interface (ATAPI), serial storage architecture (SSA), integrated drive electronics (IDE), and/or any combination thereof. Network270and its various components may be implemented using hardware, software, or any combination thereof.

As discussed above, when a particular system board120requires service, CMC110may determine which of the nodes operating on the particular system board120may be powered down and which may be transferred to other system boards120. Consider, for example, that system board120A requires service. Once CMC110has identified the nodes of system board120A that will be transferred instead of powered down, CMC110may identify target processors130on system boards120B and/or120C to which the operation of the nodes may be transferred.

Once target processors130have been identified, CMC110may capture and transfer the system state and/or system image of the nodes identified for transfer. As discussed above, in certain embodiments one or more storage resources150may be allocated to a particular system board. Where this is the case, CMC110may then transfer both the system state and system image for the nodes identified for transfer to target processors130. As discussed above, CMC110may capture the system state for each node identified for transfer. CMC110may retrieve the system image for each node identified for transfer from external storage240via network270. In other embodiments, storage resources150may be shared among system boards120. Where there is shared storage, target processors130may directly access system images from storage resources150via network270.

As discussed above, target processors130may be initialized in a pre-execution environment (PXE). The system state and system image of the nodes identified for transfer may be loaded on target processors130, thereby enabling the transferred nodes to operate on target processors130. By transferring instead of powering down or otherwise taking the nodes off-line, the disruption in service to the applications running on these nodes may be reduced or eliminated.

FIG. 3illustrates an example information handling system300including a plurality of system boards with on-board storage, in accordance with the teachings of the present disclosure. Information handling system300may include CMC110and system boards320, which may be housed in chassis360. Chassis360may fully or partially enclose the components of information handling system300.

System boards320may include, for example, one or more processors130and one or more memory devices350. System boards320may also include a voltage regulator (not expressly shown), a network interface card (not expressly shown), a board management controller (not expressly shown), and/or any other component necessary to permit each system board320to process, communicate, and/or store data. As shown inFIG. 3, each system board320may be communicatively coupled to CMC110. AlthoughFIG. 3depicts three system boards320A-320C, chassis360may include more or less than three system boards320.

Memory devices350may include any instrumentality or aggregation of instrumentalities that may retain data and/or instructions for a period of time. For example, memory devices350may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and/or flash memory; as well as communications media such as wires, optical fibers, microwaves, radio waves, and other electromagnetic and/or optical carriers; and/or any combination of the foregoing.

The system images for the nodes operating on a particular one of system boards320may be stored on memory devices350of the particular system board320. In such a configuration, memory devices350on each system board320may be allocated to the particular system board320and are not shared with the remaining system boards320. For example, the system images for nodes operating on system board320A may be stored on memory devices350A and/or350B. Similarly, the system images for nodes operating on system board320B may be stored on memory devices350C and/or350D, while the system images for nodes operating on system board320C may be stored on memory devices350E and/or350F.

Consider, for example, that system board320A requires service. CMC110may determine which of the nodes operating on processors130A-130F of system board320A may be powered down and which may be transferred to system board320B and/or system board320C. Once CMC110has identified which nodes of system board320A will be transferred, CMC110may identify target processors130on system boards320B and/or320C to which the nodes may be transferred. Because system images for each node are stored on memory devices350allocated to the particular system board320on which the nodes are operating, CMC110may capture and transfer both the system state and the system image for the nodes designated for transfer to target processors130on system boards320B and/or320C.

As discussed above, target processors130may be initialized in a pre-execution environment (PXE). The system state and system image of the nodes identified for transfer may be loaded on the target processors130, thereby enabling the transferred nodes to operate on the target processors130. By transferring instead of powering down or otherwise taking the nodes off-line, the disruption in service to the applications running on these nodes may be reduced or eliminated.

FIG. 4illustrates an example method of reducing downtime in a dense node environment by transferring nodes from one system board to another in accordance with the teachings of the present disclosure. At step410, the CMC may identify a system board that requires service. In some embodiments, service to system boards may be scheduled by a system administrator. In other embodiments, service to a system board may be necessary because of a system failure or event. At step420, the CMC may determine whether all the nodes operating on the system board requiring service may be powered down. As discussed above, a distributed application may operate on several nodes spread across several system boards. The nodes on which a distributed application is running may be configured to tolerate failure of at least one of the nodes because, when one of the several nodes experiences a failure or an error, the operations performed by that node may be assumed by the remaining nodes. Thus, in the case of a particular system board requiring service, the nodes on which a distributed application is running may be powered down instead of being transferred, while the remaining nodes may be identified for transfer. If it is determined that all the nodes operating on the system board requiring service may be powered down, the method may proceed to step435. At step435, the nodes on the system board requiring service may be powered down and the method may proceed to step490. At step490, the system board requiring service may be powered down.

If, on the other hand, it is determined that some, or all, of the nodes operating on the system board requiring service should be transferred instead of powered down, the method may proceed to step430. At step430, the CMC may identify the nodes that will be transferred to another system board. Once the CMC has identified the nodes that will be transferred, the method may proceed to step440.

At step440, the CMC may identify a target processor to which each of the nodes may be transferred. As discussed above, target processors may be unused processors on other system boards in the information handling system. When a target processor has been identified for each node identified for transfer, the method may proceed to step450. At step450, the CMC may capture the system state of the nodes identified for transfer. As discussed above, the CMC may capture the system state of a node by, for example, freezing all processes running on the node and saving a snapshot of those processes to memory of the CMC.

At step460, the system state and system image for each of the nodes identified for transfer may be transferred to the target processors. An operating system image may include all the data necessary to operate a particular node. For example, an operating system image may include the operating system and applications running on the node, as well as configuration and files for the operating system and applications running on the node.

As discussed above, an operating system image for each node may be stored on an internal or external storage resource or on a memory device of the system board. In some embodiments, each system board may include a memory device on which system images for the nodes operating on that system board are stored. In other embodiments, the chassis of an information handling system may include internal storage on which system images are stored. The internal storage may include one or more storage resources, each of which may be allocated to a particular system board or shared among the system boards. In still other embodiments, an information handling system may include external storage on which system images are stored. The external storage may be communicatively coupled to the system boards via a network and may include one or more storage resources, each of which may be allocated to a particular system board or shared among the system boards.

Where each of the storage resources or memory devices are allocated to a particular system board, both the system state and system image may be transferred by the CMC because the target processor may not otherwise have access to the system image for that node. Where, on the other hand, each of the storage resources is shared among the system boards, the system state of the nodes identified for transfer may be captured and transferred by the CMC, while the system image may be accessed directly by the target processor.

At step470, operation of the transferred nodes may begin on the target processors. As discussed above, the target processors may be initialized in a pre-execution environment (PXE) and the system state and system image of the transferred nodes may be loaded on the target processors. Once the transferred nodes have begun operation on the target processors, the method may proceed to step480.

At step480, the remaining nodes on the system board requiring service may be powered down. Once the nodes operating on the system board requiring service have been either transferred or powered down, the method may proceed to step490. At step490, the system board requiring service may be powered down for service.

Although the examples discussed in detail above address powering down or transferring nodes based on the need to service a system board on which the nodes are operating, the methods disclosed herein may be utilized to reduce or eliminate downtime in the event that a component shared by multiple nodes must be powered down or otherwise taken off-line for service.

Further, although the present disclosure has been described in detail, it should be understood that various changes, substitutions, and alterations can be made hereto without departing from the spirit and the scope of the disclosure as defined by the appended claims.