Abstract:
A method and system for autonomously rebuilding a failed one of a plurality of servers and a computer system utilizing the same is disclosed. In a first aspect, the method comprises providing a bus for allowing a recovery mechanism to access each of the plurality of servers and utilizing the recovery mechanism to rebuild the failed server onto another server. In a second aspect, the computer system comprises a plurality of servers, a management module for monitoring and managing the plurality of servers, a recovery mechanism coupled to the management module, and a bus coupling the recovery mechanism to each of the plurality of servers, wherein the recovery mechanism rebuilds a failed server onto another of the plurality of servers.

Description:
FIELD OF THE INVENTION  
         [0001]    The present invention relates generally to computer server systems and, more particularly, to a method and system for autonomously rebuilding a failed server onto another server.  
         BACKGROUND OF THE INVENTION  
         [0002]    In today&#39;s environment, a computing system often includes several components, such as servers, hard drives, and other peripheral devices. These components are generally stored in racks. For a large company, the storage racks can number in the hundreds and occupy huge amounts of floor space. Also, because the components are generally free standing components, i.e., they are not integrated, resources such as floppy drives, keyboards and monitors, cannot be shared.  
           [0003]    A system has been developed by International Business Machines Corp. of Armonk, N.Y., that bundles the computing system described above into a compact operational unit. The system is known as an IBM eServer BladeCenter.™ The BladeCenter is a 7U modular chassis that is capable of housing up to 14 individual server blades. A server blade or blade is a computer component that provides the processor, memory, hard disk storage and firmware of an industry standard server. Each blade can be “hot-plugged” into a slot in the chassis. The chassis also houses supporting resources such as power, switch, management and blower modules. Thus, the chassis allows the individual blades to share the supporting resources.  
           [0004]    Currently in the BladeCenter environment, if one of the server blades fails, an administrator must intervene to identify the failing blade, and unplug, remove and replace it with a new blade. This alone is a cumbersome task. If the administrator further wishes to retain the application and data on the failed blade&#39;s hard drive, the administrator must physically remove the hard drive from the failed blade and remount it into the new blade. This process is labor intense, time consuming, and economically costly, particularly if the failed blade is located at a remote site.  
           [0005]    Accordingly, a need exists for a system and method for rebuilding a failed blade onto another blade. The system and method should be autonomous, i.e. requiring no human intervention, and easily implemented. The present invention addresses such a need.  
         SUMMARY OF THE INVENTION  
         [0006]    A method and system for autonomously rebuilding a failed one of a plurality of servers and a computer system utilizing the same is disclosed. In a first aspect, the method comprises providing a bus for allowing a recovery mechanism to access each of the plurality of servers and utilizing the recovery mechanism to rebuild the failed server onto another server. In a second aspect, the computer system comprises a plurality of servers, a management module for monitoring and managing the plurality of servers, a recovery mechanism coupled to the management module, and a bus coupling the recovery mechanism to each of the plurality of servers, wherein the recovery mechanism rebuilds a failed server onto another of the plurality of servers. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0007]    [0007]FIG. 1 is a perspective view illustrating the front portion of a BladeCenter.  
         [0008]    [0008]FIG. 2 is a perspective view of the rear portion of the BladeCenter.  
         [0009]    [0009]FIG. 3 is a schematic diagram of the Blade system&#39;s management subsystem.  
         [0010]    [0010]FIG. 4 is a topographical illustration of the Blade system&#39;s management functions.  
         [0011]    [0011]FIG. 5 is a schematic block diagram of the Blade system  500  according to a preferred embodiment of the present invention.  
         [0012]    [0012]FIG. 6 is a flowchart illustrating a process for rebuilding a failed blade processor according to a preferred embodiment of the present invention. 
     
    
     DETAILED DESCRIPTION  
       [0013]    The present invention relates generally to server systems and, more particularly, to a method and system for autonomously rebuilding a failed server onto another server. The following description is presented to enable one of ordinary skill in the art to make and use the invention and is provided in the context of a patent application and its requirements. Although the preferred embodiment of the present invention will be described in the context of a BladeCenter, various modifications to the preferred embodiment and the generic principles and features described herein will be readily apparent to those skilled in the art. Thus, the present invention is not intended to be limited to the embodiment shown but is to be accorded the widest scope consistent with the principles and features described herein.  
         [0014]    According to a preferred embodiment of the present invention, a recovery mechanism rebuilds the hard drive of a failed server onto the hard drive of another server in response to the detection of the failed server. The recovery mechanism preferably utilizes a bus that provides access to each server and allows the data on the failed server&#39;s hard drive to be copied and transferred to a hard drive of another server. In a system and method in accordance with the present invention, the failed server is rebuilt promptly and without human intervention. An administrator no longer is required to physically remove and remount the hard drive, thereby saving time and cost. Thus, the downtime for the failed server is minimized and QoS is improved.  
         [0015]    To describe further the features of the present invention, please refer to the following discussion and Figures, which describe a computer system, such as the BladeCenter, that utilizes the preferred embodiment of the present invention. FIG. 1 is an exploded perspective view of the BladeCenter system  100 . Referring to this figure, a main chassis  102  houses all the components of the system. Up to 14 server blades  104  (or other blades, such as storage blades) are plugged into the 14 slots in the front of chassis  102 . Blades  104  may be “hot swapped” without affecting the operation of other blades  104  in the system  100 . A server blade  104   a  can use any microprocessor technology so long as it is compliant with the mechanical and electrical interfaces, and the power and cooling requirements of the system  100 .  
         [0016]    A midplane circuit board  106  is positioned approximately in the middle of chassis  102  and includes two rows of connectors  108 ,  108 ′. Each one of the  14  slots includes one pair of midplane connectors, e.g.,  108   a ,  108   a ′, located one above the other, and each pair of midplane connectors, e.g.,  108   a ,  108   a ′ mates to a pair of connectors (not shown) at the rear edge of each server blade  104   a.    
         [0017]    [0017]FIG. 2 is a perspective view of the rear portion of the BladeCenter system  100 , whereby similar components are identified with similar reference numerals. Referring to FIGS. 1 and 2, a second chassis  202  also houses various components for cooling, power, management and switching. The second chassis  202  slides and latches into the rear of main chassis  102 .  
         [0018]    As is shown in FIGS. 1 and 2, two optionally hot-plugable blowers  204   a ,  204   b provide cooling to the blade system components. Four optionally hot-plugable power modules  206  provide power for the server blades and other components. Management modules MM 1  and MM 2  ( 208   a ,  208   b ) can be hot-plugable components that provide basic management functions such as controlling, monitoring, alerting, restarting and diagnostics. Management modules  208  also provide other functions required to manage shared resources, such as multiplexing the keyboard/video/mouse (KVM) to provide a local console for the individual blade servers  104  and configuring the system  100  and switching modules  210 .  
         [0019]    The management modules  208  communicate with all of the key components of the system  100  including the switch  210 , power  206 , and blower  204  modules as well as the blade servers  104  themselves. The management modules  208  detect the presence, absence, and condition of each of these components. When two management modules are installed, a first module, e.g., MM 1  ( 208   a ), will assume the active management role, while the second module MM 2  ( 208   b ) will serve as a standby module.  
         [0020]    The second chassis  202  also houses up to four switching modules SM 1  through SM 4  ( 210   a - 210   d ). The primary purpose of the switch module is to provide interconnectivity between the server blades ( 104   a - 104   n ), management modules ( 208   a ,  208   b ) and the outside network infrastructure (not shown). Depending on the application, the external interfaces may be configured to meet a variety of requirements for bandwidth and function.  
         [0021]    [0021]FIG. 3 is a schematic diagram of the server blade system&#39;s management subsystem  300 , where like components share like identifying numerals. Referring to this figure, each management module ( 208   a ,  208   b ) has a separate Ethernet link ( 302 ), e.g., MM 1 -Enet 1 , to each one of the switch modules ( 210   a - 210   d ). In addition, the management modules ( 208   a ,  208   b ) are coupled to the switch modules ( 210   a - 210   d ) via two serial  12 C buses ( 304 ), which provide for “out-of-band” communication between the management modules ( 208   a ,  208   b ) and the switch modules ( 210   a - 210   d ). Two serial buses ( 308 ) are coupled to server blades PB 1  through PB 14  ( 104   a -  104   n ) for “out-of-band” communication between the management modules ( 208   a ,  208   b ) and the server blades ( 104   a - 104   n ).  
         [0022]    [0022]FIG. 4 is a topographical illustration of the server blade system&#39;s management functions. Referring to FIGS. 3 and 4, each of the two management modules ( 208 ) has an Ethernet port  402  that is intended to be attached to a private, secure management server  404 . The management module firmware supports a web browser interface for either direct or remote access. Each server blade ( 104 ) has a dedicated service processor  406  for sending and receiving commands to and from the management module  208 . The data ports  408  that are associated with the switch modules  210  can be used to access the server blades  104  for image deployment and application management, but are not intended to provide chassis management services. The management module  208  can send alerts to a remote console, e.g.,  404 , to indicate changes in status, such as removal or insertion of a blade  104  or module. The management module  208  also provides access to the internal management ports of the switch modules  210  and to other major chassis subsystems (power, cooling, control panel, and media drives).  
         [0023]    Referring again to FIGS. 3 and 4, the management module  208  communicates with each server blade service processor  406  via the out-of-band serial bus  308 , with one management module  208  acting as the master and the server blade&#39;s service processor  406  acting as a slave. For redundancy, there are two serial busses  308  (one bus per midplane connector) to communicate with each server blade&#39;s service processor  406 .  
         [0024]    In general, the management module ( 208 ) can detect the presence, quantity, type, and revision level of each blade  104 , power module  206 , blower  204 , and midplane  106  in the system, and can detect invalid or unsupported configurations. The management module ( 208 ) will retrieve and monitor critical information about the chassis  102  and blade servers ( 104   a -  104   n ), such as temperature, voltages, power supply, memory, fan and HDD status. If a problem is detected, the management module  208  can transmit a warning to a system administrator via the port  402  coupled to the management server  404 . If the warning is related to a failing blade, e.g.,  104   a , the system administrator must replace the failed blade  104   a . In order to preserve the information on the failed blade&#39;s  104   a  hard drive, the administrator must manually remove the hard drive and remount it into a replacement blade. This process is labor intensive and economically costly. The present invention resolves this problem.  
         [0025]    Please refer now to FIG. 5, which is a schematic block diagram of a blade system  500  according to a preferred embodiment of the present invention. For the sake of clarity, FIG. 5 depicts one management module  208  and three blades: a failed blade  504   a , an operational blade  504  and a spare blade  504   b . As is shown in FIG. 5, the failed blade  504   a and the spare blade  504   b  are mounted in the same chassis (not shown). The system administrator (not shown) preferably uses the management module  208  to designate at least one blade to be the spare blade  504   b.    
         [0026]    The spare blade  504   b  is compatible with the blade type, in this case a server blade  504   a , to which it has been designated as a spare. For example, within a chassis  102 , several blade types, e.g., servers and storage blades, can be housed. The spare blade  504   b  for a server blade  504   a  will include system components compatible with those in the server blade  504   a , i.e., the spare blade&#39;s hard drive  502  is compatible with the server blade&#39;s hard drive  502 ; whereas the spare blade for a storage blade will include system components compatible with those in the storage blade.  
         [0027]    Each blade  504  includes a service processor  508  that is coupled to a central processing unit (CPU)  506 . The management module  208  communicates with each blade&#39;s service processor  508  via the out-of-band serial bus  308 . A standard IDE or SCSI interface bus  510  couples a plurality of peripheral devices  502 ,  502 ′,  502 ″, such as the hard drive  502 , to the CPU  506 , via a select module  512 . Preferably, the select module  512  directs traffic to and from the IDE or SCSI interface bus  510  in one of two directions, to the CPU  506  or to a hard-drive-direct access (HDDA) bus  518 . As is shown, the HDDA bus  518  preferably provides direct access to the hard drive  502  of each of the blades  504 ,  504   a ,  504   b.    
         [0028]    According to a preferred embodiment of the present invention, a recovery mechanism  516  is coupled to the management module  208  and controls the select module  512  via a control bus  514 . Therefore, the recovery mechanism  516  controls whether traffic on the SCSI bus  510  flows to the CPU  506  or to the HDDA bus  518 . While the recovery mechanism  516  is preferably in the management module  208 , it can also be a stand alone system coupled to the management module  208 . Moreover, the functionality of the control bus  514  can be incorporated into the HDDA bus  518 , as those skilled in the art would readily appreciate.  
         [0029]    At power up and under normal conditions, e.g., when all blades  504  are operating, the recovery mechanism  516  disables the HDDA bus  518  so that each blade&#39;s processor  506  has exclusive access to its associated hard drive  502 . If a blade  504   a  fails, however, the recovery mechanism  516  enables the HDDA bus  518 , activates the select module  512  in the failed blade  504   a  and in the spare blade  504   b , and copies data from the hard drive  502  of the failed blade  504   a  to the hard drive  502  of the designated spare blade  504   b  via the HDDA bus  518 .  
         [0030]    [0030]FIG. 6 is a flowchart illustrating a process for rebuilding a failed blade processor according to a preferred embodiment of the present invention. If a failure occurs, the failed blade&#39;s  504   a  service processor  508  will issue a warning to the management module  208  in step  602 . Once the management module  208  receives such a warning, it invokes the recovery mechanism  516 , which disables the processor access to the hard drive  502  via the control bus  514  and enables the HDDA bus  518 , via step  604 . The recovery mechanism  516  now has direct access to the hard drive(s)  502  of the failed blade  504   a . Once the HDDA bus  504  is enabled, the recovery mechanism  516  causes the data on the hard drive  502  of the failed blade  504   a  to be copied (in step  606 ) and transferred to the hard drive  502  of the spare blade  504   b  via the HDDA bus  504  in step  608 . In a preferred embodiment, the entire hard drive content residing on the failed blade  504   a  is transferred to the spare blade  504   b.    
         [0031]    While the spare blade  504   b  can be an “extra” blade that becomes operational only when it “replaces” a failed blade  504   a , it can also be a fully operational blade  504  in a server farm. Under such circumstances, the operational blade  504  can be taken off-line and used to replace the failed blade  504   a  if the quality of service (QoS) terms required by a user of the failed blade  504   a  requires another blade and the QoS terms of a user of the operational blade  504  allow the server farm administrator to degrade overall service to the operational blade  504  user.  
         [0032]    Once the contents of the hard drive  502  of the failed blade  504   a  have been transferred to the spare blade  504   b , the recovery mechanism  516  disables the HDDA bus  518  and restores processor  506  access to the hard drive  502  of the spare blade  504   b  via the control bus  514  (in step  610 ). At this point, the recovery mechanism  516  returns control to the management module  208 , which then powers down the failed blade  504   a . In step  612 , the management module  208  reassigns any chassis resources, e.g., a virtual LAN, from the failed blade  504   a  to the spare blade  504   b , and enables the spare blade  504   b  so that it can assume the failed blade&#39;s  504   a  identity and resume the same system operation. Finally, in step  614 , the management module  208  can transmit an alert to the administrator that the failed blade  504   a  should be replaced. In a preferred embodiment, once the failed blade  504   a is replaced with a new blade, that blade can be the designated spare blade  504   b.    
         [0033]    Through aspects of the present invention, a failed blade can be rebuilt onto another blade autonomously. Upon being notified of the failed blade, the recovery mechanism causes the entire contents of the hard drive of the failed blade to be transferred to the hard drive of the spare blade, which eventually assumes the identity of the failed blade. Because the failed blade is rebuilt promptly and without human intervention, the downtime for the failed blade is minimized and QoS is improved. The administrator no longer is required to physically remove and remount the hard drive, thereby saving time and cost.  
         [0034]    While the preferred embodiment of the present invention has been described in the context of a BladeCenter environment, the functionality of the recovery mechanism  516  could be implemented in any computer environment where the servers are closely coupled. Thus, although the present invention has been described in accordance with the embodiments shown, one of ordinary skill in the art will readily recognize that there could be variations to the embodiments and those variations would be within the spirit and scope of the present invention. Accordingly, many modifications may be made by one of ordinary skill in the art without departing from the spirit and scope of the appended claims.