Abstract:
A plurality of servers arranged in a cluster, such as a plurality of blade servers, may communicate with a network over a first communication path. The servers may also communicate with each other over another communication path such as an out-of-band channel. Using the out-of-band channel, a server that is having a communication or a software failure may communication with other blade servers using the out-of-band channel. The other blade servers may act as a proxy for the server with a problem, enabling the proxy to communicate on behalf of the server having a communication problem. As a result, useful computational work may be obtained in many cases from servers that would otherwise be taken off-line.

Description:
BACKGROUND 
   This invention relates generally to servers and, particularly, to clusters or groups of servers that operate together. 
   Commonly, groups of servers are provided to execute complex tasks. Commonly server farms include large numbers of servers. These servers work together in either a peer-to-peer arrangement or in a variety of other hierarchies. 
   One type of clustered server is called a blade server. A blade server may be a thin module or electronic circuit board, usually for a single, dedicated application, such as serving web pages. A blade server is designed to be mounted in a blade server rack with a large number of other blade servers. 
   When any one of a large number of blade servers in a rack suffers failure in any of its components or its local disk, the failed blade server is simply considered a lost cause. A blade server may be taken off-line if it requires a boot from a failed local disk or network program load from a defective network interface card for its operating system loader. At this point, the failed blade server becomes inoperative hardware necessitating human interaction with the system in order to replace the component. 
   Moreover, the inability of one server to function may adversely impact the overall operation of the entire cluster of servers. Thus, the failure of even one server may have a relatively significant, if not catastrophic, impact on the overall operation of the cluster of servers. 
   Thus, there is a need for better ways to enable clusters of servers to handle defects that occur in one or more of the servers in the cluster. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a schematic depiction of one embodiment of the present invention; 
       FIG. 2  is a flow chart for software in accordance with one embodiment of the present invention; and 
       FIG. 3  is a schematic depiction of a blade server in accordance with one embodiment of the present invention. 
   

   DETAILED DESCRIPTION 
   Referring to  FIG. 1 , an array or cluster of servers may communicate through network connections  14 . In one embodiment, the array or cluster of servers may be a cluster of blade servers mounted in a rack  15  including a housing  18 . However, the present invention is not limited to any particular type of server. 
   Each level  20  in the rack  15  may include a plurality of openings  22  to receive blade servers  24 , for example in a slide fit connection. 
   Each of the servers  24  in a layer  20  may communicate through an out-of-band (OOB) back plane  16 . The back plane  16  may enable back plane communication between the blade servers  24  in a given rack  15 . At the same time, each server  24  connects through external connections  14  to an external network. The external network may include the Internet, a local area network, or any other network. 
   Thus, in some embodiments, the rack  15  of blade servers  24  may be coupled to the Internet to act as a web server. In other embodiments, the blade server rack  15  may be part of a large data center. 
   Referring to  FIG. 2 , the various external network connections  14  may couple to an external network  24 . The external network  24  may couple, in turn, to a remote configuration server  24   a  and a remote boot server  24   b  in one embodiment. The boot server  24   b  may be responsible for providing configuration information or target operating system information to facilitate booting in one embodiment. The configuration server  24   a  may be an external agent that may be remote from the rack  15  and that communicates with each of the blade servers  24  in the rack over the external network  24 , monitoring the status of the blade servers  24  in one embodiment. 
   Any number of blade servers  24  may then be coupled to the external network over the network connections  14 . Failure detect software  25  may be responsible for handling any failure of any given blade server  24  within the rack or cluster of blade servers  24 . While the following example involves a blade server, those skilled in the art will appreciate that the principles of the present invention may apply to any cluster of servers. 
   Initially, the failure detect software  25  determines that a system power-on event has occurred as indicated in block  26 . An out-of-band driver may then be installed, for example in the system management mode (SMM) memory space, as indicated in block  28  in one embodiment. The out-of-band driver enables inter-server  24  communication through the out-of-band back plane  16 . The driver may also be responsible, in some embodiments, for monitoring the out-of-band back plane  16  for various agents that may be trying to communicate information. 
   Once the driver has been installed, the boot server  24   b  may be called, as indicated in block  30 , in order to boot a particular blade server  24  running the software  25 . A check at diamond  32  determines whether a connection failure has been detected. A connection failure may be any inability to connect over the network by any blade server  24  within the cluster. For example, a blade server  24  may not be able to connect to the network boot server  24   b  over the network connections  14 . If no connection failure has been detected, the boot image is read, as indicated in block  34 , and a target image is run as indicated in block  36 . 
   If a connection failure is detected, the OOB back plane  16  is used to trigger a system management interrupt (SMI) or other notification on another blade server  24 . Thus, one blade server  24 , called a failed blade server, may have a connection failure and another blade server  24 , called a proxy blade server, may be triggered by the failed blade server over the OOB back plane  16  as indicated in block  40 . The proxy blade server proxies the boot request for the failed blade server in the case where the connection failure is a failure to communicate with the boot server  24   b . In other cases, the proxy blade server proxies a communication for the failed boot server. 
   In the face of an incoming communication failure, the requested image or data is read from the external network connection  14  by the proxy blade server  24 . This information may then be provided over the OOB back plane  16  to the failed blade server  24 . Thereafter, the boot image may be read and the flow may continue. 
   If a blade server  24  crashes or hangs for example, or it experiences any other failure, as indicated in block  38 , the failed blade server advertises its situation to other blade servers using the OOB back plane  16 . Other blade servers  24  may receive a remote reset request, indicated in block  46 , as a proxy for the failed blade server. That reset request may be communicated over the OOB back plane  16  to the failed blade server which may then be reset as indicated at block  46 . 
   The proxy blade server that receives information over the OOB back plane  16  may broadcast the situation of the failed blade server over the back plane  16 . The proxy blade server may also receive a response over the back plane  16  or connections  14  to the broadcast, for example, on behalf of the failed blade server as indicated in block  48 . Thus, a proxy blade server may handle either incoming or outgoing communications on behalf of a failed server. 
   Referring to  FIG. 3 , in one embodiment of an architecture, each blade server  24  may include a processor  50  coupled to an interface  52 . The interface  52  may couple a bus  56  and system memory  54 . The bus  56  may be coupled to an interface  58  to a hard disk drive  60 . The bus  56  may also be coupled through an interface  62  to the external network  24 . The bus  56  may also couple to a memory  64  that stores the basic input/output system (BIOS). The basic input/output system may include the failure detect software  25  in one embodiment of the present invention. The bus  56  is also coupled to the OOB interface  66  that, in turn, couples to the OOB back plane  16  in one embodiment of the present invention. 
   Thus, the OOB driver may be utilized to communicate through the OOB back plane  16 . The driver is responsible for communications onto the OOB back plane  16  and from the OOB back plane  16 . 
   By enabling another blade server  24  to proxy communications on behalf of the failed blade server  24 , the operability of the overall system may be preserved despite the failure of any one given blade server  24 . For example, a proxied restart may allow a blade server  24  to boot into an operational environment and perform useful work during the period between its device failure and an upgrade. The blade server  24  may still accept compute jobs while waiting for a service call to update its failed disk or connection. This may allow for service calls to be deferred and the owner of the blade server to continue to get useful results from the computational element. 
   In addition, the ability to have a shared state across the OOB, such as a system management interrupt, a reset, or other signaling to pure blade servers, may allow the individual mean time between failure to be extended in some cases. Specifically, any single blade server failure hang can have a peer unhang it via a remotely activated signal. In the case of a system management interrupt-based signaling, one blade server can act as a watchdog timer for its peer blade servers, with the watchdog state machine being managed transparently to the operating system at run time. 
   Finally, the peer blade-to-peer blade topology allows for any blade server to act as a manageability proxy server for its neighboring blade server, in some embodiments. So in the case of a hung blade server that cannot access its local input/output resources, such as a disk (for log-file) or a network, system management mode-based recovery firmware in the failed blade server can communicate via the OOB channel to a peer blade server, in some embodiments. In this case, a peer blade server can proxy the failed unit&#39;s crisis messages to a provisioning or other management servers. 
   While the principles of the present invention may be applicable to blade servers that are in a peer-to-peer arrangement, it is also applicable to blade servers in chassis manager/many blades topology or chassis management modules. It may also be applicable to share nothing blade topologies where the rack is “dumb” with respect to the blade servers and the rack simply provides common power and some integrated network switch. In such case, the OOB may be added, for example as a secondary network interface card on each blade server within an associated switch or hub for the blade servers. 
   While the present invention has been described with respect to a limited number of embodiments, those skilled in the art will appreciate numerous modifications and variations therefrom. It is intended that the appended claims cover all such modifications and variations as fall within the true spirit and scope of this present invention.