Patent Description:
As industry moves to a hybrid-cloud infrastructure model of a data-center architecture, there is increasing demand for high performance fault-tolerant compute systems forming a low-latency on-premises cloud that allows linear scaling of infrastructure. Among such systems, one popular implementation is a cluster having four nodes, e.g., blade servers, in one 2U rack chassis. Such an implementation allows three nodes to form a minimal cluster with one node operating as an active standby to take the load in case of any other single node failure. Each cluster includes a well-balanced amount of storage, compute and network hardware, and operates as a single unit over a high-speed local bus interconnect, such as Peripheral Component Interconnect Express (PCle), to achieve maximum performance out of a given hardware configuration. Such a configuration may be referred to as a "hyperconverged infrastructure. " Each node in the cluster also includes a baseboard management controller (BMC), which manages its respective motherboard. Unlike traditional server chassis systems, in a hyper-converged infrastructure there are no separate input/output (IO) hardware plugin modules that perform IO aggregation and that send data out of the chassis to storage enclosures connected through external top of the rack switches. This architectural shift makes chassis management "lighter" compared to traditional architectures by enabling chassis-level resource sharing including, e.g., cooling-fans and power supply units (PSUs), among other shared resources. In conventional approaches, a separate Chassis Management Controller (CMC) and its identical active standby twin are used to manage such shared resources.

<CIT> describes an information handling system and method of a master baseboard management controller election and replacement sub-system (MBMCERS) enabling decentralized resource management control via the elected master baseboard management controller (BMC). The information handling system includes a plurality of server nodes, each having a BMC capable of controlling a plurality of shared common resources among the plurality of server nodes. Each BMC has a unique BMC identification. A master register stores BMC identification that has been elected as the master BMC to control the shared common resources. The master BMC relinquishes control of the shared common resources when the master register is placed in the reset state. When the master register is in the reset state, any one of the BMCs can elect to become a replacement master BMC.

<CIT> describes systems, methods, and computer-readable storage devices for reducing the amount of management ports (and associated cabling) for a top-of-rack server environment. Whereas other server management configurations have cabling connecting each node in multiple multi-node chassis in a server rack to a top-of-rack, systems configured as described herein designate a single node as a point of communication for the multi-node chassis. The designated node forwards communications for all nodes in the chassis to a chassis management controller, which acts as a distribution point for all communications within the multi-node chassis, with the benefit of only a single connection being required between the multi-node chassis and the top of rack switch.

In one embodiment, a method is provided. In a chassis comprising a plurality of nodes, a network switch, and a programmable device configured to manage a shared resource of the chassis, the method includes establishing, using the network switch, a dedicated network among baseboard management controllers of respective nodes in the plurality of nodes; and using the dedicated network, automatically selecting a given node from the plurality of nodes to function as a master node to program the programmable device on behalf of all nodes in the plurality of nodes to manage the shared resource of the chassis on behalf of all the nodes in the plurality of nodes.

According to the claimed invention, in the event that the given node receives a communication from a further node from the plurality of nodes indicating that the further node is also, simultaneously, functioning as a master node, the method further comprises selecting as the master node the node having a lower slot ID in the chassis. Furthermore, each of the respective baseboard management controllers of nodes is configured to periodically receive node data, via a broadcasted transmission, from each other node in the plurality of nodes via the dedicated network; and each of the respective baseboard management controllers of the nodes stores, in memory, node data from each other node in the plurality of nodes.

In another embodiment, an apparatus is provided and includes: a chassis; a network switch; a programmable device configured to manage a shared resource of the chassis; and a plurality of nodes disposed in the chassis, wherein each node in the plurality of nodes comprises a baseboard management controller and a network interface to communicate with the network switch, wherein the plurality of nodes and the network switch define a dedicated network, wherein respective baseboard management controllers of each of the nodes in the plurality of nodes are configured to automatically select a given node from the plurality of nodes to function as a master node to program the programmable device on behalf of all nodes in the plurality of nodes to manage the shared resource of the chassis on behalf of all the nodes in the plurality of nodes.

According to the claimed invention, in the event that the given node receives a communication from a further node from the plurality of nodes indicating that the further node is also, simultaneously, functioning as a master node, the apparatus is further configured to select as the master node the node having a lower slot ID in the chassis. Furthermore, each of the respective baseboard management controllers of nodes is configured to periodically receive node data, via a broadcasted transmission, from each other node in the plurality of nodes via the dedicated network; and each of the respective baseboard management controllers of the nodes stores, in memory, node data from each other node in the plurality of nodes.

Presented herein are approaches and methodologies to achieve shared hardware resources management without having to rely on separate Chassis Management Controller (CMC) hardware or software, by having the BMCs of respective nodes in a cluster of nodes on a chassis actively communicate with one another to intelligently select one among them as a master node that controls and monitors the shared chassis resources. More specifically, the disclosed embodiments define a unique way of discovering dynamic node insertion onto the chassis, a methodology to contest mastership among peer BMCs (or nodes), and a unique way of requesting and handing over mastership to a requesting peer BMC (or node), without losing chassis management context. In an embodiment, fault tolerance is built into each stage of these approaches to ensure that catastrophic failures are eliminated even in the case of hardware malfunctions. A user may be notified of errors as they might occur.

Reference is first made to <FIG>, which is a schematic diagram of a multi-node chassis (or, simply, "chassis") <NUM> according to an example embodiment. Chassis <NUM> includes receptacles (not shown) to accommodate, in the example shown, four (computer, blade server, etc.) nodes <NUM>. Each node <NUM> includes, among other things, a network interface <NUM> and a BMC <NUM>. The BMC <NUM> includes a processor <NUM>, and memory <NUM>, which stores master selection logic <NUM>, which can be executed by processor <NUM>. Details of master selection logic <NUM> are described more fully below. Network interface <NUM> is configured to support multiple interfaces including, e.g., two Ethernet interfaces. A first Ethernet network interface is to a network <NUM> that is used for dedicated communication among nodes <NUM> through a multi-port switch <NUM> built onto the chassis <NUM>. Network <NUM> may also be referred to as a "dedicated network. " In the example of <FIG>, network <NUM> comprises BMC1_ETH, BMC2_ETH, BMC3_ETH and BMC4_ETH, and is used to exchange management data between the nodes <NUM>. A second Ethernet network interface is employed to manage the BMCs, through remote interfaces, via, e.g., an enterprise local area network.

Network interface <NUM> may also support an Inter-Integrated Circuit (I2C) bus BMC1_i2C, BMC2_i2C, BMC3_i2C, BMC4_i2C (including a field replaceable unit (FRU) <NUM>), and a general purpose input/output interface BMC1_GPIO(<NUM>), BMC2_ GPIO(<NUM>), BMC3_ GPIO(<NUM>), BMC4_ GPIO(<NUM>) ("GPIOs"). The I2C bus and general purpose input/output interface may be considered to be communication paths different from network <NUM>.

A programmable device <NUM> (such as a microprocessor, programmable logic device (PLD) or programmable system on a chip (PSoC)) is provided to control shared resources such as, e.g., fans <NUM>, power supply units (PSUs) <NUM>, which provide, e.g., 12V main and standby power, light emitting diodes (LEDs) on a front panel <NUM>, and temperature sensors (not shown). Programmable device <NUM> itself is programmed, or updated, by one of the BMCs <NUM> associated with a node <NUM> that becomes a designated "master" node. Programming the programmable device <NUM> may include (re)installing firmware, configuring registers, and/or changing settings, among other possible programming operations. Programming or updating of the programmable device <NUM> may be performed, e.g., via the I2C bus (or possibly via the general purpose IO interface on the programmable device <NUM>). In one implementation, the GPIOs are used to detect node presence. Upon insertion of a node into the chassis <NUM>, an associated GPIO pin will become grounded or pulled to a high voltage. That ground or high voltage signal is provided, via the GPIOs, to programmable device <NUM>, which may also include memory to store presence and/or faulty node information <NUM>. In turn, such presence and/or faulty node information <NUM> may be provided to each of the BMCs over the I2C bus. This implementation provides a fault tolerant secondary method used to detect internal network failure due to hardware malfunction, such as a bent pin, broken socket, etc. while inserting / ejecting a node.

In an embodiment, the master selection logic <NUM> of a designated master node monitors and controls shared resources by aggregating data from all nodes <NUM> on the chassis <NUM>. The methodologies described below define how to discover multiple nodes <NUM> in the chassis <NUM>, and how to elect the master node, among nodes <NUM>. Also described below is a mastership transfer procedure that can handle error scenarios such as inadvertent presence of multiple master nodes in the cluster.

In an embodiment, data transfer over network <NUM>, is carried out using two types of data packets: a node data packet (sent by a slave node, i.e., not a master node) and a master data packet (sent by a master node).

A master data packet includes chassis telemetry information such as presence of fans <NUM>, fan speed, presence of PSU <NUM>, and voltage information, and protocol version among other possible parameters. The master data packet is broadcasted only by a master node to all slave nodes. The node data packet includes node information employed by the designated master node to control chassis components. The node information might include temperature, desired fan speed, slot ID, a protocol version, among other possible parameters. The node information is broadcasted in node data packets by all nodes <NUM> to their peers. This node information may be stored in each node in, e.g., memory <NUM>. Master selection logic (or, simply, "logic") <NUM> is configured to send and receive the master data packets and the node data packets, as explained below.

<FIG> is a flow chart depicting a series of operations for multi-node discovery. At a high level, the operations shown in <FIG> enable each node's data to be populated in its own shared memory, e.g., memory <NUM>. Then, this data is broadcasted by each node to peer nodes every, e.g., <NUM>. At the same time, each node also listens for peer node data over an assigned port. Receipt of node data packets from peer nodes is also used as a primary means of detecting node presence. Information gleaned via the GPIOs, as noted above, is a secondary means of detecting node presence.

More specifically, at operation <NUM>, logic <NUM>, operating independently on each node <NUM>, creates a shared memory location to store, in respective memory segments, node data for each of a maximum number of supported nodes <NUM> in the chassis <NUM>. At <NUM>, logic <NUM> populates its own node data in the shared memory location. At <NUM>, logic <NUM> sends its own node data via broadcast node data packets over assigned ports. At <NUM>, after a delay of, e.g., <NUM>, the logic <NUM> again populates node data for itself. In this manner, each node <NUM> gathers its own node data periodically and transmits that data over network <NUM> every, e.g., <NUM>. At the same time, at <NUM>, the logic <NUM> listens for node data packets on the assigned port. At <NUM>, it is determined whether any node data packets have been received. If not, at <NUM>, it is determined that no peer node is detected. On the other hand, if node data packets have been received, then, at <NUM>, it is determined that a peer node has been detected, and data received in such node data packets is updated in the shared memory, thereby enabling logic <NUM> on each BMC <NUM> to store all data associated with each node <NUM> in network <NUM>, namely, the cluster that comprises the four nodes <NUM> shown in <FIG>. According to the claimed invention, each of the respective baseboard management controllers of nodes is configured to periodically receive node data, via a broadcasted transmission, from each other node in the plurality of nodes via the dedicated network; and each of the respective baseboard management controllers of the nodes stores, in memory, node data from each other node in the plurality of nodes.

Reference is now made to <FIG>, which is a flow chart depicting a series of operations for electing or designating one of the nodes <NUM> in the cluster to be a master node.

In general, when a node <NUM> boots up, it is configured, by default, to go into slave mode. In slave mode, the node <NUM> waits for a master data packet sent from a master node for a variable wait time (wt) defined by following formula:<MAT>.

Once the wait time elapses, if a master data packet is not received, logic <NUM> is configured to cause a slave node to acquire mastership. A different wait time for each node <NUM> ensures that no two nodes <NUM> acquire mastership at the same time. If a new node <NUM> joins an existing cluster, the new node will capture a master data packet and will continue in slave mode. If the master node is rebooted/removed, existing slave nodes will restart the master election process.

More specifically, and as shown in <FIG>, at <NUM>, logic <NUM> creates a timer and arms the timer with "wait time" seconds according to a formula such as noted above, where SlotID is a slot ID of a slot in which a given node <NUM> is mounted in the chassis <NUM>. At <NUM>, logic <NUM> listens for a master packet on network <NUM> on an assigned port. At <NUM>, logic <NUM> determines whether a master data packet has been received. If yes, this suggests that one node <NUM> in the cluster is already acting as the master node and, as such, at <NUM>, logic <NUM> rearms the timer with "wait time" seconds, and the process returns to <NUM> to listen again for a master data packet. If, on the other hand, at <NUM>, no master data packet was received, then at <NUM>, it is determined whether the timer has expired, after a predetermined amount of time, e.g., <NUM> seconds. If the time has expired, then, at <NUM>, logic <NUM> designates the node as the master node.

Reference is now made to <FIG>, which is a flow chart depicting a series of operations for a fault tolerance process. Under selected inadvertent conditions, network connectivity may be flawed, causing a given node <NUM> to disconnect from the switch <NUM> connecting the cluster nodes <NUM> together. For example, such conditions might include a bent pin on a connector, or a cable malfunction. A node <NUM> that fails to receive its own or a peer node's advertised node data packets is marked faulty, which withdraws that node from the master node election process. Programmable device <NUM> may keep track of which nodes are present and/or faulty, as mentioned previously. In an embodiment, an existing master node that is subsequently designated as faulty should leave mastership, and such an event invokes mastership arbitration logic among remaining healthy nodes.

In a scenario where no nodes <NUM> are able to communicate with each other, all the nodes <NUM> are marked faulty (in, e.g., programmable device <NUM>) and none of them can participate in the master election process of <FIG>. A brute force algorithm may then be initiated that identifies the node with a lowest slot ID present in the cluster and forces that node to be the master node. This is done to ensure that a master node is present in the cluster which can supervise chassis functionality, and manage the shared resources.

In a rare scenario in which multiple masters become present in the cluster simultaneously, the process of <FIG> may be invoked resolve the situation gracefully. That is, if a node which is sending master data packets also receives master data packets from another node, the process of <FIG> may be triggered. In such a situation, logic <NUM> is configured to cause the node <NUM> with the lowest slot ID to continue with mastership and to cause other nodes to leave mastership.

More specifically, at <NUM>, one of the nodes <NUM>, a "receiving node" for purposes of this functionality, receives a master data packet on an assigned port. At <NUM>, logic <NUM> determines if the receiving node is also sending master data packets. That is, logic <NUM> is configured to detect whether there is more than one designated master node operating in the cluster. If, at <NUM>, it is determined that the receiving node is not also sending master data packets, then logic <NUM> determines, at <NUM>, that there is only a single master node in the cluster, and no further action is needed. If, on the other hand, at <NUM>, it is determined that the receiving node is also sending master data packets then, at <NUM>, logic <NUM> determines whether a slot ID of the receiving node is greater than a slot ID of the other node asserting mastership (by virtue of sending master data packets). If not, then at <NUM>, the receiving node continues with being the designated master and nothing further needs to be done. On the other hand, if the slot ID of the receiving node is greater than the slot ID of the other node asserting mastership, then logic <NUM>, at <NUM>, is configured to cause the receiving node to leave mastership. That is, logic <NUM> causes the receiving node to no longer function as a master node.

According to the claimed invention, in the event that the given node receives a communication from a further node from the plurality of nodes indicating that the further node is also, simultaneously, functioning as a master node, the node having a lower slot ID in the chassis is selected as the master node.

Once a cluster is up and running with an active master node, there may be situations in which a given slave node requests mastership. Such a situation may come about, for example, when the given node slave node receives a firmware update. The BMC <NUM> of such a slave node will want to become the master node in order to program the programmable device <NUM>. Described below is a methodology to achieve non-disruptive transfer of mastership.

A given slave node may make a request for mastership by updating a corresponding flag in its node data packet. The designated master node, on receiving this request, decides whether mastership can be granted or not. The master node may take into consideration in making such a decision a state of critical operations which may need continued mastership.

If mastership can be granted to the requesting slave node, the master node so notifies the slave node. The slave node, on receiving a mastership grant, sends an acknowledgment confirming it is ready to take mastership. This three-way-handshake helps to ensure that mastership is transferred gracefully. On receiving an acknowledgment from the requesting node, the master node leaves mastership and the requesting slave node becomes master.

Reference is now made to <FIG>, which illustrates a series of operations for a mastership transfer process. As shown, a slave node <NUM> communicates with a master node <NUM>. More specifically at <NUM>, slave node <NUM> sends a mastership grant request to master node <NUM>. Those skilled in the art will appreciate that logic <NUM> is configured to execute the above-described and following operations. At <NUM>, master node <NUM> receives the mastership grant request and, at <NUM>, determines whether mastership can be granted. If not, then, at <NUM>, master node <NUM> ignores the mastership grant request. If mastership can be granted, then master node <NUM> sends a mastership granted message <NUM> to slave node <NUM>. If, at <NUM>, mastership cannot be granted due, e.g., to a timeout of, e.g., two seconds, then the mastership grant request may be considered declined at <NUM>. If mastership is granted, then slave node <NUM> sends a mastership acknowledgment <NUM> to master node <NUM>. At <NUM>, master node <NUM> receives the mastership acknowledgment <NUM>, and master node <NUM>, at <NUM>, then configures itself to leave mastership. In connection with leaving mastership, master node <NUM> sends a left mastership message <NUM> to slave node <NUM>. Slave node <NUM> receives the left mastership message at <NUM>, and, at <NUM>, thereafter becomes the master node for the cluster.

<FIG> is a flow chart depicting a series of operations for operating a cluster of nodes, according to an example embodiment. At <NUM>, in a chassis comprising a plurality of nodes, a network switch, and a programmable device configured to manage a shared resource of the chassis, the plurality of nodes establish, using the network switch, a dedicated network among respective baseboard management controllers of nodes in the plurality of nodes. At <NUM>, using the dedicated network, the baseboard management controllers automatically select one node from the plurality of nodes as a master node to program the programmable device on behalf of all nodes in the plurality of nodes to manage the shared resource of the chassis on behalf of all the nodes in the plurality of nodes.

Those skilled in the art will appreciate that the data exchanges described in connection with <FIG> may be carried out via, e.g., network <NUM>.

In summary, a system and method are provided for a fault-tolerant and efficient way of discovering multiple nodes and electing a master among them. The disclosed methodology features a method of handling dynamic node insertion and removal without prior user input about the event. The methodology is configured to handle network failures gracefully by reconfiguring a cluster to ensure a master is chosen among connected nodes and only one master exists at any point in time.

<FIG> depicts a device <NUM> (e.g., a node <NUM>) that master selection logic <NUM>) in accordance with an example embodiment. It should be appreciated that <FIG> provides only an illustration of one embodiment and does not imply any limitations with regard to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made.

As depicted, the device <NUM> includes a bus <NUM>, which provides communications between computer processor(s) <NUM>, memory <NUM>, persistent storage <NUM>, communications unit <NUM>, and input/output (I/O) interface(s) <NUM>. Bus <NUM> can be implemented with any architecture designed for passing data and/or control information between processors (such as microprocessors, communications and network processors, etc.), system memory, peripheral devices, and any other hardware components within a system. For example, bus <NUM> can be implemented with one or more buses.

Memory <NUM> and persistent storage <NUM> are computer readable storage media. In the depicted embodiment, memory <NUM> includes random access memory (RAM) <NUM> and cache memory <NUM>. In general, memory <NUM> can include any suitable volatile or non-volatile computer readable storage media.

One or more programs (e.g., master selection logic <NUM>) may be stored in persistent storage <NUM> for execution by one or more of the respective computer processors <NUM> via one or more memories of memory <NUM>. The persistent storage <NUM> may be a magnetic hard disk drive, a solid state hard drive, a semiconductor storage device, read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, or any other computer readable storage media that is capable of storing program instructions or digital information. For example, the one or more programs may include software instructions that, when executed by the one or more processors <NUM>, cause the computing device <NUM> to perform the operations of, e.g., <FIG>.

Communications unit <NUM>, in these examples, provides for communications with other data processing systems or devices. In these examples, communications unit <NUM> includes one or more network interface cards. Communications unit <NUM> may provide communications through the use of either or both physical and wireless communications links.

I/O interface(s) <NUM> allows for input and output of data with other devices that may be connected to computer device <NUM>. For example, I/O interface <NUM> may provide a connection to external devices <NUM> such as a keyboard, keypad, a touch screen, and/or some other suitable input device. External devices <NUM> can also include portable computer readable storage media such as database systems, thumb drives, portable optical or magnetic disks, and memory cards.

Software and data used to practice embodiments can be stored on such portable computer readable storage media and can be loaded onto persistent storage <NUM> via I/O interface(s) <NUM>. I/O interface(s) <NUM> may also connect to a display <NUM>. Display <NUM> provides a mechanism to display data to a user and may be, for example, a computer monitor.

The programs described herein are identified based upon the application for which they are implemented in a specific embodiment. However, it should be appreciated that any particular program nomenclature herein is used merely for convenience, and thus the embodiments should not be limited to use solely in any specific application identified and/or implied by such nomenclature.

The present embodiments may employ any number of any type of user interface (e.g., Graphical User Interface (GUI), command-line, prompt, etc.) for obtaining or providing information (e.g., data relating to scraping network sites), where the interface may include any information arranged in any fashion. The interface may include any number of any types of input or actuation mechanisms (e.g., buttons, icons, fields, boxes, links, etc.) disposed at any locations to enter/display information and initiate desired actions via any suitable input devices (e.g., mouse, keyboard, etc.). The interface screens may include any suitable actuators (e.g., links, tabs, etc.) to navigate between the screens in any fashion.

The environment of the present embodiments may include any number of computer or other processing systems (e.g., client or end-user systems, server systems, etc.) and databases or other repositories arranged in any desired fashion, where the present embodiments may be applied to any desired type of computing environment (e.g., cloud computing, client-server, network computing, mainframe, stand-alone systems, etc.). The computer or other processing systems employed by the present embodiments may be implemented by any number of any personal or other type of computer or processing system (e.g., desktop, laptop, PDA, mobile devices, etc.), and may include any commercially available operating system and any combination of commercially available and custom software (e.g., machine learning software, etc.). These systems may include any types of monitors and input devices (e.g., keyboard, mouse, voice recognition, etc.) to enter and/or view information.

The various functions of the computer or other processing systems may be distributed in any manner among any number of software and/or hardware modules or units, processing or computer systems and/or circuitry, where the computer or processing systems may be disposed locally or remotely of each other and communicate via any suitable communications medium (e.g., LAN, WAN, Intranet, Internet, hardwire, modem connection, wireless, etc.). For example, the functions of the present embodiments may be distributed in any manner among the various end-user/client and server systems, and/or any other intermediary processing devices. The software and/or algorithms described above and illustrated in the flow charts may be modified in any manner that accomplishes the functions described herein. In addition, the functions in the flow charts or description may be performed in any order that accomplishes a desired operation.

The software of the present embodiments may be available on a non-transitory computer useable medium (e.g., magnetic or optical mediums, magneto-optic mediums, floppy diskettes, CD-ROM, DVD, memory devices, etc.) of a stationary or portable program product apparatus or device for use with stand-alone systems or systems connected by a network or other communications medium.

The communication network may be implemented by any number of any type of communications network (e.g., LAN, WAN, Internet, Intranet, VPN, etc.). The computer or other processing systems of the present embodiments may include any conventional or other communications devices to communicate over the network via any conventional or other protocols. The computer or other processing systems may utilize any type of connection (e.g., wired, wireless, etc.) for access to the network. Local communication media may be implemented by any suitable communication media (e.g., local area network (LAN), hardwire, wireless link, Intranet, etc.).

The system may employ any number of any conventional or other databases, data stores or storage structures (e.g., files, databases, data structures, data or other repositories, etc.) to store information (e.g., data relating to contact center interaction routing). The database system may be implemented by any number of any conventional or other databases, data stores or storage structures (e.g., files, databases, data structures, data or other repositories, etc.) to store information (e.g., data relating to contact center interaction routing). The database system may be included within or coupled to the server and/or client systems. The database systems and/or storage structures may be remote from or local to the computer or other processing systems, and may store any desired data (e.g., data relating to contact center interaction routing).

The embodiments presented may be in various forms, such as a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of presented herein.

A nonexhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.

Computer readable program instructions for carrying out operations of the present embodiments may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the "C" programming language or similar programming languages. In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects presented herein.

Aspects of the present embodiments are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the embodiments.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures.

In summary, in one form, a method is provided. The method includes in a chassis comprising a plurality of nodes, a network switch, and a programmable device configured to manage a shared resource of the chassis, establishing, using the network switch, a dedicated network among respective baseboard management controllers of nodes in the plurality of nodes; and using the dedicated network, automatically selecting a given node from the plurality of nodes to function as a master node to program the programmable device on behalf of all nodes in the plurality of nodes to manage the shared resource of the chassis on behalf of all the nodes in the plurality of nodes.

The method may further include the respective baseboard management controllers communicating with an enterprise local area network over a network, different from the dedicated network, to manage the respective baseboard management controllers.

In an embodiment, the shared resource include at least one of a fan for the chassis, a power supply unit for the chassis, light emitting diodes (LEDs) on a front panel of the chassis, and a temperature sensor for the chassis.

The method may still further include providing a communication path, different from the dedicated network, between each node of the plurality of nodes and the programmable device that is used to program the programmable device.

The communication path may be one of an Inter-Integrated Circuit (I2C) bus or general purpose input output line.

In an embodiment, the respective baseboard management controllers are configured to periodically receive node data from each of the nodes in the plurality of nodes via the dedicated network, and to store the node data.

The method may also include causing the given node to function as the master node after listening for, but not receiving after a predetermined amount of time, a master data packet from any other nodes in the plurality of nodes.

In an embodiment, the predetermined amount of time is determined based on a physical slot on the chassis in which the given node is mounted.

The method also include causing the given node to no longer function as the master node when the given node receives a master data packet from another node in the plurality of nodes.

The method may still further include receiving from another node in the plurality of nodes a master grant request which causes the given node to no longer function as the master node.

In another form, a device or apparatus may also be provided in accordance with an embodiment. The device may include a chassis; a network switch; a programmable device configured to manage a shared resource of the chassis; and a plurality of nodes disposed in the chassis, wherein each node in the plurality of nodes comprises a baseboard management controller and a network interface to communicate with the network switch, wherein the plurality of nodes and the network switch define a dedicated network, wherein respective baseboard management controllers of each of the nodes in the plurality of nodes are configured to automatically select a given node from the plurality of nodes to function as a master node to program the programmable device on behalf of all nodes in the plurality of nodes to manage the shared resource of the chassis on behalf of all the nodes in the plurality of nodes.

In an embodiment each respective baseboard management controller includes another network interface to an enterprise local area network, different from the dedicated network, to manage each respective baseboard management controller.

In an embodiment, the shared resource includes at least one of a fan for the chassis, a power supply unit for the chassis, light emitting diodes (LEDs) on a front panel of the chassis, and a temperature sensor for the chassis.

The device may further include a communication path, different from the dedicated network, between each node of the plurality of nodes and the programmable device that is used to program the programmable device.

In an embodiment, the communication path may be one of an Inter-Integrated Circuit (I2C) bus or a general purpose input output line.

In an embodiment, the baseboard management controller is configured to periodically receive node data from each of the nodes in the plurality of nodes via the dedicated network, and to store the node data.

In an embodiment, the baseboard management controller of the given node in the plurality of nodes is configured to cause the given node to function as the master node after listening for, but not receiving for a predetermined amount of time, a master data packet from any other nodes in the plurality of nodes.

In still another form, a non-transitory computer readable storage media is provided that is encoded with instructions that, when executed by a processor, cause the processor to establish, using a network switch, a dedicated network among respective baseboard management controllers of nodes in a plurality of nodes; and using the dedicated network, automatically select a given node from the plurality of nodes to function as a master node to program a programmable device on behalf of all nodes in the plurality of nodes to manage shared resource of a chassis on behalf of all the nodes in the plurality of nodes.

The instructions may further include instructions that, when executed by a processor, cause the processor to communicate with an enterprise local area network over a network, different from the dedicated network, to manage the respective baseboard management controllers.

Claim 1:
A method comprising:
in a chassis (<NUM>) comprising a plurality of nodes (<NUM>), a network switch (<NUM>), and a programmable device (<NUM>) configured to manage a shared resource of the chassis (<NUM>), establishing, using the network switch (<NUM>), a dedicated network (<NUM>) among respective baseboard management controllers (<NUM>) of nodes (<NUM>) in the plurality of nodes (<NUM>); and
using the dedicated network (<NUM>), automatically selecting a given node (<NUM>) from the plurality of nodes (<NUM>) to function as a master node to program the programmable device (<NUM>) on behalf of all nodes (<NUM>) in the plurality of nodes (<NUM>) to manage the shared resource of the chassis (<NUM>) on behalf of all the nodes (<NUM>) in the plurality of nodes (<NUM>); and
characterised by
in the event that the given node (<NUM>) receives a communication from a further node (<NUM>) from the plurality of nodes (<NUM>) indicating that the further node (<NUM>) is also, simultaneously, functioning as a master node, selecting as the master node the node (<NUM>) having a lower slot ID in the chassis (<NUM>); wherein
each of the respective baseboard management controllers (<NUM>) of nodes (<NUM>) is configured to periodically receive node data, via a broadcasted transmission, from each other node (<NUM>) in the plurality of nodes (<NUM>) via the dedicated network (<NUM>); and wherein
each of the respective baseboard management controllers (<NUM>) of the nodes (<NUM>) stores, in memory (<NUM>), node data from each other node (<NUM>) in the plurality of nodes (<NUM>).