System and method for avoiding deadlock in transmission of broadcast traffic in a server system

A server system may include a plurality of internal hubs communicatively coupled to a plurality of server nodes. The plurality of internal hubs may communicate with an external hub to transmit broadcast traffic to reach a designated server node. A hub controller, a routing device coupled to the plurality of internal hubs, may select an internal hub from among a plurality of internal hubs based on a link status and a set of hub selection rules. Based on a status of active link and a relative priority of internal hubs, an internal hub is selected as a transmission channel to receive broadcast traffic from the external hub and direct the broadcast traffic to a corresponding server node.

BACKGROUND

Field

This application relates to a server device, and more particularly to a method of transmission of broadcast traffic in a server device to avoid deadlock.

Background

In a typical computer system (e.g., a rack server in a data center), a server system includes a plurality of server nodes that communicate with an external hub (e.g. external routing switch) to transmit broadcast traffic. This is accomplished, in part, through an internal hub (e.g., an internal routing switch) associated with the external hub and the plurality of server nodes. Accordingly, the internal hub may be used to transmit the broadcast traffic from the external hub to a designated server node as requested by an administrative device (e.g. a user).

Servers may be used by different data centers with different computing needs. However, typical servers are preloaded with a single internal hub that may not be optimal for increased broadcast traffic and differing needs of the administrative devices. In addition, an increased number of server nodes in a server system require more integrated system architecture to provide a stable connection and efficient traffic flow.

SUMMARY

The following presents a simplified summary of one or more embodiments in order to provide a basic understanding of present technology. This summary is not an extensive overview of all contemplated embodiments of the present technology, and is intended to neither identify key or critical elements of all examples nor delineate the scope of any or all aspects of the present technology. Its sole purpose is to present some concepts of one or more examples in a simplified form as a prelude to the more detailed description that is presented later.

In accordance with one or more aspects of the examples described herein, systems and methods are provided for transmission of broadcast traffic in a server system to avoid deadlock. The method includes determining a link status for a plurality of internal hubs communicatively coupled to a plurality of server nodes. The method includes selecting a one of the plurality of internal hubs to yield a selected internal hub based on the link status for the plurality of internal hubs and a set of hub selection rules. The method further includes restricting communications with an external hub exclusively to the selected internal hub.

In another aspect, an apparatus is provided for transmission of broadcast traffic in a server system. The apparatus includes at least one hub controller configured for determining a link status for a plurality of internal hubs communicatively coupled to a plurality of server nodes, selecting a one of the plurality of internal hubs to yield a selected internal hub based on the link status for the plurality of internal hubs and a set of hub selection rules, and restricting communications with an external hub exclusively to the selected internal hub.

In yet another aspect, a non-transitory computer-readable medium is provided for routing broadcast packets in a server system. The non-transitory computer-readable medium stores executable instructions which cause a system controller device to determine a link status for a plurality of internal hubs communicatively coupled to a plurality of server nodes, select a one of the plurality of internal hubs to yield a selected internal hub based on the link status for the plurality of internal hubs and a set of hub selection rules, and restrict communications with an external hub exclusively to the selected internal hub.

DETAILED DESCRIPTION

Various aspects of the present technology are described with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It can be evident, however, that the present technology can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing these aspects. The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

The subject disclosure provides techniques for avoiding deadlock when transmitting broadcast traffic in a server system in accordance with the present technology. The disclosure turns first toFIG. 1A, which illustrates a block diagram100that is useful for understanding various aspects of the present technology. A server system (e.g. a rack server)104can include an internal hub (e.g. an internal Ethernet routing device) that is communicatively coupled to a plurality of server nodes101,102,103. The internal hub104may reside in the server system and communicate to the external hub106located outside the server system. As shown inFIG. 1A, the internal hub104can be connected to an administrative device (e.g. a user) through the external hub106(e.g. an external Ethernet routing device) and processes the broadcast data packets as requested by the administrative device107. The administrative device can manage the server nodes by designating a specific server node and instructs the external hub to transmit the broadcast packets so as to reach the designated server node. For example, if the admin device107broadcasts to node_1101, node_1101will broadcast back to the admin device107through the internal hub_1104and external hub106. The administrative device can set a specific server node or internal hub with the highest relative priority as needed.

As the technology develops, a need to increase the number of server nodes has increased. To manage a higher number of server nodes, more than one internal hub is required for a better operation of a server system and a stable connection. One proposed solution has been to increase a number of internal hubs104,105in the system120as illustrated inFIG. 1B. However, when more than one internal hub is coupled to the number of server nodes, a deadlock problem can cause due to an endless loop back flow as indicated in bold arrows inFIG. 1B. When the admin device107broadcasts to node1101, node_1101broadcasts1to internal hub_1104at step1. The internal hub_1104broadcasts to internal hub_2105through the external Hub106as indicated as step2. The internal hub_2105may again broadcast to node_1101as step3. Thus, the node_1101will be broadcasted twice and it can cause the deadlock problem.

Due to a lack of central system that manages the traffic flow from one internal hub to the other internal hub, each server node will broadcast to each other, and cause a resource conflict in a network.

In view of the foregoing, the present disclosure provides a new methodology for avoiding deadlock issues and transmitting broadcast traffic when more than one internal hub resides in the server system. This is illustrated inFIG. 2AandFIG. 3A. In particular,FIG. 2Aillustrates a block diagram200of an exemplary embodiment of transmitting broadcast traffic using a hub controller. To avoid deadlock issues, the server system208may include a hub controller209(e.g. a complex program logic device) that detects and controls the broadcast traffic. The hub controller (e.g. CPLD) may be a logic gate configured to provide a management/administration of servers and other routing devices. In some embodiments, the hub controller209can be connected to a management controller210. The management controller210may be a baseboard management controller (BMC) that independently admins/manages a central processing unit and/or operating systems of a computing device. In some embodiments, the hub controller can be a part of management controller.

In some embodiments, the hub controller209may detect a link status for each of a plurality of internal hubs204,205. For example, each of the plurality of internal hubs (e.g. network switch) can be connected to the hub controller209through a detect link (e.g. detect-link_1for internal hub_1, detect-link_2for internal hub_2). Each detect link can be activated or de-activated by the administrative device207when requested. When detect links are active, the management controller210can assign a relative priority on each of the active detect links to prioritize the broadcast traffic. The detect link with the highest relative priority is selected as an internal hub as a main channel to direct the broadcast traffic. The highest relative priority indicates that it may be easy and efficient to send broadcast packets through the selected (associated) internal hub. Therefore, the communication between the external hub206and the designated node is restricted exclusively with the selected internal hub.

Accordingly, an output port of the selected internal hub is enabled by the hub controller to facilitate communication with the external hub206. On the other hand, an output port of an unselected internal hub is disabled to restrict the external hub communication only with the selected internal hub. This can be further discussed with respect toFIG. 2B. For example, if a detect link associated with an internal hub_1and another detect link associated with an internal hub_2are both active, then a relative priority of these two detect links will be determined. But only one of the internal hubs will be selected as a transmission channel. As illustrated in the first row of aFIG. 2Btable, both internal hub_1detect link and internal hub_2detect link are active. By way of one exemplary embodiment, a relative priority of internal hub_1is determined to be higher in this case. Therefore, an output port of the internal hub_1is enabled and an output port of the internal hub_2is disabled to restrict the communication only with the internal hub_1.

In some embodiments, a detect link can be inactive. When the detect link is inactive, a relative priority becomes less important, because it then can be difficult to send the data packets through the associated internal hub. For example, as illustrated in the second row of table inFIG. 2B, when a detect link of internal hub_1is active and a detect link of internal hub_2is inactive, an output port of the internal hub_1will be enabled and the output port of the internal hub_2will be disabled by the hub controller. Thus, the communication from the external hub will be directed to only internal hub_1and finally to a corresponding server node. In some embodiments, for example, as illustrated in the third row of table inFIG. 2B, when a detect link of internal hub_1is inactive and a detect link of the internal hub_2is active, an output port of the internal hub_1will be disabled and the output port of the internal hub_2will be enabled. Thus, the communication from the external hub will be directed to only internal hub_2and a corresponding server node.

When both internal hubs are inactive as illustrated in the fourth row of the table, then none of output ports of internal hub_1and internal hub_2will be enabled and relative priorities of these inactive hubs will not be determined. However, if one of internal hubs (e.g. internal hub_2) goes from inactive (fourth row) to active (third row), then the host controller may detect a status change, enable the output port for internal hub_2, and determine a relative priority of the internal hub_2. For example, if the link status of one internal hub changes (in either case, active to inactive or inactive to active), a change in status will be communicated to the hub controller to make an appropriate action.

FIG. 3Aillustrates an exemplary embodiment300when a switch resides in the server system. A switch311,312can be an ON/OFF mechanical signal switch that an control transmission of signal. The switch1311is associated with an internal hub_1304and switch2312is associated with an internal hub_2305. The admin device307can set either one of internal hubs304,305with a higher relative priority to yield a selected internal hub as a broadcast traffic channel using a corresponding switch associated with the selected internal hub.

FIG. 3Awill also be described with respect to a table inFIG. 3B. The hub controller309will identify an active link and determine a relative priority of the identified active link. For example, as illustrated in the first row of table inFIG. 3B, both detect links are determined to be active. Then the hub controller309will determine a relative priority for each of the identified active links. In this example, the relative priority of internal hub-1is determined to be higher than the internal hub_2. Therefore, to restrict the communication exclusively with an internal hub_1, a switch1311(associated switch) will be opened and a switch2312will be closed.

The second row of table inFIG. 3Bshows an exemplary scenario where an internal hub_1is active and an internal hub_2is inactive. In this case, the hub controller309will control both switch1311and switch2312to control the broadcast data transmission. For example, the hub controller309will open the switch1311and close the switch2312, thus, makes the internal hub_1(e.g. internal hub_1is associated with the opened switch) a selected channel. The communication therefore will go from the external hub306to switch1311, pass through internal hub_1304, and ultimately to an associated server node.

In a related aspect, if an internal hub_1is inactive and an internal hub_2is active, the hub controller309will close the switch1311and open the switch2312, thus, making the internal hub_2(e.g. internal hub_2is associated with the opened switch2312) as a selected channel. The communication therefore will go from the external hub306to switch2

When both internal hubs are inactive as illustrated in the fourth row of the table, then both switches will be closed and relative priorities of these inactive hubs will not be determined. However, if one of internal hubs (e.g. internal hub_2) goes from inactive (fourth row) to active (third row), then the host controller may detect a status change, open the switch2(internal hub_2switch), and determine a relative priority of the internal hub_2.

In accordance with one or more aspects of the implementations described herein,FIG. 4illustrates a flow chart of an exemplary method of directing broadcast traffic in a server system. The method400can start at step410when there is more than one internal hub in a server system. The server system can be a micro server system. At step420, a hub controller can identify a plurality of internal hubs that are connected to the hub controller. The hub controller can then identify internal hubs of which a connecting link (e.g. a detect link) is active.

In some instances, a connecting link status can be changing from inactive to active while the hub controller is determining an active status of the plurality of internal hubs. The changing link will be considered as an active link and a relative priority value of the changing link will be considered to make a determination to select an internal hub with the highest priority.

At step430, the hub controller will determine a set of hub selection rules to yield a selected internal hub as a transmission channel. The set of hub selection rules can be stored in the server system. In some embodiments, the admin device can send the set of hub selection rules to the server system for use. There may be many different hub selection rules that can be considered when choosing a set of hub selection rules to apply to a particular server configuration. One hub selection rule may be a relative priority rule.

The hub controller will determine a relative priority value of the plurality of internal hubs with respect to each other. Any internal hub with the highest relative priority may be selected as an internal hub. The relative priority can be determined in many ways such as bandwidth of internal hub or server node, specific value assigned by the admin device, or a number of server nodes connected to an internal hub. There can be other ways to select an internal hub, and the illustrated selection methods are not intended to be an exclusive list for selecting an internal hub.

For example, in some embodiments, a server node may be associated with a high priority, but an internal hub associated with that server node may be associated with a low priority. In this case, the hub selection may specify that the server node with the high priority can override the internal hub with the low priority. Thus, the internal node with the low priority may be selected as a transmission channel over other internal nodes with higher priorities to ensure the high priority server nodes gets access to the external hub.

In addition, a number of server nodes may be considered when the set of rules are to be determined. Even if a single internal hub may be associated with a high relative priority, it will be weighted differently if that internal hub is associated with more number of server nodes than the internal hub can optimally accommodate. Because a bandwidth capacity for each server node may not be as optimal, the internal hub will be less preferred than other internal hubs with less number of server nodes.

At step440, the hub controller will select an internal hub based at least in part on the link status and a set of hub selection rules. At step450, the hub controller will restrict the communication from the external hub only to the selected internal hub by managing operation of the internal hubs or switches associated with the internal hubs. The transmission process can resume at step460.

FIG. 5is a block diagram of exemplary system architecture500implementing the features and processes ofFIGS. 1-4. The architecture500can be implemented on any electronic device that runs software applications derived from compiled instructions, including without limitation personal computers, servers, smart phones, media players, electronic tablets, game consoles, email devices, etc. In some implementations, the architecture500can include one or more processors502, one or more input devices504, one or more display devices506, one or more network interfaces508and one or more computer-readable mediums510. Each of these components can be coupled by bus512.

Display device506can be any known display technology, including but not limited to display devices using Liquid Crystal Display (LCD) or Light Emitting Diode (LED) technology. Processor(s)502can use any known processor technology, including but are not limited to graphics processors and multi-core processors. Input device504can be any known input device technology, including but not limited to a keyboard (including a virtual keyboard), mouse, track ball, and touch-sensitive pad or display. Bus512can be any known internal or external bus technology, including but not limited to ISA, EISA, PCI, PCI Express, NuBus, USB, Serial ATA or FireWire.

Computer-readable medium510can be any medium that participates in providing instructions to processor(s)502for execution, including without limitation, non-volatile storage media (e.g., optical disks, magnetic disks, flash drives, etc.) or volatile media (e.g., SDRAM, ROM, etc.). The computer-readable medium (e.g., storage devices, mediums, and memories) can include, for example, a cable or wireless signal containing a bit stream and the like. However, when mentioned, non-transitory computer-readable storage media expressly exclude media such as energy, carrier signals, electromagnetic waves, and signals per se.

Computer-readable medium510can include various instructions for implementing an operating system514(e.g., Mac OS®, Windows®, Linux). The operating system514can be multi-user, multiprocessing, multitasking, multithreading, real-time and the like. The operating system514performs basic tasks, including but not limited to: recognizing input from input device504; sending output to display device506; keeping track of files and directories on computer-readable medium510; controlling peripheral devices (e.g., disk drives, printers, etc.) which can be controlled directly or through an I/O controller; and managing traffic on bus512. Network communications instructions516can establish and maintain network connections (e.g., software for implementing communication protocols, such as TCP/IP, HTTP, Ethernet, etc.).

A graphics processing system518can include instructions that provide graphics and image processing capabilities. Application(s)520can be an application that uses or implements the processes described in reference toFIGS. 1-4. The processes can also be implemented in operating system514.

Service controller522can be a controller that operates independently of processor(s)522and/or operating system514. In some implementations, service controller522can be powered and operational before processor(s)502are powered on and operating system514is loaded into processor(s)502. For example, service controller522can provide for pre-OS management of the computing device through a dedicated network interface or other input device. For example, service controller522can be a baseboard management controller (BMC) that monitors device sensors (e.g., voltages, temperature, fans, etc.), logs events for failure analysis, provides LED guided diagnostics, performs power management, and/or provides remote management capabilities through an intelligent platform management interface (IPMI), keyboard, video, and mouse (KVM) redirection, serial over LAN (SOL), and/or other interfaces. Service controller522can be implement the processes described with reference toFIGS. 1-4above. For example, service controller522can be configured to manage power supply units coupled to server rack.

One or more features or steps of the disclosed embodiments can be implemented using an API. An API can define on or more parameters that are passed between a calling application and other software code (e.g., an operating system, library routine, function) that provides a service, that provides data, or that performs an operation or a computation.

The API can be implemented as one or more calls in program code that send or receive one or more parameters through a parameter list or other structure based on a call convention defined in an API specification document. A parameter can be a constant, a key, a data structure, an object, an object class, a variable, a data type, a pointer, an array, a list, or another call. API calls and parameters can be implemented in any programming language. The programming language can define the vocabulary and calling convention that a programmer will employ to access functions supporting the API.

In some implementations, an API call can report to an application the capabilities of a device running the application, such as input capability, output capability, processing capability, power capability, communications capability, etc.

A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, other steps may be provided, or steps may be eliminated, from the described flows, and other components may be added to, or removed from, the described systems. Accordingly, other implementations are within the scope of the following claims.