Abstract:
A computer program product, apparatus and method for providing a performance neutral heartbeat in a computer communication system, the computer program product including a tangible storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method including maintaining a send flag, maintaining a receive flag, determining that a heartbeat timer has activated, checking a state of the send flag to determine if packets have been sent since a prior heartbeat timer activation and checking a state of the receive flag to determine if packets have been received since a prior heartbeat timer activation.

Description:
BACKGROUND OF THE INVENTION 
     1. Field of Invention 
     The present disclosure relates generally to multi-tasking multiprocessor systems, and in particular, to a performance neutral heartbeat for a multi-tasking multiprocessor environment. 
     2. Description of Background 
     In many mainframe computers, multiple processors are joined into a single unit, sharing the same name and data sets. Such multi-tasking, multi-processor systems represent an instance of a computer system running on one or more physical computers. These multiple mainframes may act as a single mainframe. Such systems can be broken down into LPARs, or logical partitions, each running a different operating system. 
     InfiniBand (IB), which is a form of System Area Network (SAN), defines a multicast facility that allows a Channel Adapter (CA) to send a packet to a single address and have it delivered to multiple ports. Each multicast group is assigned a unique address, and end-nodes that wish to participate in a multicast group do so via a ‘Join’ process initiated by the candidate participant with the Subnet Manager. The InfiniBand architecture is described in the InfiniBand standard, which is available at http://www.infinibandta.org and also hereby incorporated by reference. 
     Currently, many computer communications systems attempt to ensure that any particular connection is still viable. Ensuring for viability can typically be performed at any layer, from hardware through upper level software. A common software approach involves sending heartbeat messages and ensuring that a response is received across the link. In one approach, one end point takes on the role of master, generating the heartbeat messages and checking for responses. The slave end simply receives a heartbeat message responds by sending a heartbeat response message. In another approach, both end points take on both the master and slave roles, generating heartbeat messages, checking for heartbeat responses, and responding to received heartbeat messages by sending a heartbeat response message. One problem that arises from either of these approaches occurs when the traffic level is high. During high traffic times, the heartbeat messages themselves add to the overall congestion on the link. 
     BRIEF SUMMARY OF THE INVENTION 
     An exemplary embodiment includes a computer program product for providing a performance neutral heartbeat in a computer communication system, the computer program product including a tangible storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method including maintaining a send flag, maintaining a receive flag, determining that a heartbeat timer has activated, checking a state of the send flag to determine if packets have been sent since a prior heartbeat timer activation and checking a state of the receive flag to determine if packets have been received since a prior heartbeat timer activation. 
     Another exemplary embodiment includes a performance neutral heartbeat apparatus for a computer communication system, the apparatus including a communication handler, a send operation residing on the communication handler and configured to set a data sent flag, a receive operation residing on the communication handler and configured to set a data received flag and a timer handler function coupled to the data sent flag and the data receive flag. 
     A further exemplary embodiment includes a method for providing a performance neutral heartbeat in a computer communication system, the method including maintaining a send flag, maintaining a receive flag, determining that a heartbeat timer has activated, checking a state of the send flag to determine if packets have been sent since a prior heartbeat timer activation and checking a state of the receive flag to determine if packets have been received since a prior heartbeat timer activation. 
     Other articles of manufacture, apparatuses, and/or methods according to embodiments will be or become apparent to one with skill in the art upon review of the following drawings and detailed description. It is intended that all such additional articles of manufacture, apparatuses, and/or methods be included within this description, be within the scope of the present invention, and be protected by the accompanying claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which: 
         FIG. 1A  illustrates an exemplary embodiment of a multi-tasking multi-processor InfiniBand system; 
         FIG. 1B  illustrates an example of a multi-tasking multi-processor environment; 
         FIG. 1C  illustrates an example of a multi-tasking multi-processor environment in accordance with an exemplary embodiment; 
         FIG. 1  D illustrates an example of a multi-tasking multi-processor environment in accordance with an exemplary embodiment; 
         FIG. 2  illustrates a state machine diagram illustrating a flow for a performance neutral heartbeat in accordance with an exemplary embodiment; 
         FIG. 3  illustrates a system level block diagram of a performance neutral heartbeat system implemented in a multi-tasking multi-processor environment in accordance with an exemplary embodiment; 
         FIG. 4  illustrates a flow chart for a neutral heartbeat method  400  in accordance with an exemplary embodiment; and 
         FIG. 5  depicts one embodiment of an article of manufacture incorporating one or more aspects of the present invention. 
     
    
    
     The detailed description explains the preferred embodiments of the invention, together with advantages and features, by way of example with reference to the drawings. 
     DETAILED DESCRIPTION OF THE INVENTION 
     In accordance with an aspect of the present invention, two flags are maintained at each end point, one for send and one for receive. When the system timer pops indicating that it is time to post a heartbeat, the systems and methods described herein perform two operations. First, the software checks the send flag to determine whether any packets have been sent since the last heartbeat timer pop. If the flag is set, then another packet has been sent, so no new message is sent and the software just clears the flag. If the flag is not set, then a heartbeat message is built and sent. Second, the software checks the receive flag to determine whether any packet has been received since the last heartbeat timer pop. If the flag is set, then a packet has been received, so the flag is cleared and the heartbeat state is moved back to idle. If the flag is not set, then no packet has been received, so the heartbeat state is advanced toward a timeout condition. After a predetermined number of consecutive heartbeat intervals with no packets received, the link is determined to no longer be viable. In an exemplary embodiment, the appropriate flags are set whenever messages are sent or received on the link. The systems and methods described herein can be applied to many different environments that need to monitor communications channel viability and can also be applied at many different layers in a given environment. In exemplary embodiments, the health of a channel having multiple connections can also be monitored. For example, a physical link which supports multiple channels that are running across the same fiber can be monitored via the systems and methods described herein. 
     The systems and methods described herein provide the ability to monitor the viability of the communications link at regular intervals without impacting the performance of that link. When there is little or no data traffic on the link, heartbeat messages will be sent at regular intervals, but at those times there is no data traffic on the link to be impacted. 
       FIG. 1A  illustrates an exemplary embodiment of a multi-tasking multi-processor InfiniBand system  100 . The system  100  can include an operating system  105  (e.g., IBM zOS) having a top layer including a relational database  101  a logging facility  102 , a locking facility  103  and a cross system coupling facility (XCF). The operating system  105  can further include a multiple virtual storage (MVS) services layer  107  and a message facility layer  108 . The system  100  can further include a coupling facility (CF)  110  having a CF structures layer  111 , a link subsystem  112  and a message architecture extensions layer  113 . In an exemplary embodiment, a transport layer  115  is disposed between and couples the operating system  105  and the CF  110 . In an exemplary implementation, the transport layer  115  implements InfiniBand.  FIG. 1B  illustrates an example of a multi-tasking multi-processor environment. The example shows three mainframes A, B, C connected into a two different Parallel Sysplex environments using the previous generation of coupling transports. zOS A, zOS B and zOS F are all tied together through a Coupling Facility (CF 1 ). zOS C and zOS D are tied together through CF 2 . Meanwhile zOS E is a stand alone operating system. In this configuration separate channels are connected through separate adapters in the separate frames. It is appreciated that multiple internal channels  116 ,  117 ,  118  include separate external connections  120 .  FIG. 1C  illustrates an example of a multi-tasking multi-processor environment in accordance with an exemplary embodiment. In this example, multiple internal channels  121 ,  122 ,  123  share the same physical connection  125 .  FIG. 1  D illustrates an example of a multi-tasking multi-processor environment  150  in accordance with an exemplary embodiment. The environment  150  can include one or more channels  155 , each channel including command/response areas  156 , data buffers  157 , receive/send queues  158  and adapters  160  for mapping the channels  155  to ports  161  and ultimately communication links  162 , as discussed further herein. The channels  155  can further include queue pairs  159  as discussed further herein. The system  150  can further include control code  165  having functions including but not limited to: rendezvous  166 , auxiliary queue  167 , channel  168 , discovery  169  and subnet administrator  170 . 
       FIG. 2  illustrates a state machine diagram illustrating a flow for a performance neutral heartbeat in accordance with an exemplary embodiment. In an exemplary embodiment, when a link is active (i.e., heavy traffic), a heartbeat state starts in an Aux_HB_Idle state. If a message is sent over the link, the state changes to an Aux_Idle_MsgSent state. If a message is then received, the state advances to an Aux_MsgsSentRcvd state. However, if a message is received first, then the state changes to an Aux_MsgRcvd state. Then when a message is sent, the state changes to an Aux_MsgsSentRcvd state. In either of these cases, when the heartbeat timer pops, the state changes to the Aux_HB_Idle state. 
     In an exemplary embodiment, if the link is quiet, the sequence also starts from the Aux_HB_Idle state, but may proceed down a different path. If no message is sent or received prior to the heartbeat timer pop, then a heartbeat message is sent and the heartbeat state changes to an Aux — 1out state. If the link then receives a heartbeat message from the other end point, the state changes to an Aux_MsgRcvd state. The following heartbeat timer pop will then change the state back to Aux_HB_Idle. 
     In an exemplary embodiment, a fourth heartbeat timer pop can be implemented without receipt of a message from the other end point to declare the link not viable and took action to recover the link. The operations are identical for Aux_HB_Idle, Aux — 1out, Aux — 2out and Aux — 3out. These four states represent the stages moving toward the link not viable condition. 1out indicates that we have had 1 timer pop without receiving a packet. 2out indicates that we have had 2 timer pops without receiving a packet. 3out indicates that we have had 3 timer pops without receiving a packet. Therefore, Aux_Idle_MsgSent, Aux — 1out_MsgSent, and Aux — 2out_MsgSent are all essentially the same, but carry along the identification of how long it has been since we received the last packet. The nomenclature in  FIG. 2  is in the form event/action. The events are Msg_Sent—a message has been sent on the channel, Msg_Rcvd—a message has been received on the channel, Aux_Timer_Pop—the timer associated with the heartbeat has popped, and Aux_Queue_Active—indicates that something has requested that the heartbeat be enabled. The actions that are taken for the specified events are: Enable heartbeat—initialize the controls and set the timer, Aux_Snd_Adv—set a new state to indicate that a message has been sent on the channel, Aux_Rcv_Adv—set a new state to indicate that a message has been received on the channel, Aux_Send_HB—build and send a heartbeat packet and set a new state and clear the flags, Aux_State_Advance—set a new state to indicate that a heartbeat has been processed and clear the flags, and Drive_LOL_NOS—proceed to disable the channel to prevent further use and turn off the heartbeat timer. 
       FIG. 3  illustrates a system level block diagram of a performance neutral heartbeat system  300  implemented in a multi-tasking multi-processor environment in accordance with an exemplary embodiment. In an exemplary embodiment, the system  300  includes a first communication handler  305  and a second communication handler  360 . Each communication handler  305 ,  360  includes a send operation  310 ,  365  and a receive operation  320 ,  375 . The send operation  310 ,  365  is configured to set a data sent flag  315 ,  370  and the receive operation is configured to set a data received flag  325 ,  380 . The system  300  further includes timer handlers  330 ,  385  coupled to respective data sent flags  315 ,  370  and the respective data received flags  325 ,  380 . Each side  305 ,  360  further includes a state  335 ,  390  that is updated as described with respect to  FIG. 2  above. In an exemplary embodiment, when either side  305 ,  360  sends a packet, the send operation  310 ,  365  sets the data sent flag  315 ,  370 . Similarly, when either side  305 ,  360  receives a packet, the receive operation  320 ,  375  sets the data received flag  325 ,  380 . In an exemplary embodiment, when the timer pops, the timer handler functions  330 ,  385  first reads the data sent flags  315 ,  370  and the data received flags  325 ,  380 . If the respective data sent flag  315 ,  370  is zero, a heartbeat message is sent over the link. In an exemplary embodiment, the state  335 ,  390  is updated according to the previously described state machine in  FIG. 2 . In addition, the data sent flags  315 ,  370  and the data received flags  325 ,  380  are set to zero. In exemplary embodiments, the systems and methods described herein therefore limit the amount of traffic generated when a link is already busy, while ensuring that the link is still viable in both directions. If either direction fails to carry data for a prescribed amount of time, the link is deemed to be in error, and recovery actions are taken. For example, the recovery action may be as simple as taking action to prevent further use of the channel, or as complex as shutting down the resources associated with the channel, resetting them, and then attempting to re-establish the channel. 
       FIG. 4  illustrates a flow chart for a neutral heartbeat method  400  in accordance with an exemplary embodiment. At block  410 , the method  400  maintains a send flag, and at block  420  the method  400  maintains a receive flag. At block  430 , the method  400  determines that a heartbeat timer has been activated. At block  440 , the method  400  checks a state of the send flag to determine if packets have been sent since a prior heartbeat timer activation. If the send flag is set at block  441 , then another packet has been sent, so there is no need to send a new message, and the software clears the send flag at block  442 . If the send flag is not set at block  441 , then a heartbeat message is built and sent at block  443 . At block  450 , the method  400  checks a state of the receive flag to determine if packets have been received since a prior heartbeat timer activation. If the receive flag is set at block  451 , then a packet has been received, so the receive flag is cleared and the heartbeat state is moved back to idle at block  452 . If the flag is not set at block  451 , then no packet has been received, so the heartbeat state is advanced toward a timeout condition at block  453 . After a predetermined number of consecutive heartbeat intervals with no packets received, the link is determined to no longer be viable at block  454 . 
     Technical effects of exemplary embodiments include the ability to avoid sending heartbeat messages when traffic is already flowing on the link. The embodiments described herein eliminate any need for heartbeat messages to be sent or received when the link is already being used for other messages. The heartbeat function in that case becomes one of monitoring the activity. However, when the level of traffic drops, the heartbeat messages are generated, and only in the direction required, to ensure that the link remains viable. 
     As described above, embodiments can be embodied in the form of computer-implemented processes and apparatuses for practicing those processes. In exemplary embodiments, the invention is embodied in computer program code executed by one or more network elements. Embodiments include a computer program product  500  as depicted in  FIG. 5  on a computer usable medium  502  with computer program code logic  504  containing instructions embodied in tangible media as an article of manufacture. Exemplary articles of manufacture for computer usable medium  502  may include floppy diskettes, CD-ROMs, hard drives, universal serial bus (USB) flash drives, or any other computer-readable storage medium, wherein, when the computer program code logic  504  is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. Embodiments include computer program code logic  504 , for example, whether stored in a storage medium, loaded into and/or executed by a computer, or transmitted over some transmission medium, such as over electrical wiring or cabling, through fiber optics, or via electromagnetic radiation, wherein, when the computer program code logic  504  is loaded into and executed by a computer, the computer becomes an apparatus for practicing the invention. When implemented on a general-purpose microprocessor, the computer program code logic  504  segments configure the microprocessor to create specific logic circuits. 
     While the invention has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed as the best mode contemplated for carrying out this invention, but that the invention will include all embodiments falling within the scope of the appended claims. Moreover, the use of the terms first, second, etc. do not denote any order or importance, but rather the terms first, second, etc. are used to distinguish one element from another. Furthermore, the use of the terms a, an, etc. do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item.