Abstract:
Exemplary embodiments for a distributed system for determining a server&#39;s health are disclosed. The systems and methods provide for a network controller to direct one or more servicing nodes to check the health of one or more servers, and report a health score to the network controller. The network controller may then calculate, update and maintain a health score for each server in the network from the various health scores reported to it from the servicing nodes. This allows a distributed system to be used to facilitate network operations, as a single device is not relied on for periodically determining each server&#39;s health.

Description:
BACKGROUND OF THE INVENTION 
       [0001]    1. Field of the Invention 
         [0002]    The present invention relates generally to data networks, and more particularly to components of a data network to determine a health score of a server. 
         [0003]    2. Description of the Related Art 
         [0004]    The tremendous growth of networked clients such as smartphones and tablets has led to a large growth of network services. Many network services require deployment of a substantial number of servers and server load balancers to handle the service load. It is not uncommon to see thousands of servers used for a single popular application such as a mobile messaging application, a mobile photo application, an Internet radio or music streaming services, or a multi-person online game. In such service deployment situations, the load put on the servers is not necessarily even and is often a function of service policies of connected server load balancers (SLB), traffic managers, or application delivery controllers (ADC). When a server becomes busy, it is desirable for an ADC to detect the situation and select a more available server for new service requests. In addition, servers may fail or be taken off the service by a network administrator for maintenance reasons. It is important for an ADC to be aware of such situations and to select other servers to handle the traffic. 
         [0005]    Typically, an ADC is connected to all servers and often detects the health of all servers directly. Upon knowing the servers&#39; health after the detection, the SLB or ADC would adjust their service application processing accordingly. Such direct detection mechanism works well when there are few servers. However, when there are thousands of servers, the computing and resource effort to check the servers&#39; health will consume a significant part of available resources of the ADC. Since a typical service network may have another ADC, it would be advantageous to share the load of detecting the servers&#39; health. An ADC can be assigned to check the health of a portion of the servers while it relies on other ADCs to determine the health of other servers. 
         [0006]    It should be apparent from the foregoing that there is a need to provide a distributed method to determine the health of a server. 
       SUMMARY 
       [0007]    This summary is provided to introduce a selection of concepts in a simplified form that are further described in the Detailed Description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. 
         [0008]    The present disclosure is related to approaches for a distributed system to determine a server&#39;s health. A method for determining, by a network controller, a health score for a server may comprise receiving a request for a service session from a server, directing at least one servicing node to determine a health score of a server, receiving a health score of the server from each of the at least one servicing nodes, calculating an aggregated health score using the received health scores, and sending the aggregated health score to a servicing node designated to process the service session. The servicing node may use the aggregated health score to select or not select the server for a service session. The servicing node that receives the aggregated health score may be a different servicing node than one that reported a health score for the selected server. 
         [0009]    A distributed system for determining a health score for a server is also disclosed. The system may comprise a plurality of servicing nodes, and a network controller that is configured to receive a request for a service session from a server, direct at least one servicing node to determine a health score of a server, receive a health score of the server from each of the at least one servicing nodes, calculate an aggregated health score using the received health scores, and send the aggregated health score to a servicing node designated the process the service session. 
         [0010]    In further example embodiments of the present disclosure, the method steps are stored on a machine-readable medium comprising instructions, which when implemented by one or more processors, perform the recited steps. In further example embodiments, hardware systems, or devices, can be adapted to perform the recited steps. Other features, examples, and embodiments are described below. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0011]    Embodiments are illustrated by way of example and not by limitation in the figures of the accompanying drawings, in which like references indicate similar elements. 
           [0012]      FIG. 1  illustrates an exemplary embodiment of a data network servicing a service session based on server health. 
           [0013]      FIG. 2  illustrates an exemplary embodiment of a network node. 
           [0014]      FIG. 3  illustrates an exemplary embodiment of a network controller collecting a server&#39;s health scores. 
           [0015]      FIG. 4  illustrates an exemplary embodiment of a server health score. 
           [0016]      FIG. 5  illustrates an exemplary embodiment of a network controller calculating an aggregated health score for a server. 
           [0017]      FIG. 6  illustrates an exemplary embodiment of a servicing node using a health score to process a service session. 
       
    
    
     DETAILED DESCRIPTION 
       [0018]    The following detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show illustrations in accordance with example embodiments. These example embodiments, which are also referred to herein as “examples,” are described in enough detail to enable those skilled in the art to practice the present subject matter. The embodiments can be combined, other embodiments can be utilized, or structural, logical, and electrical changes can be made without departing from the scope of what is claimed. The following detailed description is therefore not to be taken in a limiting sense, and the scope is defined by the appended claims and their equivalents. 
         [0019]      FIG. 1  illustrates an exemplary embodiment of a data network handling a service session using a service based on server health. Client  105  conducts a service session  110  with a server, such as server  115 , server  120 , or server  125 . Although three servers are depicted in  FIG. 1 , there can be any number of servers. In one embodiment, data packets of service session  110  are processed by service network  130  prior to being delivered to server  115 . Service network  130  comprises one or more servicing nodes, such as servicing nodes  135 ,  140 ,  145 , and  150 . Additionally, service network  130  connects to one or more servers  115 ,  120 , and  125 . In some embodiments, service network  130  connects to a network controller  155 . In various embodiments, network controller  155  may also perform functions of a servicing node. 
         [0020]    The one or more servicing nodes may examine the health of the servers. In an exemplary embodiment, servicing node  150  is assigned to check the health of server  115  and report a health score for server  115  to network controller  155 . In some embodiments, servicing node  150  is not assigned to examine server  115  and does not report the health score of server  115  to network controller  155 . Network controller  155  collects reported health scores of server  115  from the one or more servicing nodes, calculates server health score  160  for server  115 , and stores server health score  160 . Network controller  155  also collects reported health scores of servers  120  and  125 , and calculates server health scores for those servers as well. 
         [0021]    In various embodiments, network controller  155  sends server health score  160  to a servicing node, such as servicing node  135 . Servicing node  135  determines if server  115  is healthy according to server health score  160 . Servicing node  135  processes service session  110  and selects server  115  to process service session  110  if server  115  is determined to be healthy. Servicing node  135  then sends service session  110  to server  115  for processing. 
         [0022]    Alternatively, network controller  155  may send server health score  160  for server  115  to a different servicing node, such as servicing node  150 . Servicing node  150  determines server health score  160  to be poor and thus server  115  not healthy according to server health score  160 . Servicing node  150  processes service session  110  and does not select server  115  based on the server health score  160 . Instead, servicing node  150  selects a different server, such as server  120 , to handle the service session  110 . 
         [0023]    In various embodiments, service network  130  includes additional network elements such as routers, switches, Ethernet switches, ATM switching, wireless network elements, broadband gateways, top of rack switches, access points, base stations, mobile network elements, data traffic controller, application delivery controllers, firewalls, optical switches, remote access gateways and other data network gateways, or any other network elements. Additionally, service network  130  may utilize network protocols such as IP protocol, TCP/UDP, ATM, Frame Relay, Ethernet, VLAN, WLAN, MPLS, HTTP, and other data communication protocols. 
         [0024]      FIG. 2  illustrates an exemplary embodiment of a network node  205  in a service network. In various embodiments, network node  205  is a client device, a server device, a servicing node, or a network controller. Network node  205  may include a processor module  210 , a network module  215 , and a computer storage module  220 . Processor module  210  can include one or more processors, including a micro-processor, an Intel processor, an AMD processor, a MIPS processor, an ARM-based processor, a RISC processor, or any other type of processor. Processor module  210  can also include one or more processor cores embedded in a processor. Additionally, processor module  210  can include one or more embedded processors, or embedded processing elements in a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), or Digital Signal Processor (DSP). 
         [0025]    In exemplary embodiments, network module  215  comprises a network interface such as Ethernet, optical network interface, a wireless network interface, T1/T3 interface, or a WAN or LAN interface. Furthermore, network module  215  can include a network processor. Computer storage module  220  comprises RAM, DRAM, SRAM, SDRAM, or any other memory utilized by processor module  210  or network module  215 . Computer storage module  220  stores data utilized by processor module  210 . Computer storage module  220  can include a hard disk drive, a solid state drive, an external disk, a DVD, a CD, a readable external disk, or any other type of storage. Additionally, computer storage module  220  stores one or more computer programming instructions, which when executed by processor module  210  or network module  215 , implements one or more of the functionalities of the present invention. Network node  205  can also include an input/output (I/O) module  225 , such as a keyboard, a keypad, a mouse, a gesture-based input sensor, a microphone, a physical or sensory input peripheral, a display, a speaker, a physical or sensory output peripheral, or any other input or output module. 
         [0026]    Network node  205  may also include service application  230 . When network node  205  is a client, it performs a client-side function of service application  230 . A client-side function of service application  230  comprises file downloading, file uploading, web browsing, video streaming, e-book reading, game playing, gaming, photo applications, messaging, voice calling, music listening, conferencing, document editing, map navigation, online shopping, and other client-networking activities. On the other hand, service application  230  may include server-side functionalities when network node  205  functions as a server. Server-side functions of service application  230  comprise file transfer, HTTP server functions, voice call services, video and audio streaming, online games, collaboration, enterprise work flow, enterprise application, enterprise resource planning (ERP), customer relationship management (CRM), conferencing, e-commerce, transaction processing, business to business (B2B) applications, machine to machine (M2M) application, telemetry, remote access, identity authentication, and other server application functionalities. 
         [0027]    In various embodiments, service application  230 , when executed in network node  205  as a servicing node, comprise functions such as a network proxy application, security gateway, firewall, protocol gateway application, server load balancing, content caching, network optimization, data de-duplication, network cached storage access, browser optimization, and other network-based service applications. Service application  230  of a servicing node may also include functionalities implementing a part of the present invention. 
         [0028]    Service application  230 , when executed in network node  205  as a network controller, comprises network management functionalities over service network  130  such as provisioning, network node configuration, event logging, network center operation, network statistics collection and/or analysis, alert handling, and/or other network operation and management applications. 
         [0029]      FIG. 3  further illustrates an exemplary embodiment of network controller  155  working in conjunction with one or more servicing nodes to determine a server health score. The one or more servicing nodes may include servicing nodes  135 ,  140 ,  145 , and  150  and are connected to one or more servers, such as server  115 . 
         [0030]    In some embodiments, network controller  155  informs servicing nodes  135 ,  140 , and  145  to report the health score of server  115 . In other embodiments, network controller  155  informs servicing node  150  to not report the health score of server  115 , or servicing node  150  does not receive any instruction from network controller  155  to report the health score of server  115 , and therefore servicing node  150  does not report the health score of server  115 . In various embodiments, servicing node  135 , similarly for servicing node  140  or  145 , checks the health of server  115  by querying server  115  and then generates server health score  305  for server  115 . 
         [0031]    Server health score  305 , as illustrated in an exemplary embodiment in  FIG. 4 , comprises a score for connectivity or reachability, resource utilization such as CPU and memory usage, load of a service application, or various measures for the load of the service application. The various measures for the load of the service application comprise the number of active sessions or connections, pending sessions, or completed sessions; performance of the service application such as page loading time; or average, maximum, or minimum response time. Server health score  305  may also include one or more internal measurements within a servicing node, related to the server. Such internal measurements comprise data traffic sent from the servicing node to the server, data traffic from a client device using the servicing node directed to the server, or data traffic processed by the servicing node for a service application of the server. 
         [0032]    In various embodiments, a servicing node uses different means to determine the measures indicated in server health score  305 . For example, servicing node  135  may connect to server  115  using a network link. Servicing node  135  checks or measures the health of the network link to determine connectivity indicated in server health score  305 . Service node  135  may also connect to server  115  over service network  130 . Servicing node  135  checks reachability to server  115  by sending a query using a protocol, such as ping packet, echo or a proprietary protocol, to server  115  and examining if server  115  responds to the query. Servicing node  135  measures and records the reachability status into server health score  305 . 
         [0033]    In some embodiments, servicing node  135  sends a request to server  115  to obtain resource utilization information. Servicing node  135  sends a piece of software or computer programming instructions to server  115  which executes the sent software to collect and report system resource utilization to servicing node  135 . In various embodiments, to obtain a load of the service application, servicing node  135  sends a service request, such as a HTTP request, for the service application and calculates the service response from server  115  with a pre-stored response. Servicing node  135  compares the performance of the service request with a pre-stored expectation of the performance. Server  115  sends to servicing node  135  a measure about the working load of the service application in a response to a client service request, separately, or in a HTTP response to servicing node  135 . 
         [0034]    In some embodiments, a servicing node  135  sends all the information in server health score  305  to network controller  155 . Servicing node  135  calculates a summary server health score  305 , such as “Good”, “Bad”, “Poor”, “Failed”, “1,” or “0,” and sends server health score  305  to network controller  155 . 
         [0035]    In an exemplary embodiment illustrated in  FIG. 5 , network controller  155  maintains server health score  160  which indicates health of server  115 . Server health score  160  stores a list of health scores reported by one or more servicing nodes such as servicing nodes  135 ,  140 ,  145 , or  150 . 
         [0036]    In an exemplary embodiment, network controller  155  receives server health score  305  reported from servicing node  135  about health of server  115 . Network controller  155  stores server health score  305  into the corresponding entry in server health score  160 . Entry for servicing node  150  is not applicable when network controller  155  determines servicing node  150  does not need to report health score for server  115 . 
         [0037]    Network controller  155  determines if there is sufficient information in server health score  160 , and network controller  155  calculates aggregated health score  505  for server health score  160 . In various embodiments, network controller  155  determines there is sufficient information in server health score  160  when it receives a server health score  305  from at least one servicing node, from a majority of servicing nodes, from at least one servicing node within a specified time frame (e.g. last 30 seconds, last 5 seconds), from at least two servicing nodes within a specified time frame, from all servicing nodes, or any similar variation. Aggregated health score  505  may indicate combined health information about server  115 . Network controller  155  sends aggregated health score  505  for server  115  to servicing nodes  135 ,  140 ,  145 , or  150 . 
         [0038]    Network controller  155  may calculate aggregated health score  505  by computing an average, weighted average, mean, median, or other arithmetic operation of the received server health scores from the at least one servicing node. As such, the aggregate health score  505  may be different from the server health score  160 . In some embodiments, aggregate health score  505  calculated by the network controller  155  is also included in server health score  160 . In various embodiments, network controller  155  determines aggregated health score  505  to be good, “1”, or “yes” when a majority of received server health scores indicate good health for server  115 . On the other hand, network controller  155  determines aggregated health score  505  to be poor, “0”, or “no” when a majority of received server health scores indicate poor health for server  115 . Network controller  155  considers different weighing factors to different reported server health scores depending on the reporting servicing node. In some embodiments, the calculation is based on unanimous agreement of reported server health scores. In other embodiments, network controller  155  discards a reported health score prior to the calculation, or when a reported health score is not trusted. 
         [0039]    In an exemplary embodiment, network controller  155  receives server health score  305 , replaces an earlier reported health score from servicing node  135 , and calculates an updated aggregated health score  505 . Network controller  155  calculates an updated aggregated health score  505  when network controller  155  receives a sufficient number of newly received server health scores. Network controller  155  may include a timer (not shown) and discards all current reported health scores. After receiving another collection of reported server health scores from the one or more servicing nodes, network controller  155  may calculate an updated aggregated health score  505 . 
         [0040]    In some embodiments, network controller  155  stores a history of reported server health scores. Network controller  155  determines from a history of reported server health scores from servicing node  135  whether the reported server health scores are not consistent and whether servicing node  135  is not trusted for further reporting server health score for a particular server, such as server  115 . Then, network controller  155  instructs servicing node  135  not to further report server health score, and network controller  155  discards reported health scores from servicing node  135  in calculating aggregated health score  505 . Additionally, network controller  155  instructs a different servicing node, such as servicing node  150 , to report server health for server  115 , so as to replace the role of servicing node  135 . 
         [0041]    In some embodiments, network controller  155  determines servicing node  135  is no longer needed to report server health for server  115 , and network controller  155  instructs servicing node  135  not to report server health for server  115 . Servicing node  135  may inform network controller  155  that it no longer will report server health for server  115 . Upon determining that servicing node  135  is no longer needed to report server health for server  115 , network controller  155  removes an entry in server health score  160  corresponding to servicing node  135 . 
         [0042]    In an exemplary embodiment illustrated in  FIG. 6 , servicing node  135  determines a final server health score  605  for server  115 . Servicing node  135  determines server health score  305  for server  115 , reports server health score  305  to network controller  155 , or receives server health score  610  from network controller  155 . In various embodiments, server health score  610  is the same as server health score  160 , or aggregate health score  505 . Servicing node  135  combines received server health score  610  and server health score  305  to get final server health score  605 . 
         [0043]    In some embodiments, servicing node  135  may receive server health score  610  from network controller  155 , and detect server health score  305  directly from the server. The servicing node makes its own determination as to which server health score to use as final server health score  605 . In an exemplary embodiment, network controller  155  receives a “good” server health score from servicing nodes  135  and  140 , and a “bad” server health score from servicing node  145 . Network controller  155  determines aggregate health score  505  to be “good”, since the majority of the received server health score are good. Servicing node  145  receives aggregate health score  505  of “good”, which conflicts with its own detected value of “bad”, and then makes its own determination of which value to use as final server health sore  605 . Conflicting server health scores for server  115  may be due to the servicing nodes checking the health of server  115  at different times, using different means. Network controller  155  also calculates and updates aggregated health score  505  using scores received at different times from different servicing nodes. 
         [0044]    In some embodiments, servicing node  135  does not use server health score  610  received from network controller  155  and considers only server health score  305  to determine final server health score  605 . In other embodiments, server health score  610  is more accurate, and servicing node  135  considers only server health score  610 . If server health score  610  is much worse than server health score  305 , conflicts with server health score  305 , or indicates that server  115  is not available or failed, servicing node  135  may use only server health score  610 . In various embodiments, servicing node  135  is not assigned to report health score for server  115 , and servicing node  135  uses only server health score  610  to determine final server health score  605 . 
         [0045]    In some embodiments, servicing node  135  processes service session  110  using final server health score  605  and a service policy  615 . According to service policy  615 , servicing node  135  selects server  115  to process service session  110 . Servicing node  135  checks final server health score  605  of server  115 . If final server health score  605  indicates server  115  is available, servicing node  135  selects server  115  to process service session  110 . If final server health score  605  indicates server  115  is not available, busy, congested, or failed, servicing node  135  does not select server  115 . In various embodiments, health server score  305  may indicate that server  115  is available such that servicing node  135  would have selected server  115 , while final health server score  605  indicates server  115  is not available. In those embodiments, servicing node  135  does not select server  115  to process service session  110 . In other embodiments, servicing node  135  does not detect a health score for server  115  such that servicing node  135  would have selected server  115  while final health server score  605  indicates server  115  is not available. In those embodiments, servicing node  135  does not select  115  to process service session  110 . 
         [0046]    Servicing node  135  examines service policy  615  and select another server such as server  120  to process service session  110 . In some embodiments, servicing node  135  removes server  115  from service policy  615  based on final server health score  605  so that servicing node  135  would not consider server  115  when processing service session  110 . Alternatively, servicing node  135  may add server  115  to service policy  615  based on final server health score  605  so that servicing node  135  would consider server  115  when processing service session  110 . 
         [0047]    In various embodiments, servicing node  135  receives an updated server health score  610  from network controller  155  and calculates an updated final server health score  605 . In other embodiments, servicing node  135  determines server health score  305  and calculate an updated final server health score  605 . Servicing node  135  updates service policy  615  based on an updated final server health score  605 . 
         [0048]    In some embodiments, network controller  155  assigns different servicing nodes to collect server health scores for different servers. Network controller  155  determines that server  115  is more important than server  120  and assigns more servicing nodes for server  115  than for server  120 . If network controller  155  observes that aggregated health scores of server  115  are inconsistent over a period of time, network controller  155  may assign additional servicing nodes to collect health scores of server  115 , so as to better determine the aggregated health of server  155 . However, if network controller  155  observes that aggregated health scores of server  115  are stable over a period of time, network controller  155  may reduce a number of servicing nodes to monitor server  115 , so as to reduce the processing load of the servicing nodes, the server, and the network. In various embodiments, network controller  155  increases the frequency of health data collection by the servicing node. Alternatively, network controller  155  may decrease the frequency of health data collection, depending on the trend of aggregated health of server  115  over a period of time. 
         [0049]    In some embodiments, network controller  155  connects to a display and displays the aggregated health score of server  115  and the trend of aggregated health of server  115 . Network controller  155  generates a report of the history of aggregated health scores of one or more servers. Network controller  155  applies one or more rules to analyze the history of aggregated health scores to determine a trend of server loads, a trend of service application load, or a trend of client traffic.