Abstract:
Techniques for automatically monitoring a computer network and notifying an attendant upon detection of problems are described. A system for monitoring the computer network comprises a monitoring server connected to the network and operative to communicate with a plurality of monitored servers belonging to the network and a monitor program hosted on the monitor server and operative to test the performance and correct functioning of selected ones of the monitored servers and the presence or absence of problems related to applications running on the monitored servers.

Description:
BACKGROUND OF INVENTION  
         [0001]    The present invention relates generally to improved techniques for monitoring computer systems. More particularly, the invention relates to methods and apparatus for automatically examining selected computer system components and applications and notifying an attendant if a server or application fails to return satisfactory responses.  
           [0002]    Computer systems are widely used and benefit innumerable organizations of all types and sizes. In many cases, a computer system includes a large number of relatively widely distributed components and applications to provide services to users. A frequently encountered example of a large computer system is a computer network, which may include many servers running a number of different applications. These applications may run on different servers, and, particularly in large organizations, the servers may be spread over a large geographic area.  
           [0003]    Design of a network such that applications are distributed over a number of servers prevents any one server from being overwhelmed and unable to serve users in a timely manner. However, increasing the number of servers in a network naturally increases the number of locations where problems may arise. In order to insure the smooth functioning of a network, it is important to monitor the status of critical applications running on the network, and to alert a network administrator or other responsible person in order to solve problems which are detected. Increasing the number of applications being executed by a network and the number of servers executing those applications also increases the scope of the task of monitoring the applications and servers. Many applications produce status logs which can be examined in order to detect problems, but prior art systems typically require that these status logs be examined by a human operator in order to detect problems. Such examination by a human operator occupies the time of that operator and, moreover, frequently reveals that the application producing the log is functioning normally.  
           [0004]    A typical network administrator is frequently very busy solving problems with the computer network. It would be highly beneficial and a great saving of the time of the network administrator and his or her assistants if network problems could be automatically detected and a human operator notified upon detection of a problem.  
         SUMMARY OF INVENTION  
         [0005]    An illustrative system for monitoring a computer network and notifying an attendant if a problem is detected according to one aspect of the present invention comprises a monitoring server connected to the network and operative to communicate with a plurality of monitored servers belonging to the network and a monitor program hosted on the monitor server and operative to test the performance and correct functioning of selected ones of the monitored servers and the presence or absence of problems related to applications running on the monitored servers.  
           [0006]    An illustrative process of monitoring a network and automatically notifying an attendant of network problems according to an alternative aspect of the invention comprises the steps of testing for the presence of one or more monitored servers, testing a data transfer rate of the one or more monitored servers, examining logs maintained by one or more monitored applications for the presence of entries indicating problems and automatically notifying an attendant if one or more tests is failed or if a problem entry is present.  
           [0007]    A more complete understanding of the invention, as well as further features and advantages of the invention, will be apparent from the following Detailed Description and from the claims which follow below. 
       
    
    
     BRIEF DESCRIPTION OF DRAWINGS  
       [0008]    [0008]FIG. 1 illustrates a network performing self-monitoring according to an aspect of the present invention; and  
         [0009]    [0009]FIG. 2 illustrates a process of automatic monitoring of a network according to an aspect of the present invention. 
     
    
     DETAILED DESCRIPTION  
       [0010]    [0010]FIG. 1 illustrates a network  100  according to an aspect of the present invention.  
         [0011]    The network  100  includes a plurality of servers  102 A- 102 C. For purposes of illustration, the servers  102 A- 102 C will be described here as a security server, a file server and a print server, respectively. While only three servers are shown as exemplary in FIG. 1, it will be recognized that a network may include a large number of servers which are not shown here for ease of illustration. The network  100  also includes a number of client computers  104 A- 104 E communicating with the servers  102 A- 102 C through a communication center  106 . The communication center  106  is illustrated here as a single entity, but it will be recognized that the communication center  106  may be any system capable of receiving messages from a computer such as the servers  102 A- 102 C and the clients  104 A- 104 E and routing the messages to the proper destination. Such a communication center  106  may comprise a single hub, a series of hubs, a collection of hubs and routers, the Internet, or whatever other combination is needed or useful for correct routing of messages. Communication between the servers  102 A- 102 C, clients  104 A- 104 E and the communication center  106  will be described here as employing transmission control protocol/Internet protocol (TCP/IP), but it will be recognized that any suitable communication technique may be employed.  
         [0012]    The security server  102 A runs a security and authentication application  108 A, the file server  102 B runs a file manager application  108 B and the print server  102 C runs a print manager application  108 C. The security and authentication application  108 A produces a security event log  110 A, the file manager application  108 B produces a file event log  108 B and the print manager application  108 C produces a print event log  110 C. The logs  110 A- 110 C are preferably stored as ordinary text in order to make them more easily readable. The system  100  also includes a monitoring server  112 , running a monitor program  114 . It will be recognized that the monitor program  114  may reside on any of the servers  102 A- 102 C, but is shown here as residing on a separate server in the interest of clarity. It will also be recognized that each of the servers  102 A- 102 C may run numerous applications which can be monitored, but in the interests of avoiding repetition, only the applications  108 A- 108 C and their logs  110 A- 110 C will be described here.  
         [0013]    The smooth functioning of a network such as the network  100  depends in large part on the proper functioning of all critical servers. This includes the ability of a server to send and receive messages and to maintain a proper data transfer rate. In addition, all critical applications must perform correctly. In the system  100 , the applications  108 A- 108 C enter all significant events in the logs  110 A- 110 C, so that examination of the logs  110 A- 110 E will show any improper event.  
         [0014]    The monitor program  114  periodically monitors each of the servers  102 A- 102 C and the applications  108 A- 108 C. The monitor program  114  preferably takes information about what servers and functions are to be monitored and what conditions indicate problems from a monitor information database  116 .  
         [0015]    The monitor program  114  extracts information from the database  116  to create a script  118  to govern the testing of the network  100 . The monitor program  114  then tests the response and data transfer capabilities of the servers  102 A- 102 C in accordance with the instructions in the script  118 . The monitor program also examines the event logs  110 A- 110 C, also in accordance with the instructions in the script. The monitor application  114  may suitably modify the script in accordance with the results that it receives, in order to perform testing in the most efficient and useful manner possible.  
         [0016]    In order to monitor a server, the monitor program  114  pings the server. Pinging is the sending of a request for a response to a server. In a system using TCP/IP protocol, the request is sent to the TCP/IP address of the server and includes the address to which the response is to be sent. For example, suppose the monitor program  114  pings the print server  102 C. If the monitor program  114  does not receive a proper response, it concludes that the print server  102 C is not responding and prepares a message for the network administrator or other responsible person. The message is preferably sent in the form of a page, for example by automatically telephoning a paging center  120  and sending a predetermined message retrieved from a message library  122 . Preferably, the monitor program  114  does not report that the print server  102 C is faulty based on a single failure to respond to a ping, but instead pings the print server  102 C repeatedly, for example over a period of several minutes, and pages the administrator only if the print server  102 C fails to respond satisfactorily to the series of pings.  
         [0017]    As an alternative to managing paging of an attendant directly, the monitor program  114  may employ message queuing for transmission of messages. In such an implementation, the monitor program  114  places appropriate messages in a message queue  124  whenever sending of a message is desired. A message manager  126  periodically monitors the message queue  124 . Whenever the message manager  126  detects a message in the message queue, the message manager  126  telephones the paging center  120  and relays the message which has been detected in the message queue.  
         [0018]    In order to prevent hackers and other malicious users from impersonating one of the servers  102 A- 102 C or the server  112 , the system  100  preferably employs proper security precautions. Such precautions may, for example, take the form of an authentication signature appended to each message sent between the server  112  and one of the servers  102 A- 102 E.  
         [0019]    If the print server  102 C has responded properly to a ping, the monitor program  114  then verifies that the print server  102 C can maintain a proper data transfer rate. The monitor program  114  sends a series of pings to the print server  102 C at a frequency selected to properly exercise the print server  102 C. The time of each response from the print server  102 C is recorded and then the timing of the responses is evaluated. If the print server  102 C is unable to receive or respond to the pings at an acceptable rate, the network administrator is paged so that the problem may be investigated.  
         [0020]    In order to avoid undue repetition, the monitor program  114  has been discussed as testing communication with the print server  102 C, but it will be recognized that the monitor program  114  tests communication with each of the servers  102 A- 102 C, in whatever sequence is desired. Moreover, if desired, the monitor program  114  may be designed to adjust the testing sequence based on results received. For example, if the print server  102 C is detected to respond at a rate that is slower than usual, but not slow enough so that an attendant needs to be summoned, the monitor program  114  may increase the frequency with which communication with the print server  102 C is tested so that if further slowing requiring attention occurs, it will be promptly addressed. If the response returns to normal, however, the monitor program  114  may then decrease the frequency of testing until it once again reaches the default rate.  
         [0021]    In addition to testing communication with the servers  102 A- 102 C, the monitor program  114  also reviews the logs  110 A- 110 C in order to make sure that the applications  108 A- 108 C are operating properly. The server  114  runs the script  118  in order to establish communication with the servers  102 A- 102 C and to review desired logs. The logs  110 A- 110 C are preferably saved and maintained under known names which change infrequently, if at all. If the name of one of the logs  110 A- 110 C does change, or if a new application is added so that the monitor program  114  needs to review the operation of the new application, this information is added to the database  116  and can then be used to modify the script  118  to include the new or added names. Operating under control of the script  118 , the monitor program  114  establishes communication with, for example, the security server  102 A. The monitor program  114  retrieves the log  110 A and compares the entries against the database  116 . The database  116  preferably contains, for each log maintained by a monitored application, all entries whose presence indicates problems. The monitor program  114  may also evaluate the frequency and timing of such entries. If a questionable entry or series of entries is found, the monitor program  114  pages the network administrator with an appropriate message. For example, if evaluation of the security event log revealed a series of failed login attempts over a short period of time, the monitor program  114  pages the network administrator in order to alert him or her of a possible attempt to breach security. The monitor program  114  establishes communication with the servers  102 A- 102 C and reviews the logs  110 A- 110 C in whatever sequence is desired, and may be designed to adjust the review cycle in light of results, as described above.  
         [0022]    It is possible for the logs  110 A- 110 C to be implemented in the form of databases. Such an implementation adds power and flexibility to a search for errors and other noteworthy conditions. With the logs  110 A- 110 C implemented as databases, it is possible for the monitor program  114  to construct queries to search for specific conditions or combinations of conditions. If a particular combination of conditions is worthy of special note, the monitor program  114  can periodically query the logs  110 A- 110 C using a database query defining that combination of conditions. The query may suitably be constructed using structured query language or any other suitable form of query implementation consistent with the design of the logs  110 A- 110 C.  
         [0023]    It will be recognized that each of the servers  102 A- 102 C may host numerous applications, any number of which may maintain logs to be reviewed by the monitor program  114 , and that, as noted above, the network  100  may include numerous other servers which may be monitored. Moreover, monitor programs similar to the monitor program  114  may run on multiple servers throughout the network  100 , each monitoring a group of servers and applications running on those servers, in order to distribute the monitoring tasks and thereby to avoid overburdening any single server or monitor program.  
         [0024]    While monitoring of a network  100  has been illustrated here by way of example, it will be recognized that the techniques of the present invention may be employed to monitor computer systems which are not part of a network, for example by running a monitor program such as the program  114  on an individual computer system in order to monitor the operation of that system. In addition, techniques similar to those illustrated here may be employed to monitor the availability and data transfer rate of components which communicate with one another but which are not part of a computer network as the term is commonly understood.  
         [0025]    [0025]FIG. 2 illustrates a process  200  of network evaluation according to the present invention. At step  202 , a network is evaluated to determine which server or servers should be monitored, what functions should be monitored, what event logs, if any, are maintained by functions which should be monitored and what events or sequence of events indicate problems which require notifying an attendant. At step  204 , the results of the evaluation are analyzed and the identities and addresses of the servers to be monitored are stored in a database, along with the names of functions to be monitored, the names of the logs maintained by those functions, which log entries or sequences of log entries indicate problems and the expected rate at which the servers should receive and return data. The database also preferably maintains a set of messages to be transmitted to an attendant, with an appropriate message or messages being associated with each event requiring notification.  
         [0026]    At step  206 , a script is prepared using the information in the database, in order to govern the methods and sequence of evaluation of various network elements and to determine what responses are to be expected from each network element which is tested. At step  208 , authentication and security keys are exchanged between the elements testing and being tested, in order to prevent spurious commands from being acted on and to prevent receipt of information by unauthorized parties. At step  210 , the response capability of each of a plurality of hardware elements, such as servers, executing monitored functions is tested. This testing is preferably done by pinging the elements and noting whether a correct response is received. If a correct response is received from all elements, the process proceeds to step  214 . If a correct response is not received from an element, the process proceeds to step  212  and further testing is performed to determine whether a response can be obtained. If a response is obtained from each element, the process proceeds to step  214 . If an element fails to provide a response obtained, the process proceeds to step  250 , an appropriate message is prepared and an attendant is paged with the message. The process then proceeds to step  260  and the script is modified so that the operator will not be sent further messages about the element which failed to respond. This approach prevents numerous duplicate pages resulting from repeated testing of an element which has failed to return a response. The process then proceeds to step  214 .  
         [0027]    Turning now to step  214 , the data transfer rate of each of a plurality of hardware elements is tested, preferably by repeatedly pinging each entity and evaluating the timing of the responses. If all elements achieve a satisfactory data transfer rate, the process proceeds to step  216 . If an element has failed to achieve a satisfactory data transfer rate, the process proceeds to step  218  and further testing is performed to determine whether a satisfactory transfer rate can be achieved. If the subsequent testing produces a satisfactory transfer rate, the process proceeds to step  216 . If a satisfactory transfer rate is still not achieved, the process proceeds to step  270 , an appropriate message is prepared and an attendant is paged with the message. The process then proceeds to step  280  and the script is modified so that the transfer rate of the failed element will not be repeated. This is to prevent numerous duplicate pages resulting from repeated testing of an entity which has failed to return a response. However, the testing of the presence of the element will continue, so that the operator will still be able to receive messages if the entity fails to perform at all, and designated tests other than the test of the transfer rate will be conducted. The process then proceeds to step  216 .  
         [0028]    Turning now to step  216 , the event log associated with each function being monitored is examined to determine if it contains an entry or sequence of entries indicating problems. If no problem is detected, the process proceeds to step  220 . If desired, the event log may be constructed as a database and examination of the event log may include preparing and submitting a query defining conditions of particular interest.  
         [0029]    If a problem is detected, the process proceeds to step  290 , a message is prepared indicating the nature of the problem and sent to an attendant. Preparation of the message and paging of the attendant may suitably be accomplished by submitting a message to a message queue and subsequent retrieval of the message for transmission to the attendant. The process then proceeds to step  220 .  
         [0030]    At step  220 , the results of testing, as well as any previously stored results, are examined to determine if the script needs to be modified to change the methods and sequence of testing. At step  222 , the results of the last round of testing are stored and any needed changes to the script are made. The process then proceeds to step  210 .  
         [0031]    While the present invention is disclosed in the context of aspects of a presently preferred embodiment, it will be recognized that a wide variety of implementations may be employed by persons of ordinary skill in the art consistent with the above discussion and the claims which follow below.