Patent Publication Number: US-7716739-B1

Title: Subjective and statistical event tracking incident management system

Description:
BACKGROUND OF THE INVENTION 
   1. Field of the Invention 
   The present invention relates to the protection of computer systems. More particularly, the present invention relates to an incident management system and method. 
   2. Description of the Related Art 
   Incident management systems are capable of detecting actual and suspected internal and external intrusions, e.g., stealth scans, and denial-of-service attacks. The actual and suspected internal and external intrusions and denial-of-service attacks are referred to as threats. The detection of a threat by an incident management system is referred to as an event. Generally, an event is an occurrence of some importance, e.g., has been identified as an occurrence that is to be monitored, and frequently one that has antecedent cause, e.g., is associated with malicious code. 
   Typically, the incident management system forwards the events to a central event manager. The central event manager may also receive events from other incident management systems. This provides an administrator of the central event manager information on all of the events on a network or a plurality of networks being monitored by the central event manager. However, administrators presented with a set of events will rarely find the requisite course of action is obvious. 
   SUMMARY OF THE INVENTION 
   In accordance with one embodiment, a method includes logging past event sequences in a knowledge base, receiving a real-time event sequence, comparing the real-time event sequence to the past event sequences to determine a predicted event sequence for the real-time event sequence, and providing the predicted event sequence. 
   In one embodiment, the method further includes logging past courses of action taken in response to the past event sequences and associated outcomes in the knowledge base, characterizing at least one of the associated outcomes as a positive outcome and providing the course of action associated with the positive outcome as a suggested course of action. Further, a recommendation of a user who previously encountered the event sequence is provided in one embodiment. 
   In the above manner, use of subjectively characterized and/or raw historic data to match real-time event sequences provides users of an incident management system in accordance with one embodiment of the present invention with both subjective advice from those who had previously faced similar conditions as well as raw statistical predictions as to what is expected to come next. This allows the users to take a course of action that results in the most desirable outcome. 
   Embodiments in accordance with the present invention are best understood by reference to the following detailed description when read in conjunction with the accompanying drawings. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
       FIG. 1  is a diagram of a computer system that includes a plurality of networks in accordance with one embodiment of the present invention; 
       FIG. 2  is a flow diagram of an incident management system submission process in accordance with one embodiment of the present invention; 
       FIG. 3  is a flow diagram of a central event manager collection process in accordance with one embodiment of the present invention; 
       FIG. 4  is a flow diagram of an incident management system prediction process in accordance with one embodiment of the present invention; 
       FIG. 5  is a flow diagram of a central event manager prediction process in accordance with one embodiment of the present invention; 
       FIG. 6  is an exemplary diagram of a real-time event sequence and knowledge base in accordance with one embodiment of the present invention; and 
       FIG. 7  is a diagram of a client-server system that includes an event tracking application executing on a computer system in accordance with one embodiment of the present invention. 
   

   Common reference numerals are used throughout the drawings and detailed description to indicate like elements. 
   DETAILED DESCRIPTION 
     FIG. 1  is a diagram of a computer system  100  that includes a plurality of networks  102 A,  102 B, . . . ,  102   n , collectively networks  102 , in accordance with one embodiment of the present invention. Referring to network  102 A, network  102 A includes a plurality of interconnected computer systems  104 A- 1 ,  104 A- 2 , . . . ,  104 A-n, collectively computer systems  104 A. Network  102 A further includes an incident management system (IMS)  106 A also coupled to computer systems  104 A- 1 ,  104 A- 2 , . . . ,  104 A-n. 
   Similarly, networks  102 B, . . . ,  102   n  also include a plurality of interconnected computer systems  104 B- 1 ,  104 B- 2 , . . . ,  104 B-n, . . . ,  104   n - 1 ,  104   n - 2 , . . . ,  104   n - n , respectively. Computer systems  104 B- 1 ,  104 B- 2 , . . . ,  104 B-n, . . . ,  104   n - 1 ,  104   n - 2 , . . . ,  104   n - n , are collectively referred to as computer systems  104 B, . . . ,  104   n , respectively. 
   Networks  102 B, . . . ,  102   n  further include incident management systems  106 B, . . . ,  106   n  also coupled to computer systems  104 B, . . . ,  104   n , respectively. 
   Computer systems  104 A,  104 B, . . . ,  104   n  and incident management systems  106 A,  106 B, . . . ,  106   n  are collectively referred to as computer systems  104  and incident management systems  106 , respectively. Illustratively, computer systems  104  and/or incident management systems  106  include security software such as firewall, antivirus, host and network intrusion detection software capable of gathering, consolidating and correlating events. 
   The particular type of and configuration of networks  102 , computer systems  104  and incident management systems  106  are not essential to this embodiment of the present invention. Further, incident management systems such as incident management systems  106  and events are well known to those of skill in the art. 
   Networks  102 , and, more particularly, incident management systems  106  are coupled to a central event manager computer system  108  by a network  110 . Network  110  is any network or network system that is of interest to a user. An event tracking application  112  is executing on central event manager computer system  108 . 
     FIG. 2  is a flow diagram of an incident management system submission process  200  in accordance with one embodiment of the present invention. Execution of an event tracking application on an incident management system  106  results in the operations of incident management system submission process  200  as described below in one embodiment. 
   In accordance with one embodiment, a course of action was taken in response to the detection of an event sequence. This course of action led to an outcome, e.g., a positive outcome, a negative outcome or some other outcome. Incident management system submission process  200  is performed after the outcome. Thus, the event sequence, course of action and outcome are sometimes referred to herein as a past event sequence, a past course of action and a past outcome, respectively. However, in another embodiment, incident management system submission process  200  is performed in real-time, i.e., during the event sequence, while the course of action is taken, and/or during the outcome. 
   Referring now to  FIGS. 1 and 2  together, from an ENTER OPERATION  202 , flow moves to a COLLECT EVENT SEQUENCE OPERATION  204 . In COLLECT EVENT SEQUENCE OPERATION  204 , an event sequence is collected. In one embodiment, an event sequence is a sequence of events, e.g., a series of events that have occurred over a period of time. Generally, an event is an occurrence of some importance, e.g., has been identified as an occurrence that is to be monitored, and frequently one that has antecedent cause, e.g., is associated with malicious code. An event can be a summarization of several occurrences, e.g., an event is that a denial of service attack has occurred, and an event can be a single occurrence, e.g., a logon attempt. 
   The event sequence generally involves at least one computer system or systems. For example, the event sequence occurs on a network  102 . Collection of an event sequence is well known to those of skill in the art and the particular technique used to collect the event sequence is not essential to this embodiment of the present invention. In one embodiment, an incident management system  106  collects an event sequence occurring on a network  102  and/or on a single computer system  104 . It another embodiment, a computer system  104  collects an event sequence occurring on a network  102  and/or on a single computer system  104 , e.g., on itself or another computer system  104 . 
   Process flow moves from COLLECT EVENT SEQUENCE OPERATION  204  to a COLLECT COURSE OF ACTION OPERATION  206 . In COLLECT COURSE OF ACTION OPERATION  206 , a course of action is collected. The course of action is taken in response to the event sequence collected in COLLECT EVENT SEQUENCE OPERATION  204 . The course of action is sometimes called the response to the event sequence. 
   Process flow moves from COLLECT COURSE OF ACTION OPERATION  206  to a COLLECT OUTCOME OPERATION  208 . In COLLECT OUTCOME OPERATION  208 , an outcome is collected. The outcome is the result of the course of action collected in COLLECT COURSE OF ACTION OPERATION  206 . For example, a course of action X taken at a particular point in the event sequence led to an outcome Y. 
   In one embodiment, an outcome is a positive outcome, e.g., the threat which generated the event sequence is defeated or otherwise a desirable result is obtained. Alternatively, an outcome is a negative outcome, e.g., the threat which generated the event sequence is undefeated or other problems are created or otherwise an undesirable result is obtained. In yet another embodiment, an outcome is a neutral outcome, e.g., the threat is defeated but other problems are created or the result is otherwise marginal or indeterminate. 
   From COLLECT OUTCOME OPERATION  208 , process flow moves, optionally, to a COLLECT RECOMMENDATION OPERATION  210 . In COLLECT RECOMMENDATION OPERATION  210 , a recommendation is collected. The recommendation, e.g., by the administrator of the network upon which the event sequence was detected, is related to the event sequence. For example, the recommendation is a different course of action that the administrator would have taken for the particular event sequence or a characterization of the outcome. Generally, after taking a course of action and observing the outcome, an administrator often learns what the administrator did right/wrong and provides this lesson as a recommendation. However, COLLECT RECOMMENDATION OPERATION  210  is optional and in one embodiment is not performed. 
   From COLLECT RECOMMENDATION OPERATION  210  (or directly from COLLECT OUTCOME OPERATION  208  in the event that COLLECT RECOMMENDATION OPERATION  210  is not performed), process flow moves, optionally, to COLLECT PERSONAL CHARACTERISTICS OPERATION  212 . In COLLECT PERSONAL CHARACTERISTICS OPERATION  212 , the personal characteristics of the computer system, computer systems, network, and/or organization associated with the event sequence is collected. In one embodiment, a personal characteristic is the industry segment to which the organization which took the course of action in response to the detected event sequence belongs. 
   For example, financial institutions of a particular size may encounter similar attacks (similar event sequences) and may take similar courses of action. Accordingly, in one embodiment, a personal characteristic is that the organization is a financial institution. Another personal characteristic is the size of the organization, e.g., financial institution. 
   However, COLLECT PERSONAL CHARACTERISTICS OPERATION  212  is optional and in one embodiment is not performed. 
   From COLLECT PERSONAL CHARACTERISTICS OPERATION  212  (or from OPERATIONS  208  or  210  depending upon which of optional OPERATIONS  210 ,  212  are performed), process flow moves to a SUBMIT INFORMATION TO CENTRAL EVENT MANAGER COMPUTER SYSTEM OPERATION  214 . In SUBMIT INFORMATION TO CENTRAL EVENT MANAGER COMPUTER SYSTEM OPERATION  214 , information is submitted to central event manager computer system  108 . This information includes: (1) the event sequence collected in COLLECT EVENT SEQUENCE OPERATION  204 ; (2) the course of action collected in COLLECT COURSE OF ACTION OPERATION  206 ; and (3) the outcome collected in COLLECT OUTCOME OPERATION  208 . Optionally, the information also includes the recommendation collected in COLLECT RECOMMENDATION OPERATION  210  and/or the personal characteristics collected in COLLECT PERSONAL CHARACTERISTICS OPERATION  212 . From SUBMIT INFORMATION TO CENTRAL EVENT MANAGER COMPUTER SYSTEM OPERATION  214 , process flow moves to and exits at an EXIT OPERATION  216 . 
     FIG. 3  is a flow diagram of a central event manager collection process  300  in accordance with one embodiment of the present invention. Referring now to  FIGS. 1 and 3  together, execution of event tracking application  112  on central event manager computer system  108  results in the operations of central event manager collection process  300  as described below in one embodiment. 
   From an ENTER OPERATION  302 , flow moves to a RECEIVE INFORMATION OPERATION  304 . In RECEIVE INFORMATION OPERATION  304 , central event manager computer system  108  receives information. As described above, this received information is submitted to central event manager computer system  108  during performance of SUBMIT INFORMATION TO CENTRAL EVENT MANAGER COMPUTER SYSTEM OPERATION  214 . As discussed above, the received information includes: (1) an event sequence; (2) a course of action; and (3) an outcome. Optionally, the received information also includes a recommendation and/or personal characteristics. 
   From RECEIVE INFORMATION OPERATION  304 , process flow moves, optionally, to a SANITIZE INFORMATION OPERATION  306 . In SANITIZE INFORMATION OPERATION  306 , the received information is sanitized, sometimes called data cleansed. In one embodiment, sensitive data such as the name or other confidential information of the organization submitting the information is cleansed from the received information. Generally, sensitive data is purged from the receive information and/or otherwise protected from improper disclosure. 
   Of course, if the received information does not contain any sensitive data and/or there are no restrictions on disclosure of the sensitive data, SANITIZE INFORMATION OPERATION  306  is unnecessary, and in one embodiment, is not performed. 
   From SANITIZE INFORMATION OPERATION  306  (or from RECEIVE INFORMATION OPERATION  304  if OPERATION  306  is not performed), process flow moves, optionally, to a SIMPLIFY INFORMATION OPERATION  308 . In SIMPLIFY INFORMATION OPERATION  308 , the received information is simplified. For example, unnecessary or redundant events are collapsed or deleted. As another example, particular actions of the course of action taken that are unique to the organization and not the event sequence are deleted. SIMPLIFY INFORMATION OPERATION  308  is optional, and in one embodiment, is not performed. 
   From SIMPLIFY INFORMATION OPERATION  308  (or from OPERATIONS  304  or  306  depending upon which of optional OPERATIONS  306 ,  308  are performed), process flow moves, optionally, to a CHARACTERIZE INFORMATION OPERATION  310 . In CHARACTERIZE INFORMATION OPERATION  310 , the received information is characterized. For example, the event sequence is characterized as a particular type of threat or a particular threat although other characterizations are possible. As another example, the outcome is characterized, e.g., as a positive, negative or neutral outcome. CHARACTERIZE INFORMATION OPERATION  310  is optional, and in one embodiment, is not performed. 
   From CHARACTERIZE INFORMATION OPERATION  310 , (or from operation  304 ,  306 , or  308  depending upon which of optional operations  306 ,  308 , and  310  are performed), process flow moves to a LOG INFORMATION OPERATION  312 . In LOG INFORMATION OPERATION  312 , the received information is logged, e.g., stored or otherwise captured for later retrieval or use. In one embodiment, the received information is logged in a knowledge base  114  of central event manager computer system  108 . The received information is logged as received, and/or as sanitized, simplified, and characterized depending upon which of operations  306 ,  308 , and  310  are performed. 
   From LOG INFORMATION OPERATION  312 , process flow exits at an EXIT OPERATION  314 . 
   In the above process, central event manager computer system  108  receives information from a variety of different computer systems, networks, and/or organizations. The information is logged to knowledge base  114 . Accordingly, knowledge base  114  is a collection of information documenting past sequences of events, past courses of actions, past outcomes of the past courses of actions, and recommendations. As set forth below, a computer system, computer systems, network, and/or organization detecting a real-time event sequence consults knowledge base  114  to determine a predicted event sequence for the detected real-time event sequence based on past event sequences, sometimes called historical records. Further, the knowledge base  114  is consulted to determine which course of action should be taken to obtain a desired outcome. Thus, upon detection of an attack, knowledge base  114  is consulted to defeat the attack in the most optimum or desired manner. 
     FIG. 4  is a flow diagram of an incident management system prediction process  400  in accordance with one embodiment of the present invention.  FIG. 5  is a flow diagram of a central event manager prediction process  500  in accordance with one embodiment of the present invention.  FIG. 6  is an exemplary diagram  600  of a real-time event sequence  602  and knowledge base  114 A in accordance with one embodiment of the present invention. In one embodiment, incident management system prediction process  400  and central event manager prediction process  500  are complementary and are performed during the interactions between an incident management system and the central event manager computer system. 
   Referring now to  FIGS. 4 ,  5  and  6  together, from an ENTER OPERATION  402 , process flow moves to a DETECT REAL-TIME EVENT SEQUENCE OPERATION  404 . In DETECT REAL-TIME EVENT SEQUENCE OPERATION  404 , a real-time event sequence is detected. In accordance with one embodiment, a real-time event sequence, sometimes called a live event sequence or a current event sequence, is a sequence of events that are occurring in real-time, i.e., that are occurring at the present time. Detection of an event sequence is well-known to those of skill in the art and any one of a number of techniques can be used and the particular technique used is not essential to this embodiment of the present invention. 
   For purposes of illustration, a real-time event sequence  602  includes event E 1 , event E 2 , event E 3 , event E 4 , and event E 5 . Real-time event sequence  602  is detected during DETECT REAL-TIME EVENT SEQUENCE OPERATION  404 . 
   From DETECT REAL-TIME EVENT SEQUENCE OPERATION  404 , process flow moves to SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406 . In SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406 , the real-time event sequence is submitted to the central event manager computer system. In accordance with the illustration of  FIG. 6 , real-time event sequence  602  is submitted to the central event manager computer system. 
   Optionally, personal characteristics about the organization are also submitted during SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406 . In accordance with this embodiment, the real-time event sequence is compared against past event sequences from similar organizations in a manner similar to that discussed below. 
   From an ENTER OPERATION  502 , process flow moves to a RECEIVE REAL-TIME EVENT SEQUENCE OPERATION  504 . In RECEIVE REAL-TIME EVENT SEQUENCE OPERATION  504 , a real-time event sequence is received. Illustratively, the real-time event sequence is submitted in SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406  and received in RECEIVE REAL-TIME EVENT SEQUENCE OPERATION  504 . Again, referring to the illustration of  FIG. 6 , real-time event sequence  602  is received during RECEIVE REAL-TIME EVENT SEQUENCE OPERATION  504 . 
   From RECEIVE REAL-TIME EVENT SEQUENCE OPERATION  504 , process flow moves to a COMPARE REAL-TIME EVENT SEQUENCE TO PAST EVENT SEQUENCE(S) OPERATION  506 . In COMPARE REAL-TIME EVENT SEQUENCE TO PAST EVENT SEQUENCE(S) OPERATION  506 , the real-time event sequence is compared to at least one past event sequence in the knowledge base and more typically to many or all of the past event sequences in the knowledge base. In accordance with one embodiment, a past event sequence is an event sequence which has been observed in the past. Illustratively, repeated performance of incident management system submission process  200  and central event manager collection process  300  logged the past event sequences in the knowledge base as discussed above. 
   In one embodiment, a real-time event sequence is a portion, sometimes called substream, of a longer event sequence. For example, a real-time event sequence is simply the beginning of a longer event sequence. Accordingly, the real-time event sequence is compared to the past event sequence(s) to determine if the real-time event sequence is a portion of a past event sequence. For example, a Markov analysis is used to match the real-time event sequence to a past event sequence or sequences. Markov analyses are well-known to those of skill in the art and so are not discussed in detail to avoid detracting from the principles of the invention. A match is an exact match between the real-time event sequence and the portion of the past event sequence. However, in another embodiment, a certain amount of difference, e.g., a specified number of different events, between the real-time event sequence and a portion of the past event sequence is allowed while still resulting in a match between the real-time event sequence and the portion of the past event sequence. 
   In another embodiment, a real-time event sequence is the entire sequence. In accordance with this embodiment, the real-time event sequence is compared to past event sequences to determine if the real-time event sequence matches a past event sequence or sequences. A match is an exact match between the real-time event sequence and a past event sequence. However, in another embodiment, a certain amount of difference, e.g., a specified number of different events, between the real-time event sequence and a past event sequence is allowed while still resulting in a match between the real-time event sequence and a past event sequence. 
   For example, referring to  FIG. 6 , knowledgebase  114 A includes past event sequences  604 A,  604 B,  604 C, . . . ,  604   n , associated past courses of action CA 1 , CA 2 , CA 3 , . . . , CAn, associated past outcomes O 1 , O 2 , O 3 , . . . , On, and, associated recommendations R 1 , R 2 , R 3 , . . . , Rn, respectively. 
   In one embodiment, the past outcomes O 1 , O 2 , O 3 , . . . , On are subjectively classified as positive outcomes, negative outcomes, or neutral outcomes. Illustratively, past outcomes O 1 , O 2 , O 3 , . . . , On are tagged or otherwise labeled to determine their classification. 
   The real-time event sequence  602  is compared to past event sequences  604 A,  604 B,  604 C, . . . ,  604   n , collectively past event sequences  604 . 
   As a result of this comparison, a determination is made that real-time event sequence  602  matches four past event sequences of knowledgebase  114 A. More particularly, a determination is made that real-time event sequence  602  matches past event sequences  604 A,  604 B,  604 C and  604   n.    
   Specifically, the beginning portion  606 A, sometimes called prefix, of past event sequence  604 A includes events E 1 , E 2 , E 3 , E 4 , E 5  which match events E 1 , E 2 , E 3 , E 4 , E 5  of real-time event sequence  602 . Similarly, the beginning portion  606 B of past event sequence  604 B includes events E 1 , E 2 , E 3 , E 4 , E 5  which match events E 1 , E 2 , E 3 , E 4 , E 5  of real-time event sequence  602 . Past event sequences  604 A and  604 B are identical. 
   In accordance with one embodiment, a beginning portion of a past event sequence are the initial events, i.e., the first events, of the past event sequence. The beginning portion, i.e., the initial events, are followed by final events of the past event sequence. 
   A later portion  610  of past event sequence  604 C includes events E 1 , E 2 , E 3 , E 4 , E 5  which match events E 1 , E 2 , E 3 , E 4 , E 5  of real-time event sequence  602 . In accordance with one embodiment, a later portion of a past event sequence are the final events of the past event sequence, i.e., the events that end the past event sequence. For example, there are no events which follow events E 1 , E 2 , E 3 , E 4 , E 5  of past event sequence  604 C and thus events E 1 , E 2 , E 3 , E 4 , E 5  are the final events of past event sequence  604 C. 
   A later portion  608  of past event sequence  604   n  includes events E 1 , E 2 , E 3 , E 4 , E 5  which match events E 1 , E 2 , E 3 , E 4 , E 5  of real-time event sequence  602 . In accordance with one embodiment, a later portion of a past event sequence are the later events, i.e., the events following the initial events, of the past event sequence. A later portion of a past event sequence is followed by final events of the past event sequence. For example, past event sequence  604   n  includes final events E 9 , E 10 , E 11 , which follow later portion  608  of past event sequence  604   n.    
   From COMPARE REAL-TIME EVENT SEQUENCE TO PAST EVENT SEQUENCE(S) OPERATION  506 , flow moves, optionally, to a PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 . In PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 , a predicted event sequence is provided. A predicted event sequence is the event sequence predicted to occur based on the real-time event sequence. Stated another way, for a real-time event sequence, a following set of events is expected to occur and this following set of events is the predicted event sequence. 
   In accordance with one embodiment, there are several predicted event sequences for a real-time event sequence. The predicted event sequence having the highest probability of occurring, e.g., that has occurred with the highest frequency in the past, is provided in PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 . 
   For example, the real-time event sequence is the beginning or middle portion of a longer past event sequence. The events following the real-time event sequence of the past event sequence are the predicted event sequence. For example, if the real-time event sequence has been observed in 10,000 identical past event sequences and in one other different past event sequence, then the past event sequence which has been observed 10,000 times has a much greater probability of occurring than the past event sequence which has only occurred once. 
   To illustrate, as discussed above, real-time event sequence  602  is observed in past event sequence  604 A,  604 B, which are identical, i.e., two past event sequences. Real-time event sequence  602  is observed in past event sequence  604 C, i.e., a single past event sequence, and in past event sequence  604   n , another single past event sequence. Accordingly, all other things being equal, there is a greater probability, i.e., twice as great in this example, that past event sequence  604 A is representative of the actual event sequence of which real-time event sequence  602  is a part than past event sequence  604 C,  604   n.    
   Thus, in accordance with this illustration, the predicted event sequence having the highest probability of occurring are events E 6 , E 7 , E 8 , E 9 , E 10 , E 11  following beginning portion  606 A of past event sequence  604 A (and  604 B). This predicted event sequence is provided in PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 . 
   In accordance with another embodiment, all of the predicted event sequences are provided in PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 . The predicted event sequences are ranked, e.g., from the predicted event sequence having the highest probability of occurring to the predicted event sequence having the lowest probability of occurring or by giving the probability percentage that the predicted event sequence will occur. 
   Thus, in accordance with this embodiment, the predicted event sequence having the highest probability of occurring are events E 6 , E 7 , E 8 , E 9 , E 10 , E 11  following beginning portion  606 A of past event sequence  604 A. Other predicted event sequences having a lower probability of occurring are events E 9 , E 10 , and E 11  following later portion  608  of past event sequence  604   n  and no further events following later portion  610  of past event sequence  604 C. Accordingly, a predicted event sequence can be that there are no more events predicted to follow the real-time event sequence. These predicted event sequence are provided in PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 . 
   In another embodiment, a confidence level for the predicted event sequence is calculated. A confidence level is the probability that the predicted event sequence will occur based on the real-time event sequence. A predicted event sequence is provided only if the confidence level exceeds a certain threshold. 
   From PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508  (or directly from OPERATION  506  in the event that OPERATION  508  is not performed), flow moves, optionally, to a PROVIDE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  510 . In PROVIDE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  510 , a suggested course of action and a predicted outcome for the suggested course of action are provided. 
   For example, past outcome O 1  associated with past event sequence  604 A was more favorable than past outcome O 2  associated with past event sequence  604 B, e.g., past outcome O 1  was a positive outcome and past outcome O 2  was a negative outcome. Accordingly, past course of action CA 1  and past outcome O 1  associated with past event sequence  604 A are provided as a suggested course of action and a predicted outcome in PROVIDE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  510 . 
   In another embodiment, a confidence level for the predicted outcome is calculated. In this embodiment, a confidence level is the probability that the predicted outcome will occur based on the suggested course of action. A suggested course of action is provided only if the confidence level exceeds a certain threshold. 
   From PROVIDE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  510  (or directly from OPERATIONS  506  or  508  depending upon which of optional OPERATIONS  508 ,  510  are performed), flow moves, optionally, to a PROVIDE RECOMMENDATION OPERATION  511 . In PROVIDE RECOMMENDATION OPERATION  511 , a recommendation is provided. For example, recommendation R 1  associated with past event sequence  604 A is provided in PROVIDE RECOMMENDATION OPERATION  511 . 
   Paying particular attention now to  FIG. 4 , from SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406 , flow moves, optionally, to a RECEIVE PREDICTED EVENT SEQUENCE OPERATION  408 . In RECEIVE PREDICTED EVENT SEQUENCE OPERATION  408 , a predicted event sequence is received. For example, the received predicted event sequence is provided in PROVIDE PREDICTED EVENT SEQUENCE OPERATION  508 . 
   From RECEIVE PREDICTED EVENT SEQUENCE OPERATION  408  (or directly from SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406  in the event that OPERATION  408  is not performed), flow moves, optionally, to a RECEIVE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  410 . In RECEIVE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  410 , a suggested course of action and a predicted outcome for the suggested course of action are received. For example, the predicted outcome for the suggested course of action are provided in PROVIDE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  510 . 
   From RECEIVE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  410  (or directly from OPERATIONS  406  or  408  depending upon which of optional OPERATIONS  408 ,  410  are performed), flow moves, optionally, to a RECEIVE RECOMMENDATION OPERATION  411 . In RECEIVE RECOMMENDATION OPERATION  411 , a recommendation is received. For example, the recommendation is provided in PROVIDE RECOMMENDATION OPERATION  511 . 
   In one embodiment, operations  406 ,  408 ,  410 ,  411  occur in real-time. For example, upon viewing of the real-time event sequence in detect real-time event sequence operation  404 , operations  406 ,  408 ,  410 , and  411  occur in real-time without prompting from the user such that operations  406 ,  408 ,  410 , and  411  are transparent to the user. 
   From RECEIVE RECOMMENDATION OPERATION  411  (or directly from operation  406 ,  408 , or  410  depending on which of operations  408 ,  410 ,  411  are performed), flow moves to a PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412 . In PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412 , a determination is made as to whether the predicted outcome is acceptable. If a determination is made that the predicted outcome is acceptable, flow moves, optionally, to an IMPLEMENT COURSE OF ACTION OPERATION  418  or directly to an EXIT OPERATION  420  if operation  418  is not performed. Conversely, if a determination is made that the predicted outcome is unacceptable, flow moves to a SUBMIT COURSE OF ACTION OPERATION  414 . 
   For example, the administrator, sometimes called a user, requires a different outcome than the predicted outcome received in RECEIVE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  410 . Thus, in accordance with this embodiment of the present invention, the administrator can submit a variety of different courses of action until a desired predicted outcome is received. In this manner, the administrator can customize the course of action to receive a desired outcome. 
   More particularly, if a determination is made in PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412  that the predicted outcome is unacceptable, flow moves to SUBMIT COURSE OF ACTION OPERATION  414 . In SUBMIT COURSE OF ACTION OPERATION  414 , a course of action, sometimes called a proposed course of action, is submitted to the central event manager computer system. 
   From SUBMIT COURSE OF ACTION OPERATION  414 , flow moves to a RECEIVE PREDICTED OUTCOME OPERATION  416 . In RECEIVE PREDICTED OUTCOME OPERATION  416 , a predicted outcome for the course of action is received. Optionally, the recommendation associated the predicted outcome is also received. 
   From RECEIVE PREDICTED OUTCOME OPERATION  416 , flow returns to PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412 . In PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412 , a determination is made as to whether the predicted outcome received in RECEIVE PREDICTED OUTCOME OPERATION  416  is acceptable. If the predicted outcome is unacceptable, operations  414  and  416  are repeated. Alternatively, if the predicted outcome is acceptable, flow moves, optionally, to an IMPLEMENT COURSE OF ACTION OPERATION  418 . 
   In IMPLEMENT COURSE OF ACTION OPERATION  418 , the course of action which led to the acceptable predicted outcome is implemented. The course of action can be implemented in any one of a number of ways and the particular technique used to implement the course of action is not essential to this embodiment of the present invention. From IMPLEMENT COURSE OF ACTION OPERATION  418  (or directly from PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412  in the event that operation  418  is not performed), flow exits at EXIT OPERATION  420 . 
   In accordance with another embodiment, a predicted outcome is not received prior to PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412 . For example, RECEIVE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  410  is not performed. Accordingly, in PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  412 , a determination is made, e.g., by an administrator, whether the predicted outcome and more specifically the lack of a predicted outcome is acceptable. 
   In accordance with another embodiment, an administrator desires to try a course of action and see the predicted outcome. In accordance with this embodiment, process flow moves directly from SUBMIT REAL-TIME EVENT SEQUENCE OPERATION  406  to SUBMIT COURSE OF ACTION OPERATION  414 . Otherwise, process flow within incident management system prediction process  400  remains as discussed above. 
   Paying particular attention now to  FIG. 5 , from PROVIDE RECOMMENDATION OPERATION  511  (or directly from OPERATION  506 ,  508 , or  510  depending on which of optional OPERATIONS  508 ,  510 ,  511  are performed), flow moves to a PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  512 . In PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  512 , a determination is made as to whether the predicted outcome is acceptable. If a determination is made that the predicted outcome is acceptable, flow moves to and exits at an EXIT OPERATION  518 . Conversely, if a determination is made that the predicted outcome is unacceptable, flow moves to a RECEIVE COURSE OF ACTION OPERATION  514 . 
   For example, a different outcome than the predicted outcome provided in PROVIDE SUGGESTED COURSE OF ACTION AND PREDICTED OUTCOME OPERATION  510  is required. 
   More particularly, if a determination is made in PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  512  that the predicted outcome is unacceptable, or directly from operation  506 , flow moves to RECEIVE COURSE OF ACTION OPERATION  514 . In RECEIVE COURSE OF ACTION OPERATION  514 , a course of action is received by the central event manager computer system. 
   From RECEIVE COURSE OF ACTION OPERATION  514 , flow moves to a PROVIDE PREDICTED OUTCOME OPERATION  516 . In PROVIDE PREDICTED OUTCOME OPERATION  516 , a predicted outcome for the course of action is provided. Optionally, a recommendation associated with the predicted outcome is also provided. 
   To illustrate, assume a course of action is received in RECEIVE COURSE OF ACTION OPERATION  514 . This received course of action matches past course of action CA 2  associated with past event sequence  604 B. Past course of action CA 2  is sometimes called another past course of action. Accordingly, the associated predicted outcome is past outcome O 2 . Thus, past outcome O 2  is provided in PROVIDE PREDICTED OUTCOME OPERATION  516 . 
   From PROVIDE PREDICTED OUTCOME OPERATION  516 , flow returns to PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  512 . In PREDICTED OUTCOME ACCEPTABLE CHECK OPERATION  512 , a determination is made as to whether the predicted outcome provided in PROVIDE PREDICTED OUTCOME OPERATION  516  is acceptable. If the predicted outcome is unacceptable, operations  514  and  516  are repeated. Alternatively, if the predicted outcome is acceptable, flow exit at EXIT OPERATION  518 . 
   In the above manner, use of subjectively characterized and/or raw historic data to match real-time event sequences provides users of an incident management system in accordance with one embodiment of the present invention with both subjective advice from those who had previously faced similar conditions as well as raw statistical predictions as to what is expected to come next. This allows the users to take a course of action that results in the most desirable outcome. 
   Although various actions by a user are described above, in one embodiment, trend analysis and machine learning techniques are used. For example, a Bayesian analysis, support vector machines and/or neural network is used. 
     FIG. 7  is a diagram of a client-server system  700  that includes an event tracking application  112 A executing on a computer system  707 , e.g., a first computer system, in accordance with one embodiment of the present invention. Referring now to  FIGS. 1 and 7  together, computer system  707  is representative of computer system  108  of  FIG. 1 , an incident management system  106 , and a computer system  104  in one embodiment. Further, client-server system  700  is part of computer system  100  in one embodiment. 
   Computer system  707 , sometimes called a client or user device, typically includes a central processing unit (CPU)  708 , hereinafter processor  708 , an input output (I/O) interface  710 , and a memory  714 . Computer system  707  may further include standard devices like a keyboard  716 , a mouse  718 , a printer  720 , and a display device  722 , as well as, one or more standard input/output (I/O) devices  723 , such as a compact disk (CD) or DVD drive, floppy disk drive, or other digital or waveform port for inputting data to and outputting data from computer system  707 . In one embodiment, event tracking application  112 A is loaded into computer system  707  via I/O device  723 , such as from a CD, DVD or floppy disk containing event tracking application  112 A. 
   Computer system  707  is coupled to a server system  730  of client-server system  700  by network  110 . Server system  730  typically includes a display device  732 , a processor  734 , a memory  736 , and a network interface  738 . 
   Network  110  can be any network or network system that is of interest to a user. In various embodiments, network interface  738  and I/O interface  710  include analog modems, digital modems, or a network interface card. 
   Event tracking application  112 A is stored in memory  714  of computer system  707  and executed on computer system  707 . The particular type of and configuration of computer system  707  and server system  730  are not essential to this embodiment of the present invention. 
   Event tracking application  112 A is in computer memory  714 . As used herein, a computer memory refers to a volatile memory, a non-volatile memory, or a combination of the two. 
   Although event tracking application  112 A is referred to an application, this is illustrative only. Event tracking application  112 A should be capable of being called from an application or the operating system. In one embodiment, an application is generally defined to be any executable code. Moreover, those of skill in the art will understand that when it is said that an application or an operation takes some action, the action is the result of executing one or more instructions by a processor. 
   While embodiments in accordance with the present invention have been described for a client-server configuration, an embodiment of the present invention may be carried out using any suitable hardware configuration or means involving a personal computer, a workstation, a portable device, or a network of computer devices. Other network configurations other than client-server configurations, e.g., peer-to-peer, web-based, intranet, internet network configurations, are used in other embodiments. 
   Herein, a computer program product comprises a medium configured to store or transport computer readable code in accordance with an embodiment of the present invention. Some examples of computer program products are CD-ROM discs, DVDs, ROM cards, floppy discs, magnetic tapes, computer hard drives, and servers on a network. 
   As illustrated in  FIG. 7 , this medium may belong to the computer system itself. However, the medium also may be removed from the computer system. For example, event tracking application  112 A may be stored in memory  736  that is physically located in a location different from processor  708 . Processor  708  should be coupled to the memory  736 . This could be accomplished in a client-server system, or alternatively via a connection to another computer via modems and analog lines, digital interfaces and a digital carrier line, or wireless or cellular connections. 
   More specifically, in one embodiment, computer system  707  and/or server system  730  is a portable computer, a workstation, a two-way pager, a cellular telephone, a smart phone, a digital wireless telephone, a personal digital assistant, a server computer, an Internet appliance, or any other device that includes components that can execute the event tracking functionality in accordance with at least one of the embodiments as described herein. Similarly, in another embodiment, computer system  707  and/or server system  730  is comprised of multiple different computers, wireless devices, cellular telephones, digital telephones, two-way pagers, or personal digital assistants, server computers, or any desired combination of these devices that are interconnected to perform, the methods as described herein. 
   In view of this disclosure, the event tracking functionality in accordance with one embodiment of the present invention can be implemented in a wide variety of computer system configurations. In addition, the event tracking functionality could be stored as different modules in memories of different devices. For example, event tracking application  112 A could initially be stored in server system  730 , and then as necessary, a portion of event tracking application  112 A could be transferred to computer system  707  and executed on computer system  707 . Consequently, part of the event tracking functionality would be executed on processor  734  of server system  730 , and another part would be executed on processor  708  of computer system  707 . In view of this disclosure, those of skill in the art can implement various embodiments of the present invention in a wide-variety of physical hardware configurations using an operating system and computer programming language of interest to the user. 
   In yet another embodiment, event tracking application  112 A is stored in memory  736  of server system  730 . Event tracking application  112 A is transferred over network  110  to memory  714  in computer system  707 . In this embodiment, network interface  738  and I/O interface  710  would include analog modems, digital modems, or a network interface card. If modems are used, network  110  includes a communications network, and event tracking application  112 A is downloaded via the communications network. 
   This disclosure provides exemplary embodiments of the present invention. The scope of the present invention is not limited by these exemplary embodiments. Numerous variations, whether explicitly provided for by the specification or implied by the specification or not, may be implemented by one of skill in the art in view of this disclosure.