Abstract:
System, method, and computer program product for automatically monitoring message flow in a message queuing system in an asynchronous messaging network. A message queuing application programming interface appends, in a message descriptor field for recording and monitoring message flow, unique application identification keys of applications that process the message, operation codes applied to the message, and time stamps of operations applied during message processing. The keys are generated by the start-up of the applications and are filled in by each PUT and GET function of the API. After processing of the message, the entries included in the message descriptor field are monitored, and any deviation from a predefined message flow is automatically detected and reported.

Description:
BACKGROUND  
       [0001]     The present invention relates to message queuing in an asynchronous messaging network, and in particular to a method, system, and computer program product for monitoring message flow in a message queuing system.  
         [0002]     It is increasingly common to distribute a data processing operation over a plurality of data processing units, with each of the units communicating over a data communications network such as the Internet. One reason for this is that a particular data processing unit may be able to do a job better than another, so a first unit makes a request for a second unit to do a particular job and then to return the result to the first unit.  
         [0003]     It is also very common to have a large number of intermediate data processing units (also known as “nodes”) between the originating unit that makes a request and the destination unit that is being requested to do the work. Each intermediate unit receives the request, performs some initial processing to determine what to do with it, and then forwards it on to the next unit.  
         [0004]     A popular mechanism for carrying out such distributed data processing is called asynchronous message queuing, wherein applications communicate with each other by sending messages to queues, which can then be accessed by the receiving application at a convenient time. IBM&#39;s WebSphere MQ (trademark) software product is an example of this type of software. More and more companies are providing services based on message queuing systems as underlying information transportation infrastructure.  
         [0005]     A problem in such message queuing systems is to monitor message flow, especially to identify all of the applications involved in processing messages as well as their sequence of involvement. This problem becomes more important when using message queuing systems in a grid architecture, or whenever it is essential to ensure that a message has been processed in a predefined sequence by the authorized applications.  
         [0006]     An example would be, as shown in  FIG. 1 B , a service provider which provides business-to-business (B2B) services for other companies. A company A (Appl 1 ) sends messages to the Service Provider which will be processed and then sent to the other company B (Appl 5 ). However, there is no evident way for the two companies to ensure that the agreed applications have been used in the agreed sequence (Appl 1 , Appl 2 , Appl 3 , Appl 4 , Appl 5 ) and that no other application has used or read their messages.  
       SUMMARY  
       [0007]     The present invention provides a system, method, and computer program product for automatically monitoring message flow in a message queuing system in an asynchronous messaging network. A message queuing application programming interface (API) appends, in a message descriptor field for recording and monitoring message flow, unique application identification keys (ApplIdentKeys) of applications involved in processing the message, operation codes of operations applied to the message, and time stamps of operations applied during processing of the message. The ApplIdentKeys are automatically generated by the start-up of the applications and filled in by each PUT and GET function of the message queuing API. After processing of the message, the content of the entries included in the message descriptor field is monitored, and any deviation from a predefined message flow is automatically detected and reported. 
     
    
     BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS  
       [0008]     The invention will be best understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:  
         [0009]      FIG. 1 A  shows a prior art message queuing system;  
         [0010]      FIG. 1 B  shows a message flow in an asynchronous messaging network;  
         [0011]      FIG. 1 C  shows the structure of a message used by the prior art message queuing system;  
         [0012]      FIG. 2 A  shows the structure of a message according to the present invention;  
         [0013]      FIG. 2 B  shows a prior art message queuing system according to  FIG. 1 A  extended by the inventive message queuing API for generating the inventive structure of the message as shown in  FIG. 2 A ;  
         [0014]      FIG. 2 C  shows a method for generating the ApplIdentKeys;  
         [0015]      FIG. 2 D  shows a method for generating list of entries (content) in the message descriptor field during processing of the message;  
         [0016]      FIG. 2 E  shows a method for monitoring the message flow based on the list of entries (content) included in the message descriptor field;  
         [0017]      FIG. 2 F  shows an exemplary embodiment of a message queuing system in which the present invention may be preferably used; and  
         [0018]      FIG. 2 G  shows a messaging queuing system scenario and examples of the message flow in that messaging queuing system scenario using the present invention. 
     
    
     DETAILED DESCRIPTION  
       [0019]      FIG. 1 A  shows message queuing systems  11 ,  12  within a messaging network  2 . An application  6  generates messages and sends them to one of the message queuing systems  11 ,  12 . Each of the message queuing systems  11 ,  12  may include a queue manager  10 , Queues  22 , and communication links  30  to other message queuing systems. A queue  22  is a data structure used to store messages until they are retrieved by an application. Queues  22  are managed by a queue manager  10 . The queue manager  10  is responsible for maintaining the queues  22  it owns, and for storing all the messages it receives from application programs or other queue managers onto the appropriate queues, and retrieving the messages in response to application requests.  
         [0020]     Queues  22  exist independently of the applications that use them. A queue can exist in the main storage if it is temporary, on disk or similar auxiliary storage if it must be kept in case of recovery, or in both places if it is currently being used and must also be kept for recovery.  
         [0021]     A message queuing system is a system for asynchronous, assured exchange of information in the form of messages between applications connected to the system. The message descriptor of the message is known and interpretable by the message queuing system, but not the message body whose layout is variable and application dependent. A message queuing system stores and transports messages regardless and without knowledge of the content in the message body.  
         [0022]     Multiple message queuing systems can be interconnected in a message queuing network  2  which allows applications on different platforms and different physical locations to easily exchange data without knowledge of the underlying infrastructure.  
         [0023]      FIG. 1 C  shows the structure of a prior art message. Every message consists of a message descriptor and a message body. The message descriptor identifies the message (message ID) and contains additional control information such as the type of the message and the priority assigned to the message by the sending application. The message body includes the application data. The content and structure of the application data is defined by the application program that uses them.  
         [0024]      FIG. 2 A  shows the new structure of a message  7  that is generated according to the present invention. Compared with the prior art message structure, the message descriptor  8  is extended by field  10  for recording message flow of message in a message queuing system (hereinafter called message descriptor field). The message descriptor field  10  contains Application Identification Keys (hereinafter called ApplIdentKey) of each application getting the message from a queue or putting the message into a queue, an operation code identifying the operation applied to the message, and a time stamp of operation of the message.  
         [0025]     The message descriptor field  10  cannot be modified by the application code because the message queuing API is the interface to the messaging queuing system. The ApplIdentKey is also passed to the queue manager with each PUT or GET request. On a roll-back of a GET operation (in case of a roll-back requested by the application or in case of an abend of the application) the queue manager appends the ApplIdentKey received with the request to the message descriptor field  10  with the appropriate operation code and the time stamp of the operation.  
         [0026]      FIG. 2 B  shows the prior art message queuing system  2  as shown in  FIG. 1 A  extended by the inventive message queuing API  8  for generating input in the inventive structure of the message as shown in  FIG. 2 A .  
         [0027]     The presently available message queuing API may to be extended in two areas.  
         [0028]     During initialization (start up of the application) the ApplIdentKey may be computed by using the program code of the message processing application  6  from the file system  12  as input and applying a hash function to that input. The ApplIdentKey is preferably stored in the private memory ( 9 ; volatile memory) of the message queuing API  8  and inserted by the message queuing API  8  into message descriptor field of the messages on each PUT and GET operation of each message.  
         [0029]     On each message GET or PUT operation, the ApplIdentKey of the application is appended in the message descriptor field including the type of operation and time stamp of operation. Four types of operations are defined and used for the recording. These operations are: 
        PUT: a message has been put on the queue,     GETC: a message has been successfully taken from the queue,     GETR: a message has been taken from the queue but the operation has been rolled back (identifies problems or intruders who only want to read messages), and     GETB: a message has been read (in contrast to GETR this could be a valid processing step).        
 
         [0034]      FIG. 2 C  shows the inventive method for computing the ApplIdentKey according to  FIG. 2 B . During the start up of a message queuing application the ApplIdentKey is computed by the initialization functions of the message queuing API as shown in  FIG. 2 C  ( 10 ,  20 ,  30 ). The ApplIdentKey is computed by using the application code from the file system including all used dynamic linked libraries as input applying a hash function to that input  40 . The ApplIdentKey is preferably stored in the private memory of the message queuing API  50 . The application then connects to a message queuing system and retrieves a message from a queue ( 100 ; see  FIG. 2 D ).  
         [0035]     The content of the message descriptor field of the retrieved message is safely stored in a private memory area of the message queuing API ( 300 ) and is extended by the ApplIdentKey of the application processing the message, GET C operation code, and current time stamp ( 400 ). The updated message field content is then passed by the message queuing API to the Application ( 500 ), and the application processes the message ( 600 ). Finally, the application passes the updated message for putting on another queue to the message queuing API ( 700 ).  
         [0036]     The message queuing API uses the previously safely stored message descriptor field, and appends the ApplIdentKey computed during start up including the PUT operation code and time stamp in the message descriptor field of the message ( 800 ). The message is then passed to the queue manager of the message queuing system which puts it on the desired queue ( 900 ).  
         [0037]      FIG. 2 E  shows the inventive method for monitoring the message flow by using the content in the message descriptor field. The application is retrieving a message via the message queuing API  15 . The message queuing API appends the ApplIdentKey of the application including the GETC operation code and the current time stamp and then safely stores the content of the message descriptor field in its private memory before passing the message to the application  25 . If the application wants to check whether the message has been processed by the defined applications in the defined sequence, it reads the content of the message descriptor field and compares it with a predefined list of ApplIdentKeys. Thereby the operation codes can be also taken into account if needed  35 . The sequence of the ApplIdentKeys must be identical  45 ,  75 , otherwise the application raises an alert  85  and puts the message in doubt on another queue for further processing such in-doubt messages.  
         [0038]     If the application needs to check only whether only authorized applications have accessed the message, it reads the content of the message descriptor field and checks whether the ApplIdentKeys are in the predefined list of authorized applications  55 . If not, the application raises an alert  85  and puts the message in doubt on another queue for further processing. If the check is successful, the application continuous processing  65 .  
         [0039]     Several examples of a message flow in a messaging queuing system scenario as shown in  FIG. 2 G  with its content in the message descriptor field according to the present invention are described. The initial application Appl 1  in  FIG. 2G  is connected to the queue manager of a message queuing system QM 1 . During start-up of the application, the initialization routine of the message queuing API linked to the application reads the application code including all used dynamic loadable libraries and generates the ApplIdentKey. The ApplIdentKey is appended to the message descriptor field for recording and monitoring message flow for every GET and PUT operation on a queue. Appl 1  puts a message on queue Q 1 , and the message queuing API appends the ApplIdentKey, the operation code PUT, and the current time stamp to the message descriptor field.  
         [0040]     Content of the message descriptor field (ApplIdentKey, operation code, time stamp)  
                                           DE407645D4BA715429362C2E0F1FF23DDAB72395   PUT   TS1                  
 
 Application Appl 2  gets the message from Q 1 . As a part of the message GET-operation the message queuing API appends the ApplIdentKey of Appl 2  to message descriptor field of Appl 1 . The operation code of this message get is GETC in case of a successful get. 
 
         [0041]     Content of the message descriptor field  
                                           DE407645D4BA715429362C2E0F1FF23DDAB72395   PUT   TS1       CC35978824436FFE3C001BF352F996E74E4AC465C   GETC   TS2                  
 
         [0042]     Application Appl 2  does its processing and puts the message on Q 2 . Again, the message queuing API appends the ApplIdentKey of Appl 2  to the message descriptor field in conjunction with the operation code PUT.  
         [0043]     Content of the message descriptor field  
                                           DE407645D4BA715429362C2E0F1FF23DDAB72395   PUT   TS1       CC35978824436FFE3C001BF352F996E74E4AC465C   GETC   TS2       CC35978824436FFE3C001BF352F996E74E4AC465C   PUT   TS3                  
 
         [0044]     Now, assuming application Appl 3  is trying to get messages from Q 2 , it gets the message, reads the content, and rolls back that operation. In this case, the message queuing API appends a fourth entry to the message descriptor field consisting of the ApplIdentKey of Appl 3 , the operation code GETR, and the time stamp of that operation.  
         [0045]     Content of the message&#39;s descriptor field  
                                           DE407645D4BA715429362C2E0F1FF23DDAB72395   PUT   TS1       CC35978824436FFE3C001BF352F996E74E4AC465C   GETC   TS2       CC35978824436FFE3C001BF352F996E74E4AC465C   PUT   TS3       78243CC3549FF56E389C008F996E74E4AC465CADF   GETR   TS4                  
 
         [0046]     Now application Appl 4  gets the message successfully and checks whether this message is from Appl 1  and whether it has been processed successfully by Appl 2  and no further application. In this case, Appl 4  detects that Appl 3  did a message GET and rolled back that operation. So Appl 4  generates an intruder alert on the message so that an operator can decide what to do.  
         [0047]     Another example of this scenario is a case wherein a second application should have processed the message after Appl 2  and before Appl 4 , but not Appl 3 . So again, Appl 4  would generate an alert signaling a processing sequence error.  
         [0048]     A further example would be a case wherein a message processing agreement of two business partners and a service provider is checked. The permissible message processing flows are agreed with the service provider. The service provider provides the ApplIdentKeys of the authorized applications to the business partners. Depending on the services the partners want to use from the service provider, the sending partner may add the intended processing flow to the message by adding a list of ApplIdentKeys of the encountered applications to the message. This list is then encrypted by the sender with the public key of the responder. If the responder receives the message from the service provider, he decrypts the list of ApplIdentKeys with its private key and compares it with the content of the message descriptor field. He or she is now able to detect whether the processing steps have been performed in the indented order by the agreed applications and whether other applications have tried to read the message.  
         [0049]      FIG. 2 F  shows a specific embodiment of a message queuing system preferably used by the present invention. The message queuing system uses a content-based message tracking mechanism. The message descriptor of the messages contains a content key generated by the message generating application. The content key provides search information that is derived from the application data contained in the message body. The content key may be encrypted and authenticated as needed. Applications in the same business context are able to create appropriate search requests. This message system may be adapted to the present invention by extending the message descriptor by a message descriptor field for recording and monitoring message flow.  
         [0050]     Furthermore, each message queuing system may contain a message trace component that administers a message trace queue and add entries in that message trace queue for each operation of a message. Each entry in the message trace queue contains at least the message id, content key, the time stamp of the entry, and source queue as far as the message is not transferred to another message queuing system. If the message is transferred to another queuing system the entry contains additionally the target queue and target queuing system. According to the present invention, the message trace queue may also contain the content of the message descriptor field.  
         [0051]     Furthermore, each message queuing system may be extended by the inventive message queuing application programming interface generating the content of the message descriptor field as described in conjunction with  FIG. 1 C .  
         [0052]     Finally, each message queuing system may contain a message search component which provides the functionality to search for the current location of the message as well as to retrieve the whole trace of the movement of specified a message in messaging network. This allows an auditor to check the processing flow of a specific message.