Abstract:
A system, method, and a computer program product for processing messages in a message queuing system includes an application repository, a message store, and a queue-less application programming interface in each message queuing system.

Description:
FIELD OF THE INVENTION  
       [0001]     The present invention relates to message queuing in an asynchronous messaging network, and in particular to a system, method, and computer program product for processing messages in a message queuing system, especially in message queuing clusters.  
       BACKGROUND  
       [0002]     It is increasingly common to distribute a data processing operation over a plurality of data processing units, with each of the units communicating over a data communications network such as the Internet. One reason for this is that a particular data processing unit may be able to do a job better than another, so a first unit makes a request for a second unit to do a particular job and then to return the result to the first unit.  
         [0003]     It is also common to have a large number of intermediate data processing units, which are also known as “nodes,” between the originating unit that makes a request and the destination unit that is being requested to do the work. Each intermediate unit receives the request, performs some initial processing to determine what to do with it, and then forwards it on to the next unit.  
         [0004]     One mechanism for carrying out such distributed data processing is called asynchronous message queuing, where applications communicate with each other by sending messages to a queue, which can then be accessed by the receiving application at a time that is convenient for the receiving application. IBM&#39;s WebSphere MQ (trademark) software product, which has been on the market for a number of years, is a popular example of this type of software.  
         [0005]     Applications connect to a message queuing system and usually retrieve messages from one queue, perform their message processing with the retrieved message, and put the message on another queue in the message queuing system. From that queue, another application retrieves the message, performs its message processing, and puts it on another queue, and so on. To implement a message flow, however, each application needs to know the queue names for getting and putting the messages from or to queues. This becomes even more important in a grid where a service provider may provide services in a message queuing network.  
       SUMMARY  
       [0006]     The present invention provides a system, method, and a computer program product for processing messages in a message queuing system by introducing an Application Repository, a Message Store, and a queue-less application programming interface (API) in each message queuing system.  
         [0007]     Applications provide application identifying data like the application class, the provider name, the application name, and the version, using the connect function of the message queuing API to the Message Manager. The Message Manager stores the provided application identifying data of each application connected to the queuing system in the Application Repository, and removes them when the application disconnects from the queuing system. The application which puts messages to the queuing system provides data for identifying the target application (ApplIdentData) like application class, provider name, application name, and version, using the message PUT function of the message queuing API. The Message Manager stores each message received from the applications including ApplIdentData of the target application in the Message Store, and provides messages from the Message Store to the matching applications requesting messages from the queuing system. In a message queuing cluster, the member message queuing systems exchange their Application Repositories. If an application connects to one message queuing system in a message queuing cluster, the Message Managers of the other message queuing systems automatically forward messages dedicated for that application to that message queuing system until the application disconnects.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0008]     The invention will be best understood by reading the following detailed description together with the drawings, wherein:  
         [0009]      FIG. 1  shows a prior art message queuing system;  
         [0010]      FIG. 2  shows the structure of a message used by the inventive message queuing system;  
         [0011]      FIG. 3  shows the inventive message queuing system;  
         [0012]      FIG. 4  shows the process flow for starting and connecting an application to the message queuing system according to the present invention;  
         [0013]      FIG. 5  shows the process flow for retrieving a message from the queuing system according to the present invention;  
         [0014]      FIG. 6  shows the process flow for putting a message to the queuing system according to the present invention;  
         [0015]      FIG. 7  shows the process flow for disconnecting an application from the queuing system according to the present invention;  
         [0016]      FIG. 8  shows the process flow for receiving and processing an Application Repository update message from another member message queuing system in the message queuing network using the present invention; and  
         [0017]      FIG. 9  shows a message queuing network using the present invention.  
     
    
     DETAILED DESCRIPTION  
       [0018]      FIG. 1  shows a prior art message queuing system within a messaging network. Applications  6  generates messages and send them to message queuing systems  1 ,  2 . The message queuing systems  1 , 2  each include at least a queue manager  10 , queues  22 , and communication links  30  to other message queuing systems. A queue  22  is a data structure used to store messages until they are retrieved by an application. Queues  22  are managed by the queue managers  10 . Each queue manager  10  is responsible for maintaining the queues  22  it owns, and for storing all the messages it receives from application programs or other queue managers onto the appropriate queues, and retrieving the messages in response to application requests.  
         [0019]     Queues  22  exist independently of the applications  6  that use them. A queue  22  can exist in main storage if it is temporary, on disk or similar auxiliary storage if it must be kept for recovery, or in both places if it is currently being used and must also be kept for recovery.  
         [0020]     A message queuing system therefore is a system for asynchronous, assured exchange of information in the form of messages between applications connected to the system. The message descriptor of the message is known and interpretable by the message queuing system but not the message body whose layout is variable and application dependent. A message queuing system therefore stores and transports the messages regardless and without knowledge of the message content in the message body.  
         [0021]     Multiple message queuing systems can be interconnected in a message queuing network which allows applications on different platforms and different physical locations to easily exchange data without knowledge of the underlying infrastructure. But a source application needs to know the queue name of the target application as well as the name of the queuing system to which the target application is connected. If the target application moves from the original message queuing system to another member message queuing system of a message queuing cluster, the messages on the queue of the origin message queuing system are lost for that application until they are manually transferred to the message queuing system the application has moved to.  
         [0022]      FIG. 2  shows the structure of a message as used by the present invention. Every such message has a message descriptor  10  and a message body  20 .  
         [0023]     The message descriptor  10  identifies the message using a message ID, and contains additional control information such as the type of the message and the priority assigned to the message by the sending application.  
         [0024]     The message body  20  includes the application data. The content and structure of the application data are defined by the application program that use them.  
         [0025]      FIG. 3  shows the inventive message queuing system. The inventive message queuing system does not need a queue interface for asynchronously exchanging messages between applications. The message putting application specifies the target application on the “message put request” and the message queuing system automatically delivers the message to the respective target application. This is accomplished using a Message Store  10  for storing messages provided by a “message put request” by the applications  6 , an Application Repository  9  for storing ApplIdentData of the connected applications  6 , and a Message Manager  8 —using a queuing API (Application Programming Interface;  7 )—for processing requests (e.g., connect, put, get, and disconnect) generated by the applications  6 .  
         [0026]     The central component of the inventive message queuing system is the Message Manager  8 . It provides the interface to the applications  6 , executes requests of applications, and manages the inventive Message Store  10  and Application Repository  9  as well as the communication links  30 . In a message queuing cluster, the Message Manager  8  additionally distributes the Application Repository  9  to the members of the message queuing cluster, and distributes the messages according to the application connection data in the Application Repository  9 .  
         [0027]     Each message queuing system may have its own Application Repository  9 . The Application Repository  9  has an interface with the Message Manager  8 . The Application Repository  9  is used for storing ApplIdentData of connected applications  6  including the message queuing system name the application  6  is connected to, and the performance index. The performance index is periodically calculated by the Message Manager  8 , and is the quotient of the throughput rate of the application  6  and the current number of messages for that application  6  in the Message Store  10 . The performance index is used for workload balancing in a message queuing cluster. The ApplIdentData is provided by the application  6  with the “connect request” and preferably includes five data elements: 
        an application class which defines the function of the application (e.g. message authentication, message validation, message auditing, customer verification),     provider name of the application (e.g. IBM, SWIFT, MQSoftware),     application identifier (e.g. ‘VerifyMsg’, ‘com.ibm.MsgVerification’, ‘ICSQ002’, etc.),     application version (e.g. ‘2.1.0’, ‘5123’), and     Application Identifier Key (ApplIdentKey), which is computed during start-up of the application by the initialization function of the used message queuing API.        
 
         [0033]     The ApplIdentData  11  are permanently stored, as long as the application is connected, in the Application Repository  9 , and are concurrently available with the existing session between application and Message Manager. In a preferred embodiment, the Application Repository  9  also includes accounting information for each application connected to the message queuing system.  
         [0034]     Accounting information includes the user ID from the message descriptor of the processed messages, the number of messages processed as well as the total message size, and the time frame in which the messages have been counted. The Application Repository  9  is preferably implemented as data base.  
         [0035]     Each message queuing system may have its own Message Store  10  as shown in  FIG. 3 . The Message Store  10  has an interface to the Message Manager  8 . The Message Manager  8  stores the messages received by the connected applications  6  in the Message Store  8  and sends matching messages from the Message Store  8  to connected applications  6  requesting messages from the message queuing system  1 . The messages stored in the Message Store  10  have assigned the ApplIdentData of the target application provided with the “message put request” of the connected application as well as the source ApplIdentData of the putting application provided with a session between putting application and Message Manger  8 . The Message Store  10  is preferably implemented as data base (e.g. IBM DB2).  
         [0036]     The message matching operation is performed on the target ApplIdentData stored with the message and the ApplIdentData of a connected application requesting a message from the queuing system on several levels: 
        if the ApplIdentKey is specified with the target ApplIdentData of the message, the message is only sent to the application with that ApplIdentKey     if the ApplIdentKey is not specified, a comparison of the ApplIdentData is performed on the remaining four data elements. The application class as well as the provider name must match exactly if specified. The application ID supports wildcards, and the application version supports compare operators. Data elements not specified are treated as wildcards. 
 
 The Message Manager  8  may operate on a first-in-first-out or on a message-priority basis. 
       
 
         [0039]     In a further embodiment of the present invention, a target application list can be provided with the message which allows assigning a workflow to the message. The message is then sent to applications in the order specified in the target application list by the message queuing system.  
         [0040]     The message queuing API is used as an interface between application  6  and Message Manager  8 , and provides at least following functions: 
        establishing a connection or session between application  6  and Message Manager  8 , thereby providing the ApplIdentData of the connected application  6 ,     disconnecting a connection or session between an application  6  and a Message Manager  8  and thereby deleting the ApplIdentData of the disconnecting application  6  from the Application Repository  9 ,     putting a message to the message queuing system  1 , thereby providing the ApplIdentData of the target application  6  with the message, and     retrieving a message from the message queuing system  1 .        
 
         [0045]     Putting a message to the message queuing system  1  requires that the “message put request” include at least an application class or the ApplIdentKey of the target application. Optionally, the provider name, the Application ID, and the application version can be included. ApplIdentData of the receiving application  6  can be provided with the message descriptor for response report processing purposes.  
         [0046]     When getting a message from the message queuing system, the “message get request” may include data elements of the ApplIdentData of the source application  6 . The Message Manager  8  scans the Message Store  10  for messages having an assigned ApplIdentData that matches ApplIdentData of the requesting application  6 , and provides the matching message to the requesting application  6 .  
         [0047]     In a further embodiment of the present invention, the Message Manager  8  provides the additional functionality for operating in a message queuing cluster. The additional functionality of the Message Manager  8  comprises the distribution of the content of the Application Repository  9  to the members message queuing systems of the message queuing cluster. This allows sending messages designated for an application connected to another member of the message queuing cluster directly to that message queuing system. The process flow within such a message queuing cluster is described in connection with  FIG. 9 .  
         [0048]      FIG. 9  shows a message queuing cluster  100  comprising four inventive message queuing systems as described in connection with  FIG. 3 , each having several applications  6  which are connected to the respective message queuing system. Each message queuing system in the cluster  10  includes a Message Manager  8 , a Message Store  10 , an Application Repository  9 , and communication links  30  connecting the message queuing systems with each other. The content of each Application Repository  9  is distributed to each member of the cluster  100  and synchronized whenever an application  6  connects to or disconnects from a message queuing system  1  as described below. An application  6  puts a message to its connected queuing system  1  for another application  6  connected to any member of the message queuing cluster  100 . The Message Manager  8  of the receiving message queuing system  1  scans its Application Repository  9  for an application  6  having the same ApplIdentData as specified with the message put request. If it finds an application  6  locally connected to the same message queuing system, it stores the message in its Message Store  10  until the target application  6  on the same message queuing system  1  requests that message. If it finds a matching application  6  connected to one of the members of that message queuing cluster  100 , the Message Manager  8  sends the received message via the communication links  30  to that message queuing system. The Message Manager  8  of the receiving message queuing system  1  stores the received message in its Message Store  10  until the target application  6  requests that message.  
         [0049]     If an application disconnects from a message queuing system and the same application connects to another member of the message queuing cluster  100 , all messages for that application are automatically transferred by the Message Manager  8  to that message queuing system.  
         [0050]     In another embodiment of the present invention, a performance index of every connected application  6  is computed and stored in the Application Repository  9 . The synchronization of the Application Repository  9  in the message queuing cluster  100  is performed either if an application  6  connects or disconnects, or if a performance index deviates by more than a predefined threshold from its last stored and distributed value. If the same application  6  is connected more than one time in the message queuing cluster  100 , a work load balancer component that is part of the Message Manager  8  receiving messages for that application  6  sends the messages to the message queuing system with the highest performance index connected with the application.  
         [0051]     A preferred embodiment of the inventive message queuing system provides a monitoring component for monitoring applications, and in particular messages dedicated to specific applications. Thereby, conditions can be defined for raising an alert if messages for a specific application are not processed within a specified time.  
         [0052]      FIG. 4  shows the process flow for starting and connecting an application to the message queuing system according to the present invention. When an application is started (block  400 ) using the message queuing API, the ApplIdentKey is generated during the initialization (block  405 ) of the respective message queuing API. It is computed from the application program code by applying a HASH function, and is stored in the private memory area of the message queuing API as long as the application is running. Then, the application sends a connect request to the Message Manager of the respective queuing system. Before a connection between Message Manager and an application is established, some security checks may be performed by the Message Manager. Such security checks are know in the prior art and do not need to be described in detail. After a successful security check, the Message Manager establishes a session with the application and stores the ApplIdentData provided with the “connect request” of the application in the Application Repository as well as in the local memory of the established session. If the message queuing system is member of a message queuing cluster, then the Message Manager sends an Application Repository update request to each member of the message queuing cluster for inserting the ApplIdentData of the connecting application in their Application Repositories.  
         [0053]      FIG. 5  shows the process flow for retrieving a message from the queuing system according to the present invention. The application sends a “message get request” to the connected Message Manager and may optionally provide ApplIdentData of the source application which has put the message to the message queuing system. The source ApplIdentData may be configurable data of the application. The Message Manager then scans its Message Store for messages, matching the assigned target ApplIdentData with the ApplIdentData of the requesting application. In the case that the requesting application also provides optionally the ApplIdentData of the source application, the assigned source ApplIdentData of the message must also match with that provided ApplIdentData. If the matching is successful, then the respective message is sent to the requesting application. If the matching is not successful, then it returns with an appropriate return code.  
         [0054]      FIG. 6  shows the process flow for putting a message to the message queuing system according to the present invention.  
         [0055]     If an application sends a “message put request” to the connected Message Manager including the ApplIdentData of the target application, the Message Manager checks whether the respective target application is already locally connected to it. If there is a local connection, the Message Manager stores the provided message including the provided AppldentData of the target application as well as the ApplIdentData of the requesting application provided with the established session in the Message Store. If there is no local connection, and the message queuing system is a member of a message queuing cluster, then the Message Manager checks in its Application Repository whether there is an application with the provided target ApplIdentData already connected to a member of that message queuing cluster. If so, the Message Manager sends the provided message including the provided ApplIdentData of the target application as well as the ApplIdentData of the putting application provided with the established session to that member message queuing system where the target application is connected to. If the message queuing system is not a member of a message queuing cluster, or the target application is not connected to a member of that message queuing cluster, the Message Manager stores the provided message including the provided AppldentData of the target application as well as the ApplIdentData of the putting application provided with the established session in the Message Store.  
         [0056]      FIG. 7  shows the process flow for disconnecting an application from the queuing system according to the present invention.  
         [0057]     If the application sends a “disconnect request” to the Message Manager, the Message Manager removes the existing session with that application and deletes the ApplIdentData of that application from its Application Repository. If the message queuing system is member of a message queuing cluster, then the Message Manager sends an Application Repository update request to each member of the message queuing cluster for deleting the ApplIdentData of the disconnecting application from their application repositories.  
         [0058]      FIG. 8  shows the process flow for receiving and processing an Application Repository update message from another member message queuing system in the message queuing network using the present invention. If the Message Manager receives an Application Repository update request (new application has been connected) of another member of the message queuing cluster, the Message Manager updates its local Application Repository by inserting the ApplIdentData provided with the update request. Then it scans its Message Store for messages matching their assigned target ApplIdentData with the ApplIdentData of the newly connected application, and sends all matching messages to the message queuing system sending the update request.