Providing supplemental semantics to a transactional queue manager

In one embodiment, a computer system instantiates a queue manager configured to process a plurality of existing queue manager commands on messages in a message queue. The computer system instantiates a virtualized instance of a queue manager in a virtual layer associated with the queue manager in the computing system. The a virtualized queue manager instance provides supplemental queue manager commands usable in addition to existing queue manager commands, such that the queue manager can be used to implement the supplemental commands without substantial modification. The computer system receives an indication that a message in a message queue is to be accessed according to a specified command provided by the instantiated virtualized queue manager instance that is not natively supported by the queue manager and the virtualized queue manager performs the specified supplemental command as indicated by the received indication by performing one or more existing queue manager commands.

BACKGROUND

Computers have become highly integrated in the workforce, in the home, in mobile devices, and many other places. Computers can process massive amounts of information quickly and efficiently. Software applications designed to run on computer systems allow users to perform a wide variety of functions including business applications, schoolwork, entertainment and more. Software applications are often designed to perform specific tasks, such as word processor applications for drafting documents, or email programs for sending, receiving and organizing email.

In many cases, software applications are designed to interact with other software applications or other computer systems. Software applications often use multiple different processes and/or threads to perform tasks. These threads and processes are also configured to communicate with each other. Such communication often occurs through the use of a message queue. Message queues are typically managed by queue managers. Generally, queue mangers are configured to deliver messages on the queue and dispose of messages that have been successfully read from the queue. In this manner, the queue is kept up-to-date, even while the queue is constantly changing. Queue managers may also be configured to deliver messages between queues hosted by other queue managers.

Typically, queue managers include a set of commands or semantics that can be used by outside applications to access features of the queue manager. For example, a queue manager may include a receive-message command that accesses and returns a message from the head of the queue. Many other such commands may be included in the queue manager's set of available commands. These commands, however, are generally hard-coded into the queue manager application, and, as a result, updating or adding to the set of available commands may involve a great deal of modification to the queue manager application.

BRIEF SUMMARY

Embodiments described herein are directed to instantiating a virtualized instance of a queue manager configured to provide a plurality of supplemental queue manager commands for performing message access operations. In one embodiment, a computer system instantiates a queue manager configured to process a plurality of existing queue manager commands on messages in a message queue. The computer system instantiates a virtualized instance of a queue manager in a virtual layer associated with the queue manager in the computing system. The virtualized queue manager instance provides supplemental queue manager commands usable in addition to existing queue manager commands, such that the queue manager can be used to implement the supplemental commands without substantial modification. The computer system receives an indication that a message in a message queue is to be accessed according to a specified command provided by the instantiated virtualized queue manager instance that is not natively supported by the queue manager and the queue manager performs the specified supplemental command as indicated by the received indication by performing one or more existing queue manager commands.

In another embodiment, a computer system performs a method for replacing an internal transaction with an external transaction. The computer system receives an indication that a first message in a message queue is to be accessed according to a peek-lock command provided by an instantiated virtualized queue manager instance. The computer system calls a begin-queue-transaction command to initiate a transaction internal to the queue manager. The computer system calls a receive-message command to access the first message within the internal transaction. The computer system records the internal transaction identified with a unique identifier in a portion of virtual layer state and locks the first message in the message queue according to the received peek-lock command. The computer system also calls a replace-transaction command to replace the internal transaction with an external transaction corresponding to an external software application such that the first message is locked during the replacement, preventing other users from accessing the message during the replacement.

DETAILED DESCRIPTION

Embodiments described herein are directed to instantiating a virtualized instance of a queue manager configured to provide a plurality of supplemental queue manager commands for performing message access. In one embodiment, a computer system instantiates a queue manager configured to process a plurality of existing queue manager commands on messages in a message queue. The computer system instantiates a virtualized instance of a queue manager in a virtual layer associated with the queue manager in the computing system. The virtualized queue manager instance provides supplemental queue manager commands usable in addition to existing queue manager commands, such that the queue manager can be used to implement the supplemental commands without substantial modification. The computer system receives an indication that a message in a message queue is to be accessed according to a specified command provided by the instantiated virtualized queue manager instance that is not natively supported by the queue manager and the queue manager performs the specified supplemental command as indicated by the received indication by performing one or more existing queue manager commands.

In another embodiment, a computer system performs a method for replacing an internal transaction with an external transaction. The computer system receives an indication that a first message in a message queue is to be accessed according to a peek-lock command provided by an instantiated virtualized queue manager instance. The computer system calls a begin-queue-transaction command to initiate an internal transaction internal to the queue manager. The computer system calls a receive-message command to access the first message within the internal transaction. The computer system records the internal transaction identified with a unique identifier in a portion of virtual layer state and locks the first message in the message queue according to the received peek-lock command. The computer system also calls a replace-transaction command to replace the internal transaction with an external transaction corresponding to an external software application such that the first message is locked during the replacement, preventing other users from accessing the message during the replacement.

However, it should be understood, that upon reaching various computer system components, program code means in the form of computer-executable instructions or data structures can be transferred automatically from transmission media to physical storage media. For example, computer-executable instructions or data structures received over a network or data link can be buffered in RAM within a network interface card, and then eventually transferred to computer system RAM and/or to less volatile physical storage media at a computer system. Thus, it should be understood that physical storage media can be included in computer system components that also (or even primarily) utilize transmission media.

FIGS. 1A & 1Billustrate computer architectures100A &100B in which the principles of the present invention may be employed. Computer architecture100A includes queue manager125, which itself includes queue1(Q1)126A with messages M1, M2, M3and others (as illustrated by ellipses), collectively referred to as messages127A, queue2(Q2) with messages127B and other queues126C with their corresponding messages127C. Queue manager (QM)125also includes internal transactions130and a set of existing commands135.

In general, queue manager125may be any type of inter-application, inter-process or inter-thread communication manager. Queue manager125may be configured to pass control and/or content between such applications, processes and/or threads (hereinafter, all objects over which the queue manger has control will be referred to as processes for simplicity). The completion of such inter-process communications may be ensured by using internal and/or external transactions. External transactions span multiple transactional resources (e.g. structured query language (SQL) databases and other queuing systems. Internal transactions (e.g.130) are generated by and used within the queuing system itself (e.g. within queue manager125). Internal transactions only involve the local queuing system. Internal transactions provide distinct performance advantages over external transactions in that internal transactions do not require coordination with other transactional systems.

Transactions, in general, are series of communications that are guaranteed to be successful. For example, it may be advantageous in an asynchronous system for a process to be able to access a message at the time most convenient for the process. Thus, another process (P2) may be able to post a message (M1) in a message queue (e.g. Q1126A) that is subsequently accessed at a later time by the first process (P1). If the message is accessed within a transaction, the message will not be permanently removed from the queue until the queue manager receives notification that the message was successfully accessed. In this manner, it is assured that P2's message M1will either be successfully accessed by P1, or that the message will remain on the queue. In some cases, the message may only remain on the queue until a certain time (e.g. a timeout period) has been reached. It should be noted that processes may access messages without transactions. However, in such cases, the guarantees of transaction-based message access are not provided. In some cases, queue manager125may be located and run on a single computing system. In other cases, queue manager125may be distributed over a plurality of different computing systems, possibly in different physical locations, communicatively linked by a computing network such as a local area network (LAN) or via the internet. Queue manager125may be configured to manage virtually any number of internal or external transactions. Such transactions or other data requests may be received by queue manager125as application requests115. Other, internal requests may be received from internal transactions130created by the queue manager as requested by various software applications.

Existing commands135may include those commands, primitives or semantics that are natively supported by queue manager125. For example, queue manager125may natively support the following primitive: “Receive-message” with “Transaction” and “Out Message” as possible arguments or values that can be passed with the command (hereinafter syntactically stated as follows: “Receive-message(Transaction, out Message).” As used herein, the Receive-message command may be used to return a message from the head of the queue (e.g. Q1126A), optionally within a transaction. If no transaction is supplied then the message may be read destructively from the queue, such that the message is no longer maintained on the queue. If a transaction is supplied, the message may be read under a transaction. Once the supplied transaction commits (e.g. the transaction is completed), the message is removed from the queue. Otherwise, if the transaction aborts, the message reappears on the queue in the same position as it was before the Receive-message operation. The retrieved message is unavailable for other applications until the transaction outcome is determined.

Existing commands135may also include “Receive-message-by-lookupid(Transaction, Lookupid, out Message).” This command or operation may look for the message with the specified lookup identifier (ID) in the queue and return it optionally within a specified transaction. Implications for reading under a transaction are generally the same as in the Receive-message operation. Commands135may also include “Begin-queue-transaction(out Transaction).” This command returns a new transaction dispensed by the queuing system's (e.g. queue manager125) transaction dispenser (i.e. it returns a newly created internal transaction). “Begin-transaction(out Transaction)” may also be included in commands135and may be configured to return a new transaction dispensed by a distributed system's transaction dispenser (i.e. it returns a newly created external transaction.

Existing commands135may also include “Commit-transaction(Transaction),” which may be configured to commit the specified transaction causing the queue manager and all associated transactional resource managers to make the changes belonging to the specified transaction permanent. “Abort-transaction(Transaction)” may also be included in existing commands135and may be configured to abort the specified transaction causing the queue manager and all associated transactional resource managers to rollback any changes made within that transaction. It should be noted that other existing commands may be include in commands135and that each command listed above may include more functionality than the functions listed above.

Environment100also includes users105A,105B,105C and other users not shown (105D). Each user may be able to access a separate, corresponding virtualized queue manager instance (e.g. virtualized instances110A,110B,110C and110D, respectively). In other embodiments, as illustrated inFIG. 1B, each user may access the same virtualized queue manager instance110. Virtualized queue manager instances may include one or more supplemental commands (111A,111B,111C) that are usable in addition to existing commands135. A virtualized instance may be thought of as working on top of or in association with queue manager125and may be able to access the functionality of queue manager125. Furthermore, virtualized instances may be able to access any of queues126A-C and any of internal transactions130. Thus, virtualized queue manager instances may be configured to fully interact with queue manager125and each of its components. In some cases, a virtualized queue manager may be treated as and may be interacted with as a software application by the queue manager. Moreover, as will be explained in greater detail below, virtualized queue manager instances may be able to provide supplemental functionality to the queue manager with very little or no modification to queue manager125.

FIG. 2illustrates a flowchart of a method200for instantiating a virtualized instance of a queue manager configured to provide a plurality of supplemental queue manager commands for performing message access. The method200will now be described with frequent reference to the components and data of environments100A and100B ofFIGS. 1A and 1B, respectively.

Method200includes an act of instantiating a queue manager configured to process a plurality of existing queue manager commands on one or more messages in a message queue (act210). For example, queue manager125may be instantiated and may be configured to process multiple different existing commands135on any of messages M1-M3on message queue1(Q1126A). In some cases, queue manager125may already be running as part of an application or operating system on a computing system. Queue manager125may be instantiated as a result of receiving a command from a user or computer administrator that such a queue manager is to be started in order to manage inter-process communication.

Method200also includes an act of instantiating at least one virtualized instance of a queue manager in a virtual layer associated with the queue manager in the computing system, the at least one virtualized queue manager instance providing a plurality of supplemental queue manager commands usable in addition to one or more existing queue manager commands, such that the queue manager can be used to implement the supplemental commands without substantial modification (act220). For example, virtualized queue manager instance111A may be instantiated in a virtual layer associated with queue manager125. Instance111A may provide multiple different supplemental queue manager commands usable in addition to existing queue manger commands135. Using these supplemental commands, the processing capabilities of queue manager125may extended with little or no modification to queue manager125itself

As explained above, each supplemental command (e.g.111) may comprise a number of queue manager primitives, commands, operations, instructions or semantics. Supplemental commands may be accessed by any of users105A-C, and may interact with or use any or all of existing commands135. In some cases, each user may access their own virtualized queue manager instance (e.g. instance110B for user105B). In other cases, as inFIG. 1B, user105B and the other users may access the same virtualized queue manager instance110. In some scenarios, it may be more advantageous to instantiate multiple virtual queue manager instances. In others, it may be more advantageous to instantiate a single virtualized queue manager instance. In cases where only one virtualized queue manager instance has been instantiated, other instances may be instantiated on the fly as determined by queue manager125and/or the computer system on which manager125is operating. In other cases, the decision as to how many instances are to be instantiated may be made automatically and may be adjusted dynamically as computer resources change.

Returning toFIG. 2, method200further includes an act of receiving an indication that a message in a message queue is to be accessed according to a specified command provided by the instantiated virtualized queue manager instance that is not natively supported by the queue manager (act230). For example, queue manager125may receive an indication that message M3of Q2(126B) is to be accessed according to a specified command provided by, for example, virtualized queue manager instance110C, that is not natively supported by queue manager125.

In some embodiments, supplemental commands111(A-C) may include various locking semantics or commands including a “Peek-lock-message(out Message)” command, an “Unlock-message(Message)” command, and a “Delete-message(Message, Transaction)” command. The Peek-lock-message command may be configured to lock and return a message (e.g. M1) from the head of a queue (e.g. Q1126A). The message is still physically present in the queue but is unavailable for other applications. The Unlock-message command may be configured to unlock a previously locked message in the queue making it available for all other applications. The message may be placed in the same position as it was before the message was locked. The Delete-message command may be configured to delete a previously locked message from the queue optionally within a transaction supplied by an application. If a transaction is supplied in the command, then the actual removal of the message from the queue is tied to the supplied transaction.

Working with a virtualized queue manager instance (e.g.110C), queue manager125may be able to perform any of the supplemental commands including the Peek-lock-message command by performing one or more existing commands135. The Peek-lock-message command may be performed by implementing one or more existing queue manager commands135including the following: calling Begin-queue-transaction to initiate an internal transaction internal to the queue manager, calling

Receive-message to access the message within the internal transaction, recording the association between the internal transaction identified with a unique identifier and the accessed message in a portion of virtual layer state (e.g.112). Shared state112may include a hash table which may be used to store the internal transaction's unique identifier and the message that is accessed within this transaction.

As indicated above, other commands provided by virtualized queue manager instances110A-C may include an “Unlock-message” command. Similar to the Peek-lock command, queue manager125may work with a virtualized queue manager instance110to perform the Unlock-message command by implementing one or more existing queue manager commands135including the following: retrieving a recorded identifier from a hash table identifying an internal transaction configured to perform operations in relation to a message, and calling the Abort-transaction command to abort the identified internal transaction. Upon aborting the internal transaction, the message is automatically unlocked.

The supplemental Delete-message command may be performed in one manner when an external transaction has not been supplied by implementing one or more existing queue manager commands135including the following: determining that an external transaction has not been supplied with the delete message command, retrieving a recorded identifier from a hash table identifying an internal transaction configured to perform operations in relation to a message and calling the Commit-transaction command to commit the identified internal transaction such that all changes belonging to the internal transaction are carried out.

The supplemental Delete-message command may be performed in another manner when an external transaction has been supplied by implementing one or more existing queue manager commands135including the following: determining that an external transaction has been supplied with the delete message command, retrieving a recorded identifier identifying an internal transaction configured to perform operations in relation to a message, calling the Abort-transaction command to abort the internal transaction and unlock the message, and calling the Receive-message-by-lookupid command to look for the message with the recorded identifier in the queue and return the message within a user-specified external transaction. Thus, each of the supplemental commands provided by a virtualized queue manager instance may be performed by queue manager125using some implementation of existing commands135. This allows locking semantics and commands to be implemented on queue manager135with little or no modification to the queue manager.

Method200includes an act of the virtualized queue manager performing the specified supplemental command as indicated by the received indication by performing one or more existing queue manager commands (act240). For example, as explained above, virtualized queue manager110performs the specified supplemental command (any one or more of Peek-lock message, Unlock-message and Delete-message) as indicated by the received indication by performing one or a combination of existing queue manager commands135.

Turning now toFIG. 3,FIG. 3illustrates a flowchart of a method300for replacing an internal transaction with an external transaction. The method300will now be described with frequent reference to the components and data of environments100A and100B ofFIGS. 1A and 1B, respectively and environment400ofFIG. 4.

Method300includes an act of receiving an indication that a first message in a message queue is to be accessed according to a peek-lock command provided by an instantiated virtualized queue manager instance (act310). For example, queue manager125may receive an indication (e.g. from user105A) indicating that message M1of Q1(126A) is to be accessed according to the Peek-lock-message command provided by virtualized queue manager instance110A. Similar to scenarios described above, virtualized queue manager instance110A may provide supplemental commands usable by user105A to perform various (locking) features on messages in different queues in the queue manager by leveraging existing queue manager commands135.

Additionally or alternatively, another virtualized queue manager (e.g.110A) may be configured to call, in a second call, the Receive-message command to receive another message from the head of the queue in a second internal transaction, determine that a second, different message is at the head of the queue and receive the second message in response to the second call. It will be understood that, along these lines, multiple different messages may be received using various, subsequent and possibly concurrent command calls.

Method300includes an act of calling a begin-queue-transaction command to initiate an internal transaction internal to the queue manager (act320). For example, after receiving an indication that a Peek-lock-message command is to be performed, queue manager125may call the Begin-queue-transaction command of existing commands135to initiate an internal transaction internal to queue manager125.

Method300includes an act of calling a receive-message command to access the first message within the internal transaction (act330). For example, upon initiating the internal transaction, queue manager125may call the Receive-message command of existing commands135to access message M1within the initiated internal transaction.

Next, method300includes an act of recording the association between the internal transaction identified with a unique identifier and the accessed first message in a portion of virtual layer state (act340). For example, queue manager125may record the initiated internal transaction identified with the unique identifier in shared state112. Additionally or alternatively, shared state112may include a hash table which may be used to store the internal transaction's unique identifier and the message that is accessed within this transaction.

Method300also includes an act of locking the first message in the message queue according to the received peek-lock command (act350). For example, queue manager125may lock message M1in the message queue based on having received the Peek-lock-message command from virtualized queue manager instance110A. Turning to timeline101ofFIG. 4A, act350may coincide with time105(T105) when Consumer A (e.g. user105A) locks message M1. At a (arbitrarily) later point in time (T110), Consumer A may abort the initiated internal transaction by calling the supplemental command ‘Delete-Message’ in a transaction. The virtualized queue manager instance110may execute this command by aborting the internal transaction and re-receiving the message (e.g. M1) within the external transaction. In general, the time between this abort and re-receive is where a race condition may occur (e.g. time106).

During this time period between aborting the internal transaction and receiving message M1in an external (application) transaction (time T120), Consumer B (e.g. user105B) may access and lock message M1at time T115. The message's availability to others during the time period between an internal transaction abort and an external transaction initiation may cause communication errors in some scenarios. It should be noted that the race condition described above may only be unavoidable (without transaction replacement) in scenario100A where there are multiple virtualized queue managers. In scenario100B, since the lock state is shared (in shared state112, for example), the race condition may be avoided by having logic in the single virtualized queue manager110that checks if a consumer is holding a lock before reading the message.

Method300includes an act of calling a replace-transaction command to replace the internal transaction with an external transaction corresponding to an external software application such that the first message is locked during the replacement, preventing other users from accessing the message during the replacement (act360). For example, queue manager125may call the Replace-transaction command to replace the initiated internal transaction with an external transaction corresponding to an external software application. Thus, referring to timeline101ofFIG. 4B, Consumer A may lock message M1at time T105, as was the case inFIG. 4A. Next, at time T150A, Consumer A begins replacing the initiated internal transaction with an external (application) transaction for message M1. Message M1stays locked during the replacing operation (as indicated by the dotted box on the timeline). During this time, Consumer B is unable to access message M1(e.g. time T150B) because the message is locked to outside users.

For a single virtualized queue manager instance110, transaction replacement may be avoided, as the virtualized queue manager will ensure that Consumer B does not receive the message while consumer A is processing it. In cases where multiple virtualized queue manager instances have been instantiated, the virtualized queue manager instance may ensure that the message is unavailable to other consumers by not allowing the message to leave a transaction. Consequently, the internal transaction is replaced with an external transaction.

Thus, virtualized queue manager instances may be instantiated to provide supplemental functionality to a queue manager with very little or no modification to queue manager. Moreover, internal transactions may be replaced with external transactions to avoid race conditions that would allow other users to access messages at improper times within a communication sequence.