Source: https://patents.google.com/patent/US20170155729A1/en
Timestamp: 2019-05-23 22:33:34
Document Index: 102533252

Matched Legal Cases: ['§120', 'Application No. 14', '§119', 'art 200', 'art 200', 'art 400', 'art 400']

US20170155729A1 - Message Consumer Orchestration Framework - Google Patents
Message Consumer Orchestration Framework Download PDF
US20170155729A1
US20170155729A1 US15/430,127 US201715430127A US2017155729A1 US 20170155729 A1 US20170155729 A1 US 20170155729A1 US 201715430127 A US201715430127 A US 201715430127A US 2017155729 A1 US2017155729 A1 US 2017155729A1
US15/430,127
US10148782B2 (en
Sajithkumar Kizhakkiniyil
Narender Vattikonda
Krishnam Chapa
2013-07-11 Priority to IN3096/CHE/2013 priority Critical
2013-07-11 Priority to IN3096CH2013 priority
2013-08-29 Priority to US14/013,136 priority patent/US9614794B2/en
2017-02-10 Application filed by Apollo Education Group Inc filed Critical Apollo Education Group Inc
2017-02-10 Priority to US15/430,127 priority patent/US10148782B2/en
2017-06-01 Publication of US20170155729A1 publication Critical patent/US20170155729A1/en
2018-12-04 Publication of US10148782B2 publication Critical patent/US10148782B2/en
In a decoupled messaging system, a producer device sends produced messages to a message fabric. The message fabric either stores the received message in designated queues, from which consumer devices may retrieve the messages, or broadcasts the messages to consumer devices that subscribe to topics assigned to the messages. A message consumption orchestrator service gathers capabilities information for the message producer, the message fabric, and the message consumer, and configures resources allocated for the message consumer to more fully optimize message throughput. Further, the message consumption orchestrator identifies patterns in changes of the rate of produced messages, i.e., through analysis of historical data and/or received information. The message consumption orchestrator configures resources for the message consumer to account for predicted need based on the identified patterns.
This application claims the benefit, under 35 U.S.C. §120, as a Continuation of Application No. 14/013,136 (Attorney Docket No. 60201-0344), filed Aug. 29, 2013, titled “Message Consumer Orchestration Framework”, which claims priority, under 35 U.S.C. §119(a), to Indian Patent Appln. No. 3096/CHE/2013, filed Jul. 11, 2013, titled “Message Consumer Orchestration Framework”. The entire contents of each of the above-mentioned applications are hereby incorporated by reference as if fully set forth herein. The applicant(s) hereby rescind any disclaimer of claim scope in the parent application(s) or the prosecution history thereof and advise the USPTO that the claims in this application may be broader than any claim in the parent application(s).
The present invention relates to optimizing message throughput in decoupled message producer/consumer systems, and, more specifically, to automatically adjusting the resources being used for a message consumer of the system.
It is useful, especially in information processing, to decouple an information producer from a consumer of the produced information. Such decoupling may be useful when an information producer produces information at a rate that is faster than an information consumer can process the information.
To decouple an information producer from an information consumer, the information producer sends produced information in messages to a third party message management system (i.e., a “message fabric”) instead of sending the information directly to the information consumer. The message fabric stores messages until the information consumer requests the messages.
Many times, resources for a messaging system are configured without respect to the capabilities of the messaging system components, i.e., the information producer, message fabric, and information consumer. As such, the throughput of a messaging system may be at a sub-optimal rate, given the capabilities of the system. It would be advantageous to configure resources for a messaging system based, at least in part, on the capabilities of the messaging system.
FIG. 1 is a block diagram that depicts an example network arrangement for optimizing the throughput of a decoupled messaging system.
FIG. 2 depicts a flowchart for gathering capabilities information for a messaging system and adjusting the configuration of consumer resources based on the capabilities information.
FIG. 3 depicts an example network arrangement with a message consumer client clone.
FIG. 4 illustrates a flowchart for predicting a future change to the flow of messages from a message producer and adjusting the configuration of consumer resources based, at least in part, on the predicted change.
In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, that embodiments may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the disclosure.
In a decoupled messaging system, a producer device sends produced messages to a message fabric. The message fabric either stores the received message in designated queues, from which consumer devices may retrieve the messages, or broadcasts the messages to consumer devices that subscribe to topics assigned to the messages. According to one embodiment, the decoupled messaging system includes a message consumption orchestrator service that gathers capabilities information for the message producer, the message fabric, and the message consumer, and configures resources allocated for the message consumer to more fully optimize message throughput.
Further, the message consumption orchestrator identifies patterns in changes of the rate of produced messages, i.e., through analysis of historical data and/or received information. The message consumption orchestrator configures resources for the message consumer to account for predicted need based on the identified patterns.
Specifically, the message consumption orchestrator records collected capabilities information and/or received information as historical information. From such historical information, the message consumption orchestrator identifies changes in the rate of messages sent to a particular message queue from message producer service 152. The message consumption orchestrator further identifies patterns in changes in the rate of messages, and adjusts a configuration of consumer resources for the message consumer based, at least in part, on identified patterns.
Decoupled Messaging System Architecture
Techniques are described hereafter for optimizing the throughput of a decoupled messaging system. FIG. 1 is a block diagram that depicts an example network arrangement 100 for optimizing the throughput of a decoupled messaging system. Network arrangement 100 includes a consumer device 110, a server device 120, a message fabric 140, and a producer device 150, communicatively coupled via a network 160. Server device 120 is also communicatively coupled to a database 130. According to embodiments, part or all of example network arrangement 100 and are managed by a cloud computing service or by one or more virtual computing environments. References to cloud computing services herein are non-limiting.
Example network arrangement 100 may include other devices, including client devices, server devices, other kinds of computing devices, and display devices, according to embodiments. For example, according to embodiments, one or more additional consumer devices may be communicatively coupled to network arrangement 100, as described in further detail below. Example implementations of a display device include a monitor, a screen, a touch screen, a projector, a light display, a display of a tablet computer, a display of a telephony device, a television, etc.
Consumer device 110 may be implemented by any type of computing device that is communicatively connected to network 160. Example implementations of consumer device 110 include, without limitation, mobile phones, personal digital assistants, tablets, workstations, personal computers, laptop computers, and any other type of computing device. In network arrangement 100, consumer device 110 is configured with a Message Consumption Orchestrator (MCO) client 112, and a message consumer client 114 having one or more pollers 116A-116N. MCO client 112 is configured with a collector client 118. MCO client 112 and message consumer client 114 may be implemented by one or more logical modules, and is described in further detail below. Consumer device 110 may be configured with other mechanisms, processes and functionalities, depending upon a particular implementation, including, for example, Business Services and components.
Server device 120 may be implemented by any type of computing device that is capable of communicating with at least consumer device 110 over network 160. In network arrangement 100, server device 120 is configured with a Message Consumption Orchestrator (MCO) service 122, which includes a collector service 124. MCO service 122 and collector service 124 may be implemented by one or more logical modules, and are described in further detail below. Server device 120 may be configured with other mechanisms, processes, and functionalities, depending upon a particular implementation. According to another embodiment, MCO service 122 runs on consumer device 110. In this embodiment, MCO service 122 performs one or more functions attributed herein to MCO client 112.
Server device 120 is communicatively coupled to database 130. Database 130 may reside in any type of storage, including volatile and non-volatile storage including random access memory (RAM), one or more hard or floppy disks, or main memory. The storage on which database 130 resides may be external or internal to server device 120. Database 130 stores information gathered by collector service 124 and/or information from MCO service 122.
Message fabric 140 comprises at least one or more message queues, and may be implemented on any one of the devices depicted in FIG. 1, or may be implemented by a distinct computing device that is communicatively connected to network 160 and that is not depicted in FIG. 1. Message flow indicators 142 and 144 depict the logical flow of messages between producer device 150, message fabric 140, and consumer device 110. Message fabric 140 is communicatively coupled to producer device 150 and to consumer device 110 via network 160.
Producer device 150 may be implemented by any type of computing device that is capable of communicating with at least message fabric 140 over network 160. In network arrangement 100, producer device 150 is configured with a message producer service 152. Message producer service 152 may be implemented by one or more logical modules, and is described in further detail below. Producer device 150 may be configured with other mechanisms, processes and functionalities, depending upon a particular implementation.
Network 160 may be implemented with any type of medium and/or mechanism that facilitates the exchange of information between the devices in network arrangement 100. Furthermore, network 160 may use any type of communications protocol, and may be secured or unsecured, depending upon the requirements of a particular embodiment.
The services and clients of network arrangement 100 may receive and respond to Application Programming Interface (API) calls, Simple Object Access Protocol (SOAP) messages, requests via HyperText Transfer Protocol (HTTP), HyperText Transfer Protocol Secure (HTTPS), Simple Mail Transfer Protocol (SMTP), or any other kind of communication, e.g., from another component of network arrangement 100. Further, the services and clients of network arrangement 100 may send one or more of the following over network 160 to another component of network arrangement 100: information via HTTP, HTTPS, SMTP, etc.; XML, data; SOAP messages; API calls; and other communications according to embodiments.
In an embodiment, each of the processes described in connection with one or more of the clients and services of network arrangement 100 are performed automatically and may be implemented using one or more computer programs, other software elements, and/or digital logic in any of a general-purpose computer or a special-purpose computer, while performing data retrieval, transformation, and storage operations that involve interacting with and transforming the physical state of memory of the computer.
Decoupled Messaging System
The messaging system depicted in network arrangement 100 includes message producer service 152 running on producer device 150, message fabric 140, and message consumer client 114 running on consumer device 110. Message producer service 152 is configured to transmit information in messages to message consumer client 114 via message fabric 140. Message consumer client 114 retrieves or receives the messages from message fabric 140 and processes the messages. To retrieve messages from message fabric 140, message consumer client 114 uses a poller, such as poller 116A, which is configured to retrieve a certain number of messages from a message queue of message fabric 140 at a certain frequency, e.g., 10 messages per second.
For example, message producer service 152 sends a message with credit check information, used to validate a credit card transaction, to a particular queue in message fabric 140. Message fabric 140 stores messages in the particular queue for a configurable length of time, such as 14 days. Message consumer client 114 retrieves messages from the particular queue for processing. Thus, message consumer client 114 retrieves and processes messages from message producer service 152 asynchronously from when the messages are produced.
In order to process the messages with credit check information, message consumer client 114 performs one or more of: parsing the message information; retrieving information from a database accessible by message consumer client 114; retrieving information from external sources, such as credit card agencies (not shown in network arrangement 100); dispatch messages to devices not shown in network arrangement 100; etc.
Such a decoupled messaging system can be used to perform any task where it would be beneficial for message producers and consumers to work asynchronously, e.g., when the information producer and the information consumer have different throughput capabilities.
Gathering Capabilities Information
FIG. 2 depicts a flowchart 200 for gathering capabilities information and adjusting the configuration of consumer resources based on the gathered information. At step 202, capabilities information is gathered for at least one of: a message producer, a message fabric, and a message consumer. For example, MCO service 122 collects information from and/or derives information about the capabilities of the messaging system illustrated in example network arrangement 100. MCO service 122 receives capabilities information through collector service 124. According to an embodiment, MCO service 122 derives capabilities information from data gathered by collector service 124, as described in further detail below. According to an embodiment, MCO service 122 stores gathered capabilities information and/or derived capabilities information in database 130.
Gathering Capabilities Information For The Message Consumer
According to an embodiment, collector client 118 is configured with one or more collectors, each of which collects one or more kinds of static and/or dynamic capabilities information for message consumer client 114 and/or consumer device 110. MCO client 112 reports the collected capabilities information to MCO service 122 via collector client 124. MCO client 112 performs a handshake with MCO service 122 upon sending information to MCO service 122 to authenticate the identity of MCO client 112.
Examples of static capabilities information include the top CPU speed for consumer device 110, total RAM memory for consumer device 110, etc. Examples of dynamic capabilities information include RAM memory that is currently available on consumer device 110; current CPU usage; how many pollers 116A-116N are currently running in connection with message consumer client 114; other capabilities information for message consumer client 114; current throughput for a database (not shown in example network arrangement 100) that is being used by message consumer client 114 to process messages; capabilities information for one or more other resources that are involved in processing messages; how much network bandwidth is available at consumer device 110; the average time that it takes message consumer client 114 to process a message; etc.
Collector client 118 is, by default, configured with one or more default collectors to collect information from consumer device 110 and message consumer client 114. For example, default collectors at collector client 118 are configured to collect capabilities information about the infrastructure of collector device 110, including static and dynamic CPU and RAM data.
Custom Collectors
According to an embodiment, MCO client 112 provides a collector implementation framework by which custom collectors may be added to collector client 118. Through the collector implementation framework, an administrator of message consumer client 114 may create a custom collector, for collector client 118, e.g., to collect capabilities information that is specific to the processing of messages by message consumer client 114. As such, custom collectors may provide MCO service 122 with information about how quickly message consumer client 114 is able to process received messages, which affects message throughput at the message consumer.
For example, message consumer client 114 performs one or more queries on a database in connection with processing messages. An administrator implements a custom collector in collector client 118 that retrieves information about the average throughput of database queries on the database, e.g., by performing a query on the database that returns a processing grade for the kind of query that message consumer client 114 calls on the database. In response to the query, the database returns information indicating that the database processes a certain number of the queries per second (such as ten queries per second).
As another example, to process messages, message consumer client 114 communicates with a third party resource (not shown in network arrangement 100). The speed with which the third party resource responds to the communications from message consumer client 114 affects the throughput of messages at the message consumer. In this example, an administrator creates a custom collector at collector client 118 to collect information about how quickly the third party responds to the communications from message consumer client 114. Such a custom collector may request the information directly from the third party resource, or may derive the information from actual response times for the third party resource.
MCO client 112 sends the custom-collected information to MCO service 122, which uses the custom-collected information to calculate the speed with which message consumer client 114 processes messages. According to an embodiment, a custom collector includes one or more rules for translating the custom-collected information to a format that is usable by MCO service 122, e.g., to a standard unit of time. According to another embodiment, a custom collector produces data in a format expected by MCO service 122.
Gathering Capabilities Information for the Message Fabric
According to an embodiment, MCO service 122 gathers capabilities information of message fabric 140 by directly querying message fabric 140 for capabilities information. According to one embodiment, capabilities information for message fabric 140 includes one or more of:
how many messages message fabric 140 has in a particular queue;
how many messages message fabric 140 permits a particular poller 116 to draw from a particular queue at a time;
how much time a particular poller 116 must wait after retrieving messages from a particular queue to retrieve messages again.
Message fabric 140 may expose an API to MCO service 122, or allow SOAP queries, etc. by which MCO service 122 performs capability queries.
According to an embodiment, MCO service 122 derives capabilities information for message fabric 140 by analyzing information available at message consumer client 114 via collector client 118. To illustrate, message fabric 140 denies a request of poller 116A to retrieve eleven messages from a particular queue, and then allows a request of poller 116A to retrieve ten messages from the particular queue (all other things about the requests being similar). MCO service 122 derives, from the rejected request and the subsequent allowed request, that the limit of the number of messages that message fabric 140 permits a particular poller to draw from the particular queue at a time is ten messages.
Gathering Capabilities Information for the Message Producer
According to an embodiment, MCO service 122 gathers capabilities information—such as a rate at which message producer service 152 produces messages for message consumer client 114, a rate at which producer device 150 transmits messages to message fabric 140 for message consumer client 114, etc.—for producer device 150 and/or message producer service 152 by directly querying message producer service 152 or producer device 150. MCO service 122 may query the message producer directly, e.g., using a SOAP call.
According to an embodiment, MCO service 122 derives capabilities information associated with message producer service 152 and/or producer device 150. For example, MCO service directly queries message fabric 140 for a rate at which message fabric 140 receives messages from message producer service 152 for a particular queue. MCO service 122 derives that the rate at which producer device 150 sends messages to message fabric 140 for the particular queue directly correlates with the rate at which message fabric 140 receives messages from message producer service 152 for message consumer client 114.
Adjusting the Configuration of Consumer Resources Based on the Capabilities Information
MCO service 122 uses the information from collector client 118 to identify needed adjustments to the resources being used for message consumer client 114, i.e., the pollers and/or the devices being used to pull messages from message fabric 140, and process messages, for message consumer client 114. Returning to flowchart 200 of FIG. 2, at step 204, a configuration of consumer resources associated with the message consumer are automatically adjusted, based, at least in part, on the capabilities information, by adding at least one consumer clone executing on one or more devices that are not currently used to pull messages from the message fabric for the message consumer.
For example, MCO client 112 sends, to MCO service 122, information indicating that the number of messages, in a particular message queue from which message consumer client 114 pulls messages, is increasing at a particular rate. For purposes of this example, MCO service 122 adjusts the resources for the message consumer based on information for a single message queue. However, within embodiments, MCO service 122 adjusts resources for the message consumer based on multiple sources of messages, including message queues and/or message broadcasts. Adjusting message consumer resources based on a rate of messages from a message broadcast source is described in more detail below.
From the capabilities information sent by MCO client 112, MCO service 122 determines that message consumer client 114 should pull messages from the queue at a faster rate than the client is currently pulling messages. According to an embodiment, MCO service 122 identifies a target pull rate, for the particular queue, at which message consumer client 114 should pull messages from the particular queue. For example, MCO service 122 determines that the target pull rate for the particular queue matches the rate that MCO service 122 has either been informed or has derived that message producer service 152 is currently sending messages to the particular queue. With message consumer client 114 pulling messages at such a target pull rate, the backlog of messages in the particular queue would no longer be increasing.
As a further example, MCO service 122 calculates a target pull rate for the particular queue that is faster than the rate that MCO service 122 has either been informed or has derived that message producer service 152 is currently sending messages to the particular queue. MCO service 122 may set such a target pull rate if the particular queue has more than a threshold number of messages stored therein. With message consumer client 114 pulling messages at such a target pull rate, the backlog of messages in the particular queue would decrease. Once MCO service 122 determines that the backlog of messages in the particular queue has reached a certain target number of messages, which may or may not equal the threshold number, MCO service 122 then sets a new target pull rate for the particular queue that matches the rate that message producer service 152 is sending messages to the particular queue.
If MCO service 122 determines that the target pull rate for the particular queue in message fabric 140 is greater than the rate at which message consumer client 114 is currently pulling messages from the particular queue, then MCO service 122 identifies needed adjustments to increase the resources allocated for message consumer client 114. In other words, one or more additional pollers and/or one or more message consumer client clones should be allocated to message consumer client 114 to increase the rate of pulling messages from the queue to the target pull rate.
Adjusting the Number of Pollers
MCO service 122 may increase the number of pollers 116, associated with message consumer client 114, that simultaneously pull messages from the particular queue to increase the rate at which message consumer client 114 actually pulls messages from the particular message queue. MCO service 122 may also cause that all of pollers 116 are pulling the maximum number of messages that message fabric 140 allows, which maximum number is determined from the gathered capabilities information. For purposes of this example, it is assumed that all pollers are pulling messages from message fabric 140 at the maximum allowed rate.
Based on the capabilities information for consumer device 110, MCO service 122 determines whether consumer device 110 has the capacity to run one or more additional pollers in addition to pollers 116A-116N. Such a determination is based, at least in part, on one or more of: capabilities information indicating the capacity of the infrastructure of consumer device 110 (e.g., CPU, RAM); capabilities information indicating a speed with which message consumer client 114 processes messages; etc.
For example, if consumer device 110 has the capacity to run one or more additional pollers based on the infrastructure of the device, but message consumer client 114 cannot process messages any faster than it currently processes messages, then adding a poller to message consumer client 114 will not increase the throughput of messages for message consumer client 114. If a database that message consumer client 114 queries during message processing could not handle more queries than the database currently handles, then configuring message consumer client 114 to retrieve more messages from message fabric would not result in greater throughput of messages by the message consumer. As a result, MCO service 122 requests additional resources from a cloud computing service, as described in detail below.
Also, if a third party resource, to which message consumer client 114 sends messages during processing of messages, cannot process more messages than the resource currently receives, then increasing message throughput for message consumer client 114 would be a waste of resources
However, if consumer device 110 has the capacity to run one or more additional pollers, and message consumer client 114 has the capability to process messages faster than it currently processes messages, then adding an additional poller to message consumer client 114 would increase the throughput of messages for message consumer client 114.
If MCO service 122 determines that adding a poller to message consumer client 114 is feasible and would increase message throughput, then MCO service 122 calculates the number of additional pollers that would cause message consumer client 114 to pull messages from the particular queue at the target pull rate. Also, based, at least in part, on capabilities information sent to MCO service 122 by MCO client 112, MCO service 122 determines a number of additional pollers that MCO service 122 could feasibly and usefully add to message consumer client 114.
For example, MCO service 122 determines that four additional pollers are needed to achieve the target pull rate, and that consumer device 110 has the capacity to run four additional pollers. Thus, MCO service 122 causes four additional pollers 116 to run at message consumer client 114. As a further example, if MCO service 122 determines that three pollers are needed to achieve the target pull rate, and consumer device 110 has the capacity to run four pollers, then MCO service 122 causes three additional pollers to run at message consumer client 114. According to these examples, no additional resources are needed to achieve the target pull rate for the particular queue.
As yet a further embodiment, MCO service 122 provides, to MCO client 112, logic that allows MCO client 112 to calculate a number of pollers to instruct message consumer client 114 to run. MCO client 112 uses the logic to identify the number of pollers to run and causes message consumer client 114 to run that number of pollers.
According to an embodiment, MCO service 122 provides, to MCO client 112, logic that allows MCO service 122 to dynamically adjust the number of pollers 116A-116N running at message consumer client 114 based on the current capacity of consumer device 110 (i.e., based on gathered capabilities information for CPU, RAM, etc). For example, the logic indicates that if the system capacity of consumer device 110 increases by a certain amount, or has a certain total amount of available resources for pollers, then MCO client 112 should automatically cause message consumer client 114 to run an additional poller 116. As yet another example, the logic indicates that if the system capacity of consumer device 110 decreases to a certain total amount of available resources for pollers, or the total amount of available resources decreases by a certain amount, then MCO client 112 should automatically cause message consumer client 114 to terminate one or more of pollers 116A-116N. According to an embodiment, MCO client 112 reports such automatic addition or reduction of a poller to MCO service 122.
Adding a Consumer Clone
If the maximum number of pollers that consumer device 110 is able to run (e.g., because the available system resources, such as CPU, RAM, etc., or because of limitations during message processing, etc.) is insufficient to achieve the target pull rate, then MCO service 122 requests additional resources to be dynamically provisioned from a cloud computing service that manages consumer device 110. The request includes requested device specifications that are sufficient to run the number of additional pollers needed to achieve the target pull rate for message consumer client 114.
The cloud computing system, in response to the request, dynamically provisions one or more computing devices that collectively meet or exceed the requested specifications. In one embodiment, a cloud computing service dynamically provisions a device by selecting one or more devices, from among multiple available devices managed by the cloud computing service, based, at least in part, on one or more of:
the requested specifications of MCO service 122;
resource requests by other entities;
a projected need for resources projected by the cloud computing service.
According to an embodiment, MCO service 122 receives a rejection of its request for additional computing devices from the cloud computing service when no device at least meets the required specifications, and MCO service 122 submits a second request for multiple devices with collective capacity to meet or exceed the requested specifications.
According to an embodiment, MCO service 122 automatically identifies device specifications that are sufficient to run the number of additional pollers needed to achieve the target pull rate for message consumer client 114 based, at least in part, on historical information for the messaging system of network arrangement 100, as recorded by MCO service 122 (e.g., at database 130). For example, MCO service 122 records, in database 130, historical information that includes how many pollers a device successfully runs and how much RAM and CPU the device had available for pollers at the time. From such information, MCO service 122 calculates the average amount of RAM and CPU that is required to run a particular number of pollers.
To illustrate the provisioning of additional resources, FIG. 3 depicts an example network arrangement 300 with (a) a message consumer client clone (i.e., message consumer client 314 on consumer device 310); and (b) a subset of the entities of network arrangement 100, for simplicity. The cloud computing service that manages consumer device 110 provisions consumer device 310 in response to the request for additional resources from MCO service 122. While network arrangement 300 illustrates two consumer devices 110 and 310, the cloud computing service that manages consumer device 110 may provision any number of devices to run clones of message consumer client 114.
MCO service 122 configures consumer device 310 to serve as a clone of message consumer client 114. According to an embodiment, MCO service 122 causes the appropriate clients to be installed on consumer device 310 (i.e., at least MCO client 312, collector client 318, message consumer client 314, and one or more pollers 316A-316N). According to another embodiment, the cloud computing service managing consumer device 310 causes the appropriate clients to be installed on consumer device 310.
Message consumer client 314 serves as a message consumer clone for message consumer client 114. Specifically, message consumer client 314 retrieves or receives messages from the same sources of messages within message fabric 140 as message consumer client 114, and also processes the messages in the same manner as message consumer client 114. As such, the addition of message consumer client 314 increases the throughput of messages for message consumer client 114.
MCO service 122 identifies a number of pollers to instantiate for the new message consumer clone in order to attain the target pull rate for the particular message queue for message consumer client 114. Specifically, MCO service 122 collects capabilities information for consumer device 310 and/or message consumer client 314 via collector client 318 in a manner similar to collection of capabilities information described in connection with collector client 118 above.
Based, at least in part on the capabilities information for the new consumer clone, MCO service 122 causes a number of pollers 316A-316N to be initiated for message consumer client 314 in a manner similar to identifying a number of pollers and initiating the pollers for message consumer client 114 described above. Furthermore, MCO client 312 may be configured with logic similar to the logic described for MCO client 112 above.
Running resources, for a message consumer, which resources are not running at full capacity constitutes a waste of resources. Therefore, if MCO service 122 determines that the target pull rate for the particular queue in message fabric 140 is less than the rate at which message consumer client 114 is currently pulling messages from the particular queue, then MCO service 122 identifies needed adjustments to decrease the resources allocated for message consumer client 114.
For example, MCO service 122 determines that the target pull rate for pulling messages from the particular queue for message consumer client 114 is supportable by fewer resources than are currently allocated for message consumer client 114. Specifically, MCO service 122 identifies a number of pollers needed to achieve the target pull rate, and the maximum number of pollers that may be terminated leaving message consumer client 114 the capacity to pull messages from message fabric 140 at the target pull rate. If there is only one device pulling messages for message consumer client 114 (i.e., consumer device 110), then MCO service 122 causes termination of the excess number of pollers from pollers 116A-116N. Specifically, MCO service 122 instructs MCO client 112 to cause message consumer client 114 to terminate the excess number of pollers from pollers 116A-116N.
However, if there are multiple devices pulling messages from message fabric 140 for message consumer client 114, then MCO service 122 identifies the optimum device, currently allocated for message consumer client 114, from which to remove pollers. For example, a particular device is running at most the number of pollers that should be terminated, and MCO service 122 selects the pollers from that particular device to be terminated since termination of those pollers would result in decommissioning an allocated device. Decommissioning an allocated device releases the resource to the cloud computing service that manages the device, freeing the device for other purposes. As a further example, if a particular device is designated as a valuable resource, e.g., by a cloud computing service that manages the device, then MCO service 122 selects pollers from that particular device to be terminated in order to free up capacity on the valuable resource
Predicting Future Changes in Message Rates
According to an embodiment, MCO service 122 predicts a message consumer's future need for resources based, at least in part, on historical information recorded by MCO service 122 and/or information from a third party information source. FIG. 4 illustrates a flowchart 400 for predicting a future change to the rate of a flow of messages from a message producer and adjusting the configuration of consumer resources based, at least in part, on the predicted future change.
At step 402, a future change to a flow of messages from a message producer through a message fabric to a message consumer is predicted. For example, MCO service 122 predicts a future change in the rate of messages that message producer service 152 sends to a particular queue in message fabric 140, for message consumer client 114, by identifying a pattern in changes of the rate of messages from message producer service 152 sent to the particular queue. MCO service 122 identifies patterns in changes of the rate of produced messages by analyzing historical information recorded by MCO service 122 and/or information from a third party information source. MCO service further analyzes the identified patterns to predict future changes in the rate of messages from message producer service 152 sent to the particular queue.
Predicting Change Based on Historical Information
According to an embodiment, MCO service 122 records collected capabilities information in database 130 as historical information. From such historical information, MCO service 122 identifies changes in the rate of messages sent to a particular message queue from message producer service 152. MCO service 122 stores the identified changes in the rate of messages as further historical information in database 130. In connection with the flow change information, MCO service 122 also records one or more of: a time of day, a day of the week, a time of year, a calendar date, etc.
MCO service 122 identifies patterns in changes in the rate of messages recorded in the historical information and stores the patterns in database 130. For example, MCO service 122 determines that the rate at which message producer service 152 sends messages to a particular queue in message fabric 140 surges between 9:00 AM and 12:00 PM on weekdays (“peak time”). In this example, the surge rate during this peak time is 65-75% of a particular baseline message rate, e.g., the highest recorded message production rate for message producer service 152. Herein, a percentage rate refers to a percentage of the baseline message rate. Based, at least in part, on this identified pattern of peak times, MCO service 122 predicts that the rate of messages from message producer service 152 to the particular queue will be at 65-75% between 9:00 AM and 12:00 PM on weekdays.
Returning to flowchart 400 of FIG. 4, at step 404, a configuration of consumer resources for the message consumer is adjusted based, at least in part, on the predicted change. For example, in response to predicting that the rate of messages from message producer service 152 to the particular queue will be at 65-75% between 9:00 AM and 12:00 PM on weekdays, MCO service 122 identifies a target pull rate for message consumer client 114 for the peak times, which is based, at least in part, on the predicted message production rate (i.e., 65-75%). MCO service 122 calculates a number of pollers needed to pull messages from the particular queue at the target pull rate (as described above).
MCO service 122 then determines whether to terminate pollers or to request additional resources for message consumer client 114 in order to achieve the target pull rate for the peak time. For example, MCO service 122 schedules, for a certain amount of time before the anticipated peak time (e.g., five minutes), an adjustment of resources for message consumer client 114 to account for the predicted surge rate (i.e., 65-75%). At the scheduled time, MCO service 122 determines that message consumer client 114 is running ten pollers on one or more consumer devices, and that, in order to pull resources at the target pull rate for the predicted peak time, message consumer client 114 requires 14 running pollers. Therefore, as described above, MCO service 122 causes message consumer client 114 to add four pollers, which may involve allocating additional resources, as needed, to run the needed 14 pollers. According to an embodiment, once the peak time is past, MCO service 122 decreases the number of pollers allocated for message consumer client 114.
As a further example, MCO service 122 identifies a pattern, from rate change data stored in historical information, indicating that message producer service 152 produces almost no messages (i.e., between 0%-5%) between 2:00 AM and 6:00 AM on weekdays (“dead time”). In response to identifying this pattern, MCO service 122 schedules a decrease of the resources allocated for message consumer client 114 to an amount that MCO service 122 calculates is sufficient to handle the dead time demand. The scheduled time may be at the predicted start of dead time, or a certain amount of time before or after the predicted start of dead time.
At the scheduled time, MCO service 122 reduces the number of pollers being used for message consumer client 114 and/or clones thereof, by releasing resources that are allocated for message consumer client 114, as described above. MCO service 122 may either decommission all resources allocated for message consumer client 114, or maintain one or more pollers retrieving any messages entering message fabric 140 for message consumer client 114 during the dead time. According to an embodiment, once the dead time is past, MCO service 122 increases the resources allocated for message consumer client 114.
According to an embodiment, MCO service 122 continually records identified patterns in database 130 and capabilities information and other information as historical information in database 130. MCO service 122 also updates stored patterns based, at least in part, on historical information that was recorded after the pattern was last updated. In response to updating the established pattern, MCO service 122 schedules adjustments to the resources for message consumer client 114 based, at least in part, on the updated pattern rather than based on the previous iteration of the pattern.
For example, after determining that 9:00 AM to 12:00 PM on weekdays is a peak time for the flow of messages from message producer service 152 for the particular queue, MCO service 122 records additional historical information. MCO service 122 analyzes at least the additional historical information and, based on the analysis, determines that the peak time for the flow of messages is now 9:45 AM to 12:00 PM on weekdays and 7:00 PM to 10:30 PM on Sunday through Thursday. MCO service 122 updates the peak time pattern in database 130 accordingly, and adjusts the schedule for handling the peak time.
Adjusting Resources for Predicted Message Broadcasts
According to an embodiment, MCO service 122 adjusts resources allocated to message consumer client 114 based on predicted message broadcasts to which message consumer client 114 subscribes, e.g., through message fabric 140. To illustrate, MCO service 122 collects capabilities information that indicates when message consumer client 114 receives broadcast messages and records such as historical information in database 130. MCO service 122 analyzes the historical information to identify a pattern in the receipt of broadcast messages. For example, MCO service 122 determines a pattern that message consumer client 114 generally receives the equivalent of a 10% rate of broadcast messages on the last day of a school's semester, information about which MCO service 122 receives from a third party information source, as described in further detail below. Based on the identified pattern, MCO service 122 schedules an increase of the resources allocated for message consumer client 114 to accommodate 10% more messages on the last day of the school's semesters, as described above.
Predicting Change Based on Third Party Information Sources
According to an embodiment, MCO service 122 receives information from a third party information source (“information source”), which is not depicted in FIG. 1 or 3. A third party information source is none of the message producer, message fabric, or message consumer. An information source provides MCO service 122 with information that MCO service 122 analyzes to identify patterns of changes in the rate of messages from message producer service 152, and to anticipate resource needs based, at least in part, on the patterns.
An information source may send, to MCO service 122, any kind or format of data, e.g., documents, pdfs, tables, XML, HTML, SOAP messages, etc. For example, an information source sends, to MCO service 122, one or more of: summaries of information, such as critical dates; a projected rate of messages; potential indicators of message rate; etc. As another example, an information source sends unprocessed data for MCO service 122 to process, such as syllabuses for a school's educational courses.
In order for MCO service 122 to recognize a third party information source as a valid source of information, the source may register with MCO service 122 (with or without authenticating information) as having information relevant to message producer service 152 and/or message consumer client 114. Further, one or more of the following may register an information source with MCO service 122: MCO client 112, message consumer client 114, a cloud computing service that manages consumer device 110, etc.
An information source may communicate information with MCO service 122 via broadcast, such that MCO service 122 subscribes to a broadcast from the information source. Furthermore, an information source may implement a collector client, such as collector client 118, which is configured to communicate, to MCO service 122, information from a database associated with the information source, from a broadcast to which the information source subscribes, or from other auxiliary sources of information.
An information source may indicate an explicit association between particular communicated information and one or more queues or broadcasts within message fabric 140. Thus, MCO service 122 may use the communicated information to identify patterns for those consumers that subscribe to the one or more queues or broadcasts that the information source associated with the communicated information.
The following example illustrates MCO service 122 receiving and analyzing information from a third party information source. In this example, message consumer client 114 receives and processes messages that represent submissions of homework assignments and completed examinations for a particular school. Administrators for the school set up an information source, such as a server computing device communicatively coupled to network 160, that is registered with MCO service 122 as supplying information pertinent to the flow of messages from message producer service 152.
After it is registered with MCO service 122, the information source sends, to MCO service 122, syllabus data for one or more courses that use message producer service 152 to facilitate processing the submission of homework assignments and exams. For example, the syllabus data includes information about: the start date of courses; the duration of courses; the frequency and timing of examinations and homework assignments for the courses; due dates; vacations; the date of a final exam or project; etc.
MCO service 122 is configured to parse the syllabus data and extract data therefrom that is pertinent to identifying patterns in changes of the rate of messages from message producer service 152. To illustrate, the third party information source informs MCO service 122 that 200 of 1500 available courses have a final exam or project due on the same particular day. MCO service 122 determines, from historical data, that there is a 10-12% rise in the rate of messages produced by message producer service 152 on days where 200-225 courses have an exam or project due on the same day. From this identified pattern, MCO service 122 predicts that message producer service 152 will produce 10-12% additional messages on the particular day, and MCO service 122 schedules an increase of allocated resources for message consumer client 114 a certain amount of time before the predicted start of the predicted rise in rate. MCO service 122 adjusts the allocated resources based on the schedule, as described above.
Predicting Change Based on Both Historical Data and Third Party Information Sources
MCO service 122 may also use historical information that MCO service 122 stored at database 130 in conjunction with data received from an information source to identify patterns in the message flow. For example, based on historical data, MCO service 122 determines that, historically, the third week of a six-week course generally coincides with an uptick in messages from message producer service 152 resulting from homework submissions or exams taken, e.g., by 25% if 500-600 six week courses start at the same time. From the registered information source, MCO service knows that 556 six-week courses recently started on the same day. Therefore, MCO service 122 schedules an increase of the resources for message consumer client 114 to handle 25% more messages from message producer service 152 to anticipate the period of predicted increased activity, as described above.
As a further example, the information source identifies a particular week as a school-wide vacation week. From historical data, MCO service 122 determines that message producer service 152 generates a message output level of 5% during the first three days of a school-wide vacation week. As such, MCO service 122 predicts dead time for the first three days of the school wide vacation week and schedules a reduction of resources for message consumer client 114 in anticipation of the dead time, as described above.
As yet a further example, message consumer client 114 receives and processes messages that represent credit card transactions from message producer service 152. An information source that supplies information pertinent to the flow of messages from message producer service 152 sends, to MCO service 122, holiday shopping season dates. According to one embodiment, the data from the information source indicates that a higher-than-average amount of transactions are expected during the holiday shopping season dates. According to another embodiment, MCO service 122 determines, based on historical data, that the rate of messages during holiday shopping seasons is usually 75%-95%. Based, at least in part, on the historical data, MCO service 122 predicts a 75%-95% message flow rate during the holiday shopping season indicated in the information from the information source. Accordingly, MCO service 122 schedules resources for message consumer client 114 to handle the predicted message flow rate for the predicted dates.
Automatic Correlation of Third Party Information to Message Flow Patterns
According to an embodiment, MCO service 122 automatically determines that information received from a particular information source informs identification of patterns in changes of the rate of the flow of messages from message producer service 152 through message fabric 140 to message consumer client 114.
To illustrate, MCO service 122 receives first information from a third party information source. For example, MCO service 122 receives calendar information, indicating the dates of government holidays for the current year, from a third party information source. The third party information source is not registered with MCO service 122 as producing information relevant to a flow of messages from message producer service 152, nor does MCO service 122 have access to any other information that indicates an explicit connection between the information source and message producer service 152.
MCO service 122 detects an actual change in the rate of messages from message producer service 152 through message fabric 140 to message consumer client 114. For example, MCO service detects that message production from message producer service 152 drops by 25% on certain days, which drop is currently unexplained by analysis of information that MCO service 122 has for message producer service 152 (e.g., from information sources that are explicitly connected to message producer service 152 within database 130, in historical information for message producer service 152, etc.).
Based, at least in part, on detecting the actual change in the rate of messages from message producer service 152, MCO service 122 identifies a correlation between the first information and the actual change in the rate of messages. For example, MCO service 122 analyzes information in database 130, from information sources that are not explicitly connected to message producer service 152, to identify patterns that predict the detected actual change in the flow of messages, at least in part. According to an embodiment, MCO service 122 initiates such analysis when a threshold amount of change in the rate of message flow is detected and unexplained. The threshold for the amount of change is, according to embodiments, learned by MCO service 122 based, at least in part, on historical data; or is configured by an administrator; etc.
As a result of analyzing information in database 130 from information sources that are not explicitly connected to message producer service 152 within database 130, MCO service 122 determines that the 25% drop in message flow from message producer service 152 correlates with days during a government holiday that are not the last day of the government holiday (as indicated by the first information), and records this pattern in database 130.
MCO service 122 uses the pattern to predict a future change to the rate of the flow of information from message producer service 152. For example, MCO service 122 predicts a 25% drop in the rate of messages from message producer service 152 on the next day that is a government holiday, which is not the last day of the holiday (“the predicted rate drop day”). MCO service schedules a reduction of resources to account for the predicted 25% drop in message flow for message consumer client 114 in anticipation of the predicted rate drop day, such that the reduction of resources is implemented for when the drop in message rate is predicted to occur, as described above.
Managing Multiple Message Consumers
According to an embodiment, MCO service 122 manages the configuration of multiple message consumers, including message consumer client 114. A message consumer may register with MCO service 122 to allow MCO service 122 to optimize the throughput of messages for the associated messaging system. According to an embodiment, an MCO client, such as MCO client 112, automatically registers the associated message consumer with MCO service 122 when MCO client 112 is communicatively connected to MCO service 122 (i.e., via network 160) for the first time.
using a message consumer executing on a particular device to pull, from a message fabric, messages that were produced by a message producer;
gathering capabilities information for at least one of:
the message producer,
the message fabric, or
the message consumer;
based, at least in part, on the capabilities information, automatically executing at least one consumer clone, of the message consumer, on one or more devices, other than the particular device;
wherein, prior to executing the at least one consumer clone, the one or more devices are not being used to pull messages from the message fabric for the message consumer;
creating, via a collector implementation framework, a particular custom collector to collect capabilities information comprising message processing time at the message consumer;
using the particular custom collector to gather particular capabilities information for the message consumer.
gathering second capabilities information for the at least one consumer clone; and
causing a number of message pollers to run at the at least one consumer clone based, at least in part, on the second capabilities information.
4. The method of claim 3, wherein causing the number of message pollers to run at the at least one consumer clone comprises:
configuring a particular consumer clone of the at least one consumer clone with logic for identifying the number of message pollers to run for the particular consumer clone; and
causing the particular consumer clone to use the logic to determine the number of message pollers to run at the particular consumer clone.
the one or more devices are provisioned dynamically through at least one of: a cloud computing service, or a virtual computing environment;
the at least one consumer clone pulls messages from the same one or more sources of messages within the message fabric as the message consumer; and
the at least one consumer clone processes retrieved messages in the same manner as the message consumer.
based on historical information, identifying a pattern in timing of changes in a rate of message flow between a message producer and a message fabric;
based, at least in part, on the pattern, predicting a future change to a flow of messages from the message producer through the message fabric to a message consumer;
in response to predicting the future change to the flow of messages, automatically scheduling a future adjustment of consumer resources allocated to the message consumer;
wherein the future adjustment is scheduled to occur at a scheduled adjustment time; and
in response to reaching the scheduled adjustment time, and based, at least in part, on the predicted change, automatically adjusting consumer resources allocated to the message consumer;
7. The method of claim 6, further comprising gathering at least a portion of the historical information from one or more of the message producer, the message fabric, or the message consumer.
8. The method of claim 6, further comprising receiving at least a portion of the historical information from a third party information source.
the historical information comprises information identifying two or more critical dates of a certain type;
a first critical date of the two or more critical dates occurred in the past;
predicting a future change to a flow of messages from the message producer through the message fabric to the message consumer comprises:
determining that, on the first critical date, the flow of messages had a particular peak rate, and
based, at least in part, on the type of the first critical date being the same as the type of a second critical date, of the two or more critical dates, predicting that the second critical date of the two or more critical dates is the date of the future change to the flow of messages.
10. One or more non-transitory computer readable media storing one or more sequences of instructions which, when executed by one or more processors, cause:
wherein, prior to executing the at least one consumer clone, the one or more devices are not being used to pull messages from the message fabric for the message consumer.
11. The one or more non-transitory computer readable media of claim 10, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause:
12. The one or more non-transitory computer readable media of claim 10, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause:
causing a number of message pollers to run at the at least one consumer clone based, at least in part, on the second capabilities information;
13. The one or more non-transitory computer readable media of claim 12, wherein causing the number of message pollers to run at the at least one consumer clone comprises:
14. The one or more non-transitory computer readable media of claim 10, wherein:
15. One or more non-transitory computer readable media storing one or more sequences of instructions which, when executed by one or more processors, cause:
in response to reaching the scheduled adjustment time, and based, at least in part, on the predicted change, automatically adjusting consumer resources allocated to the message consumer.
16. The one or more non-transitory computer readable media of claim 15, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause gathering at least a portion of the historical information from one or more of the message producer, the message fabric, or the message consumer.
17. The one or more non-transitory computer readable media of claim 15, wherein the one or more sequences of instructions further comprise instructions which, when executed by one or more processors, cause receiving at least a portion of the historical information from a third party information source.
18. The one or more non-transitory computer readable media of claim 17, wherein:
US15/430,127 2013-07-11 2017-02-10 Message consumer orchestration framework Active US10148782B2 (en)
IN3096/CHE/2013 2013-07-11
IN3096CH2013 2013-07-11
US14/013,136 US9614794B2 (en) 2013-07-11 2013-08-29 Message consumer orchestration framework
US15/430,127 US10148782B2 (en) 2013-07-11 2017-02-10 Message consumer orchestration framework
US14/013,136 Continuation US9614794B2 (en) 2013-07-11 2013-08-29 Message consumer orchestration framework
US20170155729A1 true US20170155729A1 (en) 2017-06-01
US10148782B2 US10148782B2 (en) 2018-12-04
ID=52278040
US14/013,136 Active 2035-09-19 US9614794B2 (en) 2013-07-11 2013-08-29 Message consumer orchestration framework
US15/430,127 Active US10148782B2 (en) 2013-07-11 2017-02-10 Message consumer orchestration framework
US (2) US9614794B2 (en)
US20150295871A1 (en) * 2014-04-15 2015-10-15 Blanca Perper Greenstein System and method for processing incoming emails
US20100229127A1 (en) * 2009-03-04 2010-09-09 Centre Technologies Ltd. System and method for navigating posts in a discussion
US20110161973A1 (en) * 2009-12-24 2011-06-30 Delphix Corp. Adaptive resource management
US20140281057A1 (en) * 2013-03-15 2014-09-18 Lsi Corporation Unified message-based communications
JP3278865B2 (en) * 1991-06-28 2002-04-30 日本電気株式会社 Traffic control method
EP1803277A1 (en) * 2004-10-22 2007-07-04 Vidiator Enterprises Inc. System and method for mobile 3d graphical messaging
US8416746B2 (en) * 2008-07-07 2013-04-09 Apple Inc. Medium access control for wireless systems
US8429302B2 (en) * 2008-11-05 2013-04-23 At&T Intellectual Property I, L.P. Aggregate control for application-level compression
CN102469374A (en) * 2010-11-12 2012-05-23 北京信威通信技术股份有限公司 Cross-layer optimization method of video service in wireless communication
US8788654B2 (en) * 2010-12-07 2014-07-22 Cisco Technology, Inc. System and method for allocating resources based on events in a network environment
US8782211B1 (en) * 2010-12-21 2014-07-15 Juniper Networks, Inc. Dynamically scheduling tasks to manage system load
US9231897B1 (en) * 2011-01-10 2016-01-05 Amazon Technologies, Inc. Estimated value ratings for electronic messages
CN103124433A (en) * 2011-03-25 2013-05-29 北京新岸线移动多媒体技术有限公司 Device for wireless communication
WO2012166641A1 (en) * 2011-05-27 2012-12-06 Vpisystems Inc. Methods and systems for network traffic forecast and analysis
US9538528B2 (en) * 2011-10-06 2017-01-03 Nokia Solutions And Networks Oy Efficient co-existence method for dynamic spectrum sharing
US9462542B2 (en) * 2012-11-07 2016-10-04 Telefonaktiebolaget Lm Ericsson (Publ) Method and base station for a cell selection
GB2511860A (en) * 2013-03-15 2014-09-17 Vodafone Ip Licensing Ltd A Method and System of Providing Data Service According to a User's Future Location
US9459980B1 (en) * 2013-04-17 2016-10-04 Amazon Technologies, Inc. Varying cluster sizes in a predictive test load while testing a productive system
2013-08-29 US US14/013,136 patent/US9614794B2/en active Active
2017-02-10 US US15/430,127 patent/US10148782B2/en active Active
US10148782B2 (en) 2018-12-04
US20150019655A1 (en) 2015-01-15
US9614794B2 (en) 2017-04-04
US10003693B2 (en) 2018-06-19 System and method for a work distribution service
CN101448247B (en) 2012-09-05 Updating data on a remote device
US9720941B2 (en) 2017-08-01 Fully automated SQL tuning
CN104737132B (en) 2018-10-30 Demand for the service environment message queue-based resource sharing auction
US20140095486A1 (en) 2014-04-03 Aggregation source routing
US20020120703A1 (en) 2002-08-29 Cooperative location based tasks
US20080043256A1 (en) 2008-02-21 Data presentation methods and apparatus to facilitate printing and reviewing
CN102224715A (en) 2011-10-19 Optimized polling in low resource devices
US8856291B2 (en) 2014-10-07 Providing configurable workflow capabilities
US8949258B2 (en) 2015-02-03 Techniques to manage file conversions
US9380431B1 (en) 2016-06-28 Use of teams in a mobile application
US8521783B2 (en) 2013-08-27 Highly reliable and scalable architecture for data centers
US9015229B1 (en) 2015-04-21 Determining placement of user data to optimize resource utilization for distributed systems
CN103686841B (en) 2018-08-28 Prediction system and method for network congestion control