System and method for end-to-end response-time analysis

In one embodiment, a method includes selecting a group of transactions having a common end-to-end transaction path comprising a plurality of transaction-path nodes. Each transaction of the group is associated with either an acceptable category or an unacceptable category based, at least in part, on an end-to-end response time for the transaction. The method further includes separately determining, for each transaction-path node, an execution-time pair. The execution-time pair includes first aggregate information indicative of execution time by the transaction-path node for transactions associated with the acceptable category. The execution-time pair also includes second aggregate information indicative of execution time by the transaction-path node for transactions associated with the unacceptable category. Additionally, the method includes generating a visualization of the end-to-end transaction path. The visualization depicts each determined execution-time pair in relation to a corresponding transaction-path node.

BACKGROUND

Technical Field

The present disclosure relates generally to data analysis and more particularly, but not by way of limitation, to systems and methods for end-to-end response time analysis.

History of Related Art

Modern web applications process millions of transactions per day and can include multiple redundant layers. When problems occur, it can be difficult to trace the problem to a particular layer. Typical reports and alerts regarding transactions are complex and do not adequately indicate a root cause of poor-performing transactions.

SUMMARY OF THE INVENTION

In one embodiment, a method is performed by a computer system comprising physical computer hardware. The method includes selecting a group of transactions having a common end-to-end transaction path comprising a plurality of transaction-path nodes. Each transaction of the group is associated with either an acceptable category or an unacceptable category based, at least in part, on an end-to-end response time for the transaction. The method further includes separately determining, for each transaction-path node, an execution-time pair. The execution-time pair includes first aggregate information indicative of execution time by the transaction-path node for transactions associated with the acceptable category. The execution-time pair also includes second aggregate information indicative of execution time by the transaction-path node for transactions associated with the unacceptable category. Additionally, the method includes generating a visualization of the end-to-end transaction path. The visualization depicts each determined execution-time pair in relation to a corresponding transaction-path node.

In one embodiment, an information handling system includes a processing unit, wherein the processing unit is operable to implement a method. The method includes selecting a group of transactions having a common end-to-end transaction path comprising a plurality of transaction-path nodes. Each transaction of the group is associated with either an acceptable category or an unacceptable category based, at least in part, on an end-to-end response time for the transaction. The method further includes separately determining, for each transaction-path node, an execution-time pair. The execution-time pair includes first aggregate information indicative of execution time by the transaction-path node for transactions associated with the acceptable category. The execution-time pair also includes second aggregate information indicative of execution time by the transaction-path node for transactions associated with the unacceptable category. Additionally, the method includes generating a visualization of the end-to-end transaction path. The visualization depicts each determined execution-time pair in relation to a corresponding transaction-path node.

In one embodiment, a computer-program product includes a non-transitory computer-usable medium having computer-readable program code embodied therein. The computer-readable program code is adapted to be executed to implement a method. The method includes selecting a group of transactions having a common end-to-end transaction path comprising a plurality of transaction-path nodes. Each transaction of the group is associated with either an acceptable category or an unacceptable category based, at least in part, on an end-to-end response time for the transaction. The method further includes separately determining, for each transaction-path node, an execution-time pair. The execution-time pair includes first aggregate information indicative of execution time by the transaction-path node for transactions associated with the acceptable category. The execution-time pair also includes second aggregate information indicative of execution time by the transaction-path node for transactions associated with the unacceptable category. Additionally, the method includes generating a visualization of the end-to-end transaction path. The visualization depicts each determined execution-time pair in relation to a corresponding transaction-path node.

DETAILED DESCRIPTION OF ILLUSTRATIVE

Embodiments of the Invention

In various embodiments, a performance-monitoring system can track and trace end-user (EU) transactions. The performance-monitoring system can produce and store, for example, an end-to-end (E2E) response time for each EU transaction. An EU transaction, as used herein, is initiated by an EU request such as, for example, a web request, includes subsequent processing of the request by a backend-computing system, and is concluded by a web response from the backend-computing system. EU transactions can cross multiple nodes such as, for example, a web browser, a web server, an application server, a database, one or more external services, etc. An E2E response time can include, for example, a time elapsed from initiation through conclusion of an EU transaction.

One way to troubleshoot slow transaction performance is to aggregate E2E response times for a group of EU transactions and examine execution times at each node in an E2E transaction path. An E2E transaction path typically includes a set of nodes through which a given transaction passes. The E2E transaction path can be, for example, application-specific (e.g., web-application-specific). An E2E response time is typically made up of an execution time at each node in the E2E transaction path. For example, consider an exemplary E2E transaction path that includes a web browser, a network, a web server, an application server, and a database. Each E2E response time can be composed of execution times at each of the web browser, the network, the web server, the application server, and the database. For purposes of troubleshooting, execution times for a group of EU transactions can be aggregated, for example, by determining a mean, median, or mode, performing a statistical analysis, etc. From this information, it can be determined where transactions generally spend the most time. Manual root-cause analysis can subsequently occur.

A downside of the above-mentioned approach is that significant performance aberrations may become diluted in the aggregations. For example, it may be that each poor-performing transaction spends an excessively long period of time at the application server. However, until the poor-performing transactions become abundant in number, the aggregate execution times at the application server may not be indicative of a major problem and thus may not be useful in performing a root-cause analysis. In addition, it may not be immediately apparent what constitutes an excessively long execution time or to what degree a given execution time is excessive.

Various embodiments described herein facilitate a comparative performance analysis between acceptable EU transactions and unacceptable EU transactions. For example, in certain embodiments, each EU transaction can be associated with either an acceptable category or an unacceptable category based, at least in part, on whether an E2E response time exceeds a configurable threshold. The configurable threshold can be an absolute value (e.g., defined in seconds), a relative value (e.g., defined relative to a mean or median value for a transaction type), and/or the like. The configurable threshold can also be varied by type of transaction (e.g., log-in, checkout, etc.).

For purposes of aggregate analysis, an execution-time pair can be separately determined for each node along an E2E transaction path. The execution-time pair can include, for example, a first aggregate execution time for a group of EU transactions associated with the acceptable category and a second aggregate execution time for a group of EU transactions associated with the unacceptable category. Each aggregate execution time can represent a mean, median, mode, etc. Other methods of aggregation can also be used. Advantageously, in certain embodiments, this method enables simplified analysis of each node's performance in unacceptable transactions as compared with that same node's performance in acceptable transactions. In certain embodiments, when each node of the E2E transaction path is presented in this manner, it can more easily be determined which node or nodes are causes of the unacceptable transactions.

Backend performance data, as used herein, refers to data collected during runtime of a software application such as, for example, a web application, through instrumentation of the software application. EU-experience data, as used herein, refers to data collected through observation of one or more transactions from an EU perspective. For example, in various embodiments, the EU perspective may be a node between a web server and an EU information handling system, a node between a web server and an application server, or the EU information handling system.

FIG. 1illustrates a system100for generating and performing analysis of E2E response times. The system100includes at least one EU information handling system102communicating with a backend-computing system110over a network106. The at least one EU information handling system102has a client application104such as, for example, a web-browser application, resident and executing thereon. The network106may include, for example, a public intranet, a private intranet, and/or the Internet. The system100further includes a monitoring system118, an EU archive system126, and an administrative system140. The backend-computing system110, the monitoring system118, and the EU archive system126are operable to communicate over a network132. Like the network106, the network132may be representative, for example, of a public or private intranet or the Internet. In addition, the system100includes a data collector108.

For illustrative purposes, the backend-computing system110is shown to utilize a three-tier architecture that includes a presentation tier116, a logic tier114, and a data tier112. The presentation tier116includes at least one information server138such as, for example, a web server, that serves content to be rendered by the client application104. The logic tier114includes at least one application server136that operates a platform based on, for example, Java EE, ASP.NET, PHP, ColdFusion, Perl, and/or the like. The data tier112includes at least one database134that further includes, for example, data sets and a database management system that manages and provides access to the data sets.

It should be appreciated that, in various embodiments, the backend-computing system110may include any number of tiers. In addition, in various embodiments, the backend-computing system110may implement various alternative architectures such as, for example, a model-view-controller architecture. It should also be appreciated that the at least one application server136and the at least one information server138are shown separately inFIG. 1only for purposes of illustrating logically-related functionality. In various embodiments, the at least one application server136and the at least one information server138are combined into a single server that functions as web server and application server.

The backend-computing system110executes one or more distributed software applications such as, for example, a web application, from which backend-performance data is collected. In a typical embodiment, the one or more distributed software applications have been instrumented to provide the backend-performance data. Each of the one or more distributed software applications may be, for example, a collection of software components or services that make up an application stack. In various embodiments, the backend-computing system110may use an agent resident thereon to collect the backend-performance data.

The backend-performance data can include, for example, metrics related to infrastructure components (virtual or physical) such as, for example, the at least one database134, the at least one application server136, and the at least information server138. The backend-performance data can also include aggregated metrics related to infrastructure tiers such as, for example, the presentation tier116, the logic tier114, and the data tier112. In addition, the backend-performance data can include metrics related to the application stack for each of the one or more distributed software applications. In a typical embodiment, the backend-performance data can trace EU transactions through a topology of nodes that can include, for example, infrastructure components, infrastructure tiers, and/or application-stack components as described above. Metrics can include, for example, execution time at each tier or by each component or node. Examples of how backend-performance data can collected and managed is described in detail in U.S. Pat. Nos. 7,979,245 and 8,175,863, each of which is hereby incorporated by reference.

The data collector108is a software component that collects the EU-experience data for the at least one EU information handling system102. In a typical embodiment, the data collector108is situated in the system100such that the data collector108is capable of seeing all network traffic (i.e., all packets exchanged) between the at least one EU information handling system102and the backend-computing system110. In this fashion, the data collector108functions as a packet analyzer and is operable to extract the EU-experience data and transmit the EU-experience data to the EU archive system126. The EU archive system126includes at least one server computer128and at least one database130. The EU archive system126receives the EU-experience data from the data collector108and stores the EU-experience data in the at least one database130. An example of how EU-experience data can be collected is described in U.S. Pat. No. 7,941,385. U.S. Pat. No. 7,941,385 is hereby incorporated by reference.

As illustrated, the data collector108can reside at various nodes in the system100. For example, the data collector108can reside on the backend-computing system110between the presentation tier116and the logic tier114. The data collector108can also be resident on the backend-computing system110between the presentation tier116and the network106. In addition, in various embodiments, the data collector108is representative of client-side scripting that is executed on the at least one EU information handling system102. In this fashion, the data collector108can also be resident on the at least one EU information handling system102. It should be appreciated that other locations for the data collector108such as, for example, within the presentation tier116, are also contemplated.

The monitoring system118includes at least one server computer120and at least one database122. The at least one server computer120is operable to execute a correlator124. The correlator124is typically a software component that correlates the EU-experience data maintained by the EU archive system126with the backend-performance data maintained by the monitoring system118to yield E2E response times for EU transactions. Exemplary operation of the system100will be described with respect toFIG. 2.

The administrative system140includes a reporting module142. The administrative system140can include any number of server computers and/or databases. The reporting module142can include hardware and/or software for generating and/or presenting alerts, reports, and/or the like based on data stored or generated by the monitoring system118and the EU archive system126. The reports and/or alerts can be served to an administrative user using, for example, an information handling system similar to the EU information handling system102. For example, in certain embodiments, the reporting module142can facilitate a comparative performance analysis between acceptable EU transactions and unacceptable EU transactions. An example of functionality of the reporting module142will be described with respect toFIG. 4.

One of ordinary skill in the art will appreciate that each instance of a computer or computer system as described above may be representative of any number of physical or virtual server computers. Likewise, each instance of a database may be representative of a plurality of databases. In addition, it should be appreciated that, in various embodiments, each instance of a network such as, for example, the network106or the network132, can be viewed as an abstraction of multiple distinct networks. For example, the network106and the network132can each include one or multiple communications networks such as, for example, public or private intranets, a public switch telephone network (PSTN), a cellular network, the Internet, or the like. In addition, in various embodiments, the network106and the network132may overlap or refer to a same network.

FIG. 2illustrates an exemplary data flow200using the system100ofFIG. 1. The EU information handling system102initiates a transaction by directing a request such as, for example, an HTTP request, to the at least one information server138of the presentation tier116. The at least information server138forwards the request to an appropriate application server, i.e., the at least one application server136, for handling. The at least one application server136generates an identifier (e.g., a UUID) for the transaction. In a typical embodiment, the backend-computing system110uses the identifier to identify backend-performance data collected during processing of the transaction, which data is stored by the monitoring system118as described above.

A monitoring agent on the at least one application server136injects the identifier in a response to the request (i.e., a UUID-injected response), which response is directed to the at least one EU information handling system102along a transmission path that includes that at least one information server138and the at least one EU information handling system102. In this fashion, no modification of application code is required to inject the identifier. Rather, the monitoring agent, which is already being utilized for existing instrumentation of the distributed software application, injects the identifier into the response. The response may be a web response such as, for example, an HTTP response. In various embodiments, the identifier can be injected, for example, into a response header for the response. In some embodiments, the identifier may be inserted into a cookie that is sent as part of the response header. Content of the UUID-injected response is rendered on the at least one EU information handling system102via the client application104.

As noted above, the data collector108is situated on the system100so that the data collector108can observe all network traffic exchanged between the backend-computing system110and the EU information handling system102. Therefore, the data collector108is effectively a transparent node along the transmission path. The data collector108passively observes the UUID-injected response and uses the identifier to identify EU-experience data that is collected.

The correlator124is operable to extract EU-experience data not previously obtained by the correlator (i.e., new EU-experience data) from the EU archive system126. In various embodiments, the correlator124may operate on a periodic basis, on-demand, or in real-time. The correlator124is operable to correlate the EU-experience data and the backend-performance data that relates to a same transaction (i.e., a same request and response) by cross-referencing identifiers. In this manner, data resulting from instrumentation (the backend-performance data) and the EU-experience data, which is typically collected without instrumentation, can be correlated. The correlated data can be stored in the at least one database122. The correlated data can also be used to generate E2E response times for end-use transactions. In addition, on a periodic basis (e.g., every five minutes) or on demand, the correlator124may aggregate the correlated data into one or more high-level transaction categories such as, for example, log-in, search, or checkout. Therefore, problems with particular transaction categories can be readily identified and appropriate alerts generated.

FIG. 3illustrates an example of a process300for determining an acceptability categorization of an EU transaction. In various embodiments, the process300can be performed for each EU transaction handled by a backend-computing system such as the backend-computing system110. For example, the process300, in whole or in part, can be implemented by one or more of the monitoring system118, the correlator124, the EU archive system126, the administrative system140, the reporting module142, and/or the EU information handling system102. The process300can also be performed generally by the system100. Although any number of systems, in whole or in part, can implement the process300, to simplify discussion, the process300will be described in relation to specific systems or subsystems of the system100.

At block302, the system100processes an EU transaction. For example, the block302can include tracing the EU transaction as described with respect toFIGS. 1 and 2. At block304, the monitoring system118, or another component, generates an E2E response time for the EU transaction. For example, the E2E response time can be based on correlated data as described with respect toFIGS. 1-2. The E2E response time can also be based on backend-performance data and EU-experience data that is correlated in other ways such as, for example, session identifiers, session timing, and/or other information.

At block306, the monitoring system118, the reporting module142, or another component, associates the EU transaction with either an acceptable category or an unacceptable category. For example, in certain embodiments, the EU transaction can be associated with the unacceptable category if the E2E response time exceeds a configurable threshold value. Otherwise, the EU transaction can be associated with the acceptable category. The configurable threshold can be an absolute value (e.g., defined in seconds), a relative value (e.g., defined relative to a mean or median value for a transaction type), and/or the like. The configurable threshold can also be varied by type of transaction (e.g., log-in, checkout, etc.). The association can also be stored, for example, in the at least one database122, a database on the administrative system140, in memory of the monitoring system118and/or the administrative system140, etc.

FIG. 4illustrates an example of a process400for comparing E2E response-time breakdowns between acceptable and unacceptable transactions. For example, the process400, in whole or in part, can be implemented by one or more of the monitoring system118, the correlator124, the EU archive system126, the administrative system140, the reporting module142, and/or the EU information handling system102. The process400can also be performed generally by the system100. Although any number of systems, in whole or in part, can implement the process400, to simplify discussion, the process400will be described in relation to specific systems or subsystems of the system100.

At block402, the reporting module142selects a group of EU transactions that have a common E2E transaction path. For example, the group of transactions can relate to a same application or web application. The E2E transaction path includes a plurality of transaction-path nodes such as, for example, a web browser, a network, a web server, an application server, a database, an external service, and/or the like. Each transaction of the group may be of a same type or a different type. For example, in some embodiments, the selected group may be all checkout transactions over a certain period of time. By way of further example, the selected group can include a heterogeneous set of transaction types such as, for example, log-in, search, and checkout. In certain embodiments, the group of EU transactions can be transactions for which a process similar to the process300ofFIG. 3has been executed. In some embodiments, the selection can be automated. In other embodiments, the selection can be responsive to user input, for example, from an administrative user of an information handling system such as the EU information handling system102.

At block404, the reporting module142separately determines an execution-time pair for each transaction-path node of the E2E transaction path. In general, the execution-time pair includes two aggregate representations. A first aggregate representation can be indicative of execution time by the transaction-path node of those transactions of the group that are deemed unacceptable transactions. A second aggregate representation can be indicative of execution time by the transaction-path node of those transactions of the group that are deemed acceptable transactions. An example of functionality that can be performed at the block404will be described in greater detail with respect toFIG. 5.

At block406, the reporting module142generates a report of the E2E transaction path. The report can be considered a breakdown, across the plurality of transaction-path nodes, of an aggregate E2E response time for the group. In general, the report indicates, or summarizes, each determined execution-time pair in relation to a corresponding transaction-path node. In certain embodiments, the report can be a visualization of the E2E transaction path. In these embodiments, the visualization may depict each determined execution-time pair in relation to the corresponding transaction-path node. In certain other embodiments, the report can be a collection of data representative of each determined execution-time pair and its corresponding transaction-path node.

At block408, the reporting module142publishes the report. For example, the report can be transmitted to a system, entity, or user. In embodiments in which the report is a visualization, the block408can include causing the visualization to be displayed, for example, to an administrative of user of an EU information handling system such as the EU information handling system102ofFIG. 1. In some embodiments, publishing the report can include transmitting the report to another system for analysis. For example, in some cases, the report can be used to make automatic scaling decisions in a shared-resource environment such as, for example, a cloud environment.

FIG. 5illustrates an example of a process500for separately determining an execution-time pair for each transaction-path node of a transaction path. In various embodiments, the process500can be performed as all or part of the block404ofFIG. 4. For example, the process500, in whole or in part, can be implemented by one or more of the monitoring system118, the correlator124, the EU archive system126, the administrative system140, the reporting module142, and/or the EU information handling system102. The process500can also be performed generally by the system100. Although any number of systems, in whole or in part, can implement the process500, to simplify discussion, the process500will be described in relation to specific systems or subsystems of the system100.

At block502, the reporting module142ascertains a category of each EU transaction of a group of EU transactions. The group can be, for example, a group that is selected as described with respect to the block402ofFIG. 4. In certain embodiments, the category can be one of two categories: an acceptable category and an unacceptable category. Each category can be ascertained, for example, by accessing the at least one database122or other memory.

At block504, the reporting module142accesses a response-time breakdown for each EU transaction of the group. Each response-time breakdown typically includes an execution time for each transaction-path node of the transaction path. At block506, for each transaction-path node, the reporting module142aggregates corresponding execution times for those EU transactions of the group that are associated with the acceptable category. The aggregation can include computing a mean, median, or mode, performing a statistical analysis, or aggregating in another suitable fashion. At block508, for each transaction-path node, the reporting module142aggregates corresponding execution times for those EU transactions of the group that are associated with the unacceptable category. The aggregation at the block506can be performed in similar fashion to the aggregation at the block504.

FIG. 6illustrates an example of a visualization600. In various embodiments, the visualization600can be generated as described with respect to the block406ofFIG. 4and/or caused to be displayed as described with respect to the block408ofFIG. 4. As illustrated, the visualization600shows a response-time breakdown across a transaction path that includes a browser144, a network646, one or more web servers648, one or more application servers650, one or more external services652, and one or more databases654.

More particularly, the visualization600illustrates unacceptable-category aggregate representations656(1)-656(6) (collectively, unacceptable-category aggregation representations656) and acceptable-category aggregate representations658(1)-658(6) (collectively, acceptable-category aggregation representations658) as described with respect to the block404ofFIG. 4. The unacceptable-category aggregate representation656(1) and the acceptable-category aggregate representation658(1) may be considered an execution-time-pair as described above for the browser644. As illustrated, each of the network646, the one or more web servers648, the one or more application servers650, the one or more external services652, and the one or more databases654similarly have execution-time pairs.

In a typical embodiment, as depicted inFIG. 6, a visualization of each execution-time pair can be scaled to a highest value contained therein. For example, with respect to the browser144, a value for the unacceptable-category aggregation representation656(1) is higher than a value for the acceptable-category aggregate representation658(1). Therefore, a length of a horizontal bar corresponding to the acceptable-category aggregate representation658(1) is scaled in proportion to a length of a horizontal bar corresponding to the unacceptable-category aggregate representation656(1).

In various embodiments, the visualization600can facilitate improved root-cause analysis. In the illustrated embodiment, the one or more databases654is indicated to have a significant difference in response time between the unacceptable-category aggregation representation656(6) and the acceptable-category aggregate representation658(6). In contrast, the browser144, the network646, the one or more web servers648, the one or more application servers650, and the one or more external services652exhibit far less variation in performance between unacceptable transactions and acceptable transactions. For purposes of this example, the one or more databases654may be considered a potential root cause of the unacceptable transactions.

It should be appreciated that end-user transactions are described herein only for illustrative purposes. In certain embodiments, various principles described above relative toFIGS. 1-6can similarly be applied to numerous other types of timed, multi-node transactions. Another example of a timed, multi-node transaction can be database query execution (e.g., execution of SQL queries, NoSQL queries, etc.).

In the case of a database query, certain sources of delay in database-query execution can be considered nodes of a transaction path. Execution time of a database query, or of a group of database queries could be affected by CPU time, input/output operations, lock delay, latch delay, and/or the like. In an example, a given transaction path could include a node for each of CPU time, input/output operations, lock delay, latch delay, and/or the like. According to this example, overall execution time can correspond to an end-to-end response time as described above. In certain embodiments, a report as described with respect toFIG. 4can be generated for a group of database-query executions. The report can be generated as described with respect to the block406ofFIG. 4and/or published as described with respect to the block408ofFIG. 4. The group of database-query executions can relate, for example, to a particular database query, a selection of a plurality of database queries, a selection of database queries involving one or more databases, etc. The report can be, for example, a visualization similar to the visualization600ofFIG. 6.