Patent Description:
With modern software applications and systems evolving to become more sophisticated (i.e., more complex, dynamic, robust and distributed in nature), enterprises are grappling with the technical challenge of ensuring reliability and availability. Customers not only want business applications to constantly evolve to rapid changing requirements but also expect high availability and high-performance applications. If customer experiences are not sufficiently fast and reliable, the result can be a lost opportunity for a business, such as a loss in revenue or a negative brand experience.

Despite significant investments in agile engineering and automation practices, production systems continue to see critical failures. The majority of these production failures are due to causal sequences under specific usage conditions that are not easy to envisage during design and testing. Additionally, many critical production failures are not reproducible in an engineering environment.

<CIT> describes techniques for troubleshooting within computer systems using a fault tree analysis.

<CIT> describes an external request for a service provided by a runtime engine of an enterprise application that references tags stored in a tag library called by a script.

<CIT> describes managed resources having an agent for collecting performance information of the managed resource and transmitting the performance information.

Over <NUM>% of critical production failures are due to extra-functional root causes like scalability, reliability, stability, performance, fault recovery, exception handling etc. These non-functional attributes are related to architecture quality (i.e., technical design of the architecture components, interfaces and parameters). Accordingly, many critical production failures are due to architecture quality, which can degrade due to many reasons.

For example, developers and architects are not equipped to predict the runtime consequences of design and architecture choices for a particular environment. Systems can undergo critical failures under specific conditions that cannot be foreseen reliably. These design and architectural choices can cause architectural degradation in the context of specific usage conditions. Because testers do not have view of actual system usage patterns in the real world, how systems respond under those usage conditions, and the associated failure modes, test inputs do not adequately address these interactions between different systems and applications under real-world usage conditions. In addition, certain interactions between applications and/or systems initiated by a user in the environment may cause a fault or failure that may not be reproducible without knowledge of the sequence of events or workflow by the user that caused the fault or failure.

The techniques of the example embodiments described herein provide a method and system for fault analysis and prediction that uses empirical architecture analytics of an enterprise environment. These techniques provide a solution that identifies parameters that are indicative of architecture quality, employs empirical analytics of these parameters under various conditions of system usage, derives meaningful inferences about system usage and behavior under those usage conditions, and provides fault diagnosis, anomaly detection, and identification of silent failures.

The information and solutions provided by the example embodiments may be used to make architecture and design choices, remediate architecture as required, diagnose and predict failures, and design test inputs and conditions to reflect usage, anomalies and failures within the specifics of a given enterprise environment.

In one aspect, a method for fault analysis and prediction in an enterprise environment is provided, the method comprising: obtaining data from a plurality of sources in the enterprise environment, wherein the plurality of sources includes at least one or more systems, users, or applications; associating the obtained data with identifiers that include a theme selected from a set of themes and one or more keywords, wherein the keywords are specific to each theme; generating a workflow for a user based on a session identifier and/or timestamps associated with activity by the user, wherein the workflow identifies a time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment; and determining at least one fault identification or fault prediction associated with the enterprise environment based on the workflow and identifiers associated with the obtained data that corresponds to the workflow.

In another aspect, a system for fault analysis and prediction in an enterprise environment is provided, the system comprising: at least one interface configured to communicate with systems and applications in the enterprise environment; a memory; and a processor in communication with the at least one interface and the memory, wherein the processor is configured to: obtain data from a plurality of sources in the enterprise environment, wherein the plurality of sources includes at least one or more systems, users, or applications; associate the obtained data with identifiers that include a theme selected from a set of themes and one or more keywords, wherein the keywords are specific to each theme; generate a workflow for a user based on a session identifier and/or timestamps associated with activity by the user, wherein the workflow identifies a time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment; and determine at least one fault identification or fault prediction associated with the enterprise environment based on the workflow and identifiers associated with the obtained data that corresponds to the workflow.

In another aspect, one or more non-transitory computer readable storage media encoded with instructions are provided that, when executed by a processor of a fault analysis and prediction system, causes the processor to: obtain data from a plurality of sources in an enterprise environment, wherein the plurality of sources includes at least one or more systems, users, or applications; associate the obtained data with identifiers that include a theme selected from a set of themes and one or more keywords, wherein the keywords are specific to each theme; generate a workflow for a user based on a session identifier and/or timestamps associated with activity by the user, wherein the workflow identifies a time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment; and determine at least one fault identification or fault prediction associated with the enterprise environment based on the workflow and identifiers associated with the obtained data that corresponds to the workflow.

Other systems, methods, features, and advantages of the disclosure will be, or will become, apparent to one of ordinary skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description and this summary, be within the scope of the disclosure, and be protected by the following claims.

While various embodiments are described, the description is intended to be exemplary, rather than limiting, and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible that are within the scope of the embodiments. Although many possible combinations of features are shown in the accompanying figures and discussed in this detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or substituted for any other feature or element in any other embodiment unless specifically restricted.

This disclosure includes and contemplates combinations with features and elements known to the average artisan in the art. The embodiments, features, and elements that have been disclosed may also be combined with any conventional features or elements to form a distinct invention as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventions to form another distinct invention as defined by the claims.

The present embodiments provide a system and method for fault analysis and prediction that uses empirical architecture analytics of an enterprise environment. Referring now to <FIG>, an enterprise environment <NUM> is shown interacting with an example embodiment of a fault analysis and prediction system <NUM>. In some embodiments, fault analysis and prediction system <NUM> is configured to identify parameters associated with enterprise environment <NUM> that are indicative of architecture quality, perform empirical analysis of those parameters under various usage conditions, derive inferences about the usage and behavior of enterprise environment <NUM> under the various usage conditions, and provide one or more of fault diagnosis, fault prediction, anomaly detection, and/or identification of silent failures.

In addition, in some embodiments, fault analysis and prediction system <NUM> may further use the information obtained about enterprise environment <NUM> to make or recommend architecture and design choices, remediate existing system architecture, diagnose and/or prognose failures, and/or design test inputs and conditions that reflect the usage of enterprise environment <NUM>, including any anomalies or failures.

As shown in <FIG>, enterprise environment <NUM> may include a plurality of users <NUM>, a plurality of systems <NUM>, and a plurality of applications <NUM>. For example, enterprise environment <NUM> may be a business, an organization, a governmental or regulatory body, a utility, a service provider, or any other entity or entities. Plurality of users <NUM> include various people and personnel that have access to and/or interact with enterprise environment <NUM>. In some embodiments, each user of plurality of users <NUM> may have an associated user account or other identifier (e.g. user ID) that allows the user to be uniquely determined or distinguished from other users of enterprise environment <NUM>.

Plurality of systems <NUM> include various hardware components and architecture associated with enterprise environment <NUM>. For example, plurality of systems <NUM> may include one or more computer networks, servers, databases, devices, administration or business infrastructure, or other hardware or subsystems within enterprise environment <NUM> with which a user of plurality of users <NUM> may interact. Plurality of applications <NUM> include one or more applications, programs, scripts, routines, or other software-implemented mechanisms or modules that are configured to be executed on a system of plurality of systems <NUM>. In an example embodiment, one or more users of plurality of users <NUM> may interact with or implement an application of plurality of applications <NUM> within enterprise environment <NUM>. In some embodiments, plurality of applications <NUM> may also include one or more applications that interact with an administrator, other applications, and/or operate automatically within enterprise environment <NUM> without directly interacting with a user of plurality of users <NUM>.

In an example embodiment, data <NUM> associated with one or more users of plurality of users <NUM>, one or more systems of plurality of systems <NUM>, and/or one or more applications of plurality of applications <NUM> may be collected and provided to fault analysis and prediction system <NUM> from enterprise environment <NUM>. For example, as will be detailed further below, data collection agents may be used to collect, gather, and/or extract data <NUM> from the various components of enterprise environment <NUM>, including, but not limited to, plurality of users <NUM>, plurality of systems <NUM>, and/or plurality of applications <NUM>. Additionally, data <NUM> provided to fault analysis and prediction system <NUM> may include other information associated with enterprise environment <NUM> that may be used by fault analysis and prediction system <NUM> for fault analysis and prediction.

In an example embodiment, fault analysis and prediction system <NUM> includes one or more processors and/or circuitry configured to implement at least a data management and analytics engine <NUM> and a fault analysis and prediction module <NUM>. For example, data management and analytics engine <NUM> may be configured to receive data <NUM> associated with enterprise environment <NUM> and perform one or more operations to classify, sort, group, clean, and/or organize the data for further processing or analysis by fault analysis and prediction system <NUM>. Fault analysis and prediction module <NUM> may be configured to use the classified, sorted, grouped, cleaned, and/or organized data from data management and analytics engine <NUM> for fault analysis and/or prediction about potential faults, failures, anomalies, or other issues determined from data <NUM> associated with enterprise environment <NUM>. For example, fault analysis and prediction module <NUM> may provide one or more faults <NUM> to enterprise environment <NUM>. In some embodiments, faults <NUM> may be used for implementing a corrective action within enterprise environment.

Additionally, in some embodiments, fault analysis and prediction system <NUM> may include one or more databases <NUM> used to store various information associated with enterprise environment <NUM>. For example, databases <NUM> may include databases of identifiers, known faults, workflows, root causes and fixes, or other information that may be used by fault analysis and prediction system <NUM> for fault analysis and prediction. In some cases, databases <NUM> may include historical databases associated with other enterprise environments.

Referring now to <FIG>, a diagram of architecture layers in an enterprise environment is illustrated. In different embodiments, the architecture of a given enterprise environment may vary depending on the particular arrangement or type of enterprise. <FIG> illustrates an example embodiment of architecture layers in a representative enterprise environment, for example, enterprise environment <NUM>. In this embodiment, enterprise environment <NUM> includes at least a network layer <NUM>, a hardware layer <NUM>, an application and database (DB) layer <NUM>, and a business flow or user interface (Ul) layer <NUM>.

In some embodiments, parameters about enterprise environment <NUM> across one or more of network layer <NUM>, hardware layer <NUM>, application and database (DB) layer <NUM>, and business flow or user interface (Ul) layer <NUM> are monitored and analyzed by fault analysis and prediction system <NUM>. For example, information associated with various network parameters <NUM> may be monitored and/or analyzed for network layer <NUM>. Similarly, information associated with hardware parameters <NUM> may be monitored and/or analyzed for hardware layer <NUM>. For example, hardware parameters <NUM> may include information about different hardware configurations and devices in enterprise environment <NUM>, including, but not limited to central processing unit (CPU) information (e.g., processing power, speed, utilization, etc.), memory information (e.g., types, amounts, utilization, etc.), and/or input/output (I/O) information (e.g., input and/or output devices, data transfer rates, latency, etc.).

Additionally, information associated with application/DB parameters <NUM> may be monitored and/or analyzed for application and DB layer <NUM>. For example, application parameters <NUM> may include information about different applications or DBs in enterprise environment <NUM>, including, but not limited to services and/or requests, methods, transactions, Structured Query Language (SQL), threading, DB errors, cache, payload, application errors, etc. Information associated with various business/UI parameters <NUM>, for example, business transactions, workflow, Ul interactions, etc., may also be monitored and/or analyzed for business flow/UI layer <NUM>.

According to the principles of the example embodiments described herein, the parameters obtained from all of the architecture layers of enterprise environment (i.e., parameters <NUM>, <NUM>, <NUM>, <NUM> from each of layers <NUM>, <NUM>, <NUM>, <NUM>) may be used by fault analysis and prediction system <NUM>. For example, fault analysis and prediction system <NUM> may evaluate the parameters using a time series analysis <NUM> (i.e., evaluation of the parameters over time) and/or may correlate the parameters across tiers <NUM> (i.e., identify and evaluate relationships between the parameters across the various architecture layers).

In an example embodiment, parameters <NUM>, <NUM>, <NUM>, <NUM> from each of layers <NUM>, <NUM>, <NUM>, <NUM> may be provided to fault analysis and prediction system <NUM> as part of data <NUM>, as shown in <FIG>. In some embodiments, data associated with an enterprise environment (e.g., data <NUM> associated with enterprise environment <NUM>) may be collected from various sources within the enterprise environment, as well as sources outside of the enterprise environment that interact with users, systems, or applications within the enterprise environment. In an example embodiment, data from a plurality of sources in the enterprise environment may be obtained using a plurality of agents to collect the data.

For example, data may be typically collected from one or more sources, including, but not limited to: server hardware and system health logs (e.g., desktop, mobile, tablets, etc.), user interface logs, web server logs, application service logs, application monitoring trace files, database logs, and any other system capable of generating text-based logs.

In some embodiments, each agent of the plurality of agents may collect data from a different source within the enterprise environment. Referring now to <FIG>, a schematic diagram of data collection agents within enterprise environment <NUM> is illustrated. In this embodiment, data acquisition from a plurality of sources within enterprise environment is obtained by a plurality of agents, including a first agent <NUM>, a second agent <NUM>, a third agent <NUM>, and a fourth agent <NUM>.

As shown in <FIG>, first agent <NUM> is associated with a user device or machine, for example, UI machine <NUM>, and is configured to collect client-side logs associated with user-specific data. In this embodiment, first agent <NUM> obtains data from enterprise environment <NUM> associated with various user logs <NUM> of users of plurality of users <NUM> in enterprise environment <NUM> (shown in <FIG>). First agent <NUM> may collect or obtain data from different formats of user devices, including, but not limited to desktop computers, mobile computers or phones, tablets, etc. Additionally, first agent <NUM> may be deployed across multiple devices associated with the same user and/or multiple devices associated with different users with enterprise environment <NUM>.

In this embodiment, second agent <NUM> is associated with application performance monitoring (APM) tools <NUM> (e.g., AppDynamics, New Relic, and other APM tools) and is configured to obtain dynamic data associated with trace files. For example, second agent <NUM> may collect or obtain data associated with application monitoring utilization <NUM> generated by APM tools <NUM> deployed within enterprise environment <NUM> to monitor applications associated with plurality of applications <NUM> (shown in <FIG>). Third agent <NUM> obtains data from log mining <NUM> various types of application and web server logs <NUM>, including packaged application logs (e.g., SAP, Siebel, etc.), that may interact with enterprise environment <NUM>.

Finally, fourth agent <NUM> is associated with one or more databases (DB) <NUM> and is configured to extract log entries from logs generated by databases within enterprise environment <NUM>. For example, fourth agent <NUM> may collect or obtain data associated with workload repositories <NUM> within enterprise environment <NUM>. Fourth agent <NUM> may be configured to be compatible with various types of databases that may be present in an enterprise environment, including databases using relational database management systems (RDBMS), not only SQL (NoSQL) systems, and other database management systems.

In an example embodiment, the plurality of agents <NUM>, <NUM>, <NUM>, <NUM> are configured to extract log entries and obtain the data at various time intervals based, for example, on the volume of logs being generated. The data associated with enterprise environment <NUM> obtained from each of first agent <NUM>, second agent <NUM>, third agent <NUM>, and fourth agent <NUM> may be collected and ingested into fault analysis and prediction system <NUM>. For example, as shown in <FIG> above, data <NUM> may be obtained from plurality of agents <NUM>, <NUM>, <NUM>, <NUM> and ingested into data management and analytics engine <NUM> of fault analysis and prediction system <NUM>.

In an example embodiment, the data associated with enterprise environment <NUM> that is collected by the plurality of agents <NUM>, <NUM>, <NUM>, <NUM> includes real-time data. In other embodiments, additional data associated with enterprise environment <NUM> may be obtained from other sources, including, but not limited to static data <NUM>, such as user and location details <NUM> and/or service level agreement (SLA) data <NUM>, and archival data <NUM> associated with previous time periods, such as previous hour data <NUM>.

While four agents have been shown in the example embodiment illustrated in reference to <FIG>, it should be understood that an enterprise environment may have any number of different data collection agents. For example, the number and/or type of data collection agents may be specific to the architecture of a given enterprise environment.

<FIG> is a schematic diagram of an example embodiment of fault analysis and prediction system <NUM>. In this embodiment, components of fault analysis and prediction system <NUM>, including data management and analytics engine <NUM> and fault analysis and prediction module <NUM>, are shown in detail. The components of fault analysis and prediction system <NUM> described herein may be embodied in hardware, software, or a combination of both.

As shown in <FIG>, in one embodiment, data <NUM> from enterprise environment <NUM> is received at a cleanser module <NUM>. Data ingested by the various data collection agents (e.g., plurality of agents <NUM>, <NUM>, <NUM>, <NUM>) may be in different formats. Additionally, some logs may have special characters that may not be subject to analysis or have meaning outside of their function within the logs. In an example embodiment, cleanser module <NUM> is configured to clean or refine data <NUM> from enterprise environment <NUM> for use by fault analysis and prediction system <NUM>.

For example, in some embodiments, cleanser module <NUM> may perform one or more of the following actions, including, but not limited to: formatting the data to a compatible format, removing special characters in the data that may cause complications with analysis, removing duplicate files and/or log entries that are generated due to synchronization limitations or other duplicate processes, as well as other operations to organize or arrange the data from the enterprise environment (e.g., enterprise environment <NUM>).

In an example embodiment, the cleansed data (e.g., data <NUM>) after passing through cleanser module <NUM> may be passed to a data qualifier module <NUM>. In this embodiment, data qualifier module <NUM> takes the cleansed data as an input and performs one or more actions to provide contextual or other information to the data. For example, in some embodiments, data qualifier module <NUM> may perform one or more of the following actions, including, but not limited to: segregating log files into types (e.g., device, web server, app server, database, trace, and/or application monitoring), determining the specific software that generated a log file (e.g., classify a web server log as being generated by Apache Tomcat or Microsoft IIS), sequencing the logs according to timestamps associated with the data, merging files and/or eliminating redundant entries that are generated as part of merging various files, as well as other operations to associate information about the source and/or generation of the data with the data from the enterprise environment(e.g., enterprise environment <NUM>).

Fault analysis and prediction system <NUM> may also include a tag extractor module <NUM>. In an example embodiment, tag extractor module <NUM> may associate one or more tags or identifiers with the obtained data from enterprise environment (e.g., data <NUM> ingested from enterprise environment <NUM>). In some embodiments, tag extractor module <NUM> may access or use a tag corpus <NUM> to extract and/or associate identifiers with the obtained data.

Tag corpus <NUM> may include a predetermined database or list of identifiers or tags, including, but not limited to sets of industry, technology, organization, and/or application specific identifiers or tags. Additionally, if the data obtained from the enterprise environment includes identified identifiers or tags that are not already present in tag corpus <NUM>, then tag extractor module <NUM> may recommend the new identifiers or tags for inclusion in tag corpus <NUM>.

In an example embodiment, tag extractor module <NUM> may extract statistics regarding all the identifiers or tags and the correlations among the identifiers or tags. Tag extractor module <NUM> is configured to process the text from each file in the obtained data to extract identifiers or tags at two levels: themes and keywords. For example, identifiers used by tag extractor module <NUM> include a theme selected from a set of themes and one or more keywords that are specific to each theme. These themes and keywords may be associated with the data as identifiers.

Referring now to <FIG>, a process of associating themes and keywords with collected data (e.g., data <NUM> from enterprise environment <NUM>) by tag extractor module <NUM> is shown. In this embodiment, tag extractor module <NUM> receives blocks of text <NUM>, which may be generated from data <NUM> obtained from enterprise environment <NUM> and which may have passed through cleanser module <NUM> and/or data qualifier module <NUM>. Blocks of text <NUM> may include test cases, defects, requirements, log entries, etc. In an example embodiment, tag extractor module <NUM> assigns a theme to each line or block of text in block of text <NUM>. As shown in <FIG>, a theme may be selected from a set of themes <NUM> relating to different actions, processes, scenarios, or other functions or capabilities within enterprise environment <NUM>. In this embodiment, set of themes <NUM> includes a first theme <NUM>, a second theme <NUM>, and a third theme <NUM>. For example, a theme may be a business process (e.g., new employee onboarding), a subject area (e.g., employee), and/or a scenario (e.g., creating a new account).

Keywords associated with each theme of set of themes <NUM> are identified and extracted. In an example embodiment, keywords are specific to each theme. For example, as shown in <FIG>, a first group of keywords <NUM> are specific to first theme <NUM>, a second group of keywords <NUM> are specific to second theme <NUM>, and a third group of keywords <NUM> are specific to third theme <NUM>. Theme to keyword association may be identified based on previous learnings. In addition, even if an exact keyword associated with a theme does not appear in a block of text from blocks of text <NUM>, associated keywords may be used for identifying a theme.

For example, if the following block of text is analyzed: "A Religious Worker's data should be processed like a normal Employee", although the keywords "payroll" or "salary" do not appear in this sample block of text, tag extractor module <NUM> associates "Religious Worker" with "salary" and maps this block of text to the "payroll" theme.

In an example embodiment, tag extractor module <NUM> performs the following operations. First, tag extractor module <NUM> extracts themes (e.g., themes <NUM>, <NUM>, <NUM> from set of themes <NUM>) from the ingested data (e.g., data <NUM>) using a theme and keyword repository <NUM> as a lookup database. Next, tag extractor module <NUM> extracts keywords from the ingested data using theme and keyword repository <NUM> to find one or more keywords that are specific to each theme (e.g., themes <NUM>, <NUM>, <NUM> from set of themes <NUM>). In this embodiment, themes and keywords are not extracted based on text similarity but based on context similarity. Theme and keyword repository <NUM> may be specific to the industry, enterprise, applications, and/or systems associated with the enterprise environment (e.g., applications <NUM> and/or systems <NUM> associated with enterprise environment <NUM>).

Additionally, any new identifiers (e.g., themes and/or associated keywords for a theme) which are not already present in tag corpus <NUM> and/or theme and keyword repository <NUM> may be identified by tag extractor module <NUM>. An administrator <NUM> of fault analysis and prediction system <NUM> or a subject matter expert (SME) can evaluate these new identifiers (e.g., themes and/or associated keywords) for addition to tag corpus <NUM> and/or theme and keyword repository <NUM>. For example, as shown in <FIG>, third group of keywords <NUM> that are specific to third theme <NUM> may be added to theme and keyword repository <NUM>.

Referring back to <FIG>, fault analysis and prediction system <NUM> includes a tag classifier module <NUM>. In an example embodiment, tag classifier module <NUM> segregates or classifies the extracted identifiers or tags from tag extractor module <NUM> into various predefined groups, which may be stored in a tag model repository <NUM>. Examples of predefined groups in tag model repository <NUM> include, but are not limited to: Business Processes, Objects, Actors, Actions, Expected Results, Validations, Exceptions, Services and APIs, Input Parameters, Return Parameters, Methods, Exceptions, Messages, SQLs, Device Events (e.g., service start, stop, or restart), Device Attributes (e.g., CPU usage, Memory usage, Number of Threads, etc.). as well as other predefined groups which may be specific to the industry, enterprise, applications, and/or systems associated with the enterprise environment.

Classification by tag classifier module <NUM> may be performed based on the words appearing in a sentence or block of text. Proximity of the words in the block of text being analyzed is compared with the proximity of similar set of words in other blocks of text. For example, consider two blocks of text, A and B, where Block A recites "Salary for the month of January should be credited for all Employees" and Block B recites "Salary for the month of January should be credited for all Religious Workers".

In this example, tag classifier module <NUM> identifies that "Religious Worker" is used in the same context as "Employee". Accordingly, tag classifier module <NUM> treats these words as one group of words and uses this learning for classification purposes. Tag classifier module <NUM> accesses the database in tag model repository <NUM> for groups and associated identifiers or tags (e.g., themes and/or associated keywords) for appropriately classifying the identifiers in the obtained data from the enterprise environment (e.g., data <NUM> from enterprise environment <NUM>).

In this embodiment, tag classifier module <NUM> outputs one or more groups of segregated or classified tags <NUM>, including a first group <NUM> associated with Services, a second group <NUM> associated with Messages, a third group <NUM> associated with SQLs, and a fourth group <NUM> associated with Events. Additionally, groups <NUM>, <NUM>, <NUM>, <NUM> of classified tags <NUM> may be reviewed and/or reclassified through a manual process, such as by administrator <NUM> or SME. These manual changes may then be updated in tag model repository <NUM>.

Fault analysis and prediction system <NUM> includes a workflow extractor module <NUM>. In some embodiments, a workflow for a user in the enterprise environment is based on a session identifier and/or timestamps associated with the user's activity. The workflow identifies a time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment (e.g., interactions by a user of plurality of users <NUM> with one or more of plurality of systems <NUM> and/or plurality of applications <NUM> of enterprise environment <NUM>, shown in <FIG>).

In this embodiment, workflow extractor module <NUM> reconstructs application logs and/or trace file sequences based on the one or more groups of segregated or classified tags <NUM>, including groups <NUM>, <NUM>, <NUM>, <NUM>, from tag classifier module <NUM>. For example, workflow extractor module <NUM> may analyze a device log file, a web server log, and an app server log and trace files from an app server. Workflow extractor module <NUM> may first build a workflow for a user by extracting screen navigation information for the user based on timestamp information to identify a time-based sequence of interactions by the user with systems <NUM> and/or applications <NUM> of enterprise environment <NUM>.

For a given time interval under consideration, workflow extractor module <NUM> may analyze the web server logs and the activities therein may be mapped to the specific user, for example, by matching a session identifier associated with the user or a unique user identifier (user ID or login information). For the same time interval, workflow extractor module <NUM> may also extract the services, APIs, packages, methods, and/or call information from the app server. From the trace files, server parameters, such as CPU usage, memory usage, threads, heap dump, number of threads, etc., are also analyzed by workflow extractor module <NUM>. Any errors, warnings, or information entries in any of the logs for that time interval are also extracted by workflow extractor module <NUM>.

Based on the information extracted from the various data sources (e.g., logs and traces), workflow extractor module <NUM> may generate a workflow for a user and associated information. For example, workflow extractor module <NUM> may generate an end-to-end workflow for a user and the number of times these workflows were used by all users in the enterprise environment (e.g., plurality of users <NUM> in enterprise environment <NUM>). For example, a database of historical or model workflows associated with the enterprise environment may be stored in a workflow model repository <NUM>. Workflow extractor module <NUM> may use workflow model repository <NUM> to compare workflows of multiple users in the enterprise environment.

Workflow extractor module <NUM> may also generate a usage pattern associated with applications and/or systems in the enterprise environment (e.g., systems <NUM> and/or applications <NUM> of enterprise environment <NUM>). The usage pattern may indicate geographies from which users are accessing a particular application or system and the frequency of interaction, including what day of the week and time of day is the usage high, medium, and low. Workflow extractor module <NUM> may also generate information associated with error and warning patterns, including, for example, are all users facing errors or are specific users getting errors, and are errors occurring in specific applications, systems, methods or services.

An example of a workflow <NUM> for a user generated by workflow extractor module <NUM> is shown in <FIG>. Workflow <NUM> is a time-based sequence of interactions by a user within the enterprise environment. In this embodiment, workflow <NUM> begins with a first interaction <NUM> (Log). Next, a second interaction <NUM> (queue handling) leads to three actions, including a third interaction <NUM> (Add to expect_queue), a fourth interaction <NUM> (add to log-writer-queue), and a fifth interaction <NUM> (line type U). After fourth interaction <NUM>, a sixth interaction <NUM> (device logs) follows. Referring back to third interaction <NUM>, a seventh interaction <NUM> (no match - pattern) and an eighth interaction <NUM> (expect loop is evaluating) follow.

The remaining interactions associated with workflow <NUM> follow eighth interaction <NUM>. These interactions include: verifying hash integrity <NUM>, booting using the fdt blob <NUM>, loading kernel image <NUM>, loading ramdisk <NUM>, loading device tree <NUM>, starting kernel <NUM>, erasing memory <NUM>, writing and verifying address <NUM>% done <NUM>, using default environment <NUM>, secure boot not enabled <NUM>, boot from USB for mfgtools <NUM>, could not find configuration node <NUM>, press button to reboot <NUM>, and update complete <NUM>. It should be understood that workflow <NUM> is exemplary and illustrates an example of a workflow for a user that may be generated by workflow extractor module <NUM>.

In general, the sequence of steps performed by workflow extractor module <NUM> for building or generating a workflow (e.g., workflow <NUM>) are as follows below. First, workflow extractor module <NUM> establishes correlation across requirements, test cases, defects, log files and/or trace files. This may be accomplished by using identifier (ID) columns for each type of data. If traceability is maintained, for example, in a requirement to test case, then traceability is given first priority. Correlation is also established by mapping themes and associated keywords for the data.

Next, workflow extractor module <NUM> determines the sequence of interactions by the user based on following techniques in the given order: A session or transaction ID associated with the user is used to correlate interactions performed across technology layers. For example, if a workflow scenario begins with a web browser, then goes to SAP, and finally into a database then, the Session ID of the web app server is mapped to the session ID of the SAP server which in turn is mapped to the session ID of the database. When it is not possible to map a session or transaction IDs directly, then a timestamp associated with the user may be used to establish the relationship across technology layers. Additionally, occurrence of contextual similar identifiers or tags (e.g., themes and associated keywords) may be used by workflow extractor module <NUM> to establish correlations between the data.

A workflow generated for a user by workflow extractor module <NUM> may then be compared against a database of known workflows from other users in the enterprise environment stored in workflow model repository <NUM>. Workflow extractor module <NUM> may access the stored workflows in workflow model repository <NUM> to check the generated workflow for any gaps, deviations, or anomalies based on the comparison. Additionally, if the generated workflow is a new workflow, workflow extractor module <NUM> may add it to workflow model repository <NUM> after analysis and/or confirmation by administrator <NUM> or SME.

Referring back to <FIG>, fault analysis and prediction system <NUM> includes an anomaly detector module <NUM>. Anomaly detector module <NUM> is configured to analyze the one or more groups of segregated or classified tags <NUM>, including groups <NUM>, <NUM>, <NUM>, <NUM>, from tag classifier module <NUM> and identify correlations between the various types of identifiers or tags. Anomaly detector module <NUM> is also configured to identify commonly occurring correlations between the data and any outliers or anomalies in the data.

Anomaly detector module <NUM> may identify technical anomalies and/or functional anomalies. First, consider the following example of a technical anomaly. While testing a web application, testers get a "Unable to find Product Information" message <NUM>% of the time for a specific category of product although product information for this category exists in a product database. If the testers retry the same transaction after some time interval, the subsequent transaction is successful (i.e., product information is returned). Upon analyzing the web application server log file, anomaly detector module <NUM> observes that this particular transaction sends a request to a message server to get the product information. For most of the attempted transactions, the web application does not get a response from the message server within a predetermined wait time.

Upon analyzing the message server log file for the same time interval, anomaly detector module <NUM> determines that the message server is receiving more requests than it can process, and, therefore, the message server's performance is slower than normal (i.e., outside of the predetermined wait time) and the "Unable to find Product Information" message is returned to the web application. The recommended solution or fix for this anomaly is to increase the predetermined wait time on the web application server for receiving a response to a request from the message server and/or to increase the number of requests that the message server can handle (e.g., through software configuration or through a hardware upgrade).

According to the techniques of the example embodiments, anomaly detector module <NUM> analyzes the output of Workflow extractor module <NUM> and identifies this particular anomaly. In some embodiments, anomaly detector module <NUM> may send an email or other alert to administrator <NUM> or other tester with the specific lines of text extracted from the web application server log and the message server log that are associated with the generated anomaly.

Next, consider an example of a functional anomaly. For a trading application, code change data and application usage log files are analyzed by anomaly detector module <NUM>. Anomaly detector module <NUM> determines that <NUM>% of the code change data is associated with trade in Asia. However, <NUM>% of application usage is with respect to trade transactions involving Europe. Anomaly detector module <NUM> determines that the trading application enhancements (e.g., code changes), and, therefore, information technology budget, is being performed on an area of the application which is seldom used.

Additionally, consider an example of a usage anomaly. Anomaly detector module <NUM> analyzes a web server log and new infrastructure requests (e.g., through service tickets). Anomaly detector module <NUM> determines that all new infrastructure is requested to be hosted in London, but there is an <NUM>% increase in usage of the web server from China. Accordingly, anomaly detector module <NUM> determines that the new infrastructure, for example, a new server, should be provisioned in China instead of London.

In some embodiments, fault analysis and prediction system <NUM> may also include a fault correlator module <NUM>. Fault correlator module <NUM> is configured to maps or correlate errors, failures, warnings, and/or anomalies with various associated system or device attributes (e.g., CPU and/or memory usage, etc.). For example, a banking application has a web page as a user interface. When users of the banking application click on "Interest Rates" the users receive the requested information sometimes and occasionally they receive a "Requested information not found" message instead. When the application log file for the banking application is analyzed along with the messaging server logs, fault correlator module <NUM> identifies that whenever the messaging server is experiencing a high number of requests, the users of the banking application receive the "Requested information not found" message. In this case, the failure experienced by users in one layer of the system (i.e., the banking application) is actually caused by a fault in another layer of the system (i.e., the messaging server). Fault correlator module <NUM> identifies such correlations across architecture/technology layers of the system in enterprise environment <NUM>.

Fault analysis and prediction system <NUM> includes a fault modeler module <NUM> and a fault model database <NUM>. In an example embodiment, fault model database <NUM> stores one or more faults (i.e., errors, failures, warnings, and/or anomalies) identified by fault correlator module <NUM>. Fault model database <NUM> also includes a snapshot of identifiers or tags associated with the data (e.g., data <NUM>) when the identified fault happened. Fault modeler module <NUM> is configured to refer to fault model database <NUM> to generate fault predictions based on the current identifiers or tags and one or more historical patterns of identifiers or tags observed when faults occurred in enterprise environment <NUM> under similar circumstances.

Fault modeler module <NUM> may compare workflows and identifiers from multiple users in enterprise environment <NUM> to determine at least one fault identification or fault prediction. The fault prediction by fault modeler module <NUM> is based on recognizing a subset of interactions of time-based sequence of interactions by a user with the at least one or more systems or applications in enterprise environment <NUM> that have historically caused or generated a fault. Fault modeler module <NUM> may access the historical fault information included in fault model database <NUM> along with the workflows and identifiers or tags to determine that a fault may occur.

For example, fault modeler module <NUM> determines that, historically, whenever a time-based sequence of interactions occurs, Events A, B, C, D and E, a Fault F occurs in enterprise environment <NUM> within the next <NUM> minutes. Subsequently, when fault modeler module <NUM> recognizes a subset of interactions of the time-based sequence of interactions by a user with the at least one or more systems or applications in enterprise environment <NUM> occur (e.g., Events A, B, C, D and E occur), fault modeler module <NUM> will predict the occurrence of Fault F. That is, based on historical fault information in fault model database <NUM>, fault modeler module <NUM> is configured to provide a fault prediction when similar circumstances are determined.

Fault modeler module <NUM> may identify a subset of interactions that are associated with a historically known fault even when the sequence of interactions includes additional interactions that are not associated with the fault. For example, if there are other events occurring in the middle of a known subset of interactions, i.e., Events A, B, G, K, C, L, D, M, and E occur, fault modeler module <NUM> still identifies that the subset of interactions, i.e., Events A, B, C, D and E have occurred, which is recognized by fault modeler module <NUM> to generate a fault prediction that Fault F will occur as a result.

In some embodiments, fault analysis and prediction system <NUM> may include an auto healer module <NUM>. Auto healer module <NUM> may be configured to implement a corrective action automatically upon determination of a fault identification or fault prediction. For example, auto healer module <NUM> may look up a fault in a root cause and fix repository <NUM>. Root cause and fix repository <NUM> is a database that correlates fault conditions to known corrective actions based on a fault identification or fault prediction. Auto healer module <NUM> may implement a corrective action for a fault stored in root cause and fix repository <NUM> by performing a sequence of steps correlated to the fault to prevent and/or fix the fault in enterprise environment <NUM>.

In an example embodiment, various databases included in fault analysis and prediction system <NUM>, including, but not limited to: tag corpus <NUM>, tag model repository <NUM>, workflow model repository <NUM>, fault model database <NUM>, root cause and fix repository <NUM>, and theme and keyword repository <NUM>, initially may be generated manually and maintained through automatic recommendation and human reviews (e.g., by administrator <NUM> or SME).

Visualizer <NUM> may be a display or other user interface that is configured to display results and recommendations provided by fault analysis and prediction system <NUM>. For example one or more faults <NUM> (shown in <FIG>) determined by fault analysis and prediction system <NUM> based on data <NUM> may be displayed or provided as an alert or message on visualizer <NUM>. Admin <NUM> may access fault analysis and prediction system <NUM> using a comprehensive UI for all management-related tasks.

<FIG> is a flowchart of an example embodiment of a method <NUM> for fault analysis and prediction in an enterprise environment. In an example embodiment, method <NUM> may be implemented by fault analysis and prediction system <NUM> to analyze and predict faults associated with enterprise environment <NUM>.

In this embodiment, method <NUM> may begin at an operation <NUM> where data is obtained from a plurality of sources in the enterprise environment. The plurality of sources includes at least one or more systems, users, or applications. For example, as shown in <FIG>, data <NUM> obtained from enterprise environment <NUM> may include information associated with a plurality of sources, including, but not limited to: plurality of users <NUM>, plurality of systems <NUM>, and/or plurality of applications <NUM>.

Next, method <NUM> includes an operation <NUM>. At operation <NUM>, the obtained data from operation <NUM> is associated with identifiers. The identifiers include a theme selected from a set of themes and one or more keywords, where the keywords are specific to each theme. For example, as shown in <FIG>, set of themes <NUM> includes first theme <NUM>, second theme <NUM>, and third theme <NUM>, and first group of keywords <NUM> are specific to first theme <NUM>, second group of keywords <NUM> are specific to second theme <NUM>, and third group of keywords <NUM> are specific to third theme <NUM>.

Method <NUM> also includes an operation <NUM>, where a workflow is generated for a user based on a session identifier and/or timestamps associated with activity by the user. The workflow identifies a time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment. For example, as shown in <FIG>, workflow <NUM> identifies a sequence of interactions by a user of plurality of users <NUM> with systems <NUM> and/or applications <NUM> in enterprise environment <NUM>.

Next, at an operation <NUM>, at least one fault identification or fault prediction associated with the enterprise environment is determined. The fault identification or fault prediction is based on the workflow generated at operation <NUM> and identifiers associated with the obtained data that corresponds to the workflow. For example, a fault may be determined as described above with reference to fault modeler module <NUM>. In some embodiments, upon determination of the fault identification or fault prediction, a corrective action may be implemented. The corrective action may be implemented manually, for example, by administrator <NUM> or other entity, or may be implemented automatically, for example, by auto healer module <NUM>, as described above. Method <NUM> may be repeated upon ingestion of new or updated data associated with the same enterprise environment (e.g., enterprise environment <NUM>). Method <NUM> may also be repeated for data associated with a new or different enterprise environment.

The techniques for empirical fault analytics presented herein for fault analysis and prediction may be used in real time, a batch processing mode, or both, to identify one or more patterns associated with an enterprise environment. These patterns may include, but are not limited to: application usage patterns (e.g., end-to-end flows, software component usage, data combinations usage, etc.), application failures (e.g., individual failure patterns, relationships between failures, etc.), anomaly detection (e.g., rogue behavior of various application or system components, infrastructure elements, architecture attributes, etc.), failure prediction (e.g., prediction of failures, probable root causes), and/or auto healing (e.g., executing a sequence of tasks that can address root causes predicted by fault analysis and prediction system <NUM>).

Claim 1:
A method for fault analysis and prediction in an enterprise environment (<NUM>), the method comprising:
obtaining data from a plurality of sources in the enterprise environment (<NUM>), wherein the plurality of sources includes at least one or more systems, users, or applications;
associating the obtained data with identifiers that include a theme selected from a set of themes and one or more keywords, wherein the keywords are specific to each theme;
generating a workflow (<NUM>) for a user based on a session identifier and/or timestamps associated with activity by the user, wherein the workflow (<NUM>) identifies a time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment (<NUM>);
determining at least one fault identification or fault prediction associated with the enterprise environment (<NUM>) based on the workflow (<NUM>) and identifiers associated with the obtained data that corresponds to the workflow (<NUM>); and
based on the at least one fault identification or fault prediction, implementing a corrective action using a database (<NUM>) that correlates fault conditions to known corrective actions, and wherein the corrective action is implemented automatically upon determination of the at least one fault identification or fault prediction using an auto healer module (<NUM>) to look up a fault in a root cause and fix repository (<NUM>),
wherein the fault prediction is based on a subset of interactions of the time-based sequence of interactions by the user with the at least one or more systems or applications in the enterprise environment (<NUM>), and wherein the subset of interactions correspond to a known fault condition for the enterprise environment (<NUM>) stored in a fault database generated based on a previous time-based sequence of interactions by a different user, and
wherein correlating the fault conditions to known corrective actions is carried out by a fault correlator module (<NUM>) that maps errors, failures, warnings, and/or anomalies with various associated system or device attributes and identifies correlations across architecture/technology layers of the system in the enterprise environment (<NUM>) using the mapping.