Automatic software behavior identification using execution record

The automatic identification of execution behavior(s) of software. This automatic identification is based on a historical analysis of execution records to identify a particular pattern that represents an execution behavior. In order to automatically identify an execution behavior present within particular software, an execution record (or perhaps multiple execution records) representing the execution of that particular software may be accessed. Based on finding the particular pattern within the execution record (or one, some, or all of the multiple execution records) representing the execution of that particular software, the computing system may automatically identify that the execution behavior is present within the software. This may dramatically assist in modifying that execution behavior.

BACKGROUND

Computing systems operate at the direction of computer-executable code. It is important to find logical errors within the code in order to ensure that the computing system operates properly. The finding of logical errors may be quite an intensive task, especially for complex pieces of code. Some logical errors can be found when the code is being drafted, when the code is being authored and debugged. Developers may author the code using source code, and then compile that source code into intermediate or binary code in order to test for logical errors that manifest within the developer's computing system environment. The code may then be shipped for execution in various production environments. It is sometimes possible to test the code by emulating certain aspects of the anticipated production environments.

Even so, logical errors may not make themselves manifest until the code runs in the production environment. Such logical errors may be due to characteristics or errors within the production environment itself. For instance, a database or other resource of the production environment might not have been configured correctly, environmental variables specific to that production environment might not have been set properly, credentials may not be set correctly, a resource might become dereferenced, there might be a leaking resource (such as leaking memory), there might be edges on conditions that result in a logical error, and so forth. There may be countless varieties of logical errors that might be encountered in countless numbers of production environments.

BRIEF SUMMARY

At least some embodiments described herein relate to the automatic identification of execution behavior(s) of software. This automatic identification is based on a historical analysis of execution records to identify a particular pattern that represents an execution behavior. An example of the execution record might be an execution log that roughly captures execution by logging certain events that may occur during execution of software. On the other extreme, the execution record might be an execution trace that fastidiously and reproducibly represents the execution of the software within a particular execution environment, such that the execution trace may be used to rerun the execution precisely as it previously ran. The execution record might also be any record of execution with execution detail between these two extremes.

In order to automatically identify an execution behavior present within particular software, an execution record (or perhaps multiple execution records) representing the execution of that particular software may be accessed. Based on finding the particular pattern within the execution record (or one, some, or all of the multiple execution records) representing the execution of that particular software, the computing system may automatically identify that the execution behavior is present within the software. This may dramatically assist in modifying that execution behavior.

For instance, the execution record(s) might represent an execution of the software after the software is authored and deployed within its working environment (or within various working environments). The execution behavior identified may be due to particular characteristics of that environment, and how that environment is configured. For instance, if the execution behavior shows a malfunction of the software, the malfunction may be due to problems that relate to the working environment—rather than problems implicit in the software itself

Such environment-related problems might be difficult to anticipate at the software authoring phase, as the variety of environment-related problems that could arise is countless and it is difficult to anticipate what problems are actually going to arise. For instance, perhaps the environment does not have a database that is configured correctly, perhaps credentials were set wrong, perhaps environmental variables were not set correctly, and so forth almost infinitum. However, by allowing a mechanism to detect the environmental problems that have been encountered during the real execution, the software provider can address the environmental execution problems that are tending to arise in the environment, allowing the software provider to ultimately provide software that more closely meets the characteristics of the ultimate execution environment.

DETAILED DESCRIPTION

At least some embodiments described herein relate to the automatic identification of execution behavior(s) of software. This automatic identification is based on a historical analysis of execution records to identify a particular pattern that represents an execution behavior. An example of the execution record might be an execution log that roughly captures execution by logging certain events that may occur during execution of software. On the other extreme, the execution record might be an execution trace that fastidiously and reproducibly represents the execution of the software within a particular execution environment, such that the execution trace may be used to rerun the execution precisely as it previously ran. The execution record might also be any record of execution with execution detail between these two extremes.

In order to automatically identify an execution behavior present within particular software, an execution record (or perhaps multiple execution records) representing the execution of that particular software may be accessed. Based on finding the particular pattern within the execution record (or one, some, or all of the multiple execution records) representing the execution of that particular software, the computing system may automatically identify that the execution behavior is present within the software. This may dramatically assist in modifying that execution behavior.

For instance, the execution record(s) might represent an execution of the software after the software is authored and deployed within its working environment (or within various working environments). The execution behavior identified may be due to particular characteristics of that environment, and how that environment is configured. For instance, if the execution behavior shows a malfunction of the software, the malfunction may be due to problems that relate to the working environment—rather than problems implicit in the software itself

Such environment-related problems might be difficult to anticipate at the software authoring phase, as the variety of environment-related problems that could arise is countless and it is difficult to anticipate what problems are actually going to arise. For instance, perhaps the environment does not have a database that is configured correctly, perhaps credentials were set wrong, perhaps environmental variables were not set correctly, and so forth almost infinitum. However, by allowing a mechanism to detect the environmental problems that have been encountered during the real execution, the software provider can address the environmental execution problems that are tending to arise in the environment, allowing the software provider to ultimately provide software that more closely meets the characteristics of the ultimate execution environment.

FIG. 1illustrates a flowchart of a method100for automatically identifying an execution behavior of software so that the execution behavior can be modified. The method100may be performed by a computing system in response to executing computer-executable instructions that are structured to cause the computing system to perform the method100. For instance, one or more processors of the computing system may execute those instructions and cause the method100to be performed. An example of such a computing system is described below with respect to the computing system600ofFIG. 6.

The method100includes two steps110and120. First, the computing system automatically identifies an execution behavior of software (step110). Next, the computing system optionally uses the identified execution behavior to facilitate modification of the execution behavior (step120). The facilitation of the modification of the execution behavior may be in conjunction with interfacing with a user. Each of these steps may include corresponding acts that support the steps.

FIG. 2also illustrates a flowchart of a method200for automatically identifying an execution behavior of software. The method200ofFIG. 2is one example of the corresponding acts that support the step of automatically identifying an execution behavior of software (step110).FIGS. 3 and 4illustrate various execution record analysis environments300and400. As the method200ofFIG. 2may be performed in an environment in which execution records are analyzed, the method200ofFIG. 2will now be described with respect to the execution record analysis environments300and400ofFIGS. 3 and 4, respectively.

The method200includes analyzing execution records to identify a particular pattern that represents an execution behavior (act201). As an example, the execution behavior might represent a problem behavior, or a behavior that might be changed to improve software that happens to contain the particular pattern. Such execution behaviors might relate to the environment in which software is executed. For instance, the execution behavior might include difficulty connecting to or interfacing with certain environmental resources (such as a database), failure to use certain credentials set within the operating environment, encountering an error due to the failure to set one or more environmental variables, an environmental resource becoming dereferenced, edge conditionals being encountered that lead to errors, and a multitude of other possibilities.

FIG. 3illustrates an environment300in which an analyzer component310analyzes an execution record collection301of execution records301A through301G. Thus, act201may be performed in the environment300ofFIG. 3, whereas the remainder of the method200may be performed in the environment400ofFIG. 4. The analyzer component310may be structured as described below for the executable component606ofFIG. 6.

Each execution record includes a representation of how various software was executed. The principles described herein are not limited to the type of execution records. Some execution records might only roughly capture execution state. For instance, an execution log might perhaps only capture major events during execution. On the other hand, some execution records trace the execution with exact precision, such that every instruction that was executed is known, and such that the exact value(s) of data input to that instruction (and output from that instruction) are known from that trace. Such execution traces may so precisely define the execution that the execution may be performed again, exactly how it was earlier, using information from the execution trace. The principles described herein may operate with execution records, whether roughly capturing execution state, precisely capturing execution state, or any level of precision therebetween.

The execution records in the execution record collection301are each shown as having various shapes to represent that the execution records represent execution of a variety of different software. But even amongst the execution records of different software, common patterns may emerge that represent interesting execution behaviors. These interesting behaviors may be potentially problematic behaviors. The ellipsis301H represents that there may be a very large number (thousands, millions, billions, and so forth) of execution records being analyzed by the analyzer component310to thereby obtain a very comprehensive and clear historical analysis of software executions.

The result of the analysis is a collection311of patterns. In the specific case ofFIG. 3, the result is a collection of paired patterns and execution behaviors. For instance, the collection311is illustrated as having pairs311A through311C. Each pair has a circle representing a pattern found in execution records, and an associated triangle representing the execution behavior that is associated with that pattern. For instance, pair311A includes execution record pattern311A1associated with execution behavior311A2, pair311B includes execution record pattern311B1associated with execution behavior311B2, and pair311C includes execution record pattern311C1associated with execution behavior311C2.

The ellipsis311D represents that the analyzer component310may find any number (and perhaps an innumerable number) of pattern-behavior pairs. The pairs may also be identified with the assistance of observing when users are dissatisfied with particular patterns (e.g., they modify the pattern). That may help to determine which patterns are interesting to pair up with an associated behavior. The generation of pattern-behavior pairs may be an ongoing process performed over extended periods of time, such that the collection311has been accumulated over a long period of time. Thus, the act201may be performed well in advance of the remainder of the method200.

The analyzer component may use machine learning techniques in order to identify pairs. For instance, machine learning techniques may be used to automatically identify and present differences between an execution record of a prior version of software, and an execution record representing a subsequent version of the software.

Returning toFIG. 2, the method200includes accessing an execution record of an execution of the software (act211). Here we are dealing with specific software whose execution is under evaluation (hereinafter referred to as “particular software”), for instance, to potentially see how the particular software might be modified. This might be software that is currently being debugged by one or more developers. Alternatively, this particular software might have been already deployed in one or more operating environments. The remainder of method200may be performed in the environment400ofFIG. 4. Accordingly, the remainder of the method200will now be described with respect to the environment400ofFIG. 4.

Referring toFIG. 4, there is an execution record set401of one or more execution records of software that is currently under evaluation. As illustrated, the execution record set401includes two execution records401A and401B. The fact that these execution records of different executions of the same software is represented by each of the two execution records401A and401B having the same shape. Note that the shape is different than any of the shapes shown in the execution record collection301ofFIG. 3. This is to merely show that the execution record(s) for the particular software being evaluated need not have been (but may be) included within the execution record collection301that was used by the analyzer component310to extract interesting execution record patterns and corresponding execution behaviors.

The ellipsis401C represents that a pattern finder component410may search for pattern(s) within just a single execution record for particular software, or within multiple execution records for that software. Thus, the act of accessing an execution record of an execution of the software (act211) may be performed as part of a larger act of accessing multiple execution records for that particular software (also represented by act211). The pattern set that the pattern finder component410may look for is represented by the pattern set411. The pattern set411may contain as few as one of the patterns within the collection311of pattern-behavior pairs, or may continue multiple patterns from the collection311. The pattern finder component410may be structured as described below for the executable component606.

The method200then includes finding a pattern within the execution record (act212). For instance, referring toFIGS. 3 and 4, suppose that the pattern finder component410finds the pattern311B1within the execution record401A. Again, the act of finding the particular pattern within the execution record (act212) may be part of an act of finding the particular pattern in at least some of the multiple execution records of the multiple of execution records of the particular software (also represented by act212) should there be multiple execution records for that particular software. For instance, referring toFIGS. 3 and 4, suppose that the pattern finder component410finds the pattern311B1within each of at least some of the execution records (e.g., execution records401A and401B) in the execution record set401for that particular software. The pattern finder component410may look for all possible patterns, of the list of possible patterns may be reduced using input from the user, or perhaps machine learning techniques.

Evaluation of multiple execution records may be helpful where the execution of software is being evaluated for multiple post-deployment operating environments. The software will have different executions (and thus difficult execution records) in each operating environment. Thus, in will allow for comprehensive analysis of how the particular software actually executes within a wide variety of execution environments. This will allow for the detection of what problems (of the innumerable possible problems) that are trending into existence within the operating environments. Attention may then be immediate addressed towards correcting those problems, rather than problems that (while theoretically possible) will never actually happen. The end result is more efficient use of resources to improve quality of software as that software operates within various working environments.

The method200then identifies that the corresponding execution behavior is present within the software (act213) based on the act of finding the particular pattern within the execution record (act212) (or within at least one of the execution records if multiple execution records of the particular software were searched). For instance, referring toFIGS. 3 and 4, if the pattern finder component410detects the pattern311B1within the execution record(s), then the computing system may identify that the execution behavior311B2is present when the software was executed within its respective environments.

As previously mentioned, the execution record(s) might represent an execution of the software being authored. Thus, the analysis of the execution record(s) may help identify execution behaviors and/or associated patterns is execution so that they might be corrected during the debugging phase. However, the analysis of execution record(s) may be particularly helpful for analyzing executions after the software is authored and deployed within its working environment (or within various working environments). Such environment-related problems might be difficult to anticipate at the software authoring phase, as the variety of environment-related problems that could arise is countless and it is difficult to anticipate what actual problems are actually going to arise. For instance, perhaps the environment does not have a database that is configured correctly, or perhaps credentials were set wrong, perhaps environmental variables were not set correctly, and so forth almost infinitum. However, by allowing a mechanism to detect the environmental problems that actually have been encountered, the software provider can address the environmental execution problems that are tending to arise in the environment, allowing the software provider to ultimately provide software that more closely meets the characteristics of the ultimate execution environment. The principles described herein may thus greatly assist in improving the performance of deployed software.

Referring back toFIG. 1, once the identified execution behavior is automatically identified (step110), that identification may then be used to modify the execution behavior (step120).FIG. 5illustrates a flowchart of a method500for modifying the execution behavior and includes one example of corresponding acts used to support the modification of the execution behavior (step120).

Specifically, the computing system might display an identification of the execution behavior and/or the associated pattern to the user (act501). The user may then note the execution behavior and/or the associated pattern and take appropriate action themselves to modify the execution behavior or pattern if the user feels that would be appropriate.

Alternatively, or in addition, the computing system might display an identification of a modified execution behavior and/or a modified associated pattern to the user (act502). The user may then note the modified execution behavior and/or the associated modified pattern and take appropriate action themselves to modify the execution behavior or pattern if the user feels that would be appropriate. The modified execution behavior and/or modified pattern may have been previously identified by, for instance, observing what other users do when they encounter the execution behavior in order to modify that execution behavior.

Alternatively, or in addition, the computing system may offer to perform a modification of the execution behavior and/or the associated modified pattern to the user (act503). As an example, the execution behavior might be that environmental variables were not set. The suggested modification might be to add a file to the software which automatically sets the environmental variables so that the variables at least do not have a null value. This offering may take the form of a control that may be interface with by the user. In response to receiving an indication that the user has selected the modification (act504), the computing system may automatically perform the selected modification (act505). For instance, if the execution behavior was failing to set the values of environmental variables, the computing system itself might automatically add a file that sets the environmental variables. Thus, embodiments described herein may also assist in modifying the execution behavior and/or pattern to thereby improve efficiency in improving the behavior of software.

Because the principles described herein operate in the context of a computing system, a computing system will be described with respect toFIG. 6. Computing systems are now increasingly taking a wide variety of forms. Computing systems may, for example, be handheld devices, appliances, laptop computers, desktop computers, mainframes, distributed computing systems, datacenters, or even devices that have not conventionally been considered a computing system, such as wearables (e.g., glasses, watches, bands, and so forth). In this description and in the claims, the term “computing system” is defined broadly as including any device or system (or combination thereof) that includes at least one physical and tangible processor, and a physical and tangible memory capable of having thereon computer-executable instructions that may be executed by a processor. The memory may take any form and may depend on the nature and form of the computing system. A computing system may be distributed over a network environment and may include multiple constituent computing systems.

As illustrated inFIG. 6, in its most basic configuration, a computing system600typically includes at least one hardware processing unit602and memory604. The memory604may be physical system memory, which may be volatile, non-volatile, or some combination of the two. The term “memory” may also be used herein to refer to non-volatile mass storage such as physical storage media. If the computing system is distributed, the processing, memory and/or storage capability may be distributed as well.

The computing system600has thereon multiple structures often referred to as an “executable component”. For instance, the memory604of the computing system600is illustrated as including executable component606. The term “executable component” is the name for a structure that is well understood to one of ordinary skill in the art in the field of computing as being a structure that can be software, hardware, or a combination thereof. For instance, when implemented in software, one of ordinary skill in the art would understand that the structure of an executable component may include software objects, routines, methods that may be executed on the computing system, whether such an executable component exists in the heap of a computing system, or whether the executable component exists on computer-readable storage media.

The term “executable component” is also well understood by one of ordinary skill as including structures that are implemented exclusively or near-exclusively in hardware, such as within a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other specialized circuit. Accordingly, the term “executable component” is a term for a structure that is well understood by those of ordinary skill in the art of computing, whether implemented in software, hardware, or a combination. In this description, the term “component” or “vertex” may also be used. As used in this description and in the case, this term (regardless of whether the term is modified with one or more modifiers) is also intended to be synonymous with the term “executable component” or be specific types of such an “executable component”, and thus also have a structure that is well understood by those of ordinary skill in the art of computing.

In the description that follows, embodiments are described with reference to acts that are performed by one or more computing systems. If such acts are implemented in software, one or more processors (of the associated computing system that performs the act) direct the operation of the computing system in response to having executed computer-executable instructions that constitute an executable component. For example, such computer-executable instructions may be embodied on one or more computer-readable media that form a computer program product. An example of such an operation involves the manipulation of data.

The computer-executable instructions (and the manipulated data) may be stored in the memory604of the computing system600. Computing system600may also contain communication channels608that allow the computing system600to communicate with other computing systems over, for example, network610.

While not all computing systems require a user interface, in some embodiments, the computing system600includes a user interface612for use in interfacing with a user. The user interface612may include output mechanisms612A as well as input mechanisms612B. The principles described herein are not limited to the precise output mechanisms612A or input mechanisms612B as such will depend on the nature of the device. However, output mechanisms612A might include, for instance, speakers, displays, tactile output, holograms, virtual reality, and so forth. Examples of input mechanisms612B might include, for instance, microphones, touchscreens, holograms, virtual reality, cameras, keyboards, mouse or other pointer input, sensors of any type, and so forth.

Those skilled in the art will also appreciate that the invention may be practiced in a cloud computing environment, which is supported by one or more datacenters or portions thereof. Cloud computing environments may be distributed, although this is not required. When distributed, cloud computing environments may be distributed internationally within an organization and/or have components possessed across multiple organizations.

In this description and the following claims, “cloud computing” is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services). The definition of “cloud computing” is not limited to any of the other numerous advantages that can be obtained from such a model when properly deployed.

For instance, cloud computing is currently employed in the marketplace so as to offer ubiquitous and convenient on-demand access to the shared pool of configurable computing resources. Furthermore, the shared pool of configurable computing resources can be rapidly provisioned via virtualization and released with low management effort or service provider interaction, and then scaled accordingly.