SHORTENED NARRATIVE INSTRUCTION GENERATOR FOR SOFTWARE CODE CHANGE

A method, computer system, and a computer program product for generating instructions to highlight software change may be provided. In one embodiment, the technique comprises obtaining information about a requested modification to an original code of a software program and classifying it based on the type of change requested by the modification. The unit tests available are then identified. At least one of the identified unit tests are selected and customized based on classification of the type of code modification requested. Using the at least one selected and customized unit test, the differences between the original and modified code may be identified. One or more test execution stories are then generated related to the modification of the code, to highlight the changes. Test execution stories are further analyzed to provide any additional missing information.

BACKGROUND

The present invention relates generally to the field of software design and more particularly to techniques for generating narrative instructions to highlight software code change.

Many reasons may exist that require computer code to be modified in an already existing software program or application. Software designs are ever changing, and new features may have to be added or removed from the existing application or program. Modifications may be needed to adapt to new environments and requirements, or patches and bug fixes may need to be installed to address issues. Modifying computer code in an already existing software program and application may be challenging.

Code modifications require an understanding of the existing application and prior historical changes of the code. Otherwise, improper modifications may detrimentally affect the application or the program. Understanding and tracking changes may be very important but it may also be very challenging. As the program or application goes through several different types of changes over time, the complexity of its design increases and modifications become more difficult. Furthermore, in recent years collaborative environments that include many contributors globally have increased these challenges. Even when code modification's impact can be anticipated, taking a substantial amount of time to complete them may adversely impact the runtime and application development.

SUMMARY

Embodiments of the present invention disclose a method, computer system, and a computer program product for generating instructions to highlight software change. In one embodiment, the technique comprises obtaining information about a requested modification to an original code of a software program and classifying it based on the type of change requested. Any unit tests available are then identified and at least one of the available unit tests are selected and customized based on classification of the type of code change requested. Using the at least one selected and customized unit test, the differences between the original and modified code may be identified. One or more test execution stories are then generated related to the modification of the code as identified. The test execution stories highlight any changes between the original and modified code. Test execution stories are further analyzed to provide any additional missing information.

DETAILED DESCRIPTION

Detailed embodiments of the claimed structures and methods may be disclosed herein; however, it can be understood that the disclosed embodiments may be merely illustrative of the claimed structures and methods that may be embodied in various forms. This invention may, however, be embodied in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Rather, these exemplary embodiments may be provided so that this disclosure will be thorough and complete and will fully convey the scope of this invention to those skilled in the art. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

As discussed earlier, modification of software applications has been becoming increasingly complicated. Any modifications become even more intricate and challenging when the devices and the code involved resides in a distributed manner, in various locations and across different data centers. Some software modifications may include reviewing traces or logs, such as after test execution and performing test analysis of errors. However, most of these processes requires extensive and time-consuming code examinations. Sometimes these examinations are textual and easier to understand but at other times they may non-intuitive and directionless. In addition, other aspects of the software including code quality, code complexity, and code cohesion, may involve manual tasks, such as executing and understanding multiple different testing tools. These processes are costly both in terms of time and money.

InFIGS.2and3, as per one embodiment, a technique may be provided that will mitigate the complexity of code and test reviews such as by providing a visual analysis. This technique may provide live feedback to a user/programmer through an analysis and in real execution-time (using several software metrics) to address problems and acquire additional valuable information. For example, the process of blocks200and300(FIGS.2and3) when used in different embodiments including an extended reality environment, through a selected set of metrics, can allow a user/programmer/developer to check the evolution of the code. This allows the developers to test or program sections effectively and by reducing the development time.

In some examples, given a scenario in which developers and programmers are the project maintainers of a program, requesting a modification will lead to revisions of a complex code base. These modifications require a careful design and an understanding of all potential side-effects of the proposed code change. Usually, the maintainers of the program and the developer/programmers have to cooperatively resort to pencil and paper, in order to keep track of data structures modified by such a code change. They may need to even create a mental map of the code structure so they can picture how the code changes relate to that structure. As discussed earlier, these efforts are time consuming, may be costly and may even be prone to interpretation errors. The techniques as provided by process200, greatly improves the process without the usual side effects associated with code changes (including addition of bugs.)

FIGS.2-3provides a flowchart illustration of different embodiments. In one embodiment, as captured byFIG.2, a shortened narrative instruction is generated to highlight runtime impacts of software code change based on automatic transformation of test cases that may include relevant data structures, data flow, message exchanges and the like. The instructions can be used to generate videos, figures, three-dimensional (3D) objects or text to highlight the side effects on execution of software due to new source code changes. This also allows software developers to better understand how new source code impacts the execution of the existing software.

The techniques provided can be used with both two dimensional and 3D content. For example, in recent years, it has been established that code and data can be visualized in three-dimensions (3D) and better understood by means of animation and anthropomorphizing. A metaphor has been established for the human viewer to understand the visualization of abstract mathematical constructs that operate on scales beyond normal cognitive range. Some scenarios have used man-made structures such as in the real world to envision software as if in a city inhabited by many dynamic entities whose behavior could be understood by means of direct observation. Some source code editors provide visual based editing to help with debugging, syntax highlighting, code completion and the like. Some necessary functions can include theme, keyboard shortcuts, preferences, install extensions an add functionality. The present methodology as used inFIGS.2-5can also be used with these new and emerging extended realities. However, the process discussed inFIGS.2-4may not be limited to the extended realities or any (particular) source code editors and can be used in conjunction with any other scenarios.

The processes200/300as shown inFIGS.2-4provides, according to one embodiment, an automatic technique for the generation of shortened narrative instructions that emphasize how new code changes affect runtime execution of the existing software (relevant data structures, data flow, and message exchanges) based on:(i) Natural language processing (NLP) on the messages (i.e., commit messages), code comments, and Abstract Syntax Tree (AST)+ontology-based code analysis to extract code change classes (seeFIG.4410/458);(ii) Usage of code change classes to select and configure existing software test cases that can highlight new software execution flows (seeFIG.4—452and454);(iii) automatic generation of narrative texts in the absence of comments and commit messages based on code change class detection (SeeFIG.4—410/456and458).

FIG.2provides a flowchart depiction of a process200that can generate a shortened narrative instructions to highlight runtime impacts of software code change based on automatic transformation of test cases. This may be particularly useful in immersive environments that employ extended realities. The process200generates a set of narrative instructions that could be used in conjunction with a variety of media and for other purposes. These narratives can in turn be used to generate videos, figures, 3D objects or text to highlight the side effects on execution of software due to new source code changes. This may be particularly useful in immersive environments that employ extended realities.

In the embodiment ofFIG.2, a flowchart depiction of a first embodiment may be provided for generating a software change runtime. The process of providing this automatic technique with an impact narrative may be illustrated at200. In one embodiment as illustrated, the process starts at Step210.

In step210, it may be determined that a code of a software program may be changed or may be scheduled to be changed. The code can be a source code in an embodiment. The original version of the (source) code can be obtained or received from a variety of sources, such as a database. The modification to the code might have already happened or it may be scheduled to happen in the future.

In Step220, the type of modification may be determined. The type of modification will be used to classify the change. In some embodiment, there may be more than one change and more than one classification. In one embodiment, classifying the modified/revised/updated version of the source code can include selecting from a list of activities. For example, one can pick whether the new code may be a bug or problem fix or a new feature to be added. In some embodiments, the change to the code may be detected automatically by the system (processor). In other embodiments, the code can be selected by the user or a code manager using a more obvious method such as a drop-down menu, a 3D tool or the like. In the latter case, it would be easier to detect the change.

The code change request for modification, when not obvious, may be detected using a variety of strategies. In one strategy, activities may be monitored, and change will be detected using natural language processing (NLP). For example, commit messages associated with the modified/revised version can be detected and processed. Another strategy may be to use abstract syntax tree (AST) and associated ontology to search for various revised versions and changes to the code. In one embodiment, a variety of different methods may be combined. The strategies provided are to ease understanding and these and/or other strategies can be used or combined in alternate embodiments.

In Step230, different unit tests are analyzed. A unit testing may be usually defined as a type of software testing where individual components of the software are tested. Many unit tests are done automatically to isolate sections of the code and verify its correctness and monitor proper functioning of the code section and procedure. Unit tests are established to allow the developers to learn functionality may be adequately provided and integrity may be ensured. Therefore, understanding the code change through classification (previous step), allows the review of original unit tests that were set up and available to understand the code change. For example, a modified unit test that may be performed according to a classification of the revised version of the source code indicates code change blocks to be later highlighted.

In Step240, the modified/updated code may be compared and analyzed against the original code. In one embodiment, a runtime execution of the original source code may be analyzed, against the modified version (revised) of the source code. In one embodiment, analyzing the run time execution of the original source code and the revised version identifies executional divergence between them.

In Step250, a narrative may be generated associated with the analysis of the run time execution of each of the original source code and the revised version of the source code. The narrative may also be augmented selectively. In one embodiment, the narrative can be generated by highlighting differences in run time execution of the unit test by each of the original source code and revised version. In one embodiment, a narrative file may be generated that may be associated with the revised version of the original source code in a code storage system. In one embodiment, the narrative file highlights the steps performed by the revised version of the source code.

In Step260, the narrative may be reviewed and any missing information that requires to be added may be provided for a final summary. This will be discussed in more detail in conjunction with the embodiment ofFIG.3.

FIG.3provides an alternate embodiment ofFIG.2. Some of the steps inFIG.2andFIG.3are very similar. However, the embodiment ofFIG.3provides additional details and some differences.FIG.4provides a scenario that can be used in conjunction with the processes ofFIG.2or3.

FIGS.3and4can be discussed together to provide a better understanding. The scenario ofFIG.4as illustrated by the block diagram, provide a source code contributor410and a source code manager420that can request code change and interact with the system/processor. For example, the source code contributor410can provide a pull request that includes instructions for code modification. The source code contributor may not always be the source of the pull request and in other embodiments, the request may be received/obtained from other sources.

In many conventional scenarios, once the request has been received, no matter the source, the maintainers of the code (may even be source code contributor410or manager420) may need to go through the effort of carefully understanding potential side-effects of the proposed code change. As discussed earlier, this effort may be so substantial that pencil and paper has to be often utilized to keep track of data structures modified by such a code change. Such efforts may be costly, time consuming and prone to interpretation errors. The process as provided byFIGS.2-4(such as captured in box450, by narrative generator module430and source code version control service440) addresses this problem. The solution partially shown illustratively by box450, eliminates the time and effort for reviewing and accepting source code change from contributors and possible negative side-effects of these code changes, including addition of bugs. Source code version control service440can be used for obtaining source code as appropriate.

The process300inFIG.3provides what may be provided in box450ofFIG.4. The process300starts with a request or a task to be provided. For example, referring back to the example discussed inFIG.4, a pull request may initiate the process. However, in alternate embodiments, as can be appreciated by those skilled in the art, other processes that will attribute to a code change (patch file etc.) can start the process. In one embodiment, there may also be one or more input (sets of unit tests and their input data, for example). The system which can simply be comprised of a processor or a multi device network having many processors will perform the process300. For ease of understanding, the process ca be discussed below in conjunction with the scenario provided inFIG.4.

In Step310, the code change request may be received or alternatively obtained. In one example, as was discussed inFIG.4, a source code contributor (410) may submit a pull request or a patch file to a source code version control service (FIG.4at440). The processor then analyzes monitors such events to detect anything that needs handling associated with the event in order to generate a narrative or alternatively a text execution story. In one embodiment, for example, a pull request test case can be used to examine the runtime variable values and the code execution flow according to the code changes. For instance, if a given parameter changes from a scalar to a vector, our solution can execute the code to inspect the newly created vector. For pull requests with only the source code and no tests, the system can check if any existing test (previous) uses the new code and proceed accordingly.

In Step320, the type of modification (code changes) are classified. There may be several types of code changes, including but not limited to(i) an addition of a new feature,(ii) an optimization in the software execution speed,(iii) support for a new type of input file, and(iv) bug fixes such as removal of memory leaks. In this step the proposed system can have different strategies to classify such changes.

Some of the strategies that may be employed will now be discussed. However, it should be understood that these strategies are only being discussed for ease of understanding and in alternate embodiments, other or a combination of these and other strategies may be employed.

A—Natural Language Processing (NLP) of the commit messages. In this instance, the source code contributor may have included a message about the code change which could be parsed to find verbs, nouns, and actions such as: “memory leak”, “improved memory consumption”, “added support for . . . ”, “optimized . . . ”, etc.

B—NLP on code comments introduced (or modified) by the code change may be used in the same manner (same processing steps) as discussed before.

C—Abstract Syntax Tree (AST) and ontology searches. This strategy includes data types, functions, and classes of elements touched by the source code change. These may be identified based on:(i) support for reflection in programming language,(ii) on the context in which they are used in the source code, and(iii) on the metadata available in ontologies regarding such elements. An example of classification using such strategy may be, for instance, a code change that replaces a machine learning algorithm “A” with another algorithm “B”—in which case the strategy could be classified as “algorithm replacement”.

D—Identification of similar labeled code. Assume the pull request code may be similar to a previously analyzed code, in this case, the system can assign the new label according to a predefined similarity metric. The metric can use the code AST similarity or even the programmer comments+NLP to calculate this index.

E—as an alternative embodiment, a drop-down list can be provided, in which the programmer labels the new contribution using a set of predefined values (i) new feature, (ii) bug fix, (iii) performance enhancements.

In Step330, the system selects and customizes unit tests. In this step, the processor/system, may identify existing unit tests that are affected by the code change (using, e.g., AST processing or simply comparing the execution trace of the unit test before and after the introduction of the code change). The subset of unit tests that relate to the code change are then modified so that their input (and output) are reduced to the bare minimum number of elements needed to demonstrate the code change in effect. For example, if a unit test had an array of input elements [1100,1101,11070.3,2100], then this step would execute the unit test up to 4 times (one for each element in the input array). Once an array with the minimum number of elements has been produced the system proceeds to the next step. Another example may be when a code change may be related to a problem fix due to memory leak. A test case based on an array could have the array increased by a factor (example 10 times) in order to more quickly demonstrate that the memory leak was fixed.

In addition, in one embodiment, most of the existing test cases generations may try to maximize the coverage of source code. Generally, the tests employ large datasets to exercise several parts of code (e.g., stress tests). This possibility may be further used to generate test cases that minimize test data in order to simplify the understanding of code changes.

In Step340, the run time differences between the original code and the changed code may be analyzed. Using tracing and monitoring techniques (such as “strace” and “valgrind”), the system collects logs as the original code executes with the reduced unit test produced by step330. Afterwards, it repeats the process with the new code changes applied to the code base. The system then proceeds to verify the point in execution time in which the two executions start to diverge (i.e., the point in which the new code modifies the behavior of the original one: changes to the range of values attributed to a given variable, a different path taken in the execution graph, disappearance or introduction of certain variables, etc.). The execution stack trace at that point may be captured and recorded for use by the next step. The user may also set up different system change explanation according to user's preferences. For instance, the user may want to explain each changed code part or set a threshold based on runtime/static change to identify parts of the code that must be highlighted.

In Step350, one or more test execution stories (alternatively narrative instructions) may be generated. In one embodiment, this may include associating blocks of operations that are performed by the software execution to the “subtitles” or “labels” produced as output of step320. This operation applies to the execution of both the original and new code versions. As the software executes and the program counter advances, our system detects where the code has changed and creates the associations with the “labels”.

In Step360, the missing text and information may also be generated. In one embodiment, there may be a plurality of databases (db) that can be accessed by the processor to obtain and store additional information for later use. Some of these databases are shown at390-394. The databases that are used here by way of example include Code changes classification db390, operations and test execution story (narratives) db391, Unit test db392/393and Source code db394. These can be used as supporting databases selectively depending on the selected embodiment. (In one embodiment Unit test db392/393comprises both unit tests and the strategies for unit test adaptation).

Referring back to Step360, the generation of the missing text may be made easier through the access to one or more of the databases shown. For example, the software operations and narratives database391may be used with the major operations performed by the software execution that could not be mapped by the previous step350(when translated into narrative actions.) For instance, a method call such as “vector.push_back(‘foo’)” may be translated into the sentence “System pushes an element into the vector”. Similarly, a sequence of such operations may be grouped and translated into “System pushes elements into the vector”. A narrative may also include auxiliary graphical instructions such as “Move box ‘foo’ towards the right-hand side of rectangle ‘vector’”. The system can use similar labeled code parts to identify use their labels in the current code change. For instance, assume a new commit without any code comment or commit message. In this case, the system can identify similar code in the operations and test execution story (narrative) db391and label the new code.

In step370, test execution story media files are generated. A media file such as a high-level document may be assembled by the system to summarize steps performed by the software up to the point of divergence in their execution, and to highlight (by the use of different colors, for example) the point where their execution changed. The system may also generate an animated document (such as a rich Video file with transition actions) that allow a graphical depiction of classes, objects, and operations that are touched by the unit test and affected by the new code change. The system may create as many narrative media files as candidate unit tests chosen by step330.

In Step380, test execution story media file in source code may be attached. In one embodiment, the user may be given the option to choose which media files to attach to the source code change versioning (such as a GitHub's Pull Request) as supplemental material. The reviewer of the pull request may be then presented with that narrative media file next to the actual code change. Other changes to the user interface may include a playback button so that the reviewer can watch a video preview of the changes prior to analyzing the actual code changes.

In addition, as indicated inFIG.3by301, Steps310and370-380, are more provided as a supporting function and can be added or subtracted from the process alternatively. In addition, the databases Change classification (390), and operation and test execution story (narratives) database (391) may be more needed in the regular operation while the source code database (394) may be used more on an as needed basis. The Unit test database (393) may be also more critical, but it may have some elements (original test cases392) that are more selectively needed.

In alternate embodiments, some other steps and elements may be added to the process selectively. For example, the commit type guides may be provided and selected as per a template. In addition, the type of execution and possible templates may be characterized based on commit messages. In one embodiment, when a user produces a modification that actually happens to have an impact in the Graphical User Interface (GUI) or there may be another visual change, a different template may be presented for that particular case based on a selective rendering. In another embodiment, templates of animations may be incorporated to facilitate assembling media files (Step370).

FIGS.5aand5bprovide complementary processes that provide additional details to the process discussed in conjunction withFIGS.3and4.

InFIG.5a, a particular scenario may be provided that commences with the receipt of a pull request at510(arrow1). The Code change classifier520analyzes the pull request and obtains other data such as the different versions of the code from the Code Change Classification database512. In this scenario, the code change classifier520has components that analyze the commit message522and the comments524and other types and functions526to classify the change requested.

Once the code change has been classified, the classification may be provided to the source code management platform530that can both receive input from the code change classifier520and provide input and information to it to help the classification and reclassification. This information can be stored and reused when new or similar code changes are requested to help with efficiency and speed of classification.

It should be noted that the source code management platform receives information from the source code database535. It may be also in communication with the Unit test selection and customization module540so that the tests can be selected and customized accordingly as discussed earlier. The Unit test selection module540can obtain information form the Unit test database545. The result may be provided to the Execution tracer550.

The interaction of Source Code management platform530and Execution tracer550may be also shown atFIG.5b. The input provided by the source code management platform530that may be provided to the Execution tracer550. The original code552and the modified code554are obtained and also input form the runtime difference checking module560. The result may be provided by association of blocks of operation that need to be highlighted due to code change as shown at570. Input from the operation and text execution db at555may be also provided. In one embodiment this may be accomplished by labels, graphs or other means that provide such highlighting. Thereafter a test execution story may be provided through a media file at590.

With the advent of technology, immersive application platforms can be used to create an extended reality rendered view of a live graphical model of a codebase for display to make the code testing less complex. This may be especially useful when an extended reality device or environment may be used. In one embodiment, a particular immersive application platform may generate the live graphical model based on test data generated by the test, may modify the live graphical model and/or the extended reality rendered view of the live graphical model as the test may be executed on the codebase. This allows the immersive application platform to enable the user/programmers comprehend the structure of the software and how it may be exercised by the test via a more intuitive medium. The visualization may also allow the user/programmer to locate hotspots of the codebase and understand the execution times of the one or more elements, inoperable code included in the codebase, unreachable code included in the codebase, infinite loops in the codebase, and redundant code included in the codebase. Therefore, the media file can include any type of video or extended reality type files that enable this as well.

The media files may be also provided in turn to source code management platform for use and storage and historical use. The information that may be no longer needed can be de-allocated from memory at580and removed from view at582. Similarly, a push element vector may be provided at485that can move box “e” to the side of the rectangle vector at587.