INTELLIGENT TEST CASE MANAGEMENT IN COMPUTER SOFTWARE APPLICATIONS

A computing device for automatic management of test cases for testing of source code associated with computer software applications. The computing device accesses an original corpus of source code. One or more previously-generated test cases associated with the original corpus of source code are accessed. An updated corpus of source code is accessed, the updated corpus of source code an update from the original corpus of source code. The computing device generates automatically one or more new test cases for testing of the updated corpus of source code, the one or more new test cases automatically generated based upon the accessed one or more previously-generated test cases. The updated corpus of source code is compiled. The compiled updated corpus of source code is executed with the one or more new test cases.

FIELD OF THE INVENTION

The present invention relates generally to software development and more particularly to intelligent test case generation and management in computer software applications.

BACKGROUND

The present embodiments relate to test case management in computer software applications. As the software industry continues to grow in total value and the relative value of software to the public at large increases as well, the volume and complexity of software development projects is also dramatically increasing. Correspondingly, an increased need presents itself to test and confirm that these software development projects are functioning correctly for utilization by the public-at-large, or in other ways. This requires an increasing amount of effort and manhours spent. As any developer knows, new software rollouts also require a large amount of testing to ensure that the software is functioning as it should, without errors, bugs, crashes, memory management issues, or other problems. Resolving these issues requires a significant amount of manpower to be expended, particularly in the case of complex software applications. Typically, test cases are used by developers to allow testing of computer software applications, including the output of the entirety of an application, or merely the testing of newly deployed objects, classes, interfaces, libraries, etc. In sum, all testing, though time-consuming, must be performed in order to confirm the software is functioning as it should be. Since these new software development projects have become so complex and numerous, a need presents itself for an improved manner of testing software in a streamlined and expedited manner.

SUMMARY

Embodiments of the present invention disclose a method, system, and computer program product for automatic management of test cases for testing of source code associated with computer software applications. A computing device accesses an original corpus of source code. The computing device accesses one or more previously-generated test cases associated with the original corpus of source code. The computing device accesses an updated corpus of source code, the updated corpus of source code an update from the original corpus of source code. The computing device automatically generates one or more new tests cases for testing of the updated corpus of source code, the one or more new test cases automatically generated based upon the accessed one or more previously-generated test cases. The computing device compiles the updated corpus of source code. the computing device executed the compiled updated corpus of source code for utilization with one or more new test cases.

DETAILED DESCRIPTION

Test case management, as widely known by developers at large, is an important part of software development. Rigorous testing of inputs, outputs, environmental settings, system settings, memory addresses, etc. associated with computer software can be a demanding, time-consuming, and tedious task for developers, even if only a small portion of the computer software is being tested at any one time. Software development of valuable and complicated software applications can be a very substantial undertaking, considering situations such as when there may be millions of lines of source code in a computer software application, any of which can be prone to a typographical error. This process, as one of skill in the art understands, may be very prone to mistakes being made since one typographical error (for example, misspelling of terms within a call for an object), can cause the entire corpus of source code to not compile, throw exceptions, etc., or even worse compile, but not work for its intended purpose and provide bad results, etc. Considering the current role of computer software in, for example, airline flight management, the power grid, and banking, this can lead to real-world glitches which could, for example, lead to the delay of airline flights all over the world, power outages, or unavailability of credit card processing for an entire region, because of simple typographical errors.

Embodiments of the invention are directed towards one or more methods, systems, and computer program products for automatic management of test cases associated with testing of source code associated with computer software applications. The presently disclosed embodiments may function as a part of an integrated development environment, as a plug-in to an integrated development environment, function independently (via a separately executing computer application, etc.), or in some other manner while being contemplated by embodiments of the invention disclosed herein.

PROCESSOR SET110includes one, or more, computer processors of any type now known or to be developed in the future. Processor set110may be alternatively be referred to herein as one or more “computing device(s),” but computing devices may also refer to one or more CPUs, microchips, integrated circuits, embedded systems, or the equivalent, presently existing or after-arising. Processing circuitry120may be distributed over multiple packages, for example, multiple, coordinated integrated circuit chips. Processing circuitry120may implement multiple processor threads and/or multiple processor cores. Cache121is memory that is located in the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores running on processor set110. Cache memories are typically organized into multiple levels depending upon relative proximity to the processing circuitry. Alternatively, some, or all, of the cache for the processor set may be located “off chip.” In some computing environments, processor set110may be designed for working with qubits and performing quantum computing.

FIG.2is a functional block diagram illustrating modules200for automatic test case management for testing of source code associated with computer software applications, in accordance with an embodiment of the present invention. In an exemplary embodiment as displayed inFIG.2, source code repository210is operatively connected to interactive development environment240, directly or via network270. Source code repository210may be any sort of computer software (and, in various embodiments, associated computer hardware) for storing, accessing, and/or communicating one or more versions, bodies, or other corpuses of source code for utilization as discussed herein. While still contemplated as within the scope of the invention, interactive development environment240as shown inFIG.2may be replaced or supplemented with any sort of computing environment capable of automatically managing test cases associated with a corpus of source code. Source code stored in source code repository210may be in the form Java®, C++, Basic, JavaScript®, or any other presently existing or after-arising programming language that would benefit from embodiments of the presently disclosed invention. As discussed further herein, different versions of source code from source code repository210may be checked-out, read, or otherwise accessed by interactive development environment240for further utilization in connection with embodiments discussed herein. Different versions of source code available from source code repository210may be an original corpus of source code, as well as one or more updates, modifications, new versions, etc. to the original corpus of source code. As different versions of source code from source code repository210are modified by developers or in other ways, the present invention presents a streamlined way of automatically generating and managing test cases for the new versions of source code stored by the source code repository210. Even a small, seemingly inconsequential change made to source code and reflected in an “updated corpus of source code” (for example, changes to source code for one object in millions of lines of source code), may require generation of a large number of new test cases for complete and thorough testing of an important product.

In various embodiments of the invention, interactive development environment240in conjunction with accessing an original corpus of source code available from source code repository210, also accesses one or more previously-generated test cases that are associated with the original corpus of source code available from source code repository210, as well as, in other embodiments of the invention, verification events, and/or associated environmental data. As would be understood by one of skill in the art, “test cases” refers to one or more of inputs, execution conditions, testing procedures, and expected results that are related to one or more tests to be performed in connection with testing of a software application (or, in effect, compiled/interpreted/scripted source code as maintained by source code repository210). Test cases may be directed towards one portion of a corpus of source code (such as an object or an interface, etc.), or an entire corpus of source code. Also as discussed further herein, the one or more previously-generated test cases may be generated manually by a developer (or developers), auto-generated, pre-configured, created by previous iterations of embodiments of the invention, or created in any other way. Test cases, verification events, environmental data, etc. may be stored directly by source code repository210in connection with corpuses of source code, stored by interactive development environment240, or stored in another way. Interactive development environment240then accesses an updated, modified, versioned, etc. updated corpus of source code also available from the source code repository210. Interactive development environment240then, as further discussed herein, then automatically generates one or more new test cases for testing of the updated corpus of source code. The one or more new test cases, in various embodiments of the invention, are automatically generated based upon the accessed one or more previously-generated test cases, as further discussed herein.

Interactive development environment240then, in various embodiments of the invention, proceeds to test the updated corpus of source code with the automatically generated one or more new test cases. In embodiments where the updated corpus of source code must be compiled before execution, interactive development environment240may compile and execute the updated corpus of source code, then run a debugger sequence in the compiled corpus of source code using the one or more new test cases. In embodiments where the updated corpus of source code is written in a scripted or interpreted programming language, interactive development environment240may execute the scripted/interpreted language with the one or more new test cases. The results of execution/interpretation/etc. of the updated corpus of source code executed with the one or more new test cases are recorded, and, in various embodiments, a report is automatically created, the report including results of execution of the compiled/scripted/interpreted/etc. updated corpus of source code utilizing the one or more new test cases. Contents of the report generated by interactive development environment240are further discussed herein.

As further displayed inFIG.2, in various embodiments of the invention, source code repository210and interactive development environment240are connected to and via network270. In various embodiments, network270represents, for example, any sort of computer network such as a local area network (LAN), a wide area network (WAN) such as the Internet, and includes wired, wireless, or fiber optic connections. In various embodiments, network270is substantially the same as WAN102, discussed in connection withFIG.1herein. In general, network270may be any combination of connections and protocols that will support communications between source code repository210and interactive development environment240, in accordance with embodiments of the invention. In further embodiments of the invention, network270may represent an internal bus associated with a single or multicore processor executing both computer messaging software210and computer messaging software analyzer240(such as in an embodiments where source code repository210and interactive development environment240are integrated).

Discussing elements displayed inFIG.2in further detail, source code repository210represents computer software (and, in various embodiments, associated computer hardware) for storing, accessing, and/or communicating one or more versions, bodies, or other corpuses of source code for utilization as discussed herein. In various embodiments of the invention, source code repository210includes one or more of original source code module212and/or updated source code module214. In various embodiments of the invention, source code repository210is integrated as part of interactive development environment240, stand-alone software, or separated by network270.

Original source code module212represents software and/or hardware for storage, maintenance, checking-in, checking-out, etc. of an original corpus of source code (“original” as versus the “updated” source code stored in updated source code module214, as further discussed herein). As discussed elsewhere herein, source code stored by original source code module212may be in the form of Java®, C++, Basic, JavaScript®, or any other presently existing or after-arising programming language that would benefit from embodiments of the presently disclosed invention. Source code stored by original source code module212and updated source code module214may be any type of source code, presently existing or after-arising, and may contain, for example, source code for an application, library, object, interface, etc., a sandbox source code change, log file(s), a demo(s), and/or one or more environmental settings, any of which may be included in the source code or stored independently. Original source code module212makes available source code for interactive development environment240. Source code stored by original source code module212may be dynamically edited, compiled, interpreted, scripted, executed with test cases, etc. as discussed further herein. Test cases associated with testing of original corpus of source code may be stored by original source code module212, stored in connection with interactive development environment240(as further discussed) or elsewhere. Test cases stored in original source code module212may also include, for example, a range of possible inputs for inputting into compiled/interpreted/scripted source code, inputs for objects/classes/interfaces/etc.

associated with the corpus of source code, hypothetical mouse movements (or other graphical user interface interactions, one or more key test scripts used in testing of the corpus of source code (as well as one or more mappings mapping the one or more key test scripts to one or more features of the original corpus of source code), or anything of the like, as one of skill in the art would understand. Test cases are utilized for automatic generation of new tests cases, as further discussed herein.

Updated source code module214represents software and/or hardware for storage, maintenance, checking-in, checking-out, etc. of an updated, modified, versioned, etc. corpus of source code reflecting updates, changes, and/or edits to the “original” corpus of source code stored in original source code module212. As would be understood by one of skill in the art, application and software development frequently requires a large number of edits, changes, additions, subtractions, etc. to corpuses of source code before a project is completed, leading to a number of different versions of source code which would be stored by updated source code module214, and an “original” corpus of source code stored by original source code module212may be an “updated” corpus of source code from a previous iteration of the invention. Even after roll-out, new versions may be implemented. As with original source code module212, updated source code module214may be dynamically edited compiled, interpreted, scripted, executed, etc. with new test cases. Updated corpuses of source code accessed by interactive development environment240are tested with automatically generated new test cases, in various embodiments of the invention, as further discussed. In various embodiments of the invention, updated source code module214may be integrated with original source code module212, stored in connection with interactive development environment240, stored in connection with independently functioning software, etc.

Continuing with regard toFIG.2, interactive development environment240includes, in various embodiments of the invention, source code access module243, code editor245, debugging module247, compiler/interpreter249, and report generator253. As discussed elsewhere herein, interactive development environment240may be replaced or supplemented with any sort of computing environment capable of automatically managing test cases associated with a corpus of source code, with functionality as discussed herein.

Source code access module243represents software and/or hardware for accessing of corpuses of source code from source code repository210. The corpuses of source code can be checked-out, read, or otherwise accessed from original source code module212and/or updated source code module214. In various embodiments of the invention, after source code access module243accesses an original corpus of source code from original source code module212, source code access module242accesses an updated corpus of source code from updated source code module214. Corpuses of source code checked-out of source code access module243can, in various embodiments of the invention, be edited by one or more developers using code editor245, debugged by debugging module247, compiled/interpreted/scripted by compiler/interpreter249, and having relevant reports generated for user(s) by report generator253(the reports including, in various embodiments, results of execution, errors associated with execution, etc., as further discussed herein). After access of any relevant corpuses of source code by source code access module243from source code repository210, such source code becomes available to a developer in interactive development environment240, for viewing, editing, modifying, compiling and/or debugging source code.

Code editor245represents software and/or hardware for editing of various source code accessed, checked-out, read, etc. from source code repository210. As one of skill in the art understands, since source code is typically written in text, code editor245provides various facilities for developers to view and edit syntax of source code, and frequently provides functionality to highlight, comment, autocomplete, indent, request debugging (such as by debugging module247), request compilation or execution (such as by compiler/interpreter249), etc., in order to facilitate one or more developers to efficiently develop software. In various embodiments of the invention, code editor245may be a standalone application or built into integrated development environment240. Code editor245, in various embodiments of the invention, may access original source code module212in order for one or more developers to edit the original corpus of source code in order to generate a modified corpus of source code, the modified corpus of source code to be tested by debugging module247, as further discussed herein. As one of skill in the art understands, in industry-standard professional development environments, corpuses of source code are typically checked-out when a developer works on the source code with code editor245then checked-in when the work is done (in order to maintain copies of source code, determine ownership of code, who made certain changes to the source code, etc.).

Debugging module247represents software and/or hardware for automatically debugging corpuses of source code accessed, checked-out, etc. by interactive development environment240. Debugging module247may be implemented as a plug-in to interactive development environment240, as a stand-alone application, or in another manner. In various embodiments of the invention, debugging module247serves to automatically generate one or more new test cases for testing of the updated corpus of source code retrieved from updated source code module214. As discussed herein, in order to automatically generate one or more new tests cases, invention debugging module247accesses stored previously-generated test cases associated with an original corpus of source code (or other versions, roll-outs, etc. of source code). As each version, etc. of source code is rolled out by developers, and stored by source code repository210, one or more previously-generated test cases associated with each version of source code is stored by debugging module247(or becomes available to debugging module247in some way).

Previously-generated test cases accessed by debugging module247are associated with any corpus of source code previously available, and may be generated manually by a developer (or developers), auto-generated (such was with previous iterations of embodiments of the invention), pre-configured, or created in any other way. As mentioned herein, previously generated test cases may include by non-limiting example, a range of possible inputs for inputting into compiled/interpreted/scripted source code, inputs for objects/classes/interfaces/etc. associated with source code, hypothetical mouse movements (or other graphical user interface interactions), one or more key test scripts used in testing of the corpus of source code (including mappings mapping the one or more key test scripts to one or more features of the original corpus of source code), or the equivalent presently existing or after-arising, any of which may be used for testing corpuses of source code. Debugging module247may store and access such previously-generated test cases directly, store test cases in source code repository210, or test cases may be stored elsewhere in modules200. After access by debugging module247of previously-generated test cases, debugging module247automatically generates one or more new test cases for testing of the updated corpus of source code available from updated source code module214, based upon the accessed previously generated test cases.

In various embodiments of the invention, one or more machine learning models associated with debugging module247are involved in the generation of one or more new tests cases for testing of the updated corpus of source code. The machine learning models may rely, for example on a neural network to provide necessary functionality, or any other machine learning technique as is commonly understood by one of skill in the art. Other machine learning techniques specifically contemplated include long-short term memory, a multilayer perceptron, or a type of feed-forward neural network. Debugging module247acts in conjunction with compiler/interpreter249to thoroughly test corpuses of source code, as further discussed below.

In embodiments of the invention where machine learning models are utilized, machine learning models may be trained in a supervised, unsupervised, or semi-supervised manner with previously-generated test cases known to provide thorough and full testing of corpuses of source code as well as the results of execution of previously compiled/interpreted/scripted corpuses of source code. In situations where the previously generated test cases are associated with a range of possible inputs for inputting into compiled/interpreted/scripted source code, machine learning models may be trained to recognize both the various of possible inputs associated with previous corpuses of source code, as well as changes made to previous corpuses of source code, where test cases are input into, and the expected results. So, the machine learning models trained in such a way will be capable of testing an updated corpus of source code in a similar manner, but in an automated fashion and can provide results without a necessity of a human manually creating test cases. Similarly, in situations where graphical user interface interactions or key test scripts are used in testing previous corpuses of source code, historical graphical user interface interactions or historical key test scripts, as well as the previous corpuses of source code themselves, the results of testing, etc. are used in training machine learning models for testing of new corpuses of source code. Previous and current environmental settings may be accessed as well, in any embodiment of the invention, in generated new test cases, and the past or current environmental settings utilized in generating new test cases. After-arising means of testing of source code with test cases are specifically contemplated as being within the scope of the invention.

Compiler/interpreter249represents software and/or hardware for compiling, interpreting, scripting, etc. corpuses of source code provide by source code repository210. The specific action performed by compiler/interpreter249depends on the type of programming language being considered, as would be understood by one of skill in the art, and depends on whether a language is a scripting language, interpreted language, compiled language, etc. In various embodiments of the invention, after one or more new test cases are generated automatically by debugging module247(with actions as discussed elsewhere herein), compiler/interpreter249executes a compiled/scripted/interpreted/etc. version of the updated corpus of source code (from updated source code module214) with generated one or more new test cases to thoroughly test the updated corpus of source code.

Compiler/interpreter249therefore acts in conjunction with debugging module247to execute compiled/interpreted/scripted/etc. versions of the updated corpus of source code with the one or more new test cases. Generally, compiler/interpreter249and debugger247work in conjunction to assess whether the updated corpus of source code executes correctly, for its intended purpose, and provides the correct results with the generated one or more new test cases. In situations where the updated corpus of source code is compiled, for example, compiler/interpreter249and debugger247confirm that the corpus of source code is compiling correctly or refuses to compile, and records errors generated. In situations where the updated corpus of source is compiled and compiled correctly, compiler/interpreter249and debugger247confirms that an automated execution of the updated corpus of source code with the generated one or more new test cases is providing correct results (such, for example, where the updated corpus of source code is an object or an interface, confirming that the output of the object/interface is correct). In various embodiments of the invention, results of execution are utilized by report generator253to generate a report for one or more users to assess results of execution.

In various embodiments of the invention, exactly how compiler/interpreter249performs the testing depends on the nature of the generated one or more new test cases. In situations where the generated one or more new test cases are a range of inputs to be entered into the updated corpus of source code, compiler/interpreter249simply executes the source code in an appropriate manner with the inputs in place, and determines whether the program didn't compile, execute, crashed, and the errors or results generated. In embodiments where the one or more new test cases simulate graphical user interface interactions or key test scripts, the results of interactions or key test scripts are similarly recorded for errors, crashing, incorrect results, etc.

Report generator253represents software and/or hardware for automated generation of reports associated with results of execution of updated corpuses of source code, in embodiments of the invention. Report generator253, in the various embodiments of the invention, may be a standard part of interactive development environment240, an optional plug-in for interactive development environment240, or a stand-alone application in communication with interactive development environment240over network270. Report generator253provides specialized reports for developers associated with the updated corpus of source code, users, project managers, etc., to assist them in further debugging of the updated corpus of source code, with the content of the report differing in various embodiments of the invention. In embodiments of the invention report generator253may include a text (such as a logfile) or graphical user interface display providing details such as errors during compilation, results of execution of the updated corpus of source code with the one or more new test cases, associated environmental settings, memory utilization, cpu processor time, etc. The specialized reports created by report generator253may be utilized by developers, users, project managers, etc. in improving new versions of source code, or simply correcting errors in the current version. Specialized reports may simply take the form of a .txt file, another file format, an automated e-mail, a pop-up appearing on the screen of code-editor245, or the equivalent. In further embodiments of the invention, specialized reports generated by report generator253may include a “how-to” document indicating to viewers how to correct potential compilation or run-time errors, which could automatically scroll the point to the corpus of source code, and even suggest code edits which will correct errors (such as via utilization of accessing a director of similar errors, a natural language model or generative neural network). In embodiments where no errors are present during compilation or run-time, the specialized report may contain no potential errors and simply indicate “no errors found.”

FIGS.3A and3Bare a flowchart depicting operational steps that a hardware component, multiple hardware components, and/or a hardware appliance may execute, in accordance with an embodiment of the invention. As shown inFIG.3A, at step310a computing device accesses an original corpus of source code available from original source code module212of source code repository210. At step320, the computing device accesses one or more previously-generated test cases associated with the original corpus of course code, that, in an embodiment of the invention, are stored by source code repository210(in other embodiments of the invention, the one or more previously-generated test cases are stored elsewhere, such as directly by interactive development environment240). At step330, the computing device accesses an updated corpus of source code, the updated corpus of source code available from updated source code module214. The update, new version, etc. has other modifications from the original corpus of source code, as discussed elsewhere herein. At step340, the computing device generates one or more new test cases for testing of the update corpus of source code. In various embodiments of the invention, this step is performed by debugging module247. Generation of the one or more new test cases is discussed more extensively elsewhere herein, such as in connection withFIG.2.

Continuing inFIG.3B, at step350the computing device (specifically, in various embodiments of the invention, compiler/interpreter249) compiles the updated corpus of source code. As discussed elsewhere herein, in other embodiments of the invention, the updated corpus of source code may be interpreted or scripted, and execution proceeds in accordance with At step370, the computing device (specifically, compiler/interpreter249in conjunction with debugging module247) executes the updated corpus of source code for utilization with the one or more new test cases. At step390, in various embodiments of the invention, computing device automatically generates a report including results of execution of the updated corpus of source code, such as if any errors have been reported. The report may contain a how-to document on how to fix potential errors in the source code.

Based on the foregoing, a method, system, and computer program product have been disclosed. However, numerous modifications and substitutions can be made without deviating from the scope of the present invention. Therefore, the present invention has been disclosed by way of example and not limitation.