Enhanced data loading for test management tool

A computer automated method to update a data store is disclosed. The method comprises receiving a definition mapping one or more information fields in an upload file to fields in a data store, creating a first linkage between a test set identified in a first record in the upload file and a first test case based on the mapping, and creating a test set record in the data store including the first linkage and data in the information fields in the first record in the upload file based on the definition.

CROSS-REFERENCE TO RELATED APPLICATIONS

Not applicable.

REFERENCE TO A MICROFICHE APPENDIX

Not applicable.

FIELD OF THE INVENTION

The present disclosure is directed to computer based test management tools, and more particularly, but not by way of limitation, to enhanced data loading for a test management tool.

BACKGROUND OF THE INVENTION

Testing large software systems is a complicated process. Testing may include conducting long sequences of individual steps in particular orders. Testing a large software system may involve a team of testers who conduct testing over weeks or months. When testing newly added functionality, it may be desirable to be able to repeat the original testing of previously existing functionality to assure that the newly added functionality has not damaged the previously existing functionality. Maintaining a document or other artifact that can be used to guide testing may promote the repeatability of tests. Testing documents can also be used to record testing progress, for example checking off or otherwise designating tests as completed, and to generate reports for management.

The Mercury TestDirector™ is a software product that supports defining tests and requirements, tracking testing progress, and generating reports on testing activities. The TestDirector™ provides several different functionalities related to testing. In one operating environment, TestDirector™ provides a graphical user interface (GUI) that allows the user to examine and define requirements through a requirements module, to examine and define test cases through a test case module, and to examine and define test sets through a test lab module. The test case may be the smallest quantum of test activity represented in the TestDirector™. An example of a test case is logging into a customer account creation application. A test case, when executed, may involve multiple steps, for example providing a user name and a password, but this additional level of detail may not be captured in the TestDirector™ tool. A test set may comprise one or more test cases. An example of a test set is executing the test case of logging into a customer account creation application, executing the test case of providing customer information, executing the test case of confirming customer information, and executing the test case of completing customer account creation. A test set is defined within the TestDirector™ by creating links to a series of test cases.

SUMMARY OF THE INVENTION

According to one embodiment, computer automated method to update a data store is disclosed. The method comprises receiving a definition mapping one or more information fields in an upload file to fields in a data store, creating a first linkage between a test set identified in a first record in the upload file and a first test case based on the mapping, and creating a test set record in the data store including the first linkage and data in the information fields in the first record in the upload file based on the definition.

In another embodiment, an automated test documentation system is provided. The system comprises a data store containing information related to test sets and test cases, a test management tool using the information to record testing progress and to generate testing reports, a spreadsheet application operable to define and store test set definitions, and a script operable to process the test set definitions to create linkages between the test set and the test cases and to create entries in the data store, each entry containing one of the linkages between the test set and the test cases.

In still other embodiments, a method to build test sets is provided. The method comprises creating, based on an update file, a plurality of records in a data store, each record defining a linkage between a first test set and one of a plurality of test cases, displaying the linkage between the first test set and one or more test cases using a test management tool, creating a revised update file based on the update file, creating, based on the revised update file, a plurality of records in the data store, each record defining a linkage between a second test set and one of the plurality of test cases, and displaying the linkage between the second test set and one or more test cases using the test management tool.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

It should be understood at the outset that although an exemplary implementation of one embodiment of the present disclosure is illustrated below, the present system may be implemented using any number of techniques, whether currently known or in existence. The present disclosure should in no way be limited to the exemplary implementations, drawings, and techniques illustrated below, including the exemplary design and implementation illustrated and described herein.

The Mercury TestDirector™ supports limited uploading of data from Microsoft Excel™ files. Requirements may be uploaded from a first Microsoft Excel™ file to be used by the requirements module. Test cases may be uploaded from a second Microsoft Excel™ file to be used by the test case module. Mercury TestDirector™, however, does not provide support for uploading test sets from Microsoft Excel™ files. It has been suggested that it was not possible to extend the Mercury TestDirector™ to enable automated creation of test sets in the Test Lab module. Manual creation of test sets using a GUI to point and click to select each test case to link to a test set and to define data content may be a tedious, time consuming, and error prone activity when large suites of test sets are involved. The present disclosure contains a description of a script that may be executed from a spreadsheet drop down add-in menu selection. The script promotes processing the spreadsheet contents and dialogues with a user to automatically generate test sets. This enhanced data loading mechanism may reduce test set definition errors and may reduce job completion time. In an embodiment, the enhanced data loading mechanism has been found to reduce test set definition time from about 10 hours to about 3 hours.

Turning now toFIG. 1, a block diagram of a system100for test management according to an embodiment of the present disclosure is depicted. The system100includes a test management tool102in communication with a first data store104, a spreadsheet tool106in communication with a second data store108and with the test management tool102, and a data load script110in communication with the spreadsheet tool106and the test management tool102. The test management tool102, the spreadsheet tool106, and the data load script110are computer programs or scripts which may execute on a general purpose computer system. General purpose computer systems are discussed in greater detail hereinafter.

The first data store104maintains information related to one or more test documents, for example information used to define test requirements, test cases, and test sets. In an embodiment, the first data store104is an Oracle database, but in other embodiments other database or data store tools may be employed. The second data store108maintains spreadsheet information and may be one or more files in a file system or a database. In an embodiment, the test management tool102is Mercury TestDirector™. In an embodiment, the spreadsheet tool106is Microsoft Excel™.

The data load script110, which may also be referred to as a macro, accesses test set information contained in the spreadsheet tool106that defines one or more test sets, each test set defined as one or more test cases. The test set information may be defined by a user employing the spreadsheet tool106to create a spreadsheet file. The spreadsheet file may define a test set as an ordered series of rows each identifying one test case. A row may include an identity of the test set, an identity of the test case, and optionally other information associated with the particular instance of the test case, for example distinguishing data values. The spreadsheet file may define a plurality of test sets. The data load script110may be implemented as a VISUAL BASIC script.

The data load script110may be executed by selecting a menu button provided in an interface to the spreadsheet tool106. In an embodiment, the data load script110is executed by a menu selection in an Add-in portion of the Microsoft Excel™ GUI. The data load script110obtains a mapping definition from a user that the data load script110employs to associate or map entries in specific columns of the spreadsheet file to specific fields in the test set definitions. The data load script110may access the test set information by communicating with the spreadsheet tool106or may access the second data store108containing the spreadsheet file directly. The data load script110processes the test set information and communicates with the test management tool102to create definitions of test sets in the context of the test management tool102. The test set definitions may comprise a series of row entries in a date base table wherein each row establishes a linkage between the test set and one test case. The test set is then defined as the ordered sequence of test cases linked with that test set by the date base table rows. The test cases are identified and defined in a separate portion of the data store104, for example in one or more test case module data tables. The data load script110verifies that the test case identified in the date base table row entry exists in the first data store104. When the test case fails to verify, the data load script110creates an entry in an error report, for example an Exception Report, and proceeds to access more test set information.

The test management tool102may save or store the test set definitions in the first data store104. The Mercury TestDirector™ stores the test set definitions in association with a test lab module. The data load script110may communicate with the test management tool102by invoking methods or function calls on one or more application programming interfaces provided by the test management tool102. In an embodiment, the data load script110may invoke methods or functions associated with Mercury TestDirector™ application programming interfaces including TSTestFactory, TestFactory, TestSetTreeManager, TestSetFactory, TestSetFolder, Customization, Customization Lists, and CustomizationUsers. Alternatively, in an embodiment, the data load script110may directly access the first data store104to write the test set definitions into the appropriate data tables or other structures.

In an embodiment, the data load script110is further operable to define linkages between requirements and test cases. These linkages may be stored in the first data store104in a requirements-to-test case mapping data table. Alternatively, these linkages may be stored in the first data store104associated with either the requirements module or the test cases module. The data load script110enables linkage of one requirement to many test cases, and/or linkage of many test cases to one requirement. For example, the present system enables one requirement to be mapped to multiple test cases, as well as for several test cases to be mapped to a single requirement. This provides added flexibility over one-to-one mapping of test requirements to test cases that may be useful for testing. In an embodiment, the data load script110is further operable to define linkages between requirements and test sets. These linkages may be stored in the first data store104in a requirements-to-test set mapping data table. Again, the data load script110enables linkage of one requirement to many test sets, and/or linkage of many test sets to one requirement. Alternatively, these linkages may be stored in the first data store104associated with either the requirements module or the test lab module.

The test management tool102may process requirements-to-test set linkages to generate a report of requirements satisfied by each test set and process requirements-to-test case linkages to generate a report of requirements satisfied by each test case. Alternatively, in an embodiment that does not define requirements-to-test set linkages, the test management tool102may generate a report of requirements satisfied by each test set by processing the requirements-to-test case linkages and associating requirements back to test sets using the test case to test set linkages.

Turning now toFIG. 2A, a first portion of a method for performing enhanced data loading is depicted. The method may be implemented by the data load script110. In an embodiment, a test lab spreadsheet150, for example a Microsoft Excel™ spreadsheet, provides input to the data load script110. The process begins at block170where the data load script110is executed and the user logs into the test management tool102, for example the Mercury TestDirector™. The process proceeds to block172where the authority of the user to upload the test lab spreadsheet150to the test management tool102is validated by executing a method or function call using an application programming interface provided by the test management tool102, for example the Customization application programming interface in the Mercury TestDirector™.

The process proceeds to block174where an appropriate operating mode or module of the test management tool102is selected, for example the Test Lab module of Mercury TestDirector™. The process proceeds to block176/178where the user defines a mapping from the columns of the test lab spreadsheet150to the fields of row entries in a data table in the first data store104using an application programming interface provided by the test management tool102, for example the Customization application programming interface of the Mercury TestDirector™. The process proceeds to block180where the mapping provided in block176/178is validated using an application programming interface provided by the test management tool102, for example the Customization application programming interface180aand CustomizationList application programming interface180bof the Mercury TestDirector™.

The process proceeds to block182where, if an error in the mapping was found in block180, which may be referred to as a terminal error, the process proceeds to block184where an Scan Error Report or other error report is generated and the process terminates. In block182, if no error in the mapping occurred, the process proceeds to block200depicted inFIG. 2B.

Turning now toFIG. 2B, a second portion of a method for performing enhanced data loading is depicted. The method resumes at block200where a first row of the test lab spreadsheet150is selected. The process proceeds to block202where, if the folder path identified in the selected row of the test lab spreadsheet150is not verified, meaning the folder path is changed or does not exist, the process proceeds to block204. In block204the validity of the folder path is determined using an application programming interface provided by the test management tool102, for example the TestSetTreeManager application programming interface204aprovided by Mercury TestDirector™. The process proceeds to block206where if the folder path does not exist, the process proceeds to block207where the folder path is created using an application programming interface provided by the test management tool102, for example the TestSetTreeManager application programming interface provided by Mercury TestDirector™.

The process proceeds to block208where the validity and/or existence of the test set identified in the row of the test lab spreadsheet150is verified using an application programming interface provided by the test management tool102, for example the TestSetFactory application programming interface208aprovided by Mercury TestDirector™. The process proceeds to block210where, if the test set does not exist, the process proceeds to block212where the test set is created using an application programming interface provided by the test management tool102, for example the TestSetFactory application programming interface provided by the Mercury TestDirector™.

The process proceeds to block214where the test case is validated using an application programming interface provided by the test management tool102, for example the TSTestFactory application programming interface214aof Mercury TestDirector™. The process proceeds to block250inFIG. 2C.

Turning now toFIG. 2C, a third and final portion of the method for performing enhanced data loading is depicted. In block250, if the test case does exist, the process proceeds to block252where the row containing linkage information is created in the first data store104using an application programming interface provided by the test management tool102, for example the TSTestFactory application programming interface214aprovided by Mercury TestDirector™. The process proceeds to block254where the remainder of the row of information from the test lab spreadsheet150is processed and written into the row in the first data store104using an application programming interface provided by the test management tool102, for example the TSTestFactory application programming interface214aprovided by Mercury TestDirector™. In block250, if the row does not exist, the process proceeds to block256where a record is created in an Exception Report.

The process proceeds to block258where the next row in the test lab spreadsheet150is accessed. The process proceeds to block260where, if all rows in the test lab spreadsheet150have not been processed, for example if the row read in block258is a valid row, the process proceeds to block262onFIG. 2B, repeating the row processing described above. If all the rows in the test lab spreadsheet150have been processed, the process proceeds to block264where, if no records were generated in the Exception Report, the process proceeds to block268where a final report is written and the process terminates. If one or more records were generated in the Exception Report, the process proceeds to block266where the Exception Report is written and the process terminates. On the successful completion of the method or process depicted inFIGS. 2A,2B, and2C test set definitions have been uploaded from the test lab spreadsheet150and the test management tool102has defined one or more test sets which may be examined, updated with status or progress information, and used as the basis of generating testing reports.

A process for updating existing test lab records stored in the first data store104is substantially similar to the process depicted inFIGS. 2A,2B, and2C, with the following differences. In block204, if the folder path does not exist, a record is created in the Exception Report and the process proceeds to block258. In block210, if the test set does not exist, a record is created in the Exception Report and the process proceeds to block258. In block250, if the test case exists, the process updates the row information in the first data store104and then proceeds to block258.

The system100described above may be implemented on any general-purpose computer with sufficient processing power, memory resources, and network throughput capability to handle the necessary workload placed upon it.FIG. 3illustrates a typical, general-purpose computer system suitable for implementing one or more embodiments disclosed herein. The computer system380includes a processor382(which may be referred to as a central processor unit or CPU) that is in communication with memory devices including computer readable media such as secondary storage384, read only memory (ROM)386, random access memory (RAM)388, input/output (I/O) devices390, and network connectivity devices392. The processor may be implemented as one or more CPU chips.

The secondary storage384is typically comprised of one or more disk drives or tape drives and is used for non-volatile storage of data and as an over-flow data storage device if RAM388is not large enough to hold all working data. Secondary storage384may be used to store programs which are loaded into RAM388when such programs are selected for execution. The ROM386is used to store instructions and perhaps data which are read during program execution. ROM386is a non-volatile memory device which typically has a small memory capacity relative to the larger memory capacity of secondary storage. The RAM388is used to store volatile data and perhaps to store instructions. Access to both ROM386and RAM388is typically faster than to secondary storage384.

The processor382executes instructions, codes, computer programs, scripts which it accesses from hard disk, floppy disk, optical disk (these various disk based systems may all be considered secondary storage384), ROM386, RAM388, or the network connectivity devices392.

While several embodiments have been provided in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered as illustrative and not restrictive, and the intention is not to be limited to the details given herein, but may be modified within the scope of the appended claims along with their full scope of equivalents. For example, the various elements or components may be combined or integrated in another system or certain features may be omitted, or not implemented.

Also, techniques, systems, subsystems and methods described and illustrated in the various embodiments as discrete or separate may be combined or integrated with other systems, modules, techniques, or methods without departing from the scope of the present disclosure. Other items shown or discussed as directly coupled or communicating with each other may be coupled through some interface or device, such that the items may no longer be considered directly coupled to each other but may still be indirectly coupled and in communication, whether electrically, mechanically, or otherwise with one another. Other examples of changes, substitutions, and alterations are ascertainable by one skilled in the art and could be made without departing from the spirit and scope disclosed herein.