System testing using nested transactions

A computer system includes a processor and a data store coupled to the processor. An application component is operably coupled to the processor and the data store and is configured to run one or more applications stored in the data store. A test framework is coupled to the processor and the data store and is configured to perform at least one test relative to a component of the computer system that interacts with a database. A savepoint manager is configured to responsively generate at least one savepoint in the database prior to the at least one test and to roll back the at least one savepoint after the at least one test. Methods of testing the computer system are also provided.

BACKGROUND

Computer systems are currently in wide use. Some computer systems are relatively large and may include, for instance, thousands of different forms (e.g. electronic fillable documents) and other types of elements. Such computer systems are often customized (some heavily so) before they are deployed in a given implementation. Some large computer systems may include many thousands of different forms, each of which may have different controls and other user interface elements. Some example systems also include execution logic as well as workflows that allow users to access the system and perform a set of activities, or tasks, in order to carry out their duties in conducting a particular operation.

When such a computer system is deployed in a specific environment, it is common for the system to be customized to meet the functional requirements of the particular end user. By way of example, different customers may wish to have different form controls on a given form that represents a particular entity. In addition, different customers may also wish to have different fields, execution logic, or other items on a particular report form. Thus, it can be seen that a given computer system may be highly customized so that it meets the requirements of a particular end user for which the system is implemented.

When the base system is authored (before it is customized) it may also include a number of different tests that can be run to determine whether the system is working properly. Additionally, as a developer customizes or continues to develop on the base system, the developer may generate additional tests as well. Further, once the development is complete, it is also important to ensure that a system is functioning properly even when it is live and handling real data.

SUMMARY

A computer system includes a processor and a data store coupled to the processor. An application component is operably coupled to the processor and the data store and is configured to run one or more applications stored in the data store. A test framework is coupled to the processor and the data store and is configured to perform at least one test relative to a component of the computer system that interacts with a database. A savepoint manager is configured to responsively generate at least one savepoint in the database prior to the at least one test and to roll back the at least one savepoint after the at least one test. Methods of testing the computer system are also provided.

DETAILED DESCRIPTION

Testing a complex computer system has always been an important operation. Additionally, many of the tests executed upon a given computer system create traces or data in one or more databases of the system that can create conflicts to logic executing for other users or tenants at the same or subsequent time. System testing relative to database operations has generally used a database log in order to painstakingly reverse changes that were performed to the system during testing. The utilization of a database log to essentially undo such changes required significant time and created the potential locking of table rows or entire tables which could unduly impact the availability of the database for other users or tenants. Further, the process of undoing such operations was not resilient to timeouts or crashes. Thus, it was possible to leave test data behind and the state of the database could be corrupted.

Embodiments described below, generally leverage a database function known as savepoints in order to robustly and efficiently maintain system data during and after execution of testing processes. Savepoints are functions that are supported by a number of databases, such as SQL Server, available from Microsoft Corporation of Redmond, Wash.

In accordance with some embodiments, savepoints may be triggered or generated, in a “stacked” manner. When a sandboxed testing operation begins, a transaction is created in the database that will never get committed. At this point, all transaction invocations for the session owner are redirected through this specific database connection that is governed by savepoints. As the execution progresses and at specific times, markers are added to the database effectively “saving” a state. At the end of specific execution flow and to return to a previous state, the stack of savepoints is popped and the changes from that moment on are simply dismissed. When sandboxed testing is finished, the final savepoint is dismissed and the changes are reverted.

FIG. 1is a block diagram of a computer system environment100in accordance with an embodiment. Computer system200may be any suitable computer system that is generally provided as a base system and subsequently customized or otherwise developed prior to deployment. System200is shown generating user interface displays118that can include forms204. System200may have thousands of different forms204, each of which may have many controls. The user interacts with user interface displays118in order to interact with, and manipulate, system200.

System200, in one example, includes processor(s)210, data store212, application component215, user interface component216and test framework218. System200can also include other components232as well. System200includes or is coupled to savepoint manager214that includes or is coupled to savepoint store220.

Data store212, in one embodiment, includes data entities222, processes224, applications226, workflows228and other data records230. Entities component222, in one embodiment, describes entities within or otherwise used by system200. Data store212may also include one or more databases that implement the savepoint operation. However, embodiments can be practiced where the database is remote from system200and communicatively coupled to system200through any suitable communication media.

Applications226can be any suitable applications that may be executed by system200in order to perform one or more functions for which system200is deployed. Application component215, in one example, runs applications226, which can include processes224and workflows228. Processes224and workflows228, in one example, operate upon data entities222as well as other records230in order to enable the user to perform his or her operations within system200. In one example, user interface component216, either by itself, or under the control of other items in system200, generates user interface displays118.

FIG. 2is a diagrammatic view of system200performing testing relative to Form1, indicated at reference numeral250. In order to test Form1, test framework218invokes an instance of the form and interacts with various user interface elements on the form, such as “Create” user interface element252, “Modify” user interface element254, and “Save” user interface element256. As set forth above, forms are used by the system in order to allow a user to interact with data and other processes available from the system. Test framework218, using an instance of form250is able to modify data within data store212. Prior to testing form250, test framework218utilizes savepoint manager214to create a savepoint relative to data store212. Then, test framework218may perform one or a number of test operations that actually modify production data within data store212. This allows the testing function to be performed very effectively since real-world data and loads are accurately simulated during testing. Once the testing operation is complete, test framework218again engages savepoint manager214to roll back the savepoint that was created prior to the testing. Thus, any changes that occurred to the data in data store212during testing of the form are precisely and efficiently rolled back.

FIG. 3is a swim lane diagram of savepoint use in system testing in accordance with an embodiment. As shown inFIG. 3, test framework218initially makes a call to savepoint manager214in order to activate a transaction log, as indicated a reference numeral300. This causes savepoint manager214to begin a transaction with database298. Database298may be a component of data store212, a separate component of system200or it may be a remote database that is communicatively coupled to system200.

As indicated at reference numeral302, savepoint manager214initiates a transaction with database298labeled “Begin Transaction Scope.” Test framework218then commands savepoint manager214to create a first (Level 1) savepoint. This causes savepoint manager214to responsively invoke a savepoint command on database298, as indicated reference numeral306. Subsequently, test framework218commands savepoint manager214to create a second (Level 2) savepoint, as indicated at reference numeral308. This causes savepoint manager214to responsively invoke another savepoint command on database298, as indicated reference numeral309. Testing is performed after creation of the savepoints as will be described in greater detail below with respect toFIG. 4. Once the testing is complete, the associated changes to the data in database298are undone by rolling back the respective save points, as indicated by test framework218issuing restore commands to savepoint manager214. In the example shown, this includes test framework218issuing a “Restore Level 2” command to savepoint manager214, as indicated at reference numeral310followed by test framework218issuing Restore Level 1 to savepoint manager214as indicated at reference numeral312. Finally, test framework218issues “Restore Level 0” to savepoint manager214as indicated at reference numeral314. With each of these “Restore Level” commands being issued to savepoint manager214, savepoint manager214executes a corresponding rollback command on database298in order to roll back the respective savepoint. Essentially,FIG. 3illustrates that the savepoints can be nested and that they can be used to efficiently undo database operations performed during system testing.

FIG. 4is a swim lane diagram of system testing using savepoints in accordance with an embodiment. WhileFIG. 4is somewhat similar toFIG. 3,FIG. 3illustrates test framework operation and savepoint interaction relative to database298, whileFIG. 4illustrates test framework and savepoint manager interaction relative to the application layer. The application layer includes suite320as well as Test Class 1,322and Test Class 2,324. As illustrated inFIG. 4, testing initially begins with test framework218causing savepoint manager214to create savepoint1, as indicated at reference numeral325. As described above, with respect toFIG. 3, this causes savepoint manager214to interact with database298in order to create a savepoint. Next, as illustrated at reference numeral326, test framework218issues a “Create” command to application suite320. Subsequently, test framework218invokes the Suite.Setup function indicated at reference numeral328from test suite320. Within the context of the invoked suite setup, test framework218begins testing of first test class322by issuing a Create command to test class322as indicated at reference numeral330. Subsequently, test framework218invokes the SetupTestCase function for first test class322as indicated at reference numeral332. Test framework218then utilizes savepoint manager214to create savepoint2, as indicated at reference numeral334. Next, test framework218invokes the setup of first test class322, as indicated at reference numeral336. Testing of first test class322is then performed by test framework218by invoking Test1, as indicated at reference numeral338. Test1 may perform one or more suitable testing functions relative first test class322in order to validate or otherwise diagnose the behavior of first test class322in the system. Once execution of Test1 is complete, test framework218invokes a teardown of first test class322as indicated at reference numeral340. Then, test framework218rolls back to savepoint2, as indicated at reference numeral342. By rolling back to savepoint2, all of the changes to the data or traces in database298caused by operation of Test1 are removed. Accordingly, database298will revert to the state that existed when savepoint2was created at reference numeral334.

A second test method begins when test framework218again invokes the setup of first test class322, as indicated at reference numeral344. Then, at reference numeral346, test framework218invokes Test2 on first test class322. Finally, after Test2 has completed for first test class322test framework218invokes the teardown of first test class322, as indicated at reference numeral348.

Before testing second test class324, test framework218causes savepoint manager214to roll back to savepoint2, as indicated at reference numeral350. As set forth above, with respect toFIG. 3, this causes savepoint manager214to execute a rollback operation relative to database298. At reference numeral352, test framework218causes savepoint manager214to roll back to savepoint1, which was created initially at reference numeral325. Once the rollback to savepoint1has been completed, all of the database operations that occurred during the testing of first test class322have been precisely and efficiently undone.

In order to begin testing second test class324, test framework218causes savepoint manager214to create save point3, as indicated at reference numeral354. Then, test framework218invokes the setup function with respect to second test class324, as indicated at reference numeral356. Once the setup function has finished, Test1 is executed with respect to second test class324, as indicated at reference numeral358. Once Test1 of second test class324has completed, test framework218invokes the teardown of second test class324, as indicated at reference numeral360. Once the teardown function is completed, test framework218causes savepoint manager214to roll back to savepoint3, as indicated at reference numeral362thereby returning database298to the condition that existed before savepoint3was created. Accordingly, the subsequent test method, indicated at reference numeral364, can operate upon the precise database state as the previous test method.

In order to begin test method364, test framework218again invokes the setup function of second test class324as indicated at reference numeral366. Then, Test2 is performed relative to second test class324, as indicated at reference numeral368. Finally, when Test2 has completed, test framework218invokes the teardown of second test class324, as indicated at reference numeral370. Once Test2 of second test class324has completed, test framework218causes savepoint manager214to roll back to savepoint3, as illustrated at reference numeral372. Additionally, with the testing substantially complete, test framework218causes savepoint manager214to further roll back to save point1, as indicated at reference numeral374. With the rollback to save point1completed, test framework218causes test suite320to perform a teardown as indicated at reference numeral376. Finally, test framework218causes savepoint manager214to roll back to the initial state as indicated at reference numeral378. With this final rollback to the initial state, database298is returned to precisely the state it was in prior to testing. Thus, it can be appreciated that a sandboxed execution of logic for testing purposes using savepoints is provided.

The ability to make and use savepoints in the context of multi-tenant, multi-user, database-centric application testing is facilitated. The use of savepoints allows substantially instantaneous rollback of state for a specific session and by-design crashes (or connection timeout) support. The strategy used by savepoint isolation is to never commit the data changes but, instead, to create and make use of savepoints (markers in the database) which preserve, in memory, the state of the database for a particular connected session. Savepoints can be chained and nested as described with respect toFIGS. 3 and 4. This allows the controlling process to return to a prior state of the database without completely rolling back all of the changes. In the case of a crash or a connection timeout, the changes in data are simply dismissed leaving the database state intact.

Also, the figures show a number of blocks or swim lanes with functionality ascribed to each. It will be noted that fewer blocks or swim lanes can be used so the functionality is performed by fewer components. Also, more blocks or swim lanes can be used with the functionality distributed among more components.

In the embodiment shown inFIG. 5, some items are similar to those shown inFIG. 1and they are similarly numbered.FIG. 5specifically shows that200is located in cloud502(which can be public, private, or a combination where portions are public while others are private). Therefore, user112uses a user device504to access those systems through cloud502.

FIG. 5also depicts another embodiment of a cloud architecture.FIG. 5shows that it is also contemplated that some elements of system200are located in cloud502while others are not. By way of example, data store110can be located outside of cloud502, and accessed through cloud502. In another embodiment, testing framework518is also located outside of cloud502. Regardless of where they are located, they can be accessed directly by device504, through a network (either a wide area network or a local area network), they can be hosted at a remote site by a service, or they can be provided as a service through a cloud or accessed by a connection service that resides in the cloud. All of these architectures are contemplated herein.

FIG. 6is a simplified block diagram of one illustrative embodiment of a handheld or mobile computing device that can be used as a user's or client's hand held device16, in which the present system (or parts of it) can be deployed.FIGS. 7 and 8are examples of handheld or mobile devices.

FIG. 6provides a general block diagram of the components of a client device16that can run components of system200or that interacts with architecture100, or both. In the device16, a communications link13is provided that allows the handheld device to communicate with other computing devices and under some embodiments provides a channel for receiving information automatically, such as by scanning. Examples of communications link13include an infrared port, a serial/USB port, a cable network port such as an Ethernet port, and a wireless network port allowing communication though one or more communication protocols including General Packet Radio Service (GPRS), LTE, HSPA, HSPA+ and other 3G and 4G radio protocols, 1×rtt, and Short Message Service, which are wireless services used to provide cellular access to a network, as well as 802.11 and 802.11b (Wi-Fi) protocols, and Bluetooth protocol, which provide local wireless connections to networks.

Under other embodiments, applications or systems are received on a removable Secure Digital (SD) card that is connected to a SD card interface15. SD card interface15and communication links13communicate with a processor17along bus19that is also connected to memory21and input/output (I/O) components23, as well as clock25and location system27.

Memory21stores operating system29, network settings31, applications33, application configuration settings35, data store37, communication drivers39, and communication configuration settings41. Memory21can include all types of tangible volatile and non-volatile computer-readable memory devices. It can also include computer storage media (described below). Memory21stores computer readable instructions that, when executed by processor17, cause the processor to perform computer-implemented steps or functions according to the instructions. Items in data store212, for example, can reside in memory21. Similarly, device16can have a client system24which can run various applications. Processor17can be activated by other components to facilitate their functionality as well.

FIG. 8provides an additional example of devices16that can be used, although others can be used as well. In the example shown inFIG. 8, the device is a smart phone71. Smart phone71has a touch sensitive display73that displays icons or tiles or other user input mechanisms75. Mechanisms75can be used by a user to run applications, make calls, perform data transfer operations, etc. In general, smart phone71is built on a mobile operating system and offers more advanced computing capability and connectivity than a feature phone.

Note that other forms of the devices16are possible.