Computer software application self-testing

Testing a computer software application by detecting an arrival of input data provided as input to a computer software application from a source external to the computer software application, modifying the detected input data to include test data configured to test the computer software application in accordance with a predefined test, thereby creating a modified version of the detected input data, and processing the modified version of the detected input data, thereby performing the predefined test on the computer software application using the test data.

FIELD OF THE INVENTION

The invention relates to computer software testing in general.

BACKGROUND OF THE INVENTION

Dynamic analysis tools are often used by computer software developers to test computer software applications, typically by first exploring an application to discover its interfaces, including those by which data may be provided to the application, and then by interacting with the application's interfaces and monitoring the application's responses to such interactions. In one type of dynamic analysis an application is tested for security vulnerabilities by providing data with known malicious properties to the application. For example, dynamic analysis may be used to test a web application that is accessed over a computer network, such as the Internet, by employing a “black-box tester” running on one computer that sends HTTP requests via a network to the web application that is hosted by another computer. The HTTP requests are configured with malicious payloads designed to test for security vulnerabilities such as SQL injection, cross-site scripting, and command injection. Unfortunately, such testing often produces hundreds of thousands of requests and responses, resulting in a great deal of traffic between the black-box tester and the application being tested, which may negatively affect overall testing performance and duration.

SUMMARY OF THE INVENTION

In one aspect of the invention a method is provided for testing a computer software application, the method including detecting an arrival of input data provided as input to a computer software application from a source external to the computer software application, modifying the detected input data to include test data configured to test the computer software application in accordance with a predefined test, thereby creating a modified version of the detected input data, and processing the modified version of the detected input data, thereby performing the predefined test on the computer software application using the test data.

In another aspect of the invention a method is provided for testing a computer software application, the method including instrumenting a computer software application to detect an arrival of input data provided as input to a computer software application from a source external to the computer software application, modify the detected input data to include test data configured to test the computer software application in accordance with a predefined test, thereby creating a modified version of the detected input data, and process the modified version of the detected input data, thereby performing the predefined test on the computer software application using the test data, and analyzing results of the predefined test responsive to the computer software application performing the predefined test using the test data.

In other aspects of the invention systems and computer program products embodying the invention are provided.

DETAILED DESCRIPTION OF THE INVENTION

The invention is now described within the context of one or more embodiments, although the description is intended to be illustrative of the invention as a whole, and is not to be construed as limiting the invention to the embodiments shown. It is appreciated that various modifications may occur to those skilled in the art that, while not specifically shown herein, are nevertheless within the true spirit and scope of the invention.

Reference is now made toFIG. 1, which is a simplified conceptual illustration of a system for testing a computer software application, constructed and operative in accordance with an embodiment of the invention. In the system ofFIG. 1, a computer software application100, such as may be hosted by a computer102, is preferably instrumented to include an input detector104and a mutator106. Computer software application100may, for example, be a “web” application including Hypertext Markup Language (HTML)-encoded web pages, with computer102acting as a web application server. Input detector104is preferably configured to detect the arrival of input data, such as in the form of Hypertext Transport Protocol (HTTP) requests, provided as input to computer software application100from a source external to computer software application100, such as from a computer108that is configured to communicate with computer software application100at computer102via a computer network110, such as the Internet. Mutator106is preferably configured to modify the detected input data to include test data configured to test computer software application100in accordance with a predefined test, thereby creating a modified version of the detected input data. The predefined test may, for example, be designed to test for a known type of security vulnerability, such as SQL injection, where the test data includes a malicious payload designed to exploit the security vulnerability. Mutator106then provides the modified version of the detected input data for processing by computer software application100. Computer software application100processes the modified version of the detected input data, thereby performing the predefined test using the test data included in the modified version of the detected input data.

In accordance with an embodiment of the invention mutator106is configured to modify a given instance of detected input data multiple times to create multiple modified versions of the detected input data, where each version includes different test data. For example,FIG. 2Ashows detected input data in the form of an HTTP request, from which four modified versions of the detected input data are created as shown inFIG. 2B, where each version of the detected input data is modified to include different test data as represented by TEST_PAYLOAD1, TEST_PAYLOAD2, TEST_PAYLOAD3, and TEST_PAYLOAD4. Mutator106may be configured to create any number of modified versions of a given instance of detected input data for a given predefined test, and may do so for any number of different predefined tests. Mutator106then provides each of the modified versions of the detected input data for processing by computer software application100.

The system ofFIG. 1also preferably includes an instrumentor112configured to instrument computer software application100to include input detector104and mutator106, such as between identified “sources” and “sinks,” where sources are application programming interfaces (API) that introduce “untrusted” input, such as user input, into an application, and sinks are security-sensitive operations, such as where the application modifies a database.

The system ofFIG. 1also preferably includes a dynamic analyzer114configured to analyze results of predefined tests performed by computer software application100as described hereinabove. Dynamic analyzer114may, for example, be a “black-box” testing tool hosted by computer108, where dynamic analyzer114is configured to “crawl” computer software application100to discover its application interfaces, send input data, such as the detected input data described hereinabove, to computer software application100via the discovered application interfaces, and analyze results of the predefined tests performed using the modified version(s) of the detected input data as described hereinabove. Dynamic analyzer114may additionally or alternatively be a “glass-box” testing tool hosted by computer102that monitors the operation of computer software application100to detect when a predefined test is performed. Dynamic analyzer114preferably reports the results of its analysis using conventional techniques.

Any of the elements shown inFIG. 1are preferably implemented by one or more computers, such as computers102and108, in computer hardware and/or in computer software embodied in a non-transitory, computer-readable medium in accordance with conventional techniques.

Reference is now made toFIG. 3, which is a simplified flowchart illustration of an exemplary method of operation of the system ofFIG. 1, operative in accordance with an embodiment of the invention. In the method ofFIG. 3a computer software application is instrumented (step300) to detect an arrival of input data provided as input to the computer software application from a source external to the computer software application (step302), and modify the detected input data to create one or more modified versions of the detected input data in accordance with a predefined test, or for each of multiple different predefined tests, where each modified version includes different test data (step304). The computer software application processes the modified version(s) of the detected input data, thereby performing the predefined test(s) on the computer software application using the test data (step306). Results of the performed test(s) are analyzed, and the results of the analysis are reported (step308).

Referring now toFIG. 4, block diagram400illustrates an exemplary hardware implementation of a computing system in accordance with which one or more components/methodologies of the invention (e.g., components/methodologies described in the context ofFIGS. 1-3) may be implemented, according to an embodiment of the invention.

As shown, the techniques for controlling access to at least one resource may be implemented in accordance with a processor410, a memory412, I/O devices414, and a network interface416, coupled via a computer bus418or alternate connection arrangement.

It will be appreciated that any of the elements described hereinabove may be implemented as a computer program product embodied in a computer-readable medium, such as in the form of computer program instructions stored on magnetic or optical storage media or embedded within computer hardware, and may be executed by or otherwise accessible to a computer (not shown).

While the methods and apparatus herein may or may not have been described with reference to specific computer hardware or software, it is appreciated that the methods and apparatus described herein may be readily implemented in computer hardware or software using conventional techniques.