Measuring coverage of application inputs for advanced web application security testing

A computer implemented method, a data processing system, and a computer usable recordable-type medium having a computer usable program code monitor a black box web application security scan. A black box scan of a web application is initiated. The black box scan sends a test is sent to a plurality of web application inputs of the web application. A runtime analysis is performed on the black box scan of the web application. Based on the run time analysis of the black box scan, the black box scan is modified.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to a computer implemented method, a data processing system, and a computer program product. More specifically the present invention relates to a computer implemented method, a data processing system, and a computer program product for measuring coverage of application inputs for advanced web application security testing.

2. Description of the Related Art

Code coverage is a measure used in software testing for describing the extent to which source code has been exercised during the testing procedure. Code coverage measurements are a form of white box testing, wherein knowledge of the source code is required.

Typical white box code coverage measurements allow the software engineer to obtain data about the execution of the program on a line by line basis. It may also be possible for the software engineer to examine data values held in program variables on a line by line basis.

However, typical white box code coverage measurements are impractical when the analyzed code becomes too large, or when the software engineer has no knowledge of the underlying source code. With no knowledge of the underlying source code, the software engineer must typically rely on black box testing procedures. Black box testing takes an external perspective of the test object to derive test cases. The test designer selects valid and invalid input and determines the correct output. There is no knowledge of the test object's internal structure. That is, the software engineer can only determine whether a correct output was given. In black box testing, the software engineer has no direct knowledge of how the program arrived at the given output.

Black box testing works by sending a test to an application input. While black box testing can uncover unimplemented parts of the specification, the software engineer cannot be sure that all existent paths of the code are tested.

BRIEF SUMMARY OF THE INVENTION

According to one embodiment of the present invention, a computer implemented method, a data processing system, and a computer usable recordable-type medium having a computer usable program code monitor a black box web application security scan. A black box scan of a web application is initiated. The black box scan sends a test to a plurality of web application inputs of the web application. A runtime analysis is performed on the black box scan of the web application. Based on the run time analysis of the black box scan, the black box scan is modified.

DETAILED DESCRIPTION OF THE INVENTION

Program code located in network data processing system100may be stored on a computer recordable storage medium and downloaded to a data processing system or other device for use. For example, program code may be stored on a computer recordable storage medium on server104and downloaded to client110over network102for use on client110.

In some illustrative embodiments, program code216may be downloaded over a network to persistent storage208from another device or data processing system for use within data processing system200. For instance, program code stored in a computer readable storage medium in a server data processing system may be downloaded over a network from the server to data processing system200. The data processing system providing program code216may be a server computer, a client computer, or some other device capable of storing and transmitting program code216.

The different embodiments may be implemented using any hardware device or system capable of executing program code. As one example, the data processing system may include inorganic components integrated with organic components and/or may be comprised entirely of organic components excluding a human being. For example, a storage device may be comprised of an organic semiconductor.

As another example, a storage device in data processing system200is any hardware apparatus that may store data. Memory206, persistent storage208, and computer readable media218are examples of storage devices in a tangible form.

The illustrative embodiments herein describe a computer implemented method, a computer program product, and a data processing system for gathering information from a run-time analysis in order to improve a black box web application security scan. Instead of using raw code coverage metrics on code thought to be interesting from a security perspective, the illustrative embodiments utilize a run-time analysis to determine which inputs to the application are exercised during the course of the black box scan.

Information is collected during the black box web application security testing on which application inputs are exercised and not exercised during the course of a black box web application security testing. This information is instructive for refining and improving the black box scan. Black box testing works by sending a test to an application input. Therefore, information on what application inputs were not exercised during the course of a test is directly actionable. When an application input is known not to have received a test, the black box scan configuration can be refined in order to include that application input as an exercised input. Through repeated scans and feedback, the black box scan can be optimized to cover each of the application inputs of web application. Feedback control therefore enables an optimally configured black box web application security scan. Configuring an optimal black box scan without feedback control is haphazard, and unlikely to produce complete code coverage.

A computer implemented method, a data processing system, and a computer usable recordable-type medium having a computer usable program code monitor a black box web application security scan. A black box scan of a web application is initiated. The black box scan sends a test is sent to a plurality of web application inputs of the web application. A runtime analysis is performed on the black box scan of the web application. Based on the run time analysis of the black box scan, the black box scan is modified.

Referring now toFIG. 3, a dataflow diagram of a run-time analysis of a black box web application security scan is shown according to an illustrative embodiment. Data processing system310is a data processing system, such as one of clients110,112, and114ofFIG. 1. Web server312is a server, such as one of server104and server106ofFIG. 1.

In one illustrative embodiment, data processing system310and web server312can be a single data processing system. In such an embodiment, the components shown on data processing system310and the components shown on web server312may all be implemented on a single data processing system.

Data processing system310accesses web application314from web server312. Web application314is typically coded in a browser-supported language such as, but not limited to, hypertext markup language, JavaScript, and Java. Web application314may be simple interactive web pages that invoke hypertext markup language requests by a user for information on web server312.

Data processing system310includes black box scanner318. Black box scanner318is a software application capable of performing a black box security scan of web application314. In one illustrative embodiment, black box scanner318can be a commercially available black box security web application scanner, such as, for example, but not limited to, Rational AppScan, available from International Business Machines, Corp. Rational is a registered trademark of International Business Machines, Corp. AppScan is a registered trademark of International Business Machines, Corp.

Black box scanner318traverses through web application314to check for common security problems such as for example, but not limited to, cross site scripting, structured querry language injection, Directory traversal, mis-configurations, remote command execution vulnerabilities, and possible security problems. Black box scanner318systematically crawls through web application314and parses the output therefrom to identify vulnerabilities in web application314.

Data processing system further includes run-time analyzer320. Run-time analyzer320is a software application capable of performing run-time analysis on a separately executing application. In one illustrative embodiment, run-time analyzer320can be a commercially available run-time analysis software application, such as, for example, but not limited to, Rational PureCoverage, available from International Business Machines, Corp., or a freely distributed run-time analysis software application such as, for example, but not limited to, Eclipse Test and Performance Tools Platform, available from Eclipse, Corp.

Run-time analyzer320monitors the execution of web application314. In one illustrative embodiment, run-time analyzer320monitors all potential inputs to web application314. While black box scanner318is running a black box test of web application314, run-time analyzer320monitors the various inputs of web application314to determine which of those web application inputs were exercised by the receipt of incoming data, and which of those web application inputs were not exercised.

When black box scanner318has completed running its testing of web application314, run-time analyzer320prepares information on the coverage of web application inputs. Run-time analyzer320provides information such as the percentage of all application inputs exercised and the list of application inputs that were not exercised is provided. Run-time analyzer320could also prepare information on the coverage of web application314during the course of the black box scan in more of a continuous feedback loop, rather than in distinct phases.

The information provided by the run-time analyzer320is then input into feedback control322. Feedback control322is a software process that refines the tests run by black box scanner318. Feedback control322refines the black box scan such that subsequent runs of the black box scan will cover the previously unexercised application inputs of web application314. That is, a first run of tests by black box scanner318will most likely result in incomplete code coverage due to incomplete testing of the totality of application inputs. Feedback control322modifies the black box scan performed by black box scanner318, such that subsequent runs of the black box scan will cover the previously unexercised application inputs.

Through repeated scans and feedback, the black box scan can be optimized to cover each of the application inputs of web application314. Feedback control322therefore enables an optimally configured black box web application security scan. Configuring an optimal black box scan without feedback control322is haphazard, and unlikely to produce complete code coverage.

Referring now toFIG. 4, a flowchart for providing black box security scans of a web application is shown according to an illustrative embodiment. Process400is a software process, executing on a software component such as black box scanner318ofFIG. 3.

Process400begins by traversing a web application (step410). The Web application can be web application312ofFIG. 4. Process400traverses through the web application to check for common security problems such as for example, but not limited to, cross site scripting, structured querry language injection, directory traversal, mis-configurations, and remote command execution vulnerabilities.

Process400generates and sends tests to the web application (step420). The tests sent to the web application can be tests designed to elicit errors based on common vulnerabilities in web applications. The tests can include, but are not limited to, cross site scripting, structured querry language injection, directory traversal, mis-configurations, and remote command execution vulnerabilities. During the black box scan, process400systematically crawls through the web application, sends tests to the application, and parses the output therefrom to identify vulnerabilities in web application.

Responsive to generating and sending tests to the web application, process400processes web application responses to identify probably vulnerabilities (step430). Web application responses may include any error messages encountered in response to the tests sent to the web application. Probable vulnerabilities can be identified from suspicious output by the web application. For example, if JavaScript code is sent to the application in a test, and later the same JavaScript code is returned in a response from the web application, a cross site scripting vulnerability is likely to exist.

Process400then generates a report of the probable vulnerabilities (step440), with the process terminating thereafter. A software engineer can use the parsed suspicious output within the log file to address the suspected vulnerability within the web application.

Referring now toFIG. 5, a flowchart for providing run-time analysis of an executing web application is shown according to an illustrative embodiment. Process500is a software process, executing on a software component such as run-time analyzer320ofFIG. 3.

Process500begins by monitoring web application inputs during a black box security scan (step510). Process500monitors all potential inputs to web application during run time of the web application. While a black box scanner, such as black box scanner318shown inFIG. 4, is running a black box test of web application, process500monitors the various inputs of into the web application to determine which of those web application inputs were exercised by the receipt of incoming data, and which of those web application inputs were not exercised.

Process500then identifies unexercised web application inputs and the code coverage of the black box scan (step520). Instead of using raw code coverage metrics on code thought to be interesting from a security perspective, a run-time analysis of process500identifies which web application inputs are exercised during the course of the black box scan. When each possible inputs to the web application is known, web application inputs that are not exercised during the course of the black box scan can then quickly be identified.

Process500then forwards at least the unexercised application inputs to a feedback control for updating the black box scan (step530), with the process terminating thereafter. The feedback control can be feedback control322ofFIG. 3. A first run of tests by the black box scanner will most likely result in incomplete code coverage due to incomplete testing of the totality of application inputs. Through repeated scans and feedback of the unexercised application inputs, the black box scan can be optimized to cover each of the application inputs of a web application. Configuring an optimal black box scan without feedback control is haphazard, and is unlikely to produce complete code coverage.

Referring now toFIG. 6, a flowchart for updating a black box scan based on runtime analysis feedback of an executing web application is shown according to an illustrative embodiment. Process600is a software process, executing on a software component such as feedback control322ofFIG. 3.

Process600begins by receiving unexercised application inputs for updating the black box scan (step610). A first run of tests by the black box scanner will most likely result in incomplete code coverage due to incomplete testing of the totality of application inputs. Through repeated scans and feedback of the unexercised application inputs, the black box scan can be optimized to cover each of the application inputs of a web application. Configuring an optimal black box scan without feedback control is haphazard, and is unlikely to produce complete code coverage. Unexercised application inputs show those sections of the web application code that has not been subjected to the black box testing.

Responsive to receiving unexercised application inputs, process600reconfigures the black box scan so that the unexercised application inputs will receive test inputs from the black box scanner in a subsequent run of the black box scan (step620), with the process terminating thereafter. Process600modifies the black box scan performed by the black box scanner, such that subsequent runs of the black box scan will cover the previously unexercised application inputs. Through repeated scans and feedback, the black box scan can be optimized to cover each of the application inputs of the web application. Process600therefore enables an optimally configured black box web application security scan.

The illustrative embodiments herein describe a computer implemented method, a computer program product, and a data processing system for gathering meaningful information from a run-time analysis in order to improve a black box web application security scan. Instead of using raw code coverage metrics on code thought to be interesting from a security perspective, the illustrative embodiments utilize a run-time analysis to determine which inputs to the application are exercised during the course of the black box scan.

Information on which application inputs are exercised and not exercised during the course of a black box web application security testing is much more instructive for refining and improving the black box scan. Black box testing works by sending a test to an application input. Therefore, information on what application inputs were not exercised during the course of a test is directly actionable. When an application input is known not to have received a test, the black box scan configuration can be refined in order to include that application input as an exercised input.

Thus, the illustrative embodiments, provide a computer implemented method, a data processing system, and a computer usable recordable-type medium having a computer usable program code monitor a black box web application security scan. A black box scan of a web application is initiated. The black box scan sends a test is sent to a plurality of web application inputs of the web application. A runtime analysis is performed on the black box scan of the web application. Based on the run time analysis of the black box scan, the black box scan is modified.

Through repeated scans and feedback, the black box scan can be optimized to cover each of the application inputs of web application. Feedback control therefore enables an optimally configured black box web application security scan. Configuring an optimal black box scan without feedback control is haphazard, and unlikely to produce complete code coverage.