Abstract:
An apparatus and method of managing vulnerability testing of a web application is provided for running a set of one or more scripted tests against a web application, recording results of the one or more scripted tests, providing an interface for a human evaluator to review the recorded results, and accepting from the human evaluator custom test parameters based on observations of the recorded results, wherein custom test parameters include at least one context usable by a future tester in deciding whether to run the custom test, and also includes at least one instruction for automatically running custom test steps of the custom test.

Description:
FIELD OF THE INVENTION 
     The present invention relates network server security in general and in particular to web application security scanning. 
     BACKGROUND OF THE INVENTION 
     There are a number of different configurations of network client-server interfaces available today, but the most common network in use is the Internet, a global internetwork of networks and networks that use Internet protocols and/or interfaces, such as extranets, intranets, local services, and other variations. In the general case, to which inventions described herein apply, clients connect to servers over the network and clients are not always trusted computers. As a result, the designers of the servers need to ensure that untrusted clients cannot perform malicious acts or access unauthorized portions of the server through the network. 
     One approach to ensure that servers cannot be accessed in an unauthorized manner is to only provide access to secured and trusted clients. However, in many situations, that is not possible. For example, if a merchant was running an on-line store, the merchant would want to allow most anyone who has a computer to access the servers providing the on-line store functionality, but do so in a way that still prevents unauthorized interactions with the servers. 
     Server security is more than just requiring a username and password from each client before responding to client requests, since even a logged in user might try for unauthorized access and a typical service provided by a server might include content and functionality for use by unauthenticated and unlogged-in clients. One approach to server security is to review all of the code that runs on the server and verify that it does not include statements that allow for unauthorized activity and review all the files present on the server and their respective permissions, side-effects, etc. While this might be practical for a small installation, say an FTP server that serves up predefined files to all corners, it is often not practical with complex, interactive applications that have many response modes. 
     One common use of servers in this environment, but not an exclusive use, is that of a web application. As used herein, “web” refers to a collection of documents/files, some of which have references, or links, to other documents/files in the collection. One example of a web is the World Wide Web (“WWW”), a collection of files served up by WWW servers (also called “web servers”) using HTTP protocols or something similar. The “WWW” gets its name from the fact that most of these documents/files can be almost anywhere in the world and can be accessed anywhere in the world where there is Internet connectivity. 
     A web application is an application that runs on one or more server and provides some functionality or service in response to client requests received over a network using web protocols (i.e., HTTP, HTTPS, or something similar). An example of a web application is a database interface, wherein a database runs on a database system and clients can access data in that database system by sending a request for service over the network to a web application server. The web application server receives the request for service and decides, according to how it is programmed, what to do with the request. It can ignore the request, send an error message back to the client, or trigger an operation with the database system and respond to the client&#39;s request by sending the client the results of the database operation. 
     In a highly specific example, suppose a client computer system is operated by a customer seeking to configure and purchase a laptop computer. The customer would direct the client computer system to access a web application server operated by a vendor of laptop computers. The client computer system might send a request to the web application server via the network requesting a home page of the vendor. The web application server might respond with a home page that includes features allowing the client to interact with content on the home page (such as by selecting from available model names, features, etc.), send a subsequent request to the server, etc. 
     All the while, the web application server is making decisions about what is appropriate to send and what is not appropriate to send, based on its programming. For example, if the client computer sends a request for an updated page with updated pricing for new options selected by the customer, the web application server might perform some calculations, perform some database look-ups, generate a new dynamic web page and return that web page to the client computer in response to the request. However, if the client computer sends a request to see data about what someone else ordered, or internal data from the database server, the web application should properly refuse to respond to the request. 
     Because web applications are so complex, securing a web application and testing for security vulnerabilities, often involves an automated testing of the web application. Client-side web application testing refers to tests that are run from a client&#39;s point of view. For example, a client-side test suite might have logic for logging in to a web application, applying valid and invalid requests to the web application, noting the web application&#39;s responses and evaluating those responses. For example, if the test suite sends a request to the web application for ordering products where the prices have been altered and the response is “invalid order”, the test suite might note that the web application is secure in that regard, but if the response is “thank you for your order”, the test suite might note that the web application is not secure. 
     While it is a simple matter for test personnel to look at the response that a web application sends to a request and determine whether the response was appropriate or indicates a security vulnerability, it is more difficult to program a system to automatically programmatically assess a response and determine whether it is a security vulnerability. 
     In view of the above, the inventions described herein provide improvements over existing approaches. 
     BRIEF SUMMARY OF THE INVENTION 
     Embodiments of the present invention provide for gathering human insights as to potential security vulnerabilities and structure data storage to contain data supporting recordations of those insights in such a way that later security scans can use that data to trigger an analysis by computer or human when conditions are met that represent the human insight. 
     In specific embodiments, apparatus and methods of managing vulnerability testing of a web application are provided for running a set of one or more scripted tests against a web application, recording results of the one or more scripted tests, providing an interface for a human evaluator to review the recorded results, and accepting from the human evaluator custom test parameters based on observations of the recorded results, wherein custom test parameters include at least one context usable by a future tester in deciding whether to run the custom test, and also includes at least one instruction for automatically running custom test steps of the custom test. 
     In a specific example, a test suite provides a tester with an interface to enter in details of a custom test of an application. Thus, when a tester is executing manual tests on a web application to test for potential security vulnerabilities and the tester notices some indicators of a potential weakness, the tester might perform additional tests, but might also enter a custom test record into the data storage for custom tests. A custom test record would be a recordation of elements of human insight. For example, if the tester noticed that a particular file is present in a web application at a particular URI and the tester assessed that the existence of the file and/or similarly named files may be of a security concern, the tester can create a custom test record that specifies a regular expression, for example, that matches the file&#39;s name and the names of the similarly named files, add comments as to the name of the custom test, human-readable comments about when the test should be run, why it should be run, what to look for, prior experience, etc. Then, when a later automated scan is being done, the regular expressions of the custom test records are compared to the requests the scanner is making and if there is a match, the scanner might alert the later tester and present the tester with the custom test record information so that the later tester can be prompted to decide to run that custom test. 
     The following detailed description together with the accompanying drawings will provide a better understanding of the nature and advantages of the present invention. 
    
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         FIG. 1  is a block diagram illustrating a context of a test system according to aspects of the present invention. 
         FIG. 2  is a block diagram illustrating details of the text system. 
         FIG. 3  is a flowchart of one process for handling custom tests. 
     
    
    
     DETAILED DESCRIPTION OF THE INVENTION 
     An improved web application security scanner is described herein. A web application security scanner exercises a web application to find vulnerabilities and/or functional errors. For example a scanner might identify files on a web application server that are accessible but should not be, errors in web application logic, vulnerabilities wherein the web application accepts altered or unexpected inputs instead of responding with an error message. 
     The examples herein are not intended to be exhaustive, but describe embodiments of the present invention. Where individual items are shown, multiple items might be present, unless otherwise indicated. Where an indefinite number of items are shows or described, such as by parenthetical references to (1), (2), . . . , (n), the actual number is not limiting, unless otherwise specified. It should be noted that, unless otherwise indicated, the domain names used in the examples are not intended to refer to actual domains and any similarity is merely coincidental. 
     A web application test suite, or “test suite”, performs tests against a web application by sending client requests from a testing computer to the server running the web application and checking how the web application responds. A thorough web application security scan might involve thousands of checks and responses. As a result, some sort of programmatic analysis is needed. A test suite might execute a Web Application Security Scan (WASS) by issuing a series or requests to a web application in the form of test URIs Uniform Resource Identifiers) and noting the responses made by the web application. URIs can be dynamic, wherein the page referenced by the URI does not exist until a request is made and responses might also be dynamic in that they are generated to be specific to a particular request. 
       FIG. 1  is a block diagram illustrating a context of a test system according to aspects of the present invention. As shown therein, a web services system  10  under test can be tested in the environment it normally operates in, such as an environment wherein clients  12  interact with web services system  10  over a network  14 . A test system  16 , which might be a programmed computer or software running on a multi-purpose computer, can also interact with web services system  10  as if it were a client. Test system  16  generates results  18  usable for assessing whether web services system  10  provides any vulnerabilities and that are usable for an evaluator, such as a network security expert. 
     In particular, the operator of web services system  10  might be interested in vulnerabilities that allow outside entities or systems to access web applications  20  that are executed by web services system  10  or server data  22  that is maintained by web services system  10  and possibly used by web applications  20 , where the access is in ways not expected by the designers of such systems that unnecessarily expose those systems. Other vulnerabilities or potential vulnerabilities might be in the presence of some file, data, parameter and/or feature that directly indicates a vulnerability, or suggests that it is worth testing for a vulnerability. 
       FIG. 2  is a block diagram of a test system shown in greater detail. As shown there, a processor  100  is programmed to read instructions and generate requests that an I/O section  102  sends to a web application and receives responses. In an example, processor  100  might issue an HTML request, an RFC request, a telnet request, an FTP request or the like by formatting a message and passing it to I/O section  102  that transmits the message over a network to an addressed server, service and/or system, which may return a response, which I/O section  102  might return to processor  100 . It is well-known to use particular protocols to generate and transmit messages and requests and receive responses, so such details need not be mentioned here. 
     In an example test system, processor  100  might generate requests based on a reading of WASS schedules from storage  104 , send those requests, and store the corresponding results in a results log file  106 . Processor  100  might also run a test by executing a script representing that test. Processor  100  might process the results to determine vulnerabilities, but those might also be determined by a post-processor. 
     After a WASS schedule and/or test scripts are run, a test expert can use a custom test controller  110  to investigate the results, possibly using the expert&#39;s insight as to what to look for. If the expert determines that another test is warranted and/or useful in other situations, the expert can create a custom test that might then be stored in storage  104 . A custom test might comprise more than just the test script needed to execute the test, but might include other information that informs later users of the test. 
     An example of such a process is shown in  FIG. 3 , wherein a new pattern is recognized (S1), the pattern is entered into a scanner (S2) or stored such that the processor can use it, and when patterns are matched, the scanner can notify an evaluator (S3), who might choose (S4) to create a new test (S5). 
     In a specific example, a test suite provides a tester with an interface to enter in details of a custom test of an application. Thus, when a tester is executing manual tests on a web application to test for potential security vulnerabilities and the tester notices some indicators of a potential weakness, the tester might perform additional tests, but might also enter a custom test record into the data storage for custom tests. A custom test record would be a recordation of elements of human insight. For example, if the tester noticed that a particular file is present in a web application at a particular URI and the tester assessed that the existence of the file and/or similarly named files may be of a security concern, the tester can create a custom test record that specifies a regular expression, for example, that matches the file&#39;s name and the names of the similarly named files, add comments as to the name of the custom test, human-readable comments about when the test should be run, why it should be run, what to look for, prior experience, etc. Then, when a later automated scan is being done, the regular expressions of the custom test records are compared to the requests the scanner is making and if there is a match, the scanner might alert the later tester and present the tester with the custom test record information so that the later tester can be prompted to decide to run that custom test. 
     While performing a web application assessment, people recognize patterns in the web application that may lead them to perform further testing. The tester&#39;s insight might direct him or her to perform certain additional tests. For example, if a tester noticed that a file called “/private/big_secrets.txt” was accessible to a regular user, the tester might check the contents of the file, then look to see if any other files from the “/private” directory were similarly accessible. A later tester might of might not encounter that same file in another system, might not notice the file, and/or might not think to run the additional test of looking for files in the same directories. The system described herein allows for capturing these patterns, finding them during automated scans and suggesting possible tests to people engaged in the later assessment of the same systems, related systems or totally unrelated systems. 
     Patterns can be applied to all sites, some sites, or an individual site. Patterns can be regular expressions or some other form of pattern specification. Patterns might be in the form of a regular expression that matches all path/filenames of interest. A custom test record might include a pattern, test requests, commentary, and other elements. An example use of a custom test record is that a scanner checks files, requests and/or other data against the patterns of all or some of the custom test records. When there is a match, the scanner presents the commentary to the tester and suggests a number of test requests to make. Alternatively, the scanner can run some or all of the test requests and present those results with the commentary. 
     Custom tests can be applied to all sites, some sites, or an individual site. Once a custom test is created, it may be run once, a certain number of times, or during every scan. Custom tests might comprise a single pattern or multiple patterns. Custom tests might comprise a single request or multiple requests. 
     The patterns can be created by a group of one or more persons over time and accumulate in a collection of custom test records. The scanning system might provide maintenance features, so that custom tests that are never run are deleted to make room for other tests, to allow an administrator to alter custom test records to, for example, modify the regular expressions. Patterns and custom tests might be evaluated for accuracy and relevancy from time to time. 
     Patterns and custom tests that fail to meet certain requirements for relevance might be evaluated and if they are deemed to be irrelevant or incorrect they are removed from the data storage containing the custom test records. 
     While the invention has been described with respect to exemplary embodiments, one skilled in the art will recognize that numerous modifications are possible. For example, the processes described herein may be implemented using hardware components, software components, and/or any combination thereof. Thus, although the invention has been described with respect to exemplary embodiments, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.