Exploring variations of pairwise coverage for prioritizing web application test cases
Creators: name: Chaturvedi, Nilesh
Publisher: name: Maryland Shared Open Access Repository
Year: 2013
Subjects: black box testing
combinatorial testing
regression testing
software testing
test case prioritization
web application testing

description: Regression testing is a critical phase in the software development lifecycle. One of the challenges during regression testing, called regression test selection, is to identify and organize the test cases to execute, in order to evaluate the correctness of the modified software system. Test case prioritization is one of the test selection methods that aim at ordering test cases in order to find faults quickly. The ordering is obtained by subjecting test cases to criteria that can identify test cases that are likely to locate faults early in the test execution cycle. We focus on event-driven systems in this work, and in particular web applications. In prior work, several criteria are developed for event-driven systems including, pairwise coverage of parameters, frequency of windows coverage, and count-based criteria. These criteria are black-box criteria, in that they do not use the source code when prioritizing test cases. Instead, they use information about the order and frequency in which pages are accessed to prioritize test cases, where a test case is itself a sequence of pages that can be visited on the web application along with any associated data, also known as parameter-;values. The previously proposed pairwise coverage of parameters criterion, called 2-way, selects a next test case that covers the most uncovered 2-way parameter-value interactions. However, the 2-way criterion considers interactions between parameter-values across pages in a test case. The 2-way criterion also does not consider the sequence in which pages are accessed. This thesis proposes and evaluates two main hypotheses: (1) Test cases that are ordered in descending order based on the number of parameter-value interactions between consecutive pages they contain are likely to have high rate of fault detection. (2) Test cases that are ordered in descending order based on the number parameter-value interactions within a web page they contain are likely to have high rate of fault detection. Our work thus aims to explore two variations of the 2-way criteria. With the first criterion, 2-way intra, our aim is to expose faults that may occur due to parameter-value interactions within the same page in a web application. On the other hand, the 2-way-inter-consecutive criterion exposes faults based on pairwise interactions between parameter values on consecutive pages. We first implemented the criteria in CPUT, a Java-based automated tool that converts usage logs to XML based test suites, prioritizes test cases and creates test orders that was developed in prior work. To evaluate the effectiveness of these criteria, first we gathered user sessions of two open source web applications, Schoolmate and OSCommerce. Using CPUT, the test case orderings for the two criteria were obtained. We then manually seeded faults in the applications. The ordered test cases were then run against the fault seeded versions of the applications. The struct, struct++, diff, path and content oracles were used for comparing clean &amp; faulty outputs. Based on the comparison, the rate of fault detection and Average Percentage of Faults Detected (APFD) were calculated. We report and discuss the results of comparison of these metrics with that of the pre-existing criteria.