# State-Sensitive Black-Box Web Application Scanning for Cross-Site Scripting Vulnerability Detection
## 4 Evaluation
### Analysis of Results
#### 4.3.1 Filtering Method (RQ1)  
To evaluate the effectiveness of our filtering method, we present the code coverage and detected XSS vulnerabilities on the eight applications by Black Widow and the scanner with the filtering method added to Black Widow (S1).  
As shown in Table 1, we count the unique LoC executed by web applications during the operation of Black Widow and S1. Compared with Black Widow, S1 performs better in code coverage on all eight applications. S1 increases the coverage by about 57% in PrestaShop. Due to the small number of lines of code in WackoPicko, the improvement in coverage is less than 1%. Overall, S1 increases the code coverage by approximately 17% in the eight applications.  
Through further analysis of the crawling paths, we found that there were duplicate GET requests and similar form submissions in Black Widow's crawling paths. The reason for duplicate GET requests is the presence of similar administrator consoles, where there are many GET request pages that can jump to each other. Black Widow lacks strict filtering methods for this. The reason for similar submissions is that Black Widow does not fully define the filtering method for edges of \"form\". Consequently, it will perform multiple rounds of crawling on similar forms with the same functionality, thus wasting time. Repeated crawls in these two types of similar states reduces the efficiency of Black Widow.  
To further explore the effectiveness of the filtering method in reducing visiting similar states during the scanning process, Figure 2 depicts the temporal evolution of code coverage. At the same time of execution, both SSBW and S1 can almost have higher code coverage than Black Widow. We believe that this is a positive effect of filtering similar states. Sometimes the code coverage of SSBW is lower than that of Black Widow (e.g., during the 2-4 h period of Joomla). We believe this is because SSBW spends the necessary fuzzing time on forms with more input parameters.  
\begin{table}
\begin{tabular}{c|c c c} \hline
**APP/Scanner** & **SSBW** & **S1** & **Black Widow** \\ \hline WackoPicko & 1035 & 1035 & 1025 \\ Vanilla & 15,713 & 15,337 & 14,135 \\ osCommerce & 17,691 & 15,764 & 13,059 \\ Joomla & 44,539 & 44,561 & 43,947 \\ WordPress & 42,820 & 42,814 & 41,118 \\ ImpressCMS & 21,395 & 21,353 & 19,005 \\ phpBB & 11,935 & 11,900 & 8971 \\ PrestaShop & 146,674 & 147,132 & 93,556 \\ \hline \end{tabular}
\end{table}
Table 1: LoC executed by web applications during the operation of SSBW, S1 and Black Widow.  
Table 2 shows all the XSS vulnerabilities found by Black Widow and S1 on all the applications. In total, S1 finds 81 unique XSS vulnerabilities, and more vulnerabilities are detected in three applications compared to Black Widow. Especially in osCommerce and Vanilla, the total number of XSS vulnerabilities detected by S1 is three times as many as Black Widow. It is more important that S1 does not miss any XSS vulnerabilities compared to Black Widow. This indicates that our filtering method helps improve the crawling capabilities of the scanner and thus enhance vulnerability detection capabilities.  
In summary, the experimental results of code coverage and vulnerabilities detection illustrate that our filtering method is effective.