# Grey-Box Fuzzing Based on Reinforcement Learning for XSS Vulnerabilities
## 5 Evaluation
### Result and Analysis
#### 5.2.1 Answering RQ1: Effectiveness  
We test the performance of the proposed method on two target web applications. The first one is WebGoat [41], a cyber range developed by OWASP for web vulnerability detection. WebGoat provides 12 vulnerabilities, including reflected and stored XSS vulnerabilities. To make the evaluation results more accurate, we add five additional XSS vulnerabilities, which use stricter filtering rules, including specific tag filtering, key function filtering, JS function filtering, etc. To facilitate other researchers to reproduce the experiment, this customized WebGoat is publicly available on GitHub. The second target web application is Jeesns-1.4.2 [42], a social network services framework that provides functions such as forums, microblogs, and communities. We collect eight XSS vulnerabilities from Jeesns in the CVE database.  
We use the DQN, DDQN, and Policy Gradient reinforcement learning (RL) models to experiment on target web applications. As a control, random payload generation is also considered in the experiment. It randomly selects the generation rules in each round of fuzzing. Since the performance of this method is not stable, we conduct five test rounds on the two target web applications, respectively. Then, we take the average number of vulnerabilities found as the experimental results. Table 5 shows the performance of these methods on two target web applications. The time in the table refers to the time taken from finding the first XSS vulnerability to the last one. The termination condition of fuzz testing is that all input points are tested. In the WebGoat experiment, DDQN performed the best and found all the vulnerabilities. Random payload generation performed the worst, and even the poorly performing DQN model found five more vulnerabilities than it. In the Jeesns experiment, reinforcement learning-based fuzzing still performed well. Both DDQN and Policy Gradient found all vulnerabilities. In general, the experimental results demonstrate that payload generation based on reinforcement learning is better thanrandom payload generation and the DDQN model performs better than the other two models in the XSS vulnerability detection scenario.  
To further evaluate the performance of each model on XSS vulnerability detection, we directly fuzz known vulnerable input points and record the time spent. The time spent in each round of fuzzing consists of two parts, one is the execution time of the reinforcement learning model, and the other is the time for the headless browser to scan the web page to determine whether the payload is executed successfully. Tables 6 and 7 show the results of the two target applications, respectively. We found that the payload generated by the DQN can trigger all vulnerabilities, which seems inconsistent with the results in Table 5. Through trial and error, we found that the DQN model is the fastest among the three, taking an average of only 15.2 s per input point test. However, when the fuzzing is too fast, the response time of the target application will become longer with a certain probability. This makes the headless browser consider the page load to be timed out and wrongly think that the payload did not trigger the vulnerability, and eventually leads to false negatives. The Policy Gradient model has a similar situation at some input points. Therefore, we believe these three reinforcement learning models perform well in payload generation. This demonstrates that the accuracy of XSS vulnerability detection depends on generating effective payloads and on correctly judging that the vulnerability is triggered.