# Grey-Box Fuzzing Based on Reinforcement Learning for XSS Vulnerabilities
## 1 Introduction  
Web applications provide a friendly user interface for cloud services. With the large-scale implementation of cloud services in recent years, web applications have been widely used in the financial, government, communications, and industrial sectors. However, web applications face a large number of network attacks, such as injection attacks [1], distributed denial of service (DDoS) [2], and cross-site request forgery (CSRF) [3]. They seriously threaten the availability of cloud service providers and users. Cross-site scripting (XSS) vulnerability is one of the most well-known web vulnerabilities [4], which has been difficult to eradicate for a long time and has become a significant threat to web applications. An XSS vulnerability is an injection flaw in a web application caused by untrusted input flowing to a sensitive web application location. Validating all input information during development is the most effective way to eliminate XSS vulnerabilities. The XSS deatsheet series project [5] proposes rules to help developers correctly sanitize or avoid untrusted input in different contexts. In this way, the problem of XSS vulnerability is theoretically solved. However, in practice, it is difficult to correctly insert the data verification process in the program execution flow. Many XSS vulnerabilities still have not been appropriately handled. According to the CVE Program [6], 3403 newly discovered XSS vulnerabilities were assigned CVE numbers in 2022, an increase of 25.4% compared with 2021 and a rise of 56% compared with 2020. This means that the security threats faced by web applications are becoming more severe. The Open Web Application Security Project (OWASP) [7] is a community dedicated to improving software security, which provides liststo tell developers which vulnerabilities are most dangerous. In 2017, the XSS vulnerability was listed as the 7 th on the OWASP Top 10. In the latest version of the OWASP Top 10 in 2021, the risk of the XSS vulnerability rose to the third. This demonstrates that XSS vulnerabilities are a severe challenge to web applications. Many existing studies have proposed detection methods for XSS vulnerabilities. Static analysis is the earliest method used to detect XSS vulnerabilities. Through the analysis of source code, all possible execution paths of the program can be found, which provide information for detecting XSS vulnerabilities. [PERSON] et al. [8] used five static analysis tools, RIPS, PIPS, phpSafe, WAP, and WeVerca, to detect XSS vulnerabilities in 132 plugins of the WordPress content management system. Experimental results show that integrated static analysis tools perform better than individual static analysis tools. However, detection based on static analysis often has many false positives because even the most advanced static analysis techniques cannot accurately predict program execution flow. Fuzz testing is more widely used in XSS vulnerability detection than static analysis. The advantage of this approach is that it can be independent of the web application. If a vulnerability exists, it can also give a proof of concept (POC) for specific vulnerability exploitation. [PERSON] et al. [9] proposed a detection method for DOM-based XSS vulnerabilities based on a dynamic detection framework, analyzing pages with XSS vulnerability risks to obtain taint information and automatically generating attack vectors based on the information for testing. Automatic verification of vulnerabilities can be achieved by exporting attack vectors. However, this method is not perfect. Fuzzing suffers from low API coverage, so it is used with other program analysis techniques, such as dynamic taint analysis [10; 11]. In recent years, researchers have begun to apply deep learning to XSS vulnerability detection and achieved good performance. [PERSON] et al. [12] proposed an ensemble-based approach to deep learning by combining convolutional neural network (CNN) and long short-term memory (LSTM) models.