# State-Sensitive Black-Box Web Application Scanning for Cross-Site Scripting Vulnerability Detection
## 3 Approach
### State-Sensitive Crawler
#### 3.2.4 Filter  
For all edges obtained by the extractor, there may be repeated GET requests, as well as edges related to similar form submissions. The filtering algorithm is shown in Algorithm 1. In the navigation model graph, a list of unique URLs, denoted \(URL\), is maintained. This helps filter the construction of repeated GET requests. Alternatively, we have defined two functions (similarUrl and similarForm) to filter similar form submissions. After filtering out similar new edges, we add all the unique edges to the navigation model.  
**Repeated GET requests:** If the URL (only including GET requests) of page C exists on page A, and the URL of page C exists on page B, then only one edge of page C needs to be built to visit it.  
**Edges of similar form submissions:** We will illustrate similar form submissions through the following two examples. First, every product page [[http://example.com/product.php?id=X](http://example.com/product.php?id=X)]([http://example.com/product.php?id=X](http://example.com/product.php?id=X)) contains a form with action [[http://example.com/product.php?id=X&action=edit](http://example.com/product.php?id=X&action=edit)]([http://example.com/product.php?id=X&action=edit](http://example.com/product.php?id=X&action=edit)) for modifying information. We then only need to build a corresponding edge (X represents the number of the product). Second, those two URLs can be [[http://example.com/product.php/X](http://example.com/product.php/X)]([http://example.com/product.php/X](http://example.com/product.php/X)) and [[http://example.com/product.php/X/edit](http://example.com/product.php/X/edit)]([http://example.com/product.php/X/edit](http://example.com/product.php/X/edit)) after URL rewriting.  
**Function similarUrl:** The function is used to determine whether the URLs of the page where the form is located are similar. It determines similarity through the protocol of the URL, domain name, path and parameter list. The parameter list consists of parameter names passed through the \"get\" method. Except for one item in the path that can be changed, the protocol, domain name, and parameter list should be consistent, e.g., [[http://example.com/product.php?id=1](http://example.com/product.php?id=1)]([http://example.com/product.php?id=1](http://example.com/product.php?id=1)) and [[http://example.com/product.php?id=2](http://example.com/product.php?id=2)]([http://example.com/product.php?id=2](http://example.com/product.php?id=2)) are similar, [[http://example.com/product.php/1/edit](http://example.com/product.php/1/edit)]([http://example.com/product.php/1/edit](http://example.com/product.php/1/edit)) and [[http://example.com/product.php/2/edit](http://example.com/product.php/2/edit)]([http://example.com/product.php/2/edit](http://example.com/product.php/2/edit)) are similar.  
**Function similarForm:** This function is used to determine whether the forms are similar. It is mainly judged by the similar action attribute and the same parameter list of the form. The similarity judgment of the action attribute still adopts a function similar Url.  
```
Input: The list of possible new edges, \(E_{n}\)  The list of unique URLs, \(URL\)  The dictionary of forms that have been visited, \(D\) Output: The list of unique edges, \(U_{e}\) // Every edge contains two nodes and necessary information. for\(e\in E_{n}\)do \(similarEdge=False\) iftype(\(e\)) = getthen if\(e.node2.url\in URL\)then \(similarEdge=True\)  end if iftype(\(e\)) = formthen for\(url\in D\)do ifsimilarUrl(\(e.node1.url\),url)then // \(e.value.form\) contains the form information we care about;\(D[url]\) is a list. It represents all the forms in the web page whose URL is equal to \(url\). ifsimilarForm(\(e.value.form\),\(D[url]\))then \(similarEdge=True\) break  end if  end if  end if ifnotsimilarEdgethen \(D[url].append(e.value.form)\)  end if  end if  end for return\(U_{e}\)
```  
**Algorithm 1**Function filter