# Optimizing IoT Web Fuzzing by Firmware Infomation Mining
## 4 Evaluation
### Comparison with Current Work  
To illustrate the effectiveness of our tool, we compare IoTParser with W13 scan and Firmhunter [7].  
Since W13 scan has no crawler function, we add the crawler function part to W13 scan, marked as W13 scan*, to ensure that it has the same initial seed as IoTParser. Firmhunter is the first work to introduce the seed scheduling strategy into the IoT web fuzzing. Compared with the previous work, the speed of vulnerability discovery has been greatly improved, so we chose to compare it.  
The final experimental results are shown in Table 2. Since Firmhunter only relies on the web crawler for seed generation, the coverage rate is relatively low, resulting in four vulnerabilities not being discovered. Among the vulnerable seeds that can be covered, the discovery speed of IoTParser is significantly higher than that of Firmhunter, which shows the effectiveness of scheduling based on shared keywords. Since W13 scan* cannot monitor the running information of the firmware, it cannot find vulnerability without echo information but can only find XSS and unauthorized access. At the same time, since there is no scheduling method, the tests are performed in sequence, resulting in very low efficiency.  
It should be noted that unauthorized access is a logic vulnerability, so we only need to test the initial seed, no mutation is required, so there is no improvement effect. The seed corresponding to the unassigned-2 vulnerability is in the first place in order, and it is still the first after scheduling according to the weight calculation. Therefore, IoTParser and W13 scan* behave similarly here, while Firmhunter needs to perform dynamic requests and parse logs which takes a long time.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Type** & **Vender** & **Device** & **Version** \\ \hline \multirow{4}{*}{Router} & ASUS & NetgeRT-N53 & 3.0.04 \\  & Netgear & WNAP-320 & 2.0.3 \\  & Netgear & WNDR-3700 & 1.0.0.10 \\  & D-Link & DIR-825 & 2.05b \\ \multirow{2}{*}{IP Camera} & Trendnet & IP110-wn & 1.2.2.68 \\ \hline \hline \end{tabular}
\end{table}
Table 1: IoT devices for testing.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**CVE** & **Vender** & **Device** & **Vuln Type** & **IoTParser** & **Firmhunter** & **W13 scan* \\ \hline CVE-2019-20082 & asus & RT-N53 & BO & 21 m 43 s & 57 m 8 s & NA \\ unassigned-0 & asus & RT-N53 & CI & 43 m 40 s & 1 h 23 m & NA \\ CVE-2021-31655 & Trendnet & ip110 wn & XSS & 7 m 37 s & 16 m 04 s & 17 m 34 s \\ CVE-2019-11417 & Trendnet & ip110 wn & BO & 8 m 48 s & 21 m 24 s & NA \\ CVE-2018-19241 & Trendnet & ip110 wn & BO & 16 m 26 s & 47 m 58 s & NA \\ unassigned-1 & Trendnet & ip110 wn & XSS & 31 m 40 s & 37 m 17 s & 57 m 28 s \\ unassigned-2 & Trendnet & ip110 wn & XSS & 3 m 15 s & 9 m 18 s & 3 m 7 s \\ CVE-2016-1555 & netgear & wmap320 & CI & 4 m 9 s & NA & NA \\ unassigned-3 & netgear & wmap320 & unauth & 4 m 24 s & NA & 4 m 11 s \\ unassigned-4 & netgear & wmap320 & CI & 19 m 35 s & 24 m 21 s & NA \\ unassigned-5 & netgear & wndr3700 & CI & 4 h 35 m & 5 h 27 m & NA \\ unassigned-6 & netgear & wndr3700 & CI & 4 h 16 m & NA & NA \\ CVE-2019-9126 & dlink & dir-825 & unauth & 6 m 41 s & NA & 6 m 53 s \\ \hline \hline \end{tabular}
\end{table}
Table 2: Experimental results of IoTParser, Firmhunter, and W13 scan*. unassigned-num refers to the serial number of the 0-day vulnerability discovered.