# SloTFuzzer: Fuzzing Web Interface in IoT Firmware via Stateful Message Generation
## 6 Implementation and Evaluation
### Evaluation of SIoTFuzzer
#### 6.2.3 Research Questions  
Using the previous experimental setup, we would like to answer the following questions:  
* **Q1**: how effective is SloTFuzzer in finding real vulnerabilities in IoT firmware? * **Q2**: how are the suitability and effectiveness of our seed generation function and fuzzing scheduling? * **Q3**: can SloTFuzzer outperform the IoT fuzzing tool FirmFuzz in detecting vulnerabilities? **Effectiveness of Vulnerability Detection (Q1)**: Table 4 lists the vulnerabilities discovered by SloTFuzzer. For each device under test, SloTFuzzer used SMG to automatically generate initial seeds within 1 h, and then started fuzzing within 24 h. Finally, it found 12 vulnerabilities: 7 buffer overflows, 3 command injections, and 2 XSSs. These results show that SloTFuzzer can automatically detect device vulnerabilities based on our SMG mechanism and device monitor. **Effectiveness of the Optimizations (Q2)**: In order to evaluate the effectiveness of our optimizations, we set up three control groups. The specific settings are as follows: for the original \(Boofuzz\), we used the original messages that were analyzed through the front-end as the initial seed to test the device; for \(Boofuzz_{5}\), we added the SMG to test, and for \(Boofuzz_{M}\), we added the mutation strategy. The experiment time was 24 h. The results are shown in Table 5.  
We performed a further manual analysis and found the following:  
1. For comparing \(Boofuzz\) with \(Boofuzz_{5}\), when detecting buffer overflow vulnerabilities, \(Boofuzz\) is able to detect independently. However, it is unable to cause crashes that are triggered by dependency messages. 2. For comparing \(Boofuzz_{5}\) with \(Boofuzz_{M}\), through adding the mutation strategies, we can cause command injection and XSS. However, without the device monitor, command injection cannot be detected. These results show that our optimization can help us to find more vulnerabilities. In Figure 8, comparing \(Boofuzz_{M}\)(grey) with \(Boofuzz\)(blue), the stateful message and mutation strategy could improve the detection speed by 61.99%. 3. SloTFuzzer takes more time than \(Boofuzz_{M}\) to find vulnerabilities. The discovery time was increased by about 11.42%. Due to our device monitor, for every test case, we needed to read the simulation log and find the possible vulnerability. These operations will cause time consumption. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Vulnerability** & **Device** & **Exploit ID** \\ \hline \multirow{8}{*}{**Buffer Overflow**} & D-Link DSL-3782 & CVE-2019-7298 \\  & D-Link DIR-822 & CVE-2019-6258 \\  & Trendnet TEW-652 BRP & CVE-2019-11400 \\  & TP-Link WR940N & CVE-2017-13772 \\  & Netgear WNAP320 & CVE-2016-1555 \\  & D-Link DAP-2695 & CVE-2016-1558 \\  & Trendnet TV-IP110 WN & CVE-2018-19240 \\ \hline \multirow{2}{*}{**Command Injection**} & Trendnet TEW-652 BRP & CVE-2019-11399 \\  & D-Link DSL-3782 & CVE-2018-17990 \\  & D-Link DIR-823G & CVE-2019-7297 \\ \hline \multirow{2}{*}{**XSS**} & D-Link DSL-3782 & CVE-2018-17989 \\  & D-Link DIR-865L & CVE-2018-6529 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: List of discovered known vulnerabilities. **Compare with the FirmFuzz (Q3)**: In order to evaluate the efficiency and the effectiveness of SloTFuzzer, we compared it with \(FirmFuzz\). Every tool ran within 24 h.  
Table 6 lists the efficiency of vulnerability detection by \(FirmFuzz\) and SloTFuzzer. We performed a further manual analysis and found the following:  
1. \(FirmFuzz\) can only find four vulnerabilities, and the most common vulnerability found is buffer overflow. 2. In the total execution time, SloTFuzzer is 17.64% to 23.53% faster than \(FirmFuzz\). These results indicate that our work can find more vulnerabilities and detection time is reduced by about 20.57% on average.