# A Vulnerability Scanning Method for Web Services in Embedded Firmware
## 5 Evaluation
### Efficacy of Taint Analysis  
Based on reducing sensitive paths, we adopt lightweight function pollution specification which reduces the exploration path of the taint analysis and dramatically improves the efficiency of finding embedded web server vulnerabilities in firmware. As shown in Figure 6, compared with the well-known work SATC, it is found that the efficiency is improved by 40% on average; x2000r and a3002r were not completed within the specified time (12 h) due to too many sensitive call paths.  
As shown in Table 8, we calculated the proportion of true positives in alerts (TP/Alert) and the proportion of vulnerabilities in true positives in the analysis results of each sample. We believe that the value of TP/Alert only reflects the accuracy of whether the taint analysis exploration path is reachable. In fact, whether the path is reachable does not necessarily mean that a vulnerability exists, so we also calculated Vulnerability/TP. We compared the statistical results of WFinder and SaTC. It was found that WFinder outperforms SaTC on  
\begin{table}
\begin{tabular}{c} acosNvramConfig\_set \\ acosNvramConfig\_get \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 6: External data entry point recognition. BB is the boundary binary; EDIF is the external data introduction function; CSF is the settings configuration function.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Series**} & \multirow{2}{*}{**Binary**} & \multicolumn{4}{c}{**SaTC**} & \multicolumn{2}{c}{**WFinder**} \\ \cline{3-8}  & & **DF** & **IE** & **SP** & **FDF** & **IE** & **SP** \\ \hline AX1803 & tdhttpd & 649 & 2847 & 1324 & 457 & 345 & 257 \\ AX1806 & tdhttpd & 612 & 2721 & 977 & 429 & 367 & 276 \\ G1/G3 & httpd & 1268 & 4304 & 63,523 & 462 & 598 & 1236 \\ DIR823G & goahead & 254 & 670 & 672 & 208 & 48 & 143 \\ DIR816 & goahead & 316 & 427 & 1782 & 217 & 395 & 1322 \\ X2000R & boa & 909 & 1067 & 4,760,783 & 552 & 822 & 253,677 \\ A3002R & boa & 913 & 1171 & 3,524,763 & 505 & 785 & 245,343 \\ R7000 & httpd & 3813 & 5509 & 170,774 & 2907 & 1774 & 3566 \\ R7000P & httpd & 3615 & 5621 & 175,822 & 2758 & 1764 & 3687 \\ \hline \hline \end{tabular}
\end{table}
Table 7: Sensitive path reduction. DF is the dangerous function, IE is the input entry, SP is the sensitive path, and FDF is the filtered dangerous function.  
Vulnerability/TP. WFinder reduces false positives caused by factors other than the taint analysis by accurately locating the starting and ending points of the taint analysis. Therefore, true positives in WFinder results are more likely to be verified as vulnerabilities. In fact, the false positives of the taint analysis are difficult to avoid, and the cost of improving its accuracy is not worth it. We need to balance the efficiency and accuracy of the taint analysis. This is why WFinder is generally not as good as SaTC on TP/Alert. WFinder simplifies the function call specification for the taint analysis in exchange for higher analysis efficiency. But we also use accurate positioning of the starting and ending points to compensate for their false positives, reducing them to an acceptable range for manual verification. In fact, according to Table 5, the number of alerts we issued is not significantly different from SaTC, but our analysis efficiency has dramatically improved. In addition, due to the simplified function call specification, which can avoid path explosions encountered during the analysis process, we explored more vulnerabilities than SaTC.