## 5 Evaluation
### Coverage Improvement
We can see that for the QSYM method, it is hard to increase the coverage after some time, where the curve tends to be flat. On the contrary, SHFuzz still shows potential increase trends. This is because at the same time-out, since QSYM is trying to calculate all branches without any selection, it will always be stuck in the same shallow branches and cannot go deeper into the program paths. However, SHFuzz can always try to solve the most critical branches, which can guide the execution to deeper paths. \begin{table}
\begin{tabular}{l l r r r} \hline  & **Method** & **Coverage** & **Crash** & **Bug Found Time** \\ \hline vorbis-2017-12-11 & QSYM & 3028 & 0 & - \\  & SHFuzz & 3360 & 194 & 2:16:45 \\ \hline boringssl-2016-01-12 & QSYM & 1529 & 0 & - \\  & SHFuzz & 1613 & 0 & - \\ \hline harfbuzz-1.3.2 & QSYM & 11,995 & 0 & - \\  & SHFuzz & 12,093 & 0 & - \\ \hline libarchive-2017-01-04 & QSYM & 5273 & 0 & - \\  & SHFuzz & 5836 & 0 & - \\ \hline libjpeg-turbo-07-2017 & QSYM & 4135 & 0 & - \\  & SHFuzz & 4265 & 0 & - \\ \hline proj4-2017-08-15 & QSYM & 1286 & 0 & - \\  & SHFuzz & 1336 & 0 & - \\ \hline re2-2014-12-09 & QSYM & 5305 & 0 & - \\  & SHFuzz & 5456 & 0 & - \\ \hline sqlite-2016-11-14 & QSYM & 7313 & 0 & - \\  & SHFuzz & 10,835 & 0 & - \\ \hline woff2-2016-05-06 & QSYM & 1208 & 0 & - \\  & SHFuzz & 1547 & 0 & - \\ \hline openssl-1.0.1f & QSYM & 557 & 0 & - \\  & SHFuzz & 623 & 0 & - \\ \hline libpng-1.2.56 & QSYM & 1273 & 0 & - \\  & SHFuzz & 1365 & 0 & - \\ \hline readelf-2.34 & QSYM & 14,266 & 0 & - \\  & SHFuzz & 14,718 & 0 & - \\ \hline \end{tabular}
\end{table} 
Table 2: _Cont._For the negative results in nm-2.34, openssl-1.0.2d, and json-2017-02-12, each decreases coverage by 124, 50, and 37, which are rather small numbers. We can see that SHFuzz still generates more unique crashes and can find the existing bugs faster than QSYM in these three applications, showing the strength of SHFuzz in finding deeply-hidden bugs. We will explain this in detail in the next subsection. \begin{table}
\begin{tabular}{l r r r r} \hline \hline  & **Coverage** & **Percentage** & **Crash** & **Bug Found Time** \\ \hline c-ares-CVE-2016-5180 & +0 & +0\% & +4 & 20.13\(\times\) \\ \hline guetzli-2017-3-30 & +559 & +14.94\% & +17 & 2.22\(\times\) \\ \hline json-2017-02-12 & \(-\)37 & \(-\)1.16\% & +7 & 3.23\(\times\) \\ \hline llvm-libcxxabi-2017-01-27 & +899 & +8.51\% & +61 & 1.36\(\times\) \\ \hline openssl-1.0.2d & -50 & \(-\)1.57\% & +49 & 1.96\(\times\) \\ \hline pcre2-10.00 & +2512 & +10.86\% & +547 & 2.66\(\times\) \\ \hline nm-2.34 & -124 & \(-\)1.78\% & +17 & 23.42\(\times\) \\ \hline libxml2-v2.9.2 & +320 & +5.80\% & +6 & new \\ \hline vorbis-2017-12-11 & +332 & +10.96\% & +194 & new \\ \hline boringssl-2016-01-12 & 84 & +5.49\% & 0 & - \\ \hline harfbuzz-1.3.2 & +98 & +0.82\% & 0 & - \\ \hline libarchive-2017-01-04 & +563 & +10.67\% & 0 & - \\ \hline libjpeg-turbo-07-2017 & +130 & +3.14\% & 0 & - \\ \hline proj4-2017-08-15 & +50 & +3.88\% & 0 & - \\ \hline re2-2014-12-09 & +151 & +2.84\% & 0 & - \\ \hline sqlite-2016-11-14 & +3522 & +48.16\% & 0 & - \\ \hline woff2-2016-05-06 & +339 & +28.06\% & 0 & - \\ \hline openssl-1.0.1f & +66 & +11.85\% & 0 & - \\ \hline libpng-1.2.56 & +92 & +7.22\% & 0 & - \\ \hline readelf-2.34 & +452 & +3.17\% & 0 & - \\ \hline average & +497 & +8.40\% & +100 & 7.85\(\times\) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Increased number of coverage, crashes, and bugs found time by SHFuzz compared with QSYM. Figure 5: _Cont_.