## 4 System Design
### Locating RC Points and Negating Procedure
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Program** & **Edges** & **BB** & **ins** & **T/Bugs** & **Options** \\ \hline base64 & 1308 & 822 & - & 44 & -d @ \\ uniq & 1407 & 890 & 5285 & 28 & @ \\ md5 sum & 1560 & 1013 & 7397 & 57 & -c @ \\ who & 3332 & 1831 & 84,648 & 2136 & @ \\ \hline \hline \end{tabular}
\end{table} 
Table 1: LAVA-M dataset statistics. We demonstrate comparative results of our application by using effective fuzzing techniques. These include AFLFast [27] which provides effective power schedules, AFL [3] as a coverage-guided fuzzing tool, T-Fuzz [23] which is an efficient fuzzing approach based on program transformation strategy as well as other state-of-the-art hybrid testing techniques, such as DeepFuzz [47] and QSYM [22]. ### Implementation and Experiment Setup

We have implemented the DeepDiver hybrid fuzzing technique consisting three major modules, i.e., the RC detector, the hybrid fuzzing module based on AFL++ 2.60d [34] and concolic execution engine, the bug validation. The RC detector module was built using the angr tracer [48], and the detected RCs negation component was implemented using _radare2_[36] automatic reverse-engineering framework python script named _r2 pipe_[37; 38]. The concolic executor engine was implemented atop QSYM [22]. The bug validation module was built by angr [49]. We ran all experiments on a machine with 2.7 GHz Intel(R) Core (TM) i5-6400 processor and 32 GB of memory and we used Ubuntu 16.04 LTS version. We used three cores for AFL++ master, AFL++ slave and concolic execution engine, respectively. ### LVA Dataset Evaluation

The LVA dataset involves many various vulnerable programs and is commonly used to evaluate software code defect finding tools. LVA-M dataset is the part of the LVA dataset comprising four buggy programs _base64, uniq, md5 sum_ and _who_. Evaluation time for each program was three and five hours, and we tested each binary five times and summarized the average results. The statistics of LVA-M dataset target programs are illustrated in Table 1, including several edges, basic blocks, instructions, the total listed bugs and test options [50]. We have evaluated our implementation against two hybrid testing tools, DeepFuzz [47] and QSYM [22], in addition to another program transformation fuzzer T-Fuzz [23]. Unfortunately, we cannot test those programs on the Driller [18] hybrid fuzzer because LVA-M dataset applications are not DECREE binaries. Table 2 shows evaluation results in the first group dataset, arrived at by testing techniques mentioned above. Moreover, we borrowed QSYM (P) column results from the QSYM paper and the run time was only five hours. According to the Qsym [22] paper, QSYM detected 100% of vulnerabilities in three binaries of LVA-M dataset including _base64, uniq, md5 sum_ and 57.9% of bugs detected from _who_ binary. QSYM showed high-performance results in the evaluation of LVA-M dataset. Therefore, we tested LVA-M dataset by QSYM tool in our machine and QSYM (R) column represents results from our evaluation. According to the results from the three-hour run time, DeepDiver outperformed them, discovering 88.6% of _base64_, 92.8% of _uniq_ and 84.2% of _md5 sum_ listed bugs in three hours. On average, DeepDiver discovers vulnerabilities 37.6%, 48.5% and 28.8% faster than QSYM, DeepFuzz and T-fuzz, respectively. Our result in the testing binary named _who_ is 101 bugs, and it is not even comparable with the QSYM paper's original result which recorded a large number of bugs--1238 in five hours. However, when we evaluated the QSYM with optimistic solving on the LVA-M dataset five times in our machine, the average number of bugs detected was 71. Probably, one of the reasons why the test results did not match was because our machine was slower than QSYM authors' server machine, which was Intel Xeon E7-4820 8 2.0 GHz cores with 256 GB of RAM [22]. Table 3 illustrates the number of discovered crashes filtered out by Bug Validation and split floating-point instruction numbers in LVA-M dataset.