## 7 Evaluation
### Benchmarks  
The benchmarks we constructed to evaluate our system are shown in Table 3. These benchmarks are composed of six test programs, eight challenges from CTF events and four real-world applications with exposed vulnerabilities. These benchmarks are all user-mode applications running on the Linux platform in the ELF format. These programs are collected based on the following criteria.  
* Vulnerilities commonly existing in binary programs should be covered by the benchmarks. As shown in Table 3, vulnerabilities including stack overflow, integer overflow, format string, use after free, and heap overflow are all covered in this collection. These are all common vulnerabilities exhibited in binary programs.
* The benchmark programs should be protected by StackGuard, and forging a canary leakage event is a must in exploitability evaluation. As shown in Table 3, the 'Leak Type' column lists the method that should be taken to forge the desired canary leakage event. Note that 'ARR' and 'SBM' separately stand for activation record restoration and symbolic output buffer manipulation discussed in Section 4.2.
* The benchmarks should also cover as much as possible the exploitation mitigation techniques. As we can see, in the 'protection' column in Table 3, except for StackGuard, mainstream protection mechanisms including NX and ASLR are all covered in this collection. It is worth noting that for benchmark programs that run under an ASLR environment, we complete the exploitability evaluation process by merging CanaryExp with our previous work [37].  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Program** & **Vulnerability Type** & **Protections** & **Leak Type** \\ \hline \multirow{6}{*}{test programs} & stack1 & stack overflow & Canary, NX, ASLR & ARR \\ \cline{2-5}  & intovf & stack overflow, integer overflow & Canary, NX & ARR \\ \cline{2-5}  & fmtstr1 & format string & Canary & SBM \\ \cline{2-5}  & fmtstr2 & format string & Canary & SBM \\ \cline{2-5}  & heap-vul1 & use after free & Canary & SBM \\ \cline{2-5}  & heap-vul2 & heap overflow & Canary, NX & ARR \\ \hline bjdctf 2020 & babyrop2 & stack overflow, format string & Canary, NX & SBM \\ \hline moecft 2020 & baby\_canary & stack overflow, format string & Canary, NX, ASLR & SBM \\ \hline BugkuCTF & canary & stack overflow & Canary, NX & ARR \\ \hline insommihack CTF 2016 & microwave & stack overflow, format string & Canary, NX, ASLR & SBM \\ \hline
2018 anheng cup CTF & babypie & stack overflow & Canary, NX, ASLR & ARR \\ \hline CSAW Quals CTF 2017 & scv & stack overflow & Canary, NX & SBM \\ \hline \multirow{2}{*}{xctf} & Mary\_Morton & stack overflow, format string & Canary, NX & SBM \\ \cline{2-5}  & babystack & stack overflow & Canary, NX & ARR \\ \hline \multirow{6}{*}{real applications} & profitpd 1.3.1 (CVE-2006-6563) & stack overflow & Canary, ASLR & ARR \\ \cline{2-5}  & iwconfig v26 (BID-8901) & stack overflow & Canary, NX & ARR \\ \cline{2-5}  & dnsmasq 2.77 (CVE-2017-14493) & stack overflow & Canary, NX, ASLR & ARR \\ \cline{2-5}  & rsync 2.5.7 (CVE-2004-2093) & stack overflow & Canary, NX & ARR \\ \hline \hline \end{tabular}
\end{table}
Table 3: Evaluation benchmark.