# Fast Format-Aware Fuzzing for Structured Input Applications
## 4 Implementation and Evaluation
### Evaluation Setup  
**Experiment environment:** We conducted all our experiments on a server running 64-bit Ubuntu 20.04, with two Intel(R) Xeon(R) [EMAIL_ADDRESS] GHz CPUs and 128 GB of RAM. Each fuzzer on each target application was run in identical configurations, and the experiments were repeated five times in all cases.  
**Baseline:** In the evaluation, FFAFuzz was compared with Redqueen [12] and WEIZZ [15]. Table 2 lists some information about these fuzzers.  
Through padding the fragments with random values, Redqueen observes those fragments that triggered execution path changes and recognizes I2S fields within these fragments via matching the fragment value only based on colorization. WEIZZ recognizes both I2S and indirect dependencies by byte-by-byte analysis instead of value matching like Redqueen does. It observes the changing _cmp_ operand and marks the dependency between the flipped byte and the _cmp_ operand if the operand changed in value consequently after flipping this byte.  
Considering that Redqueen was developed on KAFL [27] and spotlights kernel fuzzing, we adopted the Redqueen mode of AFL++ [4] version 3.14c [28] to be fair.  
**Benchmarks:** Eleven target programs were selected as the benchmarks, representing different types of structured input binaries, including audio, image, and video processing, executable program parsing, etc. Table 3 details those targets.  
\begin{table}
\begin{tabular}{l l l l} \hline
**Fuzzer** & **Dependency Recognition via Value** & **Dependency Recognition via Value** \\  & **Matching** & **Alteration Inference** \\ \hline Redqueen & Value matching based on colorization & \(\bigtimes\) \\ WEIZZ & \(\bigtimes\) & Byte-by-byte analysis \\ FFAFuzz & _de [PERSON] matching_ & _Selective Flip_ \\ \hline \end{tabular}
\end{table}
Table 2: Baseline.  
\begin{table}
\begin{tabular}{l l l l} \hline
**Target** & **Version** & **Description** & **Binary** & **Execution Arguments** \\ \hline ffmpeg & 5.0.1 “Lorentz” & Audio parsing and processing & ffmpeg & -y-i @@ -cv mpeg4 -ca out.mp4 \\ \hline readelf & 2.38 & Executable and Linkable Format (.elf) file parsing & readelf & -a @@ \\ \hline objdump & 2.38 & Executable and Linkable Format (.elf) file parsing & objdump & -x @@ \\ \hline \multirow{2}{*}{faad} & \multirow{2}{*}{2.9.2} & Advanced Audio Coding (.aac) file parsing and processing & \multirow{2}{*}{faad} & \multirow{2}{*}{@@@ -o out.wav} \\ \cline{1-1} \cline{3-3}  & & & Portable Network Graphics (.png) file parsing and processing & \\ \hline gif2 rgb & giflib-5.2.1 & Graphics Interchange Format (.gif) file parsing and processing & gif2 rgb & \\ \hline xls2 csv & Catdoc 0.95 & Microsoft Excel 97-2003 Worksheet (xls) file parsing & xls2 csv & \\ \hline catptpt & Catdoc 0.95 & Microsoft PowerPoint Presentation (.ppt) file parsing & catptpt & @@ \\ \hline opj\_decompress & openipeg-2.4.0 & JPEG 2000 Core Coding format (.jp2) file parsing and processing & opj\_decompress & -i @@ -o out.png \\ \hline opj\_dump & openipeg-2.4.0 & JPEG 2000 Core Coding format (.jp2) file parsing & opj\_dump & -i @@ \\ \hline \multirow{2}{*}{djpeg} & libjpeg-turbo-2.1.3 & Joint Photographic Experts Group (.jpeg) file parsing and processing & \multirow{2}{*}{djpeg} & \multirow{2}{*}{-gif\_outfile out.git @@} \\ \cline{1-1} \cline{3-3}  & & & \\ \end{tabular}
\end{table}
Table 3: Targets and configurations.