## 5 Evaluation
### Experimental Setup  
**Fuzzers for evaluation. ** The main contribution of this paper is utilizing task allocation based on callgraphs in ensemble fuzzing to improve its efficiency by reducing duplicate work. Therefore, to demonstrate the effectiveness of our approach, we compare our task allocation-based ensemble fuzzing framework (TAEF) with AFILTeam, which uses task allocation in parallel fuzzing of homomorphic fuzzers, and original ensemble fuzzing framework (OEF) without task allocation. Specifically, as shown in Table 1, considering the diversity of fuzzers, in our ensemble fuzzing framework (TAEF), we use four different fuzzers including AFLFast [5], MOPT [9], QSYM [27], and radamsa [36] to conduct fuzzing using the approach described in this paper. In the original ensemble fuzzing framework (OEF), we select the same four fuzzers as in our framework, while we run the tests independently, only with periodical synchronization among the seeds to coordinate the work of fuzzers. As for AFILTeam, we use four instances of the provided fuzzer, which is highly coupled to its task division strategy, in parallel. **Programs under test. ** Due to the specific requirements of the fuzzing frameworks being evaluated, the selection of tested programs needs to adhere to stringent conditions. Firstly, AFILTeam requires the tested programs to have fixed input formats, and aflsmart needs to support these input formats. This greatly limits the number of programs that can be chosen for evaluation. After searching for commonly used benchmarks for fuzzing, we found over forty programs in addition to the four programs used by AFILTeam. However, we also need to evaluate the ensemble fuzzing frameworks, and qsym used in the ensemble fuzzing framework requires the Ubuntu 16.04 version, while other fuzzers require the Ubuntu 18.04 version. This means that the selected programs need to be able to run on both versions of the system simultaneously. Furthermore, the selected programs must have \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Frameworks** & **Number of Instances** & **Used Fuzzers** & **Time and Memory Limit** \\ \hline
**AFLTeam** & 4 & horsefuzz & -t 2000+ -m none \\
**OEF** & 4 & AFLFast, MOPT, QSYM, and radamsa & -t 2000+ -m none \\
**TAEF** & 4 & AFLFast, MOPT, QSYM, and radamsa & -t 2000+ -m none \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Fuzzers’ settings of AFLTeam, OEF, and our prototype named TAEF. identical execution paths on both versions of the Ubuntu system, otherwise, the comparison of branch coverage loses its meaning. Finally, after testing over a hundred different versions of the preselected programs, we found six programs that fully meet the aforementioned challenging conditions, including three out of the four programs used by AFLTeam. Details of these programs and the additional parameters used are shown in Table 2.  
**Parameters configuration. ** For AFLTeam, we configured it to use four cores and set the exploration phase to 2 h and the exploitation phase to 4 h with two repetitions. Both the evaluation of the original ensemble fuzzing framework (DEF) and our prototype (TAEF) were executed in docker containers due to the different environments required by the fuzzers that they contain. Each fuzzer was assigned to a docker container with a single core bound to it. For the original ensemble fuzzing, we set the time limit to 10 h, while for our prototype, which was consistent with AFLTeam, the exploration phase was set to 2 h, and the exploitation phase was set to 4 h with two repetitions. Afterward, we reported the number of covered branches as the integer average of the five testing results. It is worth noting that the chosen program for testing has a longer execution time, which may exceed the default settings of these fuzzers. So, we have to set a longer timeout for each testcase. In our experimental environment, a timeout of 2000 ms can ensure that the majority of testcases do not exceed the time limit.