## 4 Experimental Setup
### Evaluation Metrics  
We evaluate the effectiveness of our approach with two sets of metrics, including _F-score_ as well as _Accuracy_ to show the performance of classifiers and \(APFD\) to show the performance of ranking test cases. **Evaluating the classifiers in test case prioritization**. We measured the evaluation of classifiers in the ranking models in test case prioritization with typical measurements: _Precision_, _Recall_, _F-score_, and _Accuracy_. Those measurements are defined based on True Positive (TP), False Positive (FP), False Negative (FN), and True Negative (TN) for test pairs, i.e., a vector that combines a functional unit and a test case. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Dataset** & **Module or Fold as Test Set** & **Functions and Test Cases** & **Training Set** & **Test Set** \\ \hline \multirow{6}{*}{\begin{tabular}{} \end{tabular} } & \# Functions & 2992 & 192 \\ \cline{2-4}  & & \# Defective functions & 1488 & 66 \\ \cline{2-4}  & & \# Test Cases & 80 & 11 \\ \cline{2-4}  & & \# Defective functions covered by tests & 69 & 8 \\ \hline \multirow{6}{*}{\begin{tabular}{} \end{tabular} } & \# Functions & 2287 & 897 \\ \cline{2-4}  & & \# Functions with seeding defects & 1160 & 394 \\ \cline{2-4}  & & \# Test Cases & 48 & 43 \\ \cline{2-4}  & & \# Defective functions covered by tests & 36 & 33 \\ \cline{2-4}  & & \# Functions & 2607 & 577 \\ \cline{2-4}  & & \# Functions with seeding defects & 1217 & 337 \\ \cline{2-4}  & & \# Test Cases & 79 & 12 \\ \cline{2-4}  & & \# Defective functions covered by tests & 45 & 24 \\ \hline \multirow{6}{*}{\begin{tabular}{} \end{tabular} } & \# Functions & 2190 & 994 \\ \cline{2-4}  & & \# Functions with seeding defects & 1051 & 503 \\ \cline{2-4}  & & \# Test Cases & 72 & 19 \\ \cline{2-4}  & in evaluation) & \# Defective functions covered by tests & 69 & 0 \\ \hline \multirow{6}{*}{\begin{tabular}{} \end{tabular} } & \#Functions & 2548 & 636 \\ \cline{2-4}  & & \# Functions with seeding defects & 1431 & 123 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Defective functions covered by tests & 64 & 5 \\ \hline \multirow{6}{*}{\begin{tabular}{} \end{tabular} } & \#Functions & 2548 & 636 \\ \cline{2-4}  & & \# Functions with seeding defects & 11329 & 1431 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Defective functions covered by tests & 59 & 10 \\ \cline{2-4}  & & \# Functions with seeding defects & 2548 & 636 \\ \cline{2-4}  & & \# Defective functions covered by tests & 59 & 10 \\ \cline{2-4}  & & \# Functions with seeding defects & 1267 & 287 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Defective functions covered by tests & 59 & 19 \\ \hline \multirow{6}{*}{\
\begin{tabular}{} \end{tabular} } & \#Functions & 2544 & 660 \\ \cline{2-4}  & & \# Functions with seeding defects & 1038 & 516 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Functions with seeding defects & 1151 & 403 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Defective functions covered by tests & 50 & 19 \\ \hline \multirow{6}{*}{
\begin{tabular}{} \end{tabular} } & \#Functions & 2544 & 660 \\ \cline{2-4}  & & \# Functions with seeding defects & 1038 & 516 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Functions with seeding defects & 1038 & 516 \\ \cline{2-4}  & & \# Test Cases & 91 & 91 \\ \cline{2-4}  & & \# Defective functions covered by tests & 47 & 22 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Data information of training sets and test sets. We show the division of training sets and test sets of eight rounds of evaluation. # _Defective functions covered by tests_ is the number of defective functions that are covered by test cases. * TP: # of defective test pairs that are predicted as defective. * FP: # of undefective test pairs that are predicted as defective.