## 4 Experiment
### Experimental Results
#### 4.2.3 Experiment 3: Friedman-Nemenyi Test  
To verify that our algorithm has a significant performance difference compared to the baseline on multiple datasets, we conduct the Friedman-Nemenyi test on various ensemble algorithms. By ranking the performance of each algorithm on each dataset and calculating the average rank of each algorithm across all datasets, we then compute the Friedman statistic \(\chi_{F}^{2}\) using the formula shown in Equation (22). Here, \(n\) represents the number of datasets, \(k\) represents the number of algorithms, and \(rm_{j}\) represents the average rank of the \(j\)-th algorithm. Based on the degrees of freedom and the significance level, we calculated the \(p\)-value. If the \(p\)-value is less than the significance level \(\alpha\), we reject the null hypothesis, indicating that there is a significant difference in algorithm performance. After the Friedman test rejects the null hypothesis, we further use the Nemenyi test to determine which algorithms have significant differences in performance. The critical difference (CD) value is used to judge whether the difference in average ranks between two algorithms is significant. The CD value is calculated using the formula shown in Equation (23), where \(q_{\alpha,k}\) is the critical value obtained from the Nemenyi test table. If the difference in average ranks between two algorithms is greater than the CD value, their performance is considered significantly different. In our experiments, we set \(n=8\), \(k=6\), degrees of freedom \(df=k-1=5\), and \(\alpha=0.05\). From the table, we obtained \(q_{\alpha,k}=2.850\). Based on these values, we conducted the Friedman-Nemenyi test using accuracy and F1 score, as described below. \[\chi_{F}^{2}=\frac{12n}{k(k+1)}\left(\sum_{j=1}^{k}\left(rm_{j}\right)^{2}- \frac{k(k+1)^{2}}{4}\right) \tag{22}\]  
\[CD=q_{\alpha,k}\sqrt{\frac{k(k+1)}{6n}} \tag{23}\] \begin{table}
\begin{tabular}{c c c c c c c} \hline
**Accuracy (\%)** & **CTS** & **CTS-SR** & **DT** & **LightGBM** & **MLP** & **SVM** \\ \hline Australian & 86.95 & **87.24** & 82.60 & 85.79 & 81.73 & 86.66 \\ German & **75.6** & 74.2 & 67.2 & 73.6 & 68.2 & 74.0 \\ spam & **94.08** & 89.56 & 89.48 & 90.65 & 93.69 & 92.13 \\ biodeg & 85.98 & **87.12** & 79.16 & 84.84 & 85.60 & 86.93 \\ magazine & **81.04** & 80.53 & 73.79 & 76.49 & 80.87 & 80.62 \\ all-beauty & 85.82 & 86.61 & 72.83 & 84.25 & 83.46 & 86.61 \\ musk & **84.13** & 81.43 & 69.46 & 75.74 & 78.74 & 79.64 \\ Hilly & **55.94** & 50.82 & 54.78 & 53.46 & 50.19 & 49.17 \\ \hline \end{tabular}
\end{table} 
Table 6: Accuracy result compared to single classifier. \begin{table}
\begin{tabular}{c c c c c c c} \hline
**F1 Score (\%)** & **CTS** & **CTS-SR** & **DT** & **LightGBM** & **MLP** & **SVM** \\ \hline Australian & **84.94** & 85.23 & 80.00 & 82.63 & 78.70 & 84.86 \\ German & 49.58 & 37.07 & 37.95 & **49.78** & 46.37 & 44.91 \\ spam & 92.08 & 84.99 & 86.76 & **93.51** & 91.95 & 89.69 \\ biodeg & 79.89 & **80.34** & 68.64 & 79.66 & 77.26 & 80.22 \\ magazine & 87.32 & 82.65 & 82.58 & 87.38 & **87.75** & 87.39 \\ all-beauty & 82.52 & 82.52 & 73.97 & 80.78 & 79.41 & **83.16** \\ musk & **81.27** & 76.69 & 62.63 & 72.40 & 75.17 & 73.64 \\ Hilly & 59.48 & 57.53 & 58.29 & 57.88 & 63.17 & **63.24** \\ \hline \end{tabular}
\end{table} 
Table 7: F1 Score compared to single classifier. Based on the accuracy values in Table 4, the Friedman statistic was calculated to be 17.125, yielding a \(p\)-value of 0.004, which is less than \(\alpha\). This indicates that there are significant differences among the compared ensemble algorithms. Subsequently, the Nemenyi post hoc test was conducted, and the critical difference (CD) was calculated to be 2.666. Figure 1 displays the average ranks of the ensemble algorithms for accuracy. If the two horizontal line segments in the figure do not overlap, it signifies a significant performance difference between the two algorithms. It can be seen that the horizontal lines corresponding to CTS do not intersect with those of RF and AdaBoost.