# An Adaptive Active Learning Method for Multiclass Imbalanced Data Streams with Concept Drift
## 4 Experimental Evaluation and Analysis
### Ablation Study  
In the final experiment, the objective was to conduct ablation experiments to help analyze whether the various components of the AdaAL-MID could contribute to improving its ability to handle concept drift and class imbalance. The AdaAL-MID consists of four main features, and this section studied them individually by disabling each feature to form four variants. First, the first algorithm compared was the complete AdaAL-MID (V0). Secondly, the dynamic label budget strategy was removed, and a fixed label budget was used, forming V1 (Fixed budget). Then, in the uncertainty strategy based on the dual-margin dynamic threshold matrix, only the minimum margin matrix was used to obtain V2 (Only margin-min), and only the maximum margin matrix was used to obtain V3 (Only margin-max). Finally, the adaptive sampling strategy was removed, and a static sampling weight of \(SW_{\alpha}=6\) (the setting used in the classic algorithm LB) was used to form V4 (Static \(SW\)). The experiments were conducted on synthetic data streams Fixed_Mix_SIG and Var_Mix_SIG with mixed concept drift and on the real data stream connect-4. Table 10 and Figure 11 respectively present the detailed results of the ablation experiments and the real-time change curves of recall.  
Figure 10: BD test results on five metrics: (**a**) recall; (**b**) accuracy; (**c**) G-mean; (**d**) F1-score; (**e**) kappa.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Data**} & \multicolumn{6}{c}{**Supervised Learning Algorithms**} & \multicolumn{6}{c}{**Active Learning Algorithms**} \\ \cline{2-11}  & **LB** & **MOOB** & **ARFe** & **SRP** & **CSARF** & **ROSE** & **ROALE-DI** & **CALMID** & **MixFool-E** & **Proposed** \\ \hline Recall & 5.29 & 6.71 & 5.08 & 6.79 & 4.75 & 3.00 & 9.88 & 4.00 & 7.92 & **1.58** \\ Accuracy & 4.38 & 8.17 & 4.58 & 4.96 & 8.25 & 3.46 & 8.79 & 3.42 & 7.33 & **1.67** \\ G-mean & 5.21 & 6.63 & 5.33 & 7.29 & 4.42 & 3.13 & 9.63 & 3.88 & 7.71 & **1.54** \\ F1-score & 4.46 & 7.88 & 4.17 & 4.88 & 7.00 & 2.83 & 9.71 & 3.88 & 8.25 & **1.71** \\ Kappa & 4.50 & 7.46 & 4.58 & 5.21 & 8.00 & 3.08 & 9.29 & 3.54 & 7.63 & **1.71** \\ \hline \hline \end{tabular} Bold entries are used to indicate the best results.  
\end{table}
Table 9: Average ranks of the algorithms.  
As shown in Table 10, the AdaAL-MID ranks first in all five metrics compared to its four variants. Among them, the use of static sampling weights has the greatest impact on the algorithm's performance. Static sampling maintains a consistent sampling frequency for all instances in the data stream, which cannot effectively handle data distribution imbalances. The performance of the algorithm is significantly improved after the adaptive sampling strategy is added. As shown in Figure 11a,b, in the data streams with mixed concept drift, the complete AdaAL-MID algorithm performed better than its variants in terms of performance loss at the concept drift and can also recover more quickly from new instances. In particular, the variant using a fixed label budget experiences the greatest performance decline at an instance count of 150k. Furthermore, the uncertainty strategy using only the minimum or maximum margin matrix performs weaker than the complete AdaAL-MID at the concept drift, and the fluctuation amplitude during the stable phase of the data stream is also larger. This indicates that the uncertainty strategy based on the dual-margin dynamic threshold matrix used by the AdaAL-MID is effective. These two matrices complement each other, thereby better identifying the most representative instances in the data stream and labeling them. As shown in Figure 11c, on the real data stream connect-4, removing each component of the AdaAL-MID will significantly affect the algorithm's performance.  
The experimental results on synthetic and real data streams demonstrate the effectiveness of each component of the AdaAL-MID.