# Concept Drift Detection Based on Deep Neural Networks and Autoencoders
## 4 Experiment
### Experiment Results and Analysis
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline \multirow{2}{*}{**Datasets**} & \multirow{2}{*}{**DNN+AE-DD**} & \multicolumn{2}{c}{**DDM**} & \multicolumn{2}{c}{**EDDM**} & \multicolumn{2}{c}{**HDDM\_W**} & \multicolumn{2}{c}{**ADWIN**} \\ \cline{2-9}  & & **NB** & **HT** & **NB** & **HT** & **NB** & **HT** & **NB** & **HT** \\ \hline Weather & **1.00 (I)** & 0.00 (7) & 0.00 (7) & 0.33 (2.5) & 0.00 (7) & 0.00 (7) & 0.33 (2.5) & 0.17 (4) & 0.00 (7) \\ Electricity & 0.69 (3) & 0.33 (6) & 0.00 (9) & **1.0 (1.5)** & **1.0 (1.5)** & 0.67 (4.5) & 0.67 (4.5) & 0.06 (8) & 0.08 (7) \\ Usenet1 & **0.90 (I)** & 0.00 (8.5) & 0.00 (8.5) & 0.49 (2) & 0.17 (5) & 0.33 (3.5) & 0.33 (3.5) & 0.08 (7) & 0.16 (6) \\ Usenet2 & **1.0 (I)** & 0.17 (6.5) & 0.17 (6.5) & 0.33 (4.5) & 0.33 (4.5) & 0.67 (2.5) & 0.67 (2.5) & 0.06 (9) & 0.08 (8) \\ Email data & **0.96 (I)** & 0.66 (4.5) & 0.66 (4.5) & 0.33 (6.5) & 0.33 (6.5) & 0.67 (2.5) & 0.67 (2.5) & 0. In Figure 12, the orange color represents the algorithm with the best detection delay performance, while the other algorithms are depicted in blue. When detecting a single drift, the DNN+AE-DD is able to significantly reduce detection delay across most real-world datasets. However, on the NSL-KDD dataset, the delay for nearly all classifiers is notably longer compared to other datasets. By analyzing Figures 12 and 13, we observe that when detecting multiple drifts, the DNN+AE-DD algorithm performs slightly worse than detectors such as EDDM on the Electricity dataset. However, it demonstrates superior overall performance across different datasets. In the case of artificial drift points implanted in real-world datasets, the DNN+AE-DD demonstrates a clear advantage in detecting abrupt drifts. However, due to the inherent complexity of real-world data, the detection delay varies. Furthermore, we observe that the performance of the detector varies depending on the classifier chosen, highlighting the significant impact of base classifier selection on drift detection performance. For most classifiers, other detectors perform poorly, which may be attributed to the complex and dynamic nature of real-world data. Since the detector's input relies on the predictions of the base classifier, fluctuations introduced by drift points may cause the classifier to mislabel data as noise, potentially resulting in the absence of detection outputs. Based on the experimental results, the proposed DNN+AE-DD method effectively and promptly detects concept drift with minimal delay while demonstrating the capability to identify multiple types of drift. Furthermore, for the synthetic datasets in Table 10, we conducted a comparison of the false detection rates (\(FDR\)) across different algorithms, with the results presented in Table 10. For cases where the algorithm did not produce detection output in all 100 experiments, we assigned the value \"NaN\". The values in the table are expressed as percentages. Figure 11: Average rank of real-world data with multiple drifts based on the Bonferroni–Dunn test. Figure 10: Average rank of real-world data with single drift based on the Bonferroni–Dunn test. different algorithms perform optimally on different datasets, the DNN+AE-DD maintains a balanced performance. Although the DNN+AE-DD does not achieve the lowest false detection rate when compared to all algorithms, its false detection rate is comparable or even better than the comparisons. For instance, on the AGR_a, AGR_g and HYP_i datasets, most methods fail to detect drift effectively, whereas the DNN+AE-DD successfully identifies drift in all 100 experiments with a relatively low false detection rate. In summary, although the performance of the DNN+AE-DD varies across datasets compared to other algorithms, it consistently detects drift with shorter delays and strong robustness.