# Concept Drift Detection Based on Deep Neural Networks and Autoencoders
## 4 Experiment
### Experiment Results and Analysis  
To maintain consistency in the data distribution between the pretraining phase and the data before the drift, synthetic data for the pretraining phase are generated using the generators shown in Table 1. For the real data, the first \(1/5\) of the original dataset is used as pretraining data. The experiment is implemented using Python 3.6.5, and the neural network architecture is built with PyTorch 2.4.0. Details of the hyperparameter values involved in the construction of the neural network can be found in Table 5.  
To validate the effectiveness of the proposed method, its performance is evaluated using the metrics discussed in Section 4.3. The detection methods mentioned in Section 4.2 all require the base classifier to classify the data and generate the error rate. To avoid the potential bias of relying on a single classifier, which may not fully utilize the performance of the drift detection algorithm, we use two classical classifiers from skmultiflow, namely Hoeffding Tree and Naive Bayes, as baseline classifiers for comparison, providing input for the detection algorithms. The parameter settings for each detector are set to their default values. To ensure fairness, each metric is computed with n set to 100, and the average of 100 experiments is taken. It is important to note that during the pretraining phase, we first generate non-drift data for all datasets listed in Table 1. Taking SE_a as an example, its generation command in MOA is \"WriteStreamToARFFFile -s (ConceptDriftStream -s (generators.SEAGenerator -f 3 -b -n 5000 -p 0) -d (generators.SEAGenerator -f 2 -b -n 5000 -p 0) -p 5000 -w 1) -f E:\DatasetDataset\dataset.arff -m 10,000 -h\". The key parameter here is the value of 'f'. The command is designed with two phases, where 'f' takes different values. The parameter's' represents the currently generated data stream, and when 'f' is set to 3 following's', the generated data are produced using generation function 3. The parameter 'd' denotes drift data, and the data within its parentheses is generated using function 2. Correspondingly, the non-drift SEA data are the data produced when 'f' is set to 3. The same approach applies to other datasets. During the pretraining phase, Model1 in Figure 3 is trained using non-drift data generated with 'f' set to 3. In the streaming data processing phase, Model2 in Figure 4 processes the drift data generated by the above command. The detection results of synthetic data with single drift using the \(FEDP\) metric are shown in Table 6. The \(FEDP\) metric represents detection delay, with smaller values being preferred. The best results in the table are highlighted in bold. From Table 6, we can observe that the performance of the detectors varies depending on the base classifier selected for different datasets. While the DNN+AE-DD method shows slightly weaker performance on certain datasets, it outperforms other methods in overall performance. The maximum value in Table 6 is 5500, indicating that the detector did not produce any output for this dataset. This suggests that drift occurred, but the detector failed to detect it. The inability of certain detection algorithms to detect drift on datasets such as AGR_g may be due to the inherent classification difficulty of the AGR dataset. Even with multiple classifiers trained on this dataset, the classification accuracy remains poor. \begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Value** \\ \hline \(L\) & 3 \\ \(M_{F},M_{S},M_{L}\) & 256, 128, 64 \\ \(L_{A}\) & 8 \\ \(A_{p}\) & 10 \\ \(\eta_{1},\eta_{2},\eta_{A}\) & \(1\times 10^{-2}\,,\,1\times 10^{-4},\,1\times 10^{-4}\) \\ \(s_{1},s_{2}\) & 20, 10 \\ \(\gamma_{1},\gamma_{2}\) & 0.9, 1.0 \\ \(optim_{1},optim_{2}\) & SGD, Adam \\ \(\mathcal{L}_{P},\mathcal{L}_{A}\) & BCEWithLogitsLoss/MSELoss \\ \hline \hline \end{tabular}
\end{table} 
Table 5: DNN+AE-DD parameter settings. Assuming all algorithms perform equally, we obtain \(\mathcal{T}_{F}=4.66\) based on Equation (8).