# Concept Drift Detection Based on Deep Neural Networks and Autoencoders
## 2 Related Work
### Concept Drift Detection Methods Based on Deep Learning Models  
Some studies have applied deep learning techniques to address the challenge of concept drift. As a rapidly advancing technology, deep learning has become the focal point of research and application across various domains, demonstrating significant maturity in areas such as image recognition, speech recognition, and natural language processing. Compared to traditional machine learning methods, deep learning offers enhanced representational and fitting capabilities, making it well suited for handling complex data stream scenarios. Based on the classification of deep neural networks according to their network structures, deep learning models can be divided into three types: feedforward networks, memory networks, and graph networks [26]. Building on this classification, this paper categorizes deep learning-based concept drift detection methods into three groups: those based on feedforward networks, memory networks, and graph networks. Meta-ADD [16] utilizes drift data for offline pretraining to extract meta-features associated with error rates, employing prototype networks to identify different categories of concept drift. It can detect drift types without the need for pre-specified hypothesis testing. However, it lacks pretraining and suffers from a cold start problem [9]. CIDDADODNN [27] classifies imbalanced data streams using a deep neural network optimized with Adadelta and subsequently applies the ADWIN algorithm to detect concept drift. By incorporating techniques to address class imbalance, it effectively handles the challenges associated with imbalanced streaming data. WSCDD [18] trains neural network models using the Hedge backpropagation algorithm, while employing Monte Carlo methods to calculate prediction uncertainty. This uncertainty is then used as input for the ADWIN algorithm to detect concept drift. Given the difficulty of obtaining labeled data in weakly supervised environments, WSCDD can detect the occurrence of concept drift in unlabeled data. The concept drift detection model based on Bidirectional Temporal Convolutional Networks and Multi-Stacking Ensemble Learning (CD-BTMSE) [28] leverages a Bidirectional Temporal Convolutional Network (BiTCN) and a multi-stacking ensemble learning model to improve the generalization ability and accuracy of drift detection, while simultaneously addressing the issue of class imbalance. By incorporating the BiTCN model, it enhances the model's performance on time series data, an area where traditional models typically underperform, thereby overcoming the limitations of existing ensemble methods. In response to the scarcity and high acquisition costs of true labels, the Uncertainty Drift Detection (UDD) [29] algorithm combines the uncertainty provided by deep neural networks with Monte Carlo Dropout. It detects temporal structural changes using the ADWIN algorithm and retrains the model when drift is detected. SDLS [30] combines long short-term memory (LSTM) networks with a sliding window approach to predict data, selecting an appropriate window size to analyze prediction differences. It models all values within the window and assigns anomaly scores based on the probability density of these differences, thereby enabling concept drift detection. [PERSON] et al. [31] utilize the Stream Ensemble SEA algorithm, incorporating deep belief networks as the foundational model. The LSTM Drift Detector (LSTMDD) [17] proposes an attention-enhanced LSTM model, integrated with a genetic hyper-tuning drift detector, to effectively detect concept drift in non-Gaussian distributed cloud computing environments. [PERSON] et al. [32] propose the use of graph convolutional networks (GCNs) for concept drift detection. This method first converts log trajectories into activity graphs, and then applies GCNs to learn from these graphs and construct virtual nodes. Drift is assessed based on the distances between these virtual nodes.