# Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review
## 3 Concept Drift Adaptation Methods under Deep Learning
### Concept Drift Adaptation Methods Based on Discriminant Learning  
This type of deep learning technique is used in supervised or classification applications by describing the posterior distributions of conditioned visible data. A discriminative  
Figure 4: A taxonomy of concept drift adaption under deep learning framework. model is a model that learns the relationship between input data and output labels, and it predicts output labels by learning the characteristics of the input data. In classification problems, the main purpose is to assign each input vector \(a\) to label \(b\). Discriminant models attempt to directly learn the function \(f(a)\) that maps input vectors to labels. The classifier first learns the posterior class probability \(P(b=k|a)\) from the training data and assigns a new sample \(a\) to the class with the highest posterior probability based on these probabilities, where \(k\) stands for class. The general process of the discriminant concept drift adaption method is shown in Figure 5, while the two methods of active detection and parameter update mode account for a relatively large proportion. Discriminant learning mainly includes multilayer perceptrons (MLPs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their variants. * MLP-based concept drift adaptation methods  
MLP is a discriminant learning model widely adopted in decision making [32], which is often used in combination with concept drift adaptation methods to solve classification problems for unstable streaming data. However, concept drift adaptation methods are computationally expensive and converge slowly each time the model is updated due to the hyperparameter problem. Typical algorithms, such as selective ensemble-based online adaptive deep neural networks (SEOAs), bilevel online deep learning (BODL), neural networks with dynamically evolved capacity (NADINEs), and Adadelta optimizer-based deep neural networks with concept drift detection (CIDD-ADODNNs), are elaborated as follows. SEOA [33] uses a deep learning model with L network layer MLPs to form L basic classifiers. It then dynamically adjusts the parameters of each basic classifier to handle concept drift and regularly selects base classifiers with different convergence and fitting abilities. It enhances the adaptability and generalization ability of the model to data distribution, which is more suitable for gradual, incremental, and recurring drift, although less suitable for dealing with high-dimensional non-linear problems. BODL [34] mainly uses the MLP model for classification prediction and detects concept drift based on the classifier's error rate. When concept drift is detected, the model's parameters are updated through a bilevel optimization strategy and the exponential gradient descent method to adapt to the abrupt concept drift, but its limitation is that the added classes cannot be identified online. In contrast, the convergence speed of the algorithm for model structure update can be slower. NADINE [35] uses a drift detection mechanism to detect concept drift actively. The drift detection mechanism mainly adds an adaptive windowing strategy to the prominent [PERSON]'s bound detection algorithm. When the drift signal is detected, its network structure will be updated to adapt to concept drift, which mainly adjusts its network structure through the hidden unit growing strategy and hidden unit pruning strategy. The main advantage of NADINE over other algorithms is its elastic structure and online learning trait, but the training time of the model is relatively slow. It can be applied to classification and regression problems. Additionally, scholars have researched it for certain special data. CIDD-ADODNN [36] is adopted for the classification model of highly unbalanced data flow, which mainly uses an adaptive sliding window (ADWIN) drift detection algorithm to actively detect concept drift and then updates the network  
Figure 5: The general process of concept drift adaptation methods based on discriminant learning.