# Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review
## 3 Concept Drift Adaptation Methods under Deep Learning
### Concept Drift Adaptation Methods Based on Discriminant Learning
However, the algorithm is relatively expensive in terms of labor and requires the construction of annotated datasets. I-LSTM [5] combines the idea of time factor with stratified sampling. Therefore, the newer the data, the higher the weight assigned to accommodate concept drift, but there are also problems with balancing old and new data. Overall, it improves multi-classification performance for anomaly detection, mainly for IoT applications. MOMBD-CDD [45] mainly deals with high-dimensional streaming data. It mainly uses the Statistical Test of Equal Proportions method (STEPD) to detect concept drift and combines the glowworm swarm optimization (GSO) algorithm to update the bidirectional long short-term memory (Bi-LSTM) model by adjusting weights. However, it is more computationally intensive and takes up more resources. In this process, STEPD defines two windows, a recent window \(r\) and an overall window \(o\). This is also common in deep-learning-based concept drift adaptation methods. It applies the statistical test of equal proportions to compare the accuracies between the two windows as shown in Equations (4) and (5):  
\[T(v_{o},v_{r},n_{o},n_{r})=\frac{|v_{o}/n_{o}-v_{r}/n_{r}|-0.5\times(1/n_{o}+1/ n_{r})}{\sqrt{\mu+(1-\mu)\times(1/n_{o}+1/n_{r})}} \tag{4}\]  
\[\mu=(v_{0}+v_{r})/(n_{o}+n_{r}) \tag{5}\]  
Its value is compared to the percentile of the standard normal distribution to obtain the observed significance level (\(p\)-value). \(p\)-value is equivalent to the chi-square test with [PERSON]'s continuity correction, in which \(v\) is the value of accurate predictions, and \(n\) is the number of samples for the window. The calculation formula for \(\mu\) is shown in Equation (5). If \(p\)-value < \(\alpha\), STEPD predicts a concept drift. If \(p\)-value < \(\alpha_{w}\), STEPD predicts a warning that concept drift may occur. \(\alpha_{d}\) is the concept drift significance level; \(\alpha_{w}\) is the warning significance level. Fog-DeepStream [46] uses wavelet transform to reduce the dimensionality of the data and LSTM models to predict future behavior for data stream analysis on fog computing. It uses a drift detection algorithm to determine the occurrence of conceptual drift, and when a conceptual drift is detected, parameters are updated to accommodate the conceptual drift. The method tries three drift detection algorithms: cumulative sum (CUSUM), Page-Hinkley, and exponentially weighted moving average (EWMA). However, this algorithm also takes up a lot of memory. The above algorithms are used in the Internet field. Next, we introduce algorithms in other fields. For example, AD-LSTM [47] is used for predicting photovoltaic power generation. It actively detects the occurrence of concept drift through the sliding window (SDWIN) algorithm and adopts the second stage of the two-phase adaptive learning strategy (TP-ALS) to fine-tune the prediction model. DCA-DNN [48] is mainly used for industrial prognosis and is based on the LSTM-FC model, which actively detects the occurrence of concept drift through the dendritic cell algorithm. It generates synthetic data using a kernel density estimator with drift-based bandwidth, which can be used to fine-tune the weights of the last layer to achieve faster adaptation and mitigate the problem of limited new samples. Both of the above algorithms suffer from model update delays, and their concept drift detection algorithms need to be optimized. * CNN-based concept drift adaptation methods  
CNN is a feed-forward neural network in which the connections between neurons in its convolutional layer are not fully connected, and the weights and biases of connections between some neurons in the same layer are shared [49; 50]. So, the computational cost of this concept drift adaptation method is also relatively low. Typical algorithms, such as the evaluative convolutional neural network (ECNN) [51], mainly use re-weighting operation technology to dynamically update the model, so as to solve the concept drift problem in high-throughput data.