# Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review
## 3 Concept Drift Adaptation Methods under Deep Learning
### Other Concept Drift Adaptation Methods  
The deep learning framework classification mentioned above is mainly divided based on the perspective of single-class models or hybrid multiple models. It is worth noting that there are cases where other deep learning technologies [31; 78], such as deep reinforcement learning and deep federated learning, are used. For example, deep reinforcement learning was introduced as a combination of deep learning (DL) and reinforcement learning (RL) to better cope with the dynamic changes of unstable environments, leveraging the primary deep learning models to generate the target models we need [79; 80]. The general process of this concept drift adaption method is shown in Figure 8, while the structure update mode is more common in deep transfer learning. Parameter updating is more common in deep reinforcement learning. However, relatively few research studies involving concept drift adaption methods are elaborated compared to other classes. The most popular are deep transfer learning (DTL) and deep reinforcement learning (DRL). Therefore, we mainly \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Types of Deep Learning**} & \multirow{2}{*}{**Algorithms**} & \multicolumn{3}{c}{**Concept Drift Adaptation Methods**} \\ \cline{3-6}  & & **Detection Modes** & **Update Modes** & **Adaptation Drift Types** & **Limitation** \\ \hline LSTM + SNN & HSN-LSTM [71] & – & \(\surd\) & N & High resource overhead \\ \hline LSTM + AE + ORA & OAR-DLSTM [72] & – & \(\surd\) & I R & The dataset is too large, and its performance appears to degrade \\ \hline LSTM + AE & B-Detection [73] & – & \(\surd\) & A I G R & Long running time \\ \hline LSTM + CNN & CausalConvLSTM [74] & – & \(\surd\) & N & Limited log types for algorithm applications \\ \cline{2-6}  & LSTMCNNeda [75] & + & \(\surd\) & A G & \
\begin{tabular}{c} Time-series data normalization issues, \\ window size selection \\ \end{tabular} \\ \hline AE + DNN & SAE-DNN [76] & + & \(\times\) & A G & Noise interference \\ \hline RNN + ARIMA & OARIMA-RNN [39] & – & \(\surd\) & N & No quantification of conceptual drift \\  & & & & or performance during drift \\ \hline MLP + Decision tree + SVM & RACE [77] & + & \(\surd\) & A G R &
\begin{tabular}{c} Requires large amounts of memory, \\ increased integration size, \\ slows convergence \\ \end{tabular} \\ \hline \hline \end{tabular} Note: The symbol description is the same as in Table 1.  
\end{table} 
Table 3: Concept drift adaptation methods based on hybrid learning. introduce DTL and DRL. Table 4 summarizes concept drift adaptation methods based on DTL and DRL. DTL mainly uses pre-training of deep learning models to obtain relevant knowledge. Then, by transferring the acquired knowledge to a new model, it can be adapted to a new task with minimal data and can save resources [81]. Currently, there are not many algorithms that involve concept drift. Typical methods are neural network patching (NN-Patching), adaptive mechanisms for learning CNNs (AM-CNNs), and autonomous transfer learning (ATL). NN-Patching [82] is passively handled concept drift by an error estimator. It mainly constructs a discriminant classifier to identify the misclassified regions. Then, it trains a new classifier (called a patch network) on the misclassified data. The patch network uses the intermediate layer of the original neural network to extract features and representations that are critical to classification. This method keeps the original neural network quickly adaptable to concept drift, but its ability to handle concept drift is limited, and the hyperparameters need to be adjusted for the scene. AM-CNNs [83] uses the nonparametric CUSUM test to actively detect the occurrence of concept drift. It relies on a \"transfer learning\" paradigm that transfers the knowledge of the CNN running before the concept drift to the CNN running after the concept drift, but the resource overhead is relatively high.