# Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review
## 3 Concept Drift Adaptation Methods under Deep Learning
### Discussion  
According to the summary of concept drift adaptation methods, we can see that the proportion of hybrid learning and discriminant learning is relatively large, and especially  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{
\begin{tabular}{c} **Types of** \\ **Deep Learning** \\ \end{tabular} } & \multirow{2}{*}{**Algorithms**} & \multicolumn{3}{c}{**Concept Drift Adaptation Methods**} & \multirow{2}{*}{**Limitation**} \\ \cline{3-4} \cline{6-6}  & & **Detection** & & & **Adaptation** \\  & & **Nodes** & & **Drift Types** & \\ \hline \multirow{4}{*}{DTL} & NN-Patching [82] & – & \(\times\) & N & Need to adjust hyperparameters for the scene \\ \cline{2-6}  & AM-CNNs [83] & + & \(\times\) & N & High overhead \\ \cline{2-6}  & ATL [84] & – & \(\times\) & A I G & Forgetting problem \\ \cline{2-6}  & ADTCD [85] & – & \(\surd\) & A I G R & Little attention is paid to scarce anomaly data \\ \hline \multirow{4}{*}{DRL} & DeepPocket [87] & – & \(\surd\) & N & Not suitable for long-term investment strategies \\ \cline{2-6}  & RLAOSD [88] & – & \(\surd\) & N & Long model training time \\ \cline{1-1} \cline{2-6}  & OEA-RL [89] & + & \(\surd\) & N & Updating delay \\ \cline{1-1} \cline{2-6}  & DeepBreath [6] & – & \(\surd\) & N & Lack of consideration for exogenous factors \\ \hline DFL & FedHAR [90] & – & \(\surd\) & N & Scarcity of labels, with privacy \\ \hline \hline \end{tabular} Note: The symbol description is the same as in Table 1.  
\end{table}
Table 4: Other concept drift adaptation methods.  
discriminative learning is widely used. This phenomenon reflects the fact that having label information samples is beneficial for detecting changes in the distribution of data. In addition, parameter updates also account for a large part. Compared with structural updates, parameter updates reduce the convergence time and adapt well to abrupt concept drift. Secondly, in the algorithms of discriminant learning and generative learning, its active detection also accounts for a considerable part, and it is mainly conducive to explaining the occurrence of concept drift and reducing the computing resources of training, but to a certain extent, it requires additional memory and CPU storage. From this paper, it can be seen that dealing with concept drift, reducing the amount of computation, saving resources, and speeding up convergence are our main challenges at present.  
In addition, according to the above summary of drift adaptation types, we can find that there are usually more adaptation methods for abrupt, incremental, and gradual drift types. Relatively speaking, abrupt drift occurs most frequently, and its drift speed occurs the fastest, so most detection methods can be sensitive to detection, but there will also be problems such as update delay and high computational complexity. In contrast, recurring drift occurs the least often. In the case of recurring drift, previously learned models may become relevant again in the future. Online deep learning algorithms may have to relearn previous concepts. This process has a high computational burden because it means tuning or training a new model from scratch. This is also one of the main challenges at this time. Finally, it should be added that in addition to being based on deep learning algorithms, extreme learning machines are also models based on neural networks. In recent years, the use of the concept drift algorithm of ELM has also increased, and the main algorithms include Meta-RKOS-ELM [91], SSOE-FP-ELM [92], ONLAD [93], etc., which is also a worthy research direction.