# Concept Drift Adaptation Methods under the Deep Learning Framework: A Literature Review
## 3 Concept Drift Adaptation Methods under Deep Learning
### Concept Drift Adaptation Methods Based on Generative Learning
MemStream [61] is used for anomaly detection in multidimensional data and concept drift. It first uses a small portion of the training set and extracts features using the denoising autoencoder. Then, when a new sample arrives, the anomaly score is recalculated, and the weighting factor of AE is updated. If the anomaly score exceeds a user-set threshold, the memory is updated in a first in, first out (FIFO) manner, and the model is retrained to accommodate concept drift. This method effectively avoids noise disturbances and retrains quickly but with high resource overheads. GAN-based concept drift adaptation methods  
GAN mainly consists of a generator and a discriminator. The former is used to create new data with similar characteristics to the original data, and the latter is used to determine the authenticity of the given data [62]. There are few examples of GAN combined with concept drift adaptation methods compared to other deep learning techniques, such as the distributed class-incremental learning method based on generative adversarial networks (DCIGAN). DCIGAN [63] uses a GAN generator to store information about past data and constantly updates GAN parameters with new data. Meanwhile, a generative fusion method (GF), which integrates multi-node local generators into a new global generator, is adopted. Particularly, a method for monitoring and evaluating GAN during continuous learning is presented, which explains the concept drift [64]. Its main purpose is to solve the problem of classifying data streams, but different hyperparameters need to be set in different environments. RBM-based concept drift adaptation methods  
RBM is usually made up of visible and hidden nodes, each connected to every other node, which facilitates the understanding of some irregular datasets. Moreover, it is sensitive to the occurrence of concept drift because it is able to learn the probability distribution of the input [65]. RBM-I [66] and RRBM-DD [22] are two typical concept drift detection algorithms proposed by [PERSON] and [PERSON], for multi-class imbalance and the presence of adversarial attack data streams, respectively. They both use gradient descent to update the weights in order to maintain the sensitivity of concept drift detection. RBM-IM is not suitable for small data streams and is prone to overfitting. RBM-DD has limitations in identifying adversarial concept drift in dynamic classes of unbalanced data streams. In addition, the Gaussian restricted Boltzmann machine (GRBM) algorithm [67] primarily uses the Kullback-Leibler divergence distance to determine whether a concept drift has occurred, thus enabling the adaptive adjustment of the sliding window and the division of the data stream. It reduces energy consumption and saves memory but only makes judgments on data from a single source and does not adaptively divide heterogeneous data. SOM-based concept drift adaptation methods  
SOM is often applied to create low-dimensional (usually two-dimensional) representations of high-dimensional datasets, while maintaining the topology of the data [68]. The main benefit of using SOM is that it makes high-dimensional data easier to visualize and analyze for understanding patterns. As in the case of GAN, there are few examples of SOM combined with concept drift adaptation methods. An online unsupervised incremental method based on self-organizing maps (OUIM-SOM) [69] is used for multi-label stream classification in infinite delay labeling scenarios. It adopts the online update of neuronal weight vectors and dataset label cardinality to accommodate abrupt and incremental concept drifts. However, its adaptive effect on conceptual drift is limited. Table 2 summarizes the typical algorithms based on generative learning. Among them, AE-based algorithms are mainly used for anomaly detection, and depending on the characteristics of autoencoders, these algorithms use generally unsupervised or semi-supervised learning, which can enhance the flexibility of data flow methods in utilizing unlabeled samples.