# Concept Drift Adaptation with Incremental-Decremental SVM
## 2 Related Work: Concept Drift with Adaptive Shifting Windows  
In this section, we list some of the most recent concept drift models based on adaptive windows, with a focus on SVM approaches. Several models addressing concept drift on adaptive windows were proposed in recent years. Detailed overviews are given by the work of [PERSON] et al. [5], [PERSON] et al. [6], as well as [PERSON] et al. [21]. The Learn++.NSE algorithm [22; 23] and its fast version [24] generate a new classifier for each received batch of data, and add the classifier to an existing ensemble. The classifiers are later combined using dynamically weighted majority voting, based on the classifier's age. In [25], the adaptive random forest algorithm, used for the classification of evolving data streams, combines batch algorithm traits with dynamic update methods. One of the seminal papers in this field is the Drift Detection Method (DDM) described in [26]. It uses the classification error as evidence of concept drift. The classification error decreases as the model learns the newer samples. The model establishes a warning level and a drift level. When the classification error increases over the warning level, newer samples are introduced into a special window. Once the error increases over the drift level, a new model is created, starting from the samples from the special window. A later extension, the Early Drift Detection Method (EDDM) [27], uses the distance between two consecutive errors and its standard deviation instead of the simple error rate used in the DDM. It also follows the same principle of comparing the error rate against warning and drift thresholds. Both methods are designed to operate regardless of the incremental learner used. The Fast Hoeffding Drift Detection Method (FHDDM) [28] detects the drift point using a constant-size sliding window. It detects a drift if a significant variation is observed between the current and the maximum accuracy. The accuracy difference threshold is determined using [PERSON]'s inequality theorem:  
\[Pr(|\overline{X}-E[\overline{X}]\geq\epsilon_{H}])\leq\delta\,,\,\text{where} \,\epsilon_{H}=\sqrt{\frac{1}{2n}\ln\frac{2}{\delta}}\]  
The FHDDM method is extended by maintaining windows of different sizes in the Stacked Fast Hoeffding Drift Detection Method (FHDDMS) [29]. This is employed to reduce detection delays. Extensive treatment of these two methods is shown in [30; 31]. A very recent approach using the error rate is the Accurate Concept Drift Detection Method (ACDDM) [32]. The author analyzes the consistency of prequential error rate using [PERSON]'s inequality. At each step, the difference between the current error rate and the minimum error date is determined. This is compared against the threshold given by the Hoeffding inequality for a desired confidence level. The drift is detected when the error rate difference is greater than the computed deviation:  
\[\epsilon\leq\sqrt{\frac{1}{2n}\ln\frac{1}{\delta}}\]  
The ACDDM is evaluated using the Very Fast Decision Tree learning algorithm ([32], Section 3). Recently, SVMs were also used to address concept drift. [PERSON] et al. [33] proposed an SVM classification model with a learned label space that evolves in time, where novel classes may emerge or old classes may disappear. For the modelling of intricate-shape class boundaries, they used support-vector-based data description methods. [PERSON] et al. [34] used SVMs in an ensemble-based incremental learning algorithm to model the changing environments. Learning windows of variable length also appear in the papers of [PERSON] et al. [17] and [PERSON] [18]. [PERSON]'s methods use a variable width window, which is adjusted by the estimation of classification generalization error. At each time step, the algorithm builds several SVM models with various window sizes, then it selects the one that minimizes the error estimate. The appropriate window size is automatically computed, and so is the selection of samples and their weights. While the methods used by [PERSON] et al.