# Concept Drift Adaptation with Incremental-Decremental SVM
## 4 Adaptive-Window Incremental-Decremental SVM (AIDSVM)
\begin{table}
\begin{tabular}{l l l l} \hline \hline
**Method on Dataset** & **SINE1** & **CIRCLES** & **COVERTYPE** \\ \hline C-SVM & 154 \(\pm\) 12 & 115 \(\pm\) 8 & 119 \(\pm\) 11 \\ AIDSVM & 117 \(\pm\) 7 & 75 \(\pm\) 9 & 73 \(\pm\) 6 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Running time comparison between C-SVM and AIDSVM, in milliseconds per training sample. Training was carried out on 5000 samples with a window size of 1000, for all datasets. \begin{table}
\begin{tabular}{l l l l l} \hline
**Drifts Signalled** & **SINENI** & **CIRCLES** & **COVERTYPE** \\ \hline
**NB+FHDDM** & 20,048, 40, 0403, 60,408, 80,047 & 25,061, 50,063, 75,104 & 803, 1761, 2689, 3149, 3587... \\
**NB+FHDDMS** & 20,033, 40,035, 60,407, 80, 037 & 25,061, 50,023, 75,104 & 803, 1607, 2009, 2644, 3129... \\
**NB+DDM** & 20,156, 403, 80, 6106, 30, 81, 10, 82, 539, 390, 520, 75,676... & 839, 2105, 2717, 313, 3588... \\
**NB+EDDM** & 93, 21, 212, 40, 949, 619, 816, 116... & 110, 20, 311, 365, 30, 397, 50,629... & 116, 2804, 553, 703... \\
**NB+ADWIN** & 20,065, 26, 178, 277, 775, 29, 924, 00,696... & 9537, 250, 27, 843, 50, 052, 75,205... & 1025, 2818, 3747, 4772, 5637... \\
**HT+FHDDM** & 20,054, 40, 052, 41,756, 60, 052, 80, 051 & 25,061, 50,063, 75,066 & 816, 1673, 2031, 2700, 3146... \\
**HT+FHDDMS** & 20,036, 40, 042, 41,756, 60, 047, 80,038 & 25,061, 50, 203, 75,066 & 816, 1670, 2009, 3043, 123... \\
**HT+DDM** & 20,150, 404, 614, 504, 80, 164, & 25,304, 50, 297, 75,559 & 853, 1924, 51,532, 51,741, 51,775... \\
**HT+EDDM** & 93, 20,899, 40, 873, 60, 951, 61,224... & 110, 260, 27, 840, 31, 500, 31,930... & 116, 280, 364, 553, 703... \\
**HT+ADWIN** & 2877, 470, 6435, 8260, 12,869... & 1985, 5006, 25,091, 28,420, 50,053... & 993, 2786, 4675, 5636, 5797... \\ \hline
**AIDSVM** & 20,036, 40, 041, 60,044, 80, 038 & 25,055, 25,069, 50,023, 50,030, 50,037, 75,668... & 990, 996, 1006, 4715, 4751... \\ \hline \end{tabular}
\end{table} 
Table 4: Drift points detected by the compared models. Drift is detected the same regions, mostly observed for the SINENI and CIRCLES datasets. Here, we have only shown the first five detections. The presence of ellipsis shows that the sequence is longer. Clear concept drift is seen in SINE1 around theoretical positions 20,000, 40,000 and 60,000, and for CIRCLES at positions 25,000, 50,000 and 75,000. COVERTYPE dataset seems to have a mixture of abrupt and gradual concept drift. ## 6 Conclusions

We introduced AIDSVM, an incremental-decremental SVM concept drift model with adaptive shifting window. It presents two important advantages: (i) better accuracy, because irrelevant samples are discarded at the appropriate moment based on the Hoeffding test, and (ii) it is faster than a classic SVM since no retraining is needed--the model is adapted on-the-run. The results of the experiments on three frequently used datasets indicate a better adjustment of the AIDSVM model compared to other drift-detection methods. Experimental evaluation indicated that AIDSVM copes better with concept drift, and in general it has similar or better accuracy results compared to classical concept drift detectors. However, the construction of the incremental solution is generally slower; this makes AIDSVM well suited for data streams with moderate throughput, where good accuracy is required in the presence of concept drift. To the best of our knowledge, our implementation is the first online SVM classifier that copes with concept drift using dynamic adaptation of the shifting window by avoiding retrain from scratch. A further improvement to the current AIDSVM implementation would be to speed up the unlearning process. This can be carried out in two stages. First, one would determine how many samples from the beginning of the window have to be removed. This is achieved by testing the Hoeffding condition (1) on sub-windows formed by successively removing the oldest sample.