# Concept Drift Adaptation with Incremental-Decremental SVM
## 4 Adaptive-Window Incremental-Decremental SVM (AIDSVM)
The C-SVM performance is only 84.83%; because C-SVM is not suited for abrupt drift, this poor accuracy is somehow expected. The AIDSVM model was the best performer, with 88.68%, indicating good adaptability to abrupt drift. On the CIRCLES dataset, however, it can be seen that the best models seem to be on par--HT+FHIDDM with 87.16%, HT+FHIDDMS with 87.19%, C-SVM with 87.17% and AIDSVM with 87.22%. Being a dataset with gradual concept drift, C-SVM is expected to behave well, and this is supported by the experiment. For the COVERTYPE dataset, the best classic model seems to be the HT+DDM, achieving 89.90%. C-SVM performance of 91.79% suggests that this dataset seems to also be a gradual drift dataset; however, better performance of AIDSVM of 92.17% indicates a rather rapid drift. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Dataset** & **Window Size** & \(\gamma\) **(gamma)** & **C** \\ \hline SINE1 & 1500 & 6.008484 & 10 \\ CIRCLES & 1000 & 7.797753 & 1 \\ COVERTYPE & 2000 & 0.241477 & 100 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: C-SVM and AIDSVM parameters used for each data set. \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Method on Dataset** & **SINE1** & **CIRCLES** & **COVERTYPE** \\ \hline
**NB+FHIDDM** & 85.32\% & 86.24\% & 87.94\% \\
**NB+FHIDDMS** & 85.35\% & 86.26\% & 87.05\% \\
**NB+DDM** & 85.06\% & 86.06\% & 88.43\% \\
**NB+EDDM** & 82.50\% & 84.81\% & 84.92\% \\
**NB+ADWIN** & 84.72\% & 86.20\% & 86.78\% \\ \hline
**HT+FHIDDM** & 86.37\% & 87.16\% & 89.16\% \\
**HT+FHIDDMS** & 86.36\% & 87.19\% & 87.58\% \\
**HT+DDM** & 86.09\% & 86.97\% & 89.90\% \\
**HT+EDDM** & 83.69\% & 85.21\% & 84.98\% \\
**HT+ADWIN** & 84.09\% & 86.74\% & 87.02\% \\ \hline
**C-SVM** & 84.83\% & 87.17\% & 91.79\% \\
**AIDSVM** & 88.68\% & 87.22\% & 92.17\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Accuracy comparison between AIDSVM and other concept drift detectors, on given datasets. C-SVM was also evaluated, although its window size is not variable. We made a time comparison between C-SVM and AIDSVM in Table 3. We recorded the mean time and standard deviation, in milliseconds, after training on the same 1000 sample window size, on an Intel Core i5-8400 CPU with 16 GB RAM. We observed the advantage of AIDSVM without retraining from scratch. Figure 3 shows the window size dynamics for the three datasets, trained on the AIDSVM classifier. For SINE1, we can clearly observe the sudden drift changes; the window becomes almost empty. For the CIRCLES dataset, drift change is still visible at samples 25,000, 50,000, and only a little bit at 75,000. We explain this by considering the gradual drift employed by the dataset and by the fact that using a shifting window is inherently a way to cope with drift. In case of the COVERTYPE dataset, the drift here looks more like a combination of gradual and abrupt drifts; this is also supported by the point-to-point comparison among the drift methods contained in Table 4.

### Qualitative Discussion

We represented the instant accuracy of the classifier (given in Figure 4). This was computed as a mean on the next 100 samples. The C-SVM instant accuracy falls at concept drift and slowly recovers, whereas the AIDSVM accuracy recovers faster. We also computed the Exponentially Weighted Average at sample \(t\), computed as \(V_{t}=\beta V_{t-1}+(1-\beta)A_{t}\), where \(A_{t}\) is the accuracy at sample \(t\). \(\beta\) is chosen to be 0.9995, equivalent to a weighted average for about the last 2000 samples. One can see that the simple C-SVM EMA drops by about 20%, whereas the AIDSVM EMA drop is below 5%. The last metric, the mean accuracy, shows that, compared with the C-SVM, the AIDSVM mean accuracy gain is about 4%; interestingly, our mean accuracy of 88.54% is slightly better than Diversity Measure as a Drift Detection Method in a semi-supervised environment (DMDDM-S, 87.2%) presented in the most recent work of [PERSON] [46], on the same SINE1 dataset. Figure 3: Dynamic window sizes for the considered datasets.