# Directional Support Vector Machines
## 9 Experiments
\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline
**Dataset** & **vMNB** & **dLR** & **dRBF-SVM** & **cos-SVM** & **t-SVM** & **T-SVM** & **a-SVM** & **A-SVM** \\ \hline
**Colposcopy** & \(68.96\pm 11.1\) & \(79.61\pm 10.0\) & \(83.32\pm 8.9\) & \(81.21\pm 7.8\) & \(82.86\pm 9.6\) & \(83.32\pm 8.6\) & \(\mathbf{85.17}\pm 8.6\) & \(82.33\pm 9.8\) \\
**Behavier** [14] & \(22.13\pm 6.2\) & \(51.29\pm 16.9\) & \(42.47\pm 9.5\) & \(43.17\pm 8.7\) & \(83.29\pm 1.61\) & \(45.04\pm 9.7\) & \(50.63\pm 14.3\) & \(43.38\pm 10.7\) \\
**Arrhythmia** [13] & \(62.40\pm 6.3\) & \(77.80\pm 6.6\) & \(77.10\pm 5.3\) & \(77.12\pm 5.7\) & \(77.14\pm 6.5\) & \(75.21\pm 5.7\) & \(77.38\pm 6.2\) & \(75.02\pm 6.3\) \\
**eBay** [16] & \(83.64\pm 6.0\) & \(85.13\pm 5.1\) & \(\mathbf{85.57}\pm 5.1\) & \(83.64\pm 4.8\) & \(83.05\pm 6.5\) & \(81.92\pm 5.8\) & \(83.26\pm 6.6\) & \(81.07\pm 5.3\) \\
**Megasopes** [18] & \(74.29\pm 4.0\) & \(72.75\pm 5.4\) & \(71.20\pm 5.9\) & \(69.94\pm 8.1\) & \(71.93\pm 5.4\) & \(69.31\pm 5.9\) & \(71.63\pm 5.0\) & \(71.50\pm 4.8\) \\
**Characterts** [15] & \(71.73\pm 4.1\) & \(96.13\pm 17.7\) & \(97.82\pm 1.5\) & \(96.93\pm 1.5\) & \(97.09\pm 1.6\) & \(89.29\pm 1.4\) & \(96.50\pm 2.1\) & \(98.05\pm 1.3\) \\
**OnlineNews** [19] & \(45.27\pm 4.9\) & \(\mathbf{55.36}\pm 5.4\) & \(51.89\pm 6.0\) & \(47.94\pm 5.8\) & \(52.74\pm 5.7\) & \(84.88\pm 5.6\) & \(53.50\pm 5.1\) & \(48.30\pm 5.7\) \\
**Continents** & \(93.02\pm 2.8\) & \(94.57\pm 1.8\) & \(95.40\pm 16.6\) & \(95.76\pm 1.5\) & \(51.81\pm 5.4\) & \(96.99\pm 1.1\) & \(88.73\pm 3.8\) & \(\mathbf{97.34}\pm 0.9\) \\
**Wall** [20] & \(55.31\pm 2.2\) & \(71.66\pm 2.1\) & \(85.97\pm 1.7\)Hereafter, we will denote by non-Kernelized directional SVMs (nK-dSVM) the subset of proposed SVM variations with VC dimension equivalent to the one induced by the directional logistic regression; namely, the primal fixed-margin directional SVM with triangle (symmetric and asymmetric) and cosine waves. The remaining models (i.e., directional RBF, symmetric, and asymmetric kernels) will be referred to as Kernelized directional SVMs (K-dSVM). Although some datasets used here are considerably imbalanced, accuracy and macro F\({}_{1}\)-score values were fairly consistent with each other, in the sense that the best model in terms of accuracy was the top-1 model in terms of macro F\({}_{1}\)-score in 10 of 12 datasets and was among the top-2 models in all datasets. While the dLR achieved a competitive general performance, it was surpassed by at least one of the proposed SVM alternatives in most cases. nK-dSVM performed better than dLR on small datasets, given the margin regularization imposed by the SVM loss function. For larger datasets, dLR performed better since the generalization induced by the nK-dSVM margin became less relevant. However, for large datasets, K-dSVM surpassed dLR and their non-kernelized counterparts in most cases. In general, dSVM with asymmetric margins (kernelized and non-kernelized) attained the best results, obtaining the best average performance on half of the datasets. As shown in Section 5, kernels involving the triangle wave correspond to inner products in an infinite-dimensional feature space. The same is also true for the directional RBF kernel. Non-kernelized methods, on the other hand, are constructed by explicitly defining the feature transformation, having a necessarily finite VC dimension. Therefore, the former produce models with higher capacity, which may lead to overfitting in small datasets, but better accuracy for large ones. This is confirmed by our experiments: the non-kernelized models achieved the best results in small datasets, while kernelized models built on top of the triangle wave and directional RBF kernel attained the best results in large datasets. The performance gains of kernelized models on the larger datasets were small, however, which may be explained by the unimodal distribution of the angular variables.