# Directional Support Vector Machines
## Kernelized Symmetric Directional SVM  
By the representer theorem, the optimal \(f(\mathbf{x})\) in Equation (8) has the form:  
\[f(\mathbf{x})=w_{0}+\sum_{n=1}^{N}\alpha_{n}k(\mathbf{x},\mathbf{x}_{n}), \tag{11}\]  
where \(k\) is a positive-definite real-valued kernel and \(\alpha_{n}\in\mathbb{R}\). Benefiting from the decomposition of each directional feature in two, this formulation is directly applicable to the primal, fixed margin, directional SVM, when using \(g_{s}\). As such, all the conventional kernels can be applied in this extended space. When the model is instantiated with \(g_{s}\), \(x\) is mapped in a two-dimensional feature vector, \(\phi(x)=(\cos(x),\sin(x))^{\intercal}\), and the inner product between \(\phi(x)\) and \(\phi(z)\) becomes \(\langle\phi(x),\phi(z)\rangle=\cos(x)\cos(z)+\sin(x)\sin(z)=\cos(x-z)\). As such, the feature transformation can be avoided by setting as the kernel the cosine of the angular difference, \(k(x,z)=\cos(x-z)\). As seen before, in the case of \(g_{t}\), a similar conclusion does not hold. However, the result for \(g_{s}\) suggests also investigating the interest of using the function \(g_{t}(x-z)\) as a kernel. We start by presenting Theorem 1, where we show that a broad family of functions, which includes both \(g_{s}\) and \(g_{t}\), may be used to construct formally-valid kernels. **Theorem 1**. : _Let \(h:\mathbb{R}\mapsto\mathbb{R}\) be a periodic function with period \(T\) and absolutely integrable over one period. Define \(g:\mathbb{R}\mapsto\mathbb{R}\) as the autocorrelation of \(h\), i.e. :_  
\[g(x)=\left[h\star h\right](x)=\int_{0}^{T}h(t)h(x+t)\,dt. \tag{12}\]  
_Then, \(k(x,z)=g(x-z)\) is a kernel function, i.e., there exists a mapping \(\phi\) from \(\mathbb{R}\) to a feature space \(\mathbb{X}\) such that \(\langle\phi(x),\phi(z)\rangle=g(x-z)\)._  
Proof. : See Appendix A.1. **Remark 1**. : _The triangle wave \(g_{t}\) is the autocorrelation, as defined in this paper, of a square wave with amplitude \(1/\sqrt{2\pi}\) and period \(2\pi\)._  
Having proven the validity of \(g_{t}\) as a kernel, we now focus on investigating the expressiveness of the resulting SVM. The note made in Section 4.1 supports that the sum of triangle functions centered in fixed positions is not expressive enough since it cannot place the decision boundaries in arbitrary positions. However, the kernelized version in Equation (11) can still be appealing, since now, the models are centered in the training observations, and, as such, adapted in number and phase to the training data. For the purpose of this analysis, the notion of the Vapnik-Chervonenkis (VC) dimension [6], given in Definition 1, will be useful. **Definition 1**. : _A parametric binary classifier \(l(x|\mathbf{w})\), with parameters \(\mathbf{w}\), is said to shatter a dataset \(\mathbf{x}=(x_{1},...,x_{N})\) if, for any label assignment \(\mathbf{y}=(y_{1},...,y_{N})\), there exist parameters \(\mathbf{w}\) such that \(l(x|\mathbf{w})\) classifies correctly every data point in \(\mathbf{x}\). The VC dimension of \(l(x|\mathbf{w})\) is the size \(N\) of the largest dataset that is shattered by such a classifier._  
Figure 2: Example illustrating the benefits of the triangle wave kernel. (**a**) The SVM with the triangle wave in the primal form cannot learn more than two thresholds; therefore, some training points in this toy dataset are misclassified. (**b**) The SVM with the triangle wave kernel can learn an arbitrary number of thresholds, classifying every training point in this toy dataset correctly. Thus, the VC dimension of a classifier provides a measure of its expressive power. In Theorem 2, we establish a result that determines the VC dimension of the kernelized SVMs we are considering. **Theorem 2**. : _Let \(k(x,z)=\boldsymbol{\phi}(x)^{\intercal}\boldsymbol{\phi}(z)=g(x-z)\) be a kernel function where \(g\) is defined as in Theorem 1. Furthermore, suppose that \(g\) has zero mean value and its Fourier series has exactly \(N\) non-zero coefficients.