# Money Neutrality, Monetary Aggregates and Machine Learning
## 2 Methodology and Data  
### Support Vector Machines  
The support vector machines (SVM) are a set of machine learning methods for data classification and regression. The basic idea of the SVM is to define the optimal (optimal in the sense of the model generalization to unknown data) linear separator that separates the data points into two classes. The linear model is defined in two steps: Training and testing. The largest part of the dataset is used in the training process, where the linear separator is defined. In the testing step, the generalization ability of the model is evaluated by calculating the model's performance in the testing set: A small part of the dataset that was left aside in the first step. In the following, we briefly describe the mathematical derivations of the SVM technique. We consider a dataset of \(n\) vectors \(\mathbf{x}_{i}\)\(\in\)\(\mathbb{R}^{m}\) (\(i=1,\,2\,\ldots,\,n\)) belonging to two classes \(y_{i}\in\{-1,\,+1\}\). If the two classes are linearly separable, we define a boundary as:  
\[f(\mathbf{x}_{i})=\boldsymbol{w}^{\boldsymbol{T}}\mathbf{x}_{i}-b=0. \tag{5}\]  
Subject to:  
\[ \begin{split}\boldsymbol{w}^{\boldsymbol{T}}\mathbf{x}_{i}-b>0\; \forall i:&\ y_{i}=+1,\\ \boldsymbol{w}^{\boldsymbol{T}}\mathbf{x}_{i}-b<0\;\forall i:& \ y_{i}=-1,\end{split}  \tag{6}\]  
\(\boldsymbol{w}\) is the parameter vector, and \(b\) is the bias (Figure 1). So \(y_{i}f(\mathbf{x}_{i})>0,\;\forall i\). In the linearly separable case, the linear separator (line in two-variable datasets, the plane in three-variable datasets, and a hyperplane in the dataset with more dimensions) is defined as the decision boundary that classifies each data point to the correct class. The position of the separator is defined by the Support Vectors (SV), a small number of data points from the dataset. In Figure 1, the SV are represented with the pronounced black contour, the margin lines (parallel lines to the optimal separator passing from the SV) are represented by dashed lines, and the separator is represented by a continuous line. The distance between the two margin lines defines the distance between the two classes. The goal of SVM is to find the linear separator that yields the maximum distance between the two classes. In real-life phenomena, datasets are contaminated with noise and may contain outliers. In order to treat such datasets [18], introduced non-negative slack variables, \(\xi_{i}\geq 0,\forall i\), and a parameter C, describing the desired tolerance to classification errors. The solution to the problem of identifying the optimal separator can be dealt with through the Lagrange relaxation procedure of the following equation:  
\[\underset{\mathbf{w},b,\xi}{min}\underset{\mathbf{\alpha},\mathbf{\mu}}{max} \left(\frac{1}{2}\|\mathbf{w}\|^{2}+C\sum_{i=1}^{n}\xi_{i}-\sum_{j=1}^{n}a_{j} \Big{[}y_{j}\Big{(}\mathbf{w}^{T}\mathbf{x}_{j}-b\Big{)}-1+\xi_{j}\Big{]} \right)-\sum_{k=1}^{n}\mu_{k}\xi_{k}, \tag{7}\]  
where \(\xi_{i}\) measures the distance of vector \(x_{i}\) from the separator when classified erroneously, and \(\alpha_{1}\), \(\alpha_{2}\),..., \(\alpha_{n}\) are the non-negative Lagrange multipliers. The hyperplane is then defined as:  
\[f(\mathbf{x}_{i})=\ \mathbf{w}^{T}\mathbf{x}_{i}-\ b\ =\ 0, \tag{8}\]  
where:  
\[\hat{\mathbf{w}}=\sum_{i=1}^{n}a_{i}y_{i}\mathbf{x}_{i}, \tag{9}\]  
\[\hat{b}=\hat{\mathbf{w}}^{T}\mathbf{x}_{i}-y_{i},\ i\in V, \tag{10}\]  
where \(V\) is the set of the support vector indices. Figure 1: Hyperplane selection and support vectors. The SVs (represented with the pronounced black contour) define the margins that are represented with the dashed lines and outline the separating hyperplane represented by the continuous line. When the dataset is not linearly separable, the SVM model is paired with kernel projection: The initial data space is projected through a kernel function into a higher dimensionality space (called feature space) where the dataset may be linearly separable.