## 5 Classification of Selected Household Characteristics
### Classification Algorithms
#### 5.3.3 Support Vector Classification  
Support Vector learning is based on simple ideas which originated in statistical learning theory [31]. The simplicity comes from the fact that Support Vector Machines (SVMs) apply a simple linear method to the data but in a high-dimensional feature space non-linearly related to the input space. Moreover, even though we can think of SVMs as a linear algorithm in a high-dimensional space, in practice, it does not involve any computations in that high-dimensional space [28].  
SVMs use an implicit mapping \(\Phi\) of the input data into a high-dimensional feature space defined by a kernel function, i.e., a function returning the inner product \(\langle\Phi(\mathbf{x}_{i}),\Phi(\mathbf{x}_{i}^{\prime})\rangle\) between the images of two data points \(\mathbf{x}_{i},\mathbf{x}_{i}^{\prime}\) in the feature space. The learning then takes place in the feature space, and the data points only appear inside dot products with other points [32]. More precisely, if a projection \(\Phi:\mathbf{X}\rightarrow\mathbf{H}\) is used, the dot product \(\langle\Phi(\mathbf{x}_{i}),\Phi(\mathbf{x}_{i}^{\prime})\rangle\) can be represented by a kernel function \(k\) which is computationally simpler than explicitly projecting \(\mathbf{x}_{i}\) and \(\mathbf{x}_{i}^{\prime}\) into the feature space \(\mathbf{H}\)[28].  
Training an SVM involves solving a quadratic optimization problem. Using a standard quadratic problem solver for training an SVM would involve solving a big QP problem even for a moderately sized data set, including the computation of an \(n\times n\) matrix in memory (\(n\) number of training points). In general, predictions correspond to the decision function:  
\[P_{i}=\text{sign}(\langle\mathbf{w},\Phi(\mathbf{x}_{i})\rangle), \tag{14}\]  
where solution \(\mathbf{w}\) has an expansion \(\mathbf{w}=\alpha_{i}\sum_{i}\Phi(\mathbf{x}_{i})\) in terms of a subset of training patterns that lie on the margin [25].  
In the case of the L2-norm soft margin classification, the primal optimization problem takes the form:  
\[\text{minimize }(t,\mathbf{w})=\frac{1}{2}||\mathbf{w}||^{2}+\frac{C}{n}\sum_{i=1}^{n }(\xi_{i}), \tag{15}\]\[\begin{array}{c}\text{\emph{subject to }}L_{i}(\langle\Phi(\vec{x}_{i}),\vec{w} \rangle+b)\,\leq 1-\xi_{i}^{*}\\ \xi_{i}\geq 0\,\,(i=1,\ldots,n)\end{array}\,, \tag{16}\]  
where \(n\) is the number of training patterns, and \(L_{i}\
eq 1\), \(C\) is the cost parameter that controls the penalty paid by the SVM for misclassifying a training point and thus the complexity of the prediction function. A high cost value \(C\) will force the SVM to create a complex enough prediction function to misclassify as few training points as possible, while a lower cost parameter will lead to a simpler prediction function.  
To construct the support vector machine model, \(C\)-SVR from the _kernlab_ library with sequential minimal optimization (SMO) was used to solve the quadratic programming problem. A linear, polynomial (of degree 1, 2 and 3) and radial (\(\gamma\) from 0.1 to 1 by 0.2) kernel function were used, and \(\varepsilon\) (which defines the margin width for which the error function is zero) was arbitrarily taken from the following set {0.1, 0.3, 0.5, 0.7, 0.9}. The regularized parameter \(C\) that controls overfitting was arbitrarily set to one of the following values {0, 0.2, 0.4, 0.6, 0.8, 1}. Finally, as in all previous cases, the model that maximized the function (6) was chosen [29].