## 5 SVM Classification  
The Support Vector Machines, introduced in 1992 by [PERSON], [PERSON], and [PERSON] [54] in the form we know them today, are robust classification models that can leverage different kernels in order to construct non-linear decision boundaries without the need of projecting the training set into a non-linear space. These models try to fit a decision rule to the training set, aiming to maximize the distance between data points and the decision boundary (thick line in Figure 3). The distance is referred to as a margin and data points laying on the margin (dotted line) are called support vectors, hence the name. The hyperplane, also called the decision boundary, can be characterized using a normal vector (perpendicular) to that hyperplane, marked as \(\overline{\omega}\) in Figure 3. In order to see on which side of the decision boundary an unknown vector \(\overline{u}\) resides, a decision function is utilized:  
\[\overline{\omega}\cdot\overline{u}+b\geq 0. \tag{1}\]  
In hard margin classification, the training consists of maximizing the margin such that the decision rule (1) for positive data points (input vectors) yields values equal to or greater than 1 (they lay outside of the margin and not merely on the right side of the decision boundary) and smaller than or equal to \(-1\) for negative input vectors:  
\[y_{j}(\overline{\omega}\cdot\overline{u}+b)\geq 1,\text{where}y_{j}=\left\{ \begin{aligned} +1&\text{\emph{for positive samples}}\\ -1&\text{\emph{otherwise}}\end{aligned} \right. \tag{2}\]  
The margin maximization is equivalent to the minimization of  
\[\min_{\omega,b}\frac{1}{2}\|\omega\|^{2} \tag{3}\]  
Figure 3: Geometrical representation of optimal hyperplane separating negative examples (black squares) from positive examples (white triangles). with respect to the constraint given in (2). Hard margin classification is convenient when the data points are easily separable and one wants to avoid misclassifications at all costs. In case of our attack, the initial labels derived from _the comparison to the mean_ method are known to be imprecise, and the mislabeled data points would distort the decision boundary and the model might learn wrong classifications. The idea is to introduce a regularization parameter to allow for some data points to be left misclassified, keep the decision boundary smooth, and, therefore, retain a large margin. An SVM model with regularization is called a soft margin classifier and was introduced by [PERSON] and [PERSON] in 1995 [PERSON] and [PERSON] in 1995 ([PERSON] and [PERSON] in 1995). The margin maximization from (3) is modified as follows:  
\[\min_{\omega,b,\xi}\frac{1}{2}\|\omega\|^{2}+C{\sum}_{j=1}^{n-2}\xi_{j}. \tag{4}\]  
We denote each data point in the training set with the subscript \(j\) since we classify each slot to belong to either the positive or negative class. The new slack variable \(\xi_{j}\) determines the distance from the misclassified sample (slot) to the decision boundary (marked in Figure 3). It is a penalty term for every misclassification and is derived during the training processes. The regularization parameter \(C\) takes values between 0 and 1 and needs to be set before the training procedure as a hyperparameter. A value of 1 means no regularization, and values close to 0 mean high regularization and hence a smaller influence of misclassified data points on the decision boundary. The constraint from (2) takes the following form for the soft margin classifier:  
\[y_{j}(\overline{\omega}\cdot\overline{u}+b)\geq 1-\xi_{j}\forall j=1,\dots,n-2 \text{ {and} }\xi_{j}>0 \tag{5}\]  
The training algorithm optimizes (4) given the constraint from (5), using every sample from the training set \(x_{j}\) as the unknown input vector \(\overline{u}\); in other words, it tries out different hyperplanes separating the training set such that it maximizes the margin but satisfies the constraint that all samples should lay outside of the margin with some tolerance of samples laying on the wrong side of the decision boundary.