## 4 Quantitative Methods for the Detection of AD
### Support Vector Machines  
We will now introduce the support vector machine (SVM). SVMs are exceptionally common in the neuroimaging literature. They differ significantly from GLMs or PCA/SSM in that no threshold score is used to determine group designation, but rather the sign of the score is used in determining the group label. The theoretical foundation of the SVM can be simply explained by the statement \"a 2D space is clearly separable by a 1D line\". This is also intuitively true for the separation of a 3D space by a 2D plane. This property holds for spaces of arbitrary dimension: a hyperspace of \(N\)-dimensions is separable by a linear hyperplane of \(N\)-1 dimensions [123]. Consider some arbitrary dataset,  
\[(x_{1},y_{1}),(x_{2},y_{2}),\ldots,(x_{n},y_{n}) \tag{24}\]  
where \(x_{i}\) is a \(p\)-dimensional feature-vector and \(y_{i}\) is a binary label classifying the vector. An SVM will separate these feature vectors according to their class by attempting to find the geometric margin (a hyperplane) that maximizes the minimum distance between classes. The optimal hyperplane is defined as the set of points that satisfy:  
\[\mathcal{H}=\left\{\mathbf{x}\middle|\mathbf{w^{T}b}+b\right\}=0 \tag{25}\]  
where \(\mathbf{w}\) is a vector normal to the hyperplane and \(b\) is a bias. All vectors having a magnitude \(\mathcal{H}\geq 1\) or \(\mathcal{H}\leq 1\) belong to the positive and negative classes, respectively, but those with the special property \(|\mathcal{H}|=1\) are designated support vectors. The support vectors determine the placement of the hyperplane, and the SVM optimization algorithm will attempt to construct the hyperplane using the support vectors that are as far away from one another as possible, as seen in Figure 4 and visualized in Figure 5. This is a minimization problem that has the form:  
\[\min_{w,\ b}\mathbf{w^{T}w}\,s.t.\ \ \forall i\left|y_{i}\left(\mathbf{w^{T}x}+b \right)\right|\geq 1. \tag{26}\]  
Figure 4: The SVM functions here as a linear binary classifier. The data belonging to either the positive or negative class are plotted, and the optimal boundary between them clearly separates each class. The support vectors used to draw this boundary are centered in a circle of their relevant class color, while examples that violate the slack constraint are circled in black. Note that when support vectors are of unit length, the width of any hyperplane is determined by \(\frac{2}{|\mathbf{w}|}\), therefore minimizing the magnitude of \(\mathbf{w}\) maximizes the separation between classes. Most data are not linearly separable or are sufficiently low in dimensionality that there is no optimal boundary that separates all vectors. For these cases, we may introduce a \"soft-margin\" optimization by allowing a slack variable (\(\zeta\)) and a regularization constant (\(C\)) in (28), such that:  
\[\min_{\mathbf{w},\ b}\mathbf{w^{T}}\mathbf{w}+C\sum_{i=1}^{n}\zeta_{i}\ \ s.t.\ \ \forall i \left|y_{i}\left(\mathbf{w^{T}}\mathbf{x}+b\right)\right|\geq 1-\zeta_{i}\ \ s.t.\ \ \zeta_{i}\geq 0. \tag{27}\]  
In this form, the classifier will allow feature vectors to exist on the \"wrong\" side of the hyperplane without affecting the shape of the boundary. Larger values of \(C\) penalize the optimization and often make for a highly non-linear classification boundary, whereas smaller values of \(C\) allow for a simpler boundary less prone to overfitting the data. Note that in (26), the optimization considers the inner product of the vector orthogonal to the hyperplane. Any other mathematical function expressible as an inner product may be chosen in its place; thus, another solution to the problem of the non-linear boundary is to artificially increase the dimensionality of the data by appropriate choice of kernel function. This is known as the \"kernel trick\".