## 4 Least-Squares Support Vector Machine  
The least squares support vector machine is a development of the support vector machine that was suggested by [PERSON] and [PERSON] (2018). Instead of the inequality constraint in SVM, some equality constraints are used in LS-SVM. Hence, it obtains the final solution by dissolving a set of linear equations. This method has many advantages over the SVM method, which can increase computational efficiency, reduce complexity, and reduce training parameters. The following is a short overview of the LS-SVM's concepts.  
The LS-SVM uses a nonlinear function to transfer the input data \(\mathbf{z}\) into a high-dimensional feature space, where it subsequently uses a linear regressor to generate the predicted \(\hat{y}\). The regressor is described as:  
\[\hat{y}(\mathbf{z})=\mathbf{w}^{T}\phi(\mathbf{z})+b \tag{4}\]  
in which \(\mathbf{w}\) is the weights, \(b\) is a scaler bias, and \(\phi(\cdot)\) represents an unknown nonlinear function. With the given the training dataset \(\left(\mathbf{z}^{k},y^{k}\right)_{K=1}^{N}\), in which \(N\) is the number of the training samples, its training can be conducted through an optimization problem in the primal weight space:  
\[\min J_{p}(\mathbf{w},\mathbf{e})=\frac{1}{2}\mathbf{w}^{T}\mathbf{w}+\gamma \frac{1}{2}\sum_{k=1}^{N}e_{k}^{2} \tag{5}\]  
where  
\[y^{k}=\mathbf{w}^{T}\phi\left(\mathbf{z}^{k}\right)+b+e_{k} \tag{6}\]  
Using the Lagrangian function  
\[L(\mathbf{w},b,\mathbf{e};\mathbf{\alpha})=J_{p}(\mathbf{w},\mathbf{e})-\sum_{k=1}^{N} \alpha_{k}\left(\mathbf{w}^{T}\phi\left(\mathbf{z}^{k}\right)+b+e_{k}-y_{k}\right) \tag{7}\]  
where \(\alpha_{k}\) are Lagrange multipliers. The Karush-Kuhn-Tucker conditions can be used to transform the optimization problem from primal form to its dual form. The following linear set of equations can be used to obtain the parameters \(\alpha\) and \(b\) under these conditions and by omitting out the variables \(\mathbf{w}\) and \(\mathbf{e}\):  
\[\begin{bmatrix}0&1_{N}^{T}\\ 1_{N}&\Omega+\frac{1}{\gamma}\end{bmatrix}\begin{bmatrix}b\\ \mathbf{\alpha}\end{bmatrix}=\begin{bmatrix}0\\ \mathbf{y}\end{bmatrix} \tag{8}\]  
where \(\mathbf{y}=\left[y^{1},\ldots,y^{N}\right]^{T}\), \(1_{N}=\left[1,\ldots,1\right]^{T}\), \(\mathbf{\alpha}=\left[\alpha_{1},\ \ldots,\ \alpha_{N}\right]^{T}\) is called the support vector and \(\mathbf{I}\) is an identity matrix. In addition, \(\gamma\) is the regularization parameter. This parameter makes a balance between the squared error terms and weight decay in Equation (5). Additionally, the kernel trick is adopted to obtain the Gram matrix as follows:  
\[\Omega_{kl}=\phi\Big{(}\mathbf{z}^{k}\Big{)}^{T}\phi\Big{(}\mathbf{z}^{l}\Big{)} =K\Big{(}\mathbf{z}^{k},\mathbf{z}^{l}\Big{)},\ \ \ \ \ k,\ \ \ l=1,\ 2,\ \ldots,\ N \tag{9}\]  
where \(K(\cdot,\cdot)\) represents a predetermined kernel function. The kernel's purpose is to prevent the calculation of the mapping \(\phi(\cdot)\), explicitly. The LS\(-\)SVM prediction is as:  
\[\hat{y}(\mathbf{z})=\sum_{k=1}^{N}\alpha_{k}\ K\Big{(}\mathbf{z},\mathbf{z}^{ k}\Big{)}+b \tag{10}\]  
where \(\alpha_{k}\) and \(b\) are the linear set's solution of Equation (8) and \(K(\cdot,\cdot)\) is the kernel that is satisfying [PERSON]'s condition. The radial base function (RBF) that can be used as a kernel function for nonlinear mapping is defined as  
\[K\Big{(}\mathbf{z},\mathbf{z}^{k}\Big{)}=\exp\left(-\frac{\left|\left|\mathbf{ Z}-\mathbf{Z}^{k}\right|\right|^{2}}{\sigma^{2}}\right) \tag{11}\]  
where \(\left\|\cdot\right\|\) is Euclidean distance and \(\sigma\) is the width parameter. With the devoted regularization parameter \(\gamma\) and kernel parameter \(\sigma\), the goal of the training is to specify the support values \(\alpha_{k}\) and the bias \(b\) by using Equation (8). The LS\(-\)SVM model's great generalization capacity depends on the two hyperparameters, \(\gamma\) and \(\sigma\), being appropriately adjusted. The hyperparameter values in this paper are determined using the \(k-\)fold cross\(-\)validation technique.