## 2 Multi-Classification LS-SVM Survey  
SVM technique as invented by [PERSON], [PERSON] and [PERSON] was first introduced during the Computational Learning Theory (COLT) conference of 1992 [4] and since then it has established itself as one of the leading approaches in the pattern recognition and the machine learning areas, as demonstrated by the results obtained by a broad range of practical applications and recent research work [5-27]. In terms of structural or representational capacity, SVM behaves like a neural network, but it differs in the learning technique.SVM solves a quadratic programming (QP) problem and finds a computationally efficient way of learning a hyper-plane that correctly classifies the high-dimensional feature space after using a linear combination of kernel functions that have to be positive definite [28]. With kernel functions centered on the training input data, SVM minimizes the confidence interval and keeps the training error \(e\) fixed while retaining only the support vectors (SV) from the input data. For instance, given a classical linearly separable multi-classification task with attributes or feature sets \(<\!\!f_{1}\!f_{2}...f_{J}\!\!>\) defined as \(\left\{x_{i},y_{i}\right\}_{i=1}^{N},\) where \(x_{i}\in R^{f}\) represents the \(i^{th}\) input image and \(y_{i}\) the output class, an LS-SVM as introduced in [8] optimizes the objective function  
\[\text{Objective function:}\frac{1}{2}\sum_{i=1}^{c}w_{-}^{T}.w_{+} =\frac{\lambda}{c}\sum_{i=1}^{N}\sum_{n=y_{i}}^{c}(e_{i}^{m})^{2} \tag{1}\]  
where \(\lambda\) is a suitable positive penalty parameter that controls the tradeoff between the classification error \(e\) of the c different classes and the margin maximization during the training phase. The error term \(e\), often referred to as the slack variable, accounts for the non-separable data points. [PERSON] and [PERSON] [29] showed that SVM accuracy rates in general are influenced by the selection of \(\lambda\) which varies depending on the problem under investigation. The selection of \(\lambda\) can be found heuristically or by a grid search. Depending on the decomposing strategy for converting the multi-classification problem into a set of binary ones, the ML constraint can be formulated as a multi-classification objective function, one-versus-rest, pair-wise, or error-correcting output code [9]. The multi-classification objective function has probably the most compact form as it optimizes the problem into a one single step. It constructs \(c\) two-class rules, where each classifier separates training vectors of class \(y_{i}\) from the other \(m\) classes using the constraint of the hyper-planes defined by their slopes \(w\) and intercepts \(b\):  
\[\text{Constraint: }w_{y_{i}}^{T}x_{i}+b_{y_{i}}\geq w_{m}^{T}x_{i}+b_{m}-2-e_{i}  
planes [4]. _WP-SVM_ does not numerically solve for the support vectors and their nonzero Lagrange multipliers. Instead, it classifies points by assigning them to the closest parallel planes without explicitly calculating the SVs. The hyper-planes are still pushed apart by a maximum margin \(w\), but the data are clustered around the planes rather than on the planes. The uniqueness of the global solution is still valid because it is a property of the Hessian being positive definite or semi definite [28].  
Figure 1 a represents a standard multiclass SVM with the SV lying on the hyperplanes, whereas Figure 1 b illustrates the proposed WP-SVM where data points are rather clustered around the hyperplanes.  
The mathematical steps for the optimization start by solving for the partial derivatives of \({}^{L}(w,b)\) with respect to both \(w\) and \(b\).  
\[\frac{\partial L(w,b)}{\partial w_{n}}=0\,\ \frac{\partial L(w,b)}{\partial b_{n}}=0 \tag{8}\]  
Defining  
\[a_{i}= \begin{cases}1&y_{i}=n\\ 0&y_{i}\
eq n\end{cases} \]  
\[