# Fast Support Vector Machine for Power Quality Disturbance Classification
## 2 Fundamental Theory  
An SVM, a learning paradigm proposed by [PERSON] et al., is effective for both regression problems [24; 25], and pattern recognition [25; 26; 29]. An SVM can deal with linear and non-linearly separable models from the statistical learning theory [23]. In addition, an SVM is a linear machine of one output, based on theoretical derivation, the number of hidden units is equal to the SVs, closest to the separation plane for different classes. An SVM optimizes the tradeoff between the training errors and Vapnik-Chervonenkis (VC) dimension and provides a concept of complexity measure [24; 25; 26]. For complex data, the SVM is extended to work in the high dimensional space with nonlinear mapping from the K-dimensional input vector into R-dimensional feature space (R \(>\) K) through a kernel function. This structural risk minimization (SRM) framework generalizes the empirical risk minimization (ERM) principle applied for ANN training [25; 26]. For the two-class problems, SVM is based on a hyper-plane to separate the data. Consider an n-dimensional real-valued input vector \(\mathbf{x}_{i}\), \(\mathbf{x}_{i}\in\Re^{n}\) of the training set \(\mathrm{Tr}=\left\{\left(\mathbf{x}_{i},y_{i}\right)\right\}_{i=1}^{l}\), where \(y_{i}\in\left\{-1,\,+1\right\}\) is the classification that determines the class of \(\mathbf{x}_{i}\). The hyper-plane can be determined by \(\mathbf{w}\), an orthogonal vector, and a bias \(b\) satisfying \(\mathbf{w}^{T}\mathbf{x}+b=0\), as shown in Figure 1. The classification becomes the problem of finding the hyper-plane to separate the classes, i.e., to maximize the distance between classes, called the'margin of separation M'. The hyper-plane can be found using the training sets, by the nearest points to it with the largest margin, i.e., the so-called SVs. The hyper-plane depends on the SVs. In a simple form, SVMs learn the linear decision rules by  
\[f(\mathbf{x})=sign(\mathbf{w}^{T}\mathbf{x}+b) \tag{1}\]  
so that \((\mathbf{w},b)\) are needed to classify the training examples to maximize M.  
To show the theory, consider that we can always scale \(\mathbf{w}\) and \(b\) so that  
\[\mathbf{w}^{T}\mathbf{x}+b=\pm 1 \tag{2}\]  
for the SVs as in Figure 1, and by the same time, it is clear that for non-SVs, we will have  
\[\mathbf{w}^{T}\mathbf{x}+b>+1\text{ or }\mathbf{w}^{T}\mathbf{x}+b<-1 \tag{3}\]  
Using the SVs \(\mathbf{x}_{1}\) and \(\mathbf{x}_{2}\), the margin M can be calculated as  
\[\text{M}=\frac{\mathbf{w}^{T}}{\|\mathbf{w}\|}(\mathbf{x}_{1}-\mathbf{x}_{2}) =\frac{2}{\|\mathbf{w}\|} \tag{4}\]  
Maximizing M is equivalent to the primal optimization problem of minimizing  
\[2^{-1}\mathbf{w}^{T}\mathbf{w} \tag{5}\]  
subject to  
\[y_{i}\cdot[\langle\mathbf{w},\mathbf{x}_{i}\rangle+b]\geq 1\quad i=1,\,2,\, \ldots,\,l \tag{6}\]  
where \(\langle\mathbf{w},\mathbf{x}_{i}\rangle=\mathbf{w}^{T}\mathbf{x}_{i}\) is an inner product. Minimization of cost Function (5) with constrain (6) is a quadratic optimization problem, and a unique solution can be found using the Lagrange multiplier \(\alpha\), we have  
\[L(\mathbf{w},b,\boldsymbol{\alpha})=2^{-1}\mathbf{w}^{T}\mathbf{w}-\sum_{i=1 }^{l}\alpha_{i}\Big{[}y_{i}(\mathbf{w}^{T}\mathbf{x}_{i}+b)-1\Big{]}\]  
\(\boldsymbol{\alpha}=[\alpha_{1},\,\alpha_{2},\cdots,\,\alpha_{l}]^{T}\) is the vector of Lagrange multipliers. It is easier to solve the dual problem (7) with (8) instead of solving for (5) and (6), by maximizing  
\[L_{d}(\alpha)=\sum_{i=1}^{l}\alpha_{i}-2^{-1}\sum_{i=1}^{l}\sum_{j=1}^{l} \alpha_{i}\alpha_{j}y_{i}y_{j}\langle\mathbf{x}_{i},\mathbf{x}_{j}\rangle \tag{7}\]  
Figure 1: The concept of linear SVM with SVs. subject to  
\[f(\mathbf{x})=sign(\mathbf{w}^{T}\mathbf{x}+b) \tag{1}\]0  
where \(\langle xi,\,xj\rangle\) is a scalar inner product. \(C\) is a preselected positive penalty factor, acting as a trade-off between the two terms.