# Metaheuristics and Support Vector Data Description for Fault Detection in Industrial Processes
## 2 Theoretical Background  
### Support Vector Data Description  
Tax and Duin [10] proposed the SVDD classification method which determines a close boundary around the data set for a given class: a hyper-sphere characterized by a center and a radius \(R\geq 0\) that defines a separation between the inner region with high data density and the outer region with low density. The data that lie right at the limit of the hyper-sphere are called the support vectors while those outside are the outliers. Let \(\left\{\mathbf{x}_{i}:i=1,\,2,\,3,\,\ldots,\,N\right\}\) be a column vector set and \(\mathbf{x}^{2}=\mathbf{x}\cdot\mathbf{x}\) the training set for which a description must be specified and assume that \(x_{i}\)'s show variances in all given directions. The data set delimitation of the inner region of the hyper-sphere will be minimized with an error function which minimizes the possibility of accepting outliers and such a function is defined as:  
\[\begin{array}{rcl}\min F(R,\mathbf{a},\zeta_{i})&=& R^{2}+C\sum_{i} \xi_{i}\\ \\ \text{s.t.}\ \left\|\mathbf{x}_{i}-\mathbf{a}\right\|^{2}&\leq& R^{2}+\xi_{i},\ \xi_{i}\geq 0\ \forall i\end{array} \tag{1}\]  
where \(\zeta_{i}\) are the slack variables that will penalize the largest distances, and C is a control parameter of the trade-off between volume of the hyper-sphere and the errors [45]. Applying the Lagrange multipliers method, the following equations arise:  
\[\begin{array}{l}L(R,\mathbf{a},\alpha_{i},\gamma_{i},\xi_{i})=R^{2}+C\sum_{i }\xi_{i}-\sum\alpha_{i}\\ \\ \left\{R^{2}+\xi_{i}-\left(\left\|\mathbf{x}_{i}\right\|^{2}-2\mathbf{a}\cdot \mathbf{x}_{i}+\left\|\mathbf{a}\right\|^{2}\right)\right\}-\sum_{i}\gamma_{i} \xi_{i}\end{array} \tag{2}\]  
with \(\alpha_{i}\geq 0\) as Lagrangian multipliers and \(\gamma_{i}>0\).  
The dual formulation of the equations in (2) can be obtained by solving the KKT conditions, which reads  
\[\begin{array}{l}\min\qquad\sum_{i}\alpha_{i}\left(\mathbf{x}_{i},\mathbf{x} _{i}\right)-\sum_{i}\sum_{j}\alpha_{i}\alpha_{j}\left(\mathbf{x}_{i},\mathbf{ x}_{j}\right)\\ \\ \text{s.t.}\qquad 0\leq\alpha\leq C\end{array} \tag{3}\]  
In case a given \(\mathbf{x}_{i}\) satisfies \(\left\|\mathbf{x}_{i}-\mathbf{a}\right\|^{2}<R^{2}+\xi_{i}\), then \(\alpha_{i}=0\); otherwise, when \(\mathbf{x}_{i}\) satisfies \(\left\|\mathbf{x}_{i}-\mathbf{a}\right\|^{2}=R^{2}+\xi_{i}\), the corresponding Lagrange multiplier \(\alpha_{i}\) is strictly greater than zero.  
The vectors \(\mathbf{x}_{i}\) corresponding to \(\alpha_{i}>0\) represent the set of vectors necessary to characterize a set of data, and this set of vectors can be called support vectors of the description [46]. When there is a new vector \(\mathbf{z}\), the distance to the center of the sphere can be computed. If this distance is smaller than \(R\), \(\mathbf{z}\) is accepted as a new vector in the description of the data, that is,  
\[\left\|\mathbf{z}-\mathbf{a}\right\|^{2}=\left(\mathbf{z}\cdot\mathbf{z} \right)-2\sum_{i=1}^{l}\alpha_{i}\left(\mathbf{z}\cdot\mathbf{x}_{i}\right)+ \sum_{i=1}^{l}\sum_{j=1}^{l}\alpha_{i}\alpha_{j}\left(\mathbf{x}_{i}\cdot \mathbf{x}_{j}\right)\leq R^{2} \tag{4}\]  
Note that, if the inner product in Equation (2) is replaced with a kernel function \(K\left(\mathbf{x}_{i},\mathbf{x}_{i}\right)\) a description for nonlinear data sets can then be obtained. In this manner, data are mapped to a higher dimension feature space by means of the kernel function, and this makes the nonlinear data separable [47]. Thus, the problem can be reformulated as follows:  
\[\begin{array}{l}\min\qquad\sum_{i}\alpha_{i}K\left(\mathbf{x}_{i},\mathbf{ x}_{i}\right)-\sum_{i}\sum_{j}\alpha_{i}\alpha_{j}K\left(\mathbf{x}_{i}, \mathbf{x}_{j}\right)\\ \\ \text{s.t.}\qquad\qquad 0\leq\alpha\leq C\end{array} \tag{5}\]  
where \(\alpha_{i}\) remains as the Lagrange multipliers and \(K\left(\mathbf{x}_{i},\mathbf{x}_{j}\right)\) is a kernel function used as a functional mapping.