## 4 System modeling
### Partial least square regression (PLSR)
#### 4.1.8 Support vector machine (SVM)  
The principle for SVM was proposed in the 1960's, but extensive studies and applications of SVM began in the 1990's. Principally SVM is a statistical method, but in many cases it can be categorized in the neural network field, as it has the same structure as RBFNN. In contrast to BPNN which requires defining a number of hidden neurons before training, the structure of SVM is adaptively determined during a learning process. Figure 5 shows a schematic linear classification based on support vectors. The black circles and triangles are the support vectors of two classes, respectively. Basically, SVM is to find a discrimination function as shown in eq. (10), which is the same as LDA, but the calculation of the weight vector is only based on the support vectors.  
\[f(\mathbf{x}_{i})=\mathbf{w}^{T}\mathbf{x}_{i}+b \tag{10}\]  
where \(\mathbf{x}_{i}\) is a sample vector, \(\mathbf{w}\) is the weight vector and \(b\) is a threshold value.  
Assuming support vectors have been determined, the optimal separation plane is the one that can maximize the margin between two classes, which is proved to be \(2/\!\left|\mathbf{w}\right|\!\left|\right.\)[75]. Introducing Lagrange multipliers to this quadratic optimization problem can yield,  
\[\min:L(\mathbf{w},b,\alpha)=\frac{1}{2}\,\mathbf{w}^{T}\mathbf{w}-\sum_{i} \alpha_{i}\{\mathcal{V}_{i}[\mathbf{w}\mathbf{x}_{i}+b]\!-\!1\} \tag{11}\]where \(\alpha_{i}\) are Lagrange multipliers. The optimal solution is \(\mathbf{w}^{opt}=\sum_{i}\alpha_{i}^{opt}y_{i}\mathbf{x}_{i}\). \(b^{opt}\) can be easily calculated by any support vector using \(\mathbf{w}^{opt}\). Support vectors are the samples whose \(\alpha_{i}\
eq 0\). The separation process can be extended to multiple-class problems.  
Using Gaussian kernel functions to transform the original input space to a linear transitional space, the linear separation process of SVM can be easily extended to nonlinear systems. Gaussian kernel functions are also a key issue for RBFNN, so in some cases SVM for nonlinear systems can also be considered as a specific modification of RBFNN where only the support vectors (samples) are used for constructing a classifier.  
SVM has obtained successful applications in many fields such as E-nose based vapor detection for environmental monitoring [76] and alcohol identification [77]. Using Gaussian kernel function for signal space transformation, SVM also presents some inherent problems including the determination of a proper Gaussian width for each kernel function.  
In addition to the above three ANN models, probability neural network (PNN) is also popularly used for sensor signal analysis. Basically, PNN has the same principle as Bayes classifier. PNN also used kernel functions for signal space transformation, so PNN can be considered another specific modification of RBFNN or a neural network implementation of Bayes classifier. Kernel functions have been also integrated with PLS and PCA to extend the application fields of these two classical machine learning tools from linear problems to nonlinear problems. RBF-PLS discussed above is a typical kernel PLS model.