# Lean Manufacturing Soft Sensors for Automotive Industries
## 2 Methodology
### Soft Sensor Modeling
#### 2.2.6 Support Vector Machine (SVM)  
SVM is a popular regression and classification supervised learning algorithm that segregates multi-dimensional data classes using a hyperplane. SVM creates and uses a hyperplane as a decision boundary to be able to classify a future data point into its correct category. The extreme cases of each data class (known as support vectors) help in defining the class boundary, or the hyperplane. The support vectors of respective data classes lie closest to the hyperplane. The SVM algorithm explores a number of hyperplane solutions and selects the best class boundary that maximizes the separation margin between classes. SVM achieves this by attaching a penalty to every point that lies on the other side of its class across a particular hyperplane. The hyperplane dimensions are dependent on the number of data class features. For instance, for a dataset having only two features, the hyperplane takes the form of a line (Figure 8), whereas for a dataset with three features, the hyperplane takes the form of a plane to segregate the classes.  
Hence, linear SVM is useful for a dataset having linearly separable entities, whereas a nonlinear SVM classifier is required for datasets wherein linear separation is not possible.  
Figure 8: SVM Classification.  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Model Hyperparameters** & **Gaussian Naive Bayes** & **Kernel Naive Bayes** \\ \hline Preset Naive Bayes & Gaussian & Kernel \\ Distribution name for numeric predictors & Gaussian & Kernel \\ Distribution name for categorical predictors & NA & NA \\ Kernel type & NA & Gaussian \\ Support & NA & Unbounded \\ \hline \hline \end{tabular}
\end{table}
Table 8: Naive Bayes classification hyperparameters.  
SVM is more effective in solving higher-dimensional feature spaces, as they are more separable into distinct class label regions across the hyperplane. Similarly, SVM works better in datasets that have wider separation margins between data classes. This implies that SVM is more effective in datasets wherein the number of samples is lesser than the number of dimensions. Moreover, since it uses only a small part of the entire dataset, namely the support vectors, SVM is highly memory-efficient. Conversely, SVM proves relatively ineffective in larger datasets, or when multiple data classes are overlapping, and/or when the total number of data samples is lesser than the number of features per data point. The SVM loss function is defined as follows:  
\[\min_{w}\lambda\|w\|^{2}+\sum_{i=1}^{n}(1-y_{i}\langle x_{i},w\rangle)_{+} \tag{10}\]  
where \(x\), \(y\), \(\lambda\), and \(w\) represent the input parameters, class output, Kernel scale, and the regularisation parameters, respectively.  
Table 9 shows the classification hyperparameters for the various SVM models explored in the current study. Each SVM model employed a kernel function suitable to its configuration. Input data were scaled automatically (before input to the kernel function) by the Matlab software for linear, quadratic, and cubic SVMs. Progressively higher kernel input scales were employed for fine, medium, and coarse Gaussian SVMs. A box constraint level of one was defined for all SVMs, ensuring a high missclassification cost and a stricter data classification. The one-versus-one methodology enables SVM, which is a binary classifier, to execute multi-class classification. One-versus-one splits the multi class dataset into multiple binary classification problems. All input data were standardized and scaled to a standard normal distribution.