# Lean Manufacturing Soft Sensors for Automotive Industries
## 2 Methodology
### Soft Sensor Modeling
#### 2.2.7 Ensemble Models  
Ensemble machine learning models aim to achieve better prediction accuracy by strategically combining the estimations of multiple models on the same dataset. The bootstrap aggregation method (bagging) is one of the prominent ensemble techniques wherein multiple decision trees are fitted on different parts of the same dataset and the final prediction is computed as an average of all tree estimations.  
Boosting is another significant ensemble method that involves sequential application of multiple models that build upon the estimations of previous models. The successive ensemble models attempt to further boost the estimation accuracy achieved by the previous model, thus ensuring continuous correction of prediction errors. In other words, boosting ensemble iteratively moves models to focus on more and more data samples that are hard to predict. The final estimation of the boosted ensemble learning is computed as the weighted average of all model predictions. Random undersampling boosting (RUSBoost) is a boosting ensemble method that is especially adept at handling datasets that are imbalanced, having significantly asymmetric/non-uniform number of data samples across classes. Adaptive Boosting (AdaBoost), on the other hand, assigns weights to each data point to improve overall estimation, continuously adapting higher weights to the data instances that are difficult to classify.  
Subspace ensembles are yet another category of ensemble models in which a model is fitted on different subsets of features of the same dataset. In other words, the same model  
\begin{table}
\begin{tabular}{l l l l l l l l} \hline \hline
**Model** & **Linear SVM** & **Quadratic SVM** & **Cubic SVM** & **Fine Gaussian SVM** & **Medium Gaussian SVM** & **Coarse Gaussian SVM** \\ \hline Preset SVM & Linear & Quadratic & Cubic & Fine Gaussian & Medium Gaussian & Coarse Gaussian \\ Kernel Function & Linear & Quadratic & Cubic & Gaussian & Gaussian & Gaussian \\ Kernel Scale & Automatic & Automatic & Automatic & 0.61 & 2.4 & 9.8 \\ Box Constraint Level & \(1\) & \(1\) & \(1\) & \(1\) & \(1\) & \(1\) \\ Multiclass Method & One-vs-One & One-vs-One & One-vs-One & One-vs-One & One-vs-One & One-vs-One \\ Standardized data & True & True & True & True & True & True \\ \hline \hline \end{tabular}
\end{table}
Table 9: Support Vector Machine (SVM) classification hyperparameters.  
architecture is fitted on different groups of input data features from the same training dataset. The final prediction of the subspace ensemble is the mode (for classification) or mean (for regression) of the class labels estimated by the model applied on different data feature subsets. Table 10 shows the classification hyperparameters for the various ensemble models explored in this work. The type of learning methodology of each model varied as per its own unique ensemble method and preset. NA indicates the hyperparameters that were not applicable to the specific models. For instance, subspace dimensions are only applicable to subspace ensembles, whereas the number of predictors, maximum number of splits, and learning rates are only applicable to ensemble trees. The next section presents and discusses the LM soft sensing results obtained from all of the above-mentioned models.