# Lean Manufacturing Soft Sensors for Automotive Industries
## 3 Results and Discussion
### Result Analyses of the Best Performing Model
#### 3.9.2 Confusion Matrices  
Figure 14 shows the confusion matrix for the validation results of the trilayered neural network. In this figure, true classes are represented in rows whereas the predicted classifications are shown in columns. The number of observations indicates the number of companies in each cell. Blue color-filled cells indicate the cases wherein the model predictions match the true classes. For instance, there are 2 companies in the first row and first column, indicating that the true class as well as the predicted class of these companies is high LM. Similarly, there is one company shown in the third row and third column, indicating that the true class as well as the predicted class of this company is medium LM. All other predicted classifications are incorrect, as they do not match with the corresponding true classes. For instance, 5 of the 6 companies that are medium LM in reality have been misclassified by the model--3 as high and 2 as low class. Moreover, two companies having true classes low and high were misclassified by the model as medium. Overall, the validation confusion matrix of the trilayered neural network depicts a poor classification performance, as corroborated by the associated accuracy of only 30% (Table 18). Figure 15 shows the confusion matrix for the testing results. In this figure, four out of five observations (companies) lie in blue colored cells, indicating high testing accuracy of the trilayered network LM soft sensor. Only one observation having high LM true class was misidentified by the model to belong to the medium class. Hence, the testing confusion matrix of the trilayered neural network depicts a superior classification performance, as corroborated by the associated accuracy of 80% (Table 18). Figure 14: Validation Confusion Matrix for Trilayered Neural Network. The testing confusion matrix (Figure 15) was further analysed to yield the precision and recall metrics of all classes. For instance, the true positives (TP) for the high, medium, and low classes are 1, 2, 1 and false positives (FP) are 0, 1, and 0, respectively. The false negatives (FN) for these classes (in the same order) are 1, 0, and 0, respectively. Model precision is defined as a measure of prediction accuracy based on the total predicted samples, determined as follows:  
\[Precision=\frac{TP}{TP+FP} \tag{11}\]  
Whereas, recall indicates the prediction accuracy based on the actual number of samples, given as:  
\[Recall=\frac{TP}{TP+FN} \tag{12}\]  
F1 scores, on the other hand, are harmonic means of recall and precision values. F1 scores give a balanced measure of both metrics by considering both false negatives as well as false positives:  
\[F1\ Score=\frac{2}{\frac{1}{Precision}+\frac{1}{Recall}}=\frac{2\times Precision \times Recall}{(Precision+Recall)} \tag{13}\]  
Table 19 depicts the precision, recall, and F1 scores of each of the three LM classes based on the respective TP, FP, and FN values obtained during testing of the trilayered neural network. Figure 15: Testing Confusion Matrix for Trilayered Neural Network. This result confirms that the trilayered neural network soft sensor is insensitive/robust to the relatively unbalanced dataset used in the present study. As per [PERSON] et al. (2019), imbalanced class distribution deteriorates classification performance. The authors state that even the small class to dominant class sample size ratio of 1:10 may prove difficult to be successfully classified. Secondly, the authors state that class separability is one of the primary contributors to unsuccessful classification of small classes due to the presence of overlapping patterns across classes. Thirdly, the authors discuss that the networks trained on unbalanced datasets are likely to misclassify small classes due to the inadequate representation of the small class dependency patterns in the network weights. In the present work, 5 of the 46 surveyed firms belonged to the low LM class, whereas 27 and 14 companies belonged to the medium and high LM classes, respectively.