## 4 Results
### Validation of Classification Models  
After testing the developed methodology with simulated data, a validation phase is continued with data obtained directly from the process. A validation matrix of size [3 \(\times\) 2100] was formed where 100 measurements were included for each of the seven classes with magnitudes of variation classified as 1\(\sigma\), 2\(\sigma\), and 3\(\sigma\). The first phase of validation corresponds to the multivariate control charts to which the task of monitoring and surveillance of quality in the production system corresponds. Table 13 shows the level of efficiency of each of the control charts to detect the variations occurring in the process and concentrated in the validation matrix, where it is shown that the chart with the highest efficiency to detect the variability generated in the solder paste application operation is the MCUSUM chart.  
To validate the use of the artificial neural network focused on the analysis of the source of variation occurred in the process, it is important to evaluate its performance in conjunction with the multivariate control chart. Table 14 shows the probability of detecting and interpreting the source of variation in the solder paste application operation. According to the results, the integration of the MCUSUM control chart and the multilayer perceptron ANN gives the best results.  
It is important to show graphically and analytically the performance of the artificial neural networks developed in synchrony with the control charts proposed to detect and interpret the source of variation. Therefore, ROC curves were developed; see Figure 7. Figures 8 and 9 show the confusion matrices generated after validat  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Control Chart** & **1\(\sigma\)** & **2\(\sigma\)** & **3\(\sigma\)** & **Out-of-Control Signals Detected** & **Efficiency** \\ \hline MCUSUM & 685 & 691 & 698 & 2074 & 98.76\% \\ MEWMA & 667 & 688 & 697 & 2052 & 97.71\% \\ \hline \hline \end{tabular}
\end{table}
Table 13: Comparison of control charts.  
Figure 6: ROC analysis test phase.  
of the multilayer perceptron ANN and SVM integrated with the MCUSUM and MEWMA control charts, respectively.  
According to the results, the multilayer perceptron artificial neural network has a higher probability of correctly discriminating the source of variability occurring in the operation, both with the MCUSUM and MEWMA control chart, achieving a performance of 91.2% and 88.1%, respectively. The best classification result achieved with the SVM is from working together with the MCUSUM chart, reaching a probability of 87.3%. As for the confusion matrices, the success of the classification model is evaluated by comparing the predictions of the trained model against the real values; in this scheme, both multilayer perceptron ANN and SVM present a lower classification performance in the categories  
Figure 8: Confusion matrix of the multilayer perceptron ANN. (a) MCUSUM control chart; (b) MEWMA control chart.  
Figure 7: ROC analysis validation phase. (a) Multilayer perceptron ANN; (b) SVM.  
Figure 9: Confusion matrix of the SVM. (a) MCUSUM control chart; (b) MEWMA control chart.  
where variability associated with a single variable is present, which is predictable since a lower degree of variability in the operation is analyzed.