## 4 Results  
To test the performance of the multilayer perceptron artificial neural network and SVM for classifying and analyzing the source of variation detected by the control charts, their results were compared based on the performance measures: sensitivity, specificity, and accuracy (Equations (23)-(25), respectively). The probability of not incurring a type II error, (\(1-\beta\)), is known as the power of the test, equivalent to the sensitivity. The probability of not making a type I error, (\(1-\alpha\)), is known as the efficiency of the test, equivalent to the specificity. The degree of accuracy of a test is defined as the proportion of cases where the result is correct [32].  
\[\text{Sensitivity}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{23}\]  
\[\text{Specificity}=\frac{\text{TN}}{\text{TN}+\text{FP}} \tag{24}\]  
\[\text{Accuracy}\;=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{FP}+\text{FN}+ \text{TN}} \tag{25}\]  
where TP is true positive, FP is false positive, TN is true negative, and FN is false negative.  
The results generated by the ANNs used for the analysis of the statistical variation generated in the solder paste application process when introducing the test matrix are shown in Table 11.  
According to the results obtained, the multilayer perceptron ANN presents a higher average efficiency for the various levels of variation compared to the SVM ANN. The multilayer perceptron ANN achieves an average accuracy of 96.45% to detect changes with magnitudes averaging one standard deviation in the quality characteristics of the manufacturing process, which translates into a strict monitoring of the process. Regarding  
Figure 5: Cost value vs. validation error.  
the accuracy of the multilayer perceptron ANN to classify variation increments in two and three standard deviations in the operation of applying solder paste, the accuracy increases to 96.97% and 98.25%, respectively. The increase in accuracy is logical, since, by incurring a greater variation within the manufacturing process, the measurement that would originate such a warning signal in the control chart would be further away from the average, and, consequently, it would be easier to classify through the ANN.  
The ROC (receiver operating characteristic curve) analysis applied to the multilayer perceptron artificial neural network and SVM allows us to determine through the Youden index the highest sensitivity and specificity to classify the source of variation generated in the PCB manufacturing process; see Figure 6. The area under the curve (AUC) estimates the probability that the artificial neural network model can correctly discriminate the source of variability; see Table 12.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & & & & & & \\ \hline \multicolumn{2}{c}{**ANN**} & **AUC** & **Deviation** & **Lower Limit** & **Upper Limit** & **Sensitivity** & **Specificity** \\ \hline Multilayer Perceptron & 0.952 & 0.011 & 0.93044 & 0.97356 & 0.9632 & 0.9561 \\ SVM & 0.926 & 0.021 & 0.88484 & 0.96716 & 0.9175 & 0.9379 \\ \hline \hline \end{tabular}
\end{table}
Table 12: AUC, 95% confidence interval, and Youden index.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & & & & & & & \\ \hline \multicolumn{2}{c}{**Multilayer Perceptron**} & \multicolumn{1}{c}{**SVM**} \\ \hline \multicolumn{2}{c}{**ANN**} & **AUC** & **Deviation** & **Lower Limit** & **Upper Limit** & **Sensitivity** & **Specificity** \\ \hline Multilayer Perceptron & 0.952 & 0.011 & 0.93044 & 0.97356 & 0.9632 & 0.9561 \\ SVM & 0.926 & 0.021 & 0.88484 & 0.96716 & 0.9175 & 0.9379 \\ \hline \hline \end{tabular}
\end{table}
Table 12: AUC, 95% confidence interval, and Youden index.