## 3 Results and Discussion
### Cnn
#### 3.2.1 Evaluation of Over-Fitting
of unacceptable layers are located early in the build process. Most unacceptable low-density occurrences occur between layers 50 and 125. Figure 15 confirms that the majority of low-density occurrences occur earlier in the build process for sample 1. This is quite different from sample 2, shown in Figure 6, where the low-density occurrences happen more uniformly throughout the build process. Table 7 summarizes an in-depth analysis of the performance of both samples. We show the capabilities of the CNN model and determine that features can be utilized for accurate layer-wise classification with an average accuracy of 75.9% when trained on a low-quality sample and 78.4% when trained on a high-quality sample. Specificity measurements indicate that the proportion of actual negative cases was predicted correctly on average 65.9% of the time when trained on sample 1 and 12% on average when trained on sample 2. However, sensitivity measures, which are the proportion of actual positive cases predicted correctly, was measured on average to be 78.2% and 95.5% when trained on a low and high-quality sample, respectively. We point out that the sensitivity metric is more impactful  
Figure 14: Porosity inter-layer evaluation: (**a**) low-density diameter of Sample 1 (mm) and (**b**) low-density quantity of sample 1. Figure 15: Inter-layer low-density identification of sample 1 (voxel scale 0.03228 mm). here since it is responsible for the majority of cases (shown in detail in Figure 16a,b)). The results of the sensitivity/specificity indicate that the CNN model is very sensitive and effective at determining bad layers as bad which means few cases of bad layers are misidentified. The low specificity suggests that the model does identify some good layers as bad layers. This is important in this investigation since layers with defects can render a sample defective. Table 7 showcases how the accuracy of the CNN drops significantly from 88.8% to 68.1% when trained on a high-quality sample and tested on a low-quality sample, a 23% drop. Sample 1 demonstrates a 25% drop in accuracy when tested on sample 2. The highly informative F1 scores are reported in Table 8. It is demonstrated that when we train this CNN on a high-quality sample, the model performance peaks with F1 scores of 80.9% and 93.98% when tested on a low-quality sample and high-quality sample, respectively. The confusion matrices from Table 2 are shown in Figure 16a,b for training on sample 1 and 2, respectively. The majority of cases are True Positive indicating that the CNN identifies actual positive cases the most. When training and testing on the same sample, TP is the highest which was expected. When trained on the low-quality sample and tested on the high-quality sample, False Positives are the highest of all combinations. This information is important, indicating that when trained on a low-quality sample and testing on a high-quality sample a higher rate of falsely identifying positive cases occurs. This is not the case when trained on a high-quality sample. Figure 16b highlights FP percentages of 2.53 and 2.99 when tested on low-quality and high-quality samples, respectively. \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{3}{c}{**Train**} \\  & & **Sample 1** & & **Sample 2** \\ \hline Test & Sample 1 & 86.8\% \({}^{1}\) & 79.5\% \({}^{2}\) & 68.1\% \({}^{1}\) & 2.36\% \({}^{2}\) \\  & & 89.9\% \({}^{3}\) & 91.1\% \({}^{4}\) & 96.4\% \({}^{3}\) & 69.6\% \({}^{4}\) \\  & & & 52.3\% \({}^{2}\) & 88.8\% \({}^{1}\) & 21.5\% \({}^{2}\) \\  & & 66.5\% \({}^{3}\) & 92.2\% \({}^{4}\) & 96.7\% \({}^{3}\) & 91.3\% \({}^{4}\) \\  & & 75.9\% \({}^{1}\) & 65.9\% \({}^{2}\) & 78.4\% \({}^{1}\) & 12.0\% \({}^{2}\) \\  & & 78.2\% \({}^{3}\) & 91.7\% \({}^{4}\) & 96.5\% \({}^{3}\) & 80.5\% \({}^{4}\) \\ \hline \hline \end{tabular} \({}^{1}\) Accuracy \({}^{2}\) Specificity \({}^{3}\) Sensitivity \({}^{4}\) Precision.  
\end{table} 
Table 7: CNN performance on all samples. Figure 16: Confusion matrix training on (**a**) sample 1 and (**b**) sample 2.