# Metal Additive Manufacturing Parts Inspection Using Convolutional Neural Network
## 3 Results and Discussion
### Regularization  
Regularization could be used to mitigate the problem of overfitting during the training process. The implementation of L2 regularization and dropout were explored in our study. L2 regularization applies a penalty on large framework parameters and forces them to be relatively small. Dropout is a method that randomly drops units in the network. Several combinations of L2 regularization and dropout in convolutional and fully connected layers were tested. The training time and validation accuracy of eight models were presented in Table 5. The use of L2 in the convolutional layer with a ratio of \(1\times 10^{-5}\) and a dropout rate of 0.25 on all layers was demonstrated to be the most conducive for both architectures. Regarding the convolutional kernel size, the size of 5 has a 4.3% higher validation accuracy than the size of 3. Therefore, Model 11 was chosen, and the corresponding accuracy and loss plots are shown in Figure 5. It was observed that the overfitting issue was alleviated compared to Model 6 in Figure 4, but more fine tuning of regularization parameters and longer training time were needed to improve the network performance.  
Model 11 was trained for the fine tuning dropout parameters with epochs of 300, and the results are listed in Table 6. Model 16 achieved a validation accuracy of 94.3% and outperformed other models. The accuracy and loss plots of the four tested models are shown in Figure 6. The learning rate was increasing until around 100-150 epochs in Figure 6a. The training and validation accuracy climbed until about 50-100 epochs and then plateaued as seen in Figure 6c,e,g, but overfitting was displayed as worse in Models 17 and 18. Thus, Model 16 was chosen as our final model that converged fast and had high validation accuracy of 94.3% with a training time of about 1 h and 46 min. The schematic architecture of Model 16 is shown in Figure 3.  
Figure 5: (**a**) The accuracy and (**b**) the loss values of the training and validation dataset for Model 11 using data augmentation, L2 regularization, and dropout.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Model \#** & **Architecture** & **L2** & **Dropout** & **Time (b:ms)** & **Val. Acc. (\%)** \\ \hline
7 & & Y(\(1\times 10^{-5}\))N 1 & Y(0.25)Y(0.25) & 0:10:43 & 84.4 \\
8 & C 3 x 3/32, C 3 x 3/64, C 3 x 3/128, & Y(\(1\times 10^{-5}\))Y(\(1\times 10^{-5}\)) & Y(0.25)Y(0.25) & 0:11:05 & 82.5 \\
9 & FC 512, FC 64 & Y(\(1\times 10^{-5}\))Y(\(1\times 10^{-5}\)) & NIN & 0:10:47 & 77.6 \\
10 & & NIN & Y(0.25)Y(0.25) & 0:10:56 & 81.2 \\ \hline
11 & & Y(\(1\times 10^{-5}\))N & Y(0.25)Y(0.25) & 0:10:24 & 88.7 \\
12 & C 5 x 5/32, C 5 x 5/64, C 5 x 5/128, & Y(\(1\times 10^{-5}\))Y(\(1\times 10^{-5}\)) & Y(0.25)Y(0.25) & 0:10:29 & 87.5 \\
13 & FC 512, FC 64 & Y(\(1\times 10^{-5}\))Y(\(1\times 10^{-5}\)) & NIN & 0:10:17 & 73.2 \\
14 & & NIN & Y(0.25)Y(0.25) & 0:10:10 & 87.8 \\ \hline \hline \end{tabular}  
* The left side of “\(|\)” indicates the use of regularization in the convolutional layer, and the right side of “\(|\)” means the use of regularization in the fully connected layer. “N” means no regularization. “Y” means the use of regularization, the number in “\(0\)” means the regularization parameter.  
\end{table}
Table 5: Experimental results of different L2 regularization and dropout parameters. The convolutional layers’ parameters are denoted as “C kernel size/number of kernels”. The fully connected layers are denoted as “FC number of hidden units”. Epoch = 30 and learning rate = \(1\times 10^{-4}\).