# Detection of Bad Stapled Nails in Wooden Packages
## 3 Results
#### Changes in Image Capturing Conditions  
Two experiments are conducted to test the classification process robustness based on CNN. The first experiment measures the effects of changing the angle of view of the crates. Captured images are shown in Figure 8a. Since the image's angle of view changes, the appearance of the crate changes. In this case, 100 images are captured for each angle of view shown in Figure 8a.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **Confidence** & \multicolumn{4}{c}{**SSD**} & \multicolumn{2}{c}{**FASTER-RCNN**} \\ \cline{3-8}  & **Matrix** & **AlexNET** & **MOBILENET** & **RESNET** & **RESNET** & **INCEPTION** \\ \hline \multirow{4}{*}{Changing View angle} & C\({}_{11}\) & 0.7250 & 0.7050 & 0.6050 & 0.6550 & 0.6800 \\  & C\({}_{22}\) & 0.7500 & 0.6800 & 0.5800 & 0.5550 & 0.5250 \\  & C\({}_{12}\) & 0.2750 & 0.2950 & 0.3950 & 0.3450 & 0.3200 \\  & C\({}_{21}\) & 0.2500 & 0.3200 & 0.4200 & 0.4450 & 0.4750 \\  & C\({}_{11}\) & 0.6800 & 0.5850 & 0.6550 & 0.6750 & 0.6800 \\  & C\({}_{22}\) & 0.7050 & 0.6100 & 0.5700 & 0.6050 & 0.6100 \\  & C\({}_{12}\) & 0.3200 & 0.4150 & 0.3450 & 0.3250 & 0.3900 \\  & C\({}_{21}\) & 0.2950 & 0.3900 & 0.4350 & 0.3950 & 0.3200 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results of the confidence matrix when image acquisition conditions change. The first four rows are the results when the angle of view changes. The second four rows show the results of changing the illumination conditions. Four hundred images are used to test the angle of view, and 200 with illumination conditions changed. Compared with Table 1, the results are worse.  
Figure 7: A graphical representation of the results is shown in Table 1. AlexNet can classify all nails correctly and faster, but the user needs to define areas in the image where nails are. From the point of view of the methods that do not need predefined ROIs to classify nails, the SSD MobileNet model obtains better performance with both well and badly stapled nails. Coefficients C\({}_{11}\) and C\({}_{22}\) should be closer to 1 and C\({}_{12}\) and C\({}_{21}\) closer to 0 to obtain a good classification performance.  
For using the Alexnet model, ROIs, where nails are in the crate, are defined previously. Classification results are shown in Table 2. As before, better performance is computed with the Alexnet model with predefined ROIs in the image. With no predefined ROIs, the SSD MobileNet model obtains better performance with both well and poorly stapled nails.  
The second experiment, conducted to test the robustness of the classifier, changes the illumination conditions. The scene's lighting changes generate shadows and changes in the crate illumination. Figure 8b shows two samples, one with changes in the illumination crate using direct lighting and the other with shadows in the crate using lateral lighting. One hundred images with each illumination are used to compare the results. Results are shown in Table 2. As before, the Alexnet and MobileNet models obtain better performance.  
Figure 8: Images were captured under different conditions to test the robustness of the badly stapled nails classification method using CNN. (**a**) Crate images are captured under a different angle of view than the one used to train the CNN. (**b**) Lighting conditions change, and shadows appear. Also, crate illumination is not uniform.  
To compare the results of Alexnet and MobileNet models in a graph when image acquisition conditions change, Figure 9 shows a bar graph of the confidence matrix elements. Best performance is computed when the angle of view and lighting conditions are similar to the training step.