## 3. Tests and Results
### Tests  
With the above sample images from simulations and experiments, a series of hybrid training for deep learning is performed. Table 1 shows five training results of YOLOv4 only with the experimental dataset. From Test 1 to Test 5, the experimental samples are successively cut down to half, but randomly separated in every test as a training set, validation set and test set with proportions of 20%, 40% and 40%, respectively. Accuracies on the validation sets for all the five tests are satisfied, being always larger than 92%, but the accuracies on some test sets sharply decreased when the number of sample images was reduced. There is a wide gap in Test 4 between the accuracy on the validation set, which is 99.02%, and that of the test set, which is only 62.38%. This indicates that the YOLOv4 network trained with 238 experimental images is over-fitted and has poor generalization. Similarly, Test 5, with only 95 sample images, obtains 92.31% on the validation set and 48.65% on the test set, which shows worse generalization of the deep learning network. This is consistent with what know: the size of the dataset in deep learning should not be too small for high detection accuracy and good generalization. When the training dataset in digital shearography is large enough, defect detection using deep learning is robust and applicable. However, when the number of training images is smaller than 250 in digital shearography, which is the common situation in real engineering applications, defect detection using deep learning is unstable and inapplicable. The simulation dataset prepared in Section 3.1 is then joined to improve the generalization of deep learning in digital shearography. Table 2 shows the results of YOLOv4 with hybrid simulation and experimental dataset. The numbers of experimental samples in Test 6 to Test 10 are the same as those of Test 1 to Test 5, respectively, and the experimental images in every test are also randomly separated as training set, validation set and test set with proportions of 20%, 40% and 40%, respectively. The extra 5117 simulated images are added to the training set. Compared with Test 1, 2 and 3, there are small drops in the accuracies on the validation set or the test set in Test 6, 7 and 8, respectively. This is caused by the appearance difference of the simulated images in Figure 7 and the experimental images in Figure 10. The joining of simulated images damages the dataset's consistency and makes no improvement. Similar results can also be found in Test 11, where the 5117 simulated images are taken only as a training set, and the 1900 experimental images are separated only as validation and test sets with proportions of 50% and 50%. Accuracies on the validation and test sets are only 43.00% and 43.60%. The simulated shearography images are not always helpful. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Test**} & **Simulation** & **Experimental** & **Accuracy on** & **Accuracy on Test** \\  & **Images** & **Images** & **Validation Set** & **Set** \\ \hline
1 & 0 & 1900 (0.2:0.4:0.4) & 98.84\% & 99.37\% \\
2 & 0 & 950 (0.2:0.4:0.4) & 98.84\% & 98.09\% \\
3 & 0 & 475 (0.2:0.4:0.4) & 98.74\% & 98.28\% \\
4 & 0 & 238 (0.2:0.4:0.4) & 99.02\% & 62.38\% \\
5 & 0 & 95 (0.2:0.4:0.4) & 92.31\% & 48.65\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Training results with only experimental dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Test**} & **Simulation** & **Experimental** & **Accuracy on** & **Accuracy on Test** \\  & **Images** & **Images** & **Validation Set** & **Set** \\ \hline
6 & 5117 (1:0:0) & 1900 (0.2:0.4:0.4) & 96.44\% & 96.00\% \\
7 & 5117 (1:0:0) & 950 (0.2:0.4:0.4) & 87.71\% & 88.88\% \\
8 & 5117 (1:0:0) & 475 (0.2:0.4:0.4) & 85.34\% & 90.31\% \\
9 & 5117 (1:0:0) & 238 (0.2:0.4:0.4) & 76.25\% & 85.00\% \\
10 & 5117 (1:0:0) & 95 (0.2:0.4:0.4) & 81.34\% & 75.46\% \\
11 & 5117 (1:0:0) & 1900 (0.5:0.5) & 43.00\% & 43.60\% \\ \hline \hline \end{tabular}
\end{table}