# Generative Adversarial Networks for Data Augmentation in Structural Adhesive Inspection
## 3 Preliminary Results
### Automated Defect Detection Using Deep Learning  
Following the method specified in Section 2.3, one of the YOLOv4-Tiny models was trained on the real training set and the other on the augmented one, as described in Section 2.1. To further facilitate the comparison, a third model was trained only on the synthetic dataset, with a fourth model being trained on a dataset augmented with a previous alternative approach based on simulation [19]. A summary of the results from this process is provided in Table 1.  
In both cases, the model trained on the training set augmented using the proposed approach attained superior results. Furthermore, the impact of the augmentation becomes even more evident when testing the model with the holdout set. While the validation set contains images with defects that are similar to those present in the real training set, the ones held out are considerably different, for instance with excess defects occurring in parts of the bead that were previously unseen (in both GAN and object detector training sets). Several examples are showcased in Figure 4, in which the first column (left) depicts results from the model trained only on real data, while the second column (right) shows the detections of the augmented model on the same image. \begin{table}
\begin{tabular}{l c c c} \hline
**Dataset** & **[EMAIL_ADDRESS]** & **[EMAIL_ADDRESS]** & **[EMAIL_ADDRESS]** \\ \hline \multicolumn{4}{c}{**Validation Set (60 discontinuity, 60 excess)**} \\ \hline Synthetic & 0.8141 & 0.7724 & 0.6755 \\ Real & 0.8662 & 0.8239 & 0.7221 \\ Augmented (Simulation) & 0.8870 & 0.8265 & 0.7339 \\ Augmented (GAN) & **0.9131** & **0.8569** & **0.7708** \\ \hline \multicolumn{4}{c}{**Holdout Set (40 discontinuity, 17 excess)**} \\ \hline Synthetic & 0.5969 & 0.5154 & 0.3344 \\ Real & 0.5986 & 0.5249 & 0.4125 \\ Augmented (Simulation) & 0.6631 & 0.6189 & 0.4406 \\ Augmented (GAN) & **0.7675** & **0.7256** & **0.5788** \\ \hline \end{tabular}
\end{table} 
Table 1: Evaluation metrics for the models trained with synthetic set (536 images), the scarce real set (58 images) and the augmented sets (594 images for both). Models were first validated on a set of 58 real images, then on a holdout set of 19 real images. The number of instances of each defect is listed in the table. The mean Average Precision (mAP) is computed for different Intersection over Union (IoU) thresholds, namely 0.15, 0.30 and 0.50. Best results are highlighted in bold. Figure 4: Examples from the output of the YOLOv4-Tiny models running inference on the holdout set. Left column depicts the output from the model trained only on real data, while the right column represents the corresponding result from the model trained on the augmented set. In addition to this, given the imbalanced nature of the original (real) dataset, it is interesting to take a closer look at the Average Precision (AP) for each class (discontinuity and excess) between the different training sets. These results are presented in Table 2.  
From the analysis of Table 2 it becomes evident that there is a clear difference in AP between the classes for the model trained only on the real imbalanced dataset. More specifically, the difference amounts to ~10% in the validation set, and ~26% in the holdout set. The considerably larger gap in the latter can be explained once more due to the more pronounced difference between the defects observed in the holdout set and those of the initial dataset. Clear examples of this are provided in Figure 4, with excess defects occurring at varied parts of the bead. In contrast, the performance of the model trained on the balanced augmented set is not only superior but also much more harmonized across classes, with a difference of only ~4.5% in the validation set and ~1.5% in the holdout set, clearly showcasing the impact of the augmentation in terms of model performance for this task.