# Generative Adversarial Networks for Data Augmentation in Structural Adhesive Inspection
## 4 Discussion  
Recent advances in the field of GANs have made it possible to train these models with limited data while often achieving results close to previous state-of-the-art approaches. While generally a few thousand training images are still required for this purpose, we show that the usage of a much smaller dataset (under 200 images) still yielded results capable of greatly improving the performance of object detection models in the specific task of structural adhesive inspection, particularly in the case of imbalanced training data. Based on the original recommendation it can be hypothesized that results could be further improved by either increasing the size of the original dataset, or by employing for instance smaller partially overlapping crops. These results suggest that the proposed approach has the potential to significantly mitigate the issues of data availability in such tasks, as it not only reduces the costs associated with the generation of images (energy, personnel and resources), but also the time required for this process. Naturally, it is generally unfeasible to dedicate a production line solely for the purpose of generating specific defects to create sufficiently large datasets. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Dataset** & **AP Discontinuity** & **AP Excess** \\ \hline \multicolumn{3}{c}{**Validation Set (60 discontinuity, 60 excess)**} \\ \hline Synthetic & 0.7577 & 0.5934 \\ Real & 0.7735 & 0.6707 \\ Augmented (Simulation) & 0.7720 & 0.6958 \\ Augmented (GAN) & **0.7933** & **0.7483** \\ \hline \multicolumn{3}{c}{**Holdout Set (40 discontinuity, 17 excess)**} \\ \hline Synthetic & 0.4404 & 0.2284 \\ Real & 0.5421 & 0.2829 \\ Augmented (Simulation) & **0.5812** & 0.3001 \\ Augmented (GAN) & 0.5711 & **0.5864** \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Average Precision (AP) per defect class with an IoU threshold of 50%, validated first on a set of 58 real images, then on a holdout set of 19 real images. The number of instances of each defect is listed in the table. Even in cases where a pilot line can be used for this purpose, such as the case presented herein, considerable effort is still required to manually trigger and supervise the process, whilst adjusting the parameters for defects with sufficient variety to be generated. Hence, these results reveal the potential of the proposed approach to improve the viability and likelihood of adoption of such a solution at an industrial level. Particularly regarding Table 1, the results from the holdout set suggest that the key contribution of the augmentation is not only the increased volume of data, but also the variety of the defects generated by the GAN. This increased variety enables the object detector to perform better in previously unseen settings. This conclusion is further corroborated by Table 2, which highlights the performance differences between training with the small imbalanced set of real images, and the the larger balanced set of both real and synthetic images. Nevertheless, some limitations remain to be tackled. On the one hand, the possibility to further increase the variety of the generated images with a finer degree of control on the type of defects generated would be tremendously useful. Besides increasing the size of the dataset, recent methods to explore the model's latent space such as GANSpace [27] could be experimented with to achieve this, on top of fine-tuning or extending the base architecture. On the other hand, the effort required to annotate the synthetic dataset is still considerable, as in this case this process was performed manually. A possible direction to explore here could be the usage of semi-supervised approaches to automatically label newly generated images and include them in the training loop, hence reducing the annotation effort. In addition to this, future work will further explore the comparison of this approach with those based on simulated data (e.g., images generated in simulations such as those created in CoppeliaSim [28] or the Unity engine).