# Methods for Corrosion Detection in Pipes Using Thermography: A Case Study on Synthetic Datasets
## 4 Case Study
### Results and Discussion
#### 4.3.2 Results  
The UNet+ model was trained under two scenarios to evaluate the impact of dataset composition on defect detection performance. Initially, the model was trained solely with 20 experimental images. The second scenario included 2000 synthetic images generated via FEM simulations, along with the 20 experimental images. Validation against unused experimental images provided a clear performance comparison. The performance of the UNET+ model was evaluated using metrics such as the F1 score, loss, and Mean Intersection over Union (Mean IoU). The F1 score is the harmonic mean of precision and recall, providing a balance between the two. Loss measures the difference between the predicted values and the actual values, indicating how well the model is performing. Mean IoU is a metric used for evaluating the accuracy of object detection models, calculated by dividing the intersection of the predicted and ground truth regions by their union [53; 54]. Table 5 presents a comparison of these metrics for the model trained on the experimental dataset alone versus the combined dataset. The results indicate that incorporating synthetic data significantly improved the model's performance, enhancing its accuracy and ability to generalize across unseen defect types and conditions.  
The inclusion of synthetic data in the training process addressed the common challenge of limited experimental data, resulting in a more robust and accurate model. The improved performance metrics demonstrate the value of data augmentation through FEM simulations. As shown in the learning curves (Figure 7), the UNET+ model trained on the FEM + experimental dataset exhibits steady improvement in accuracy throughout the training steps, reaching a final accuracy of approximately 0.88. In contrast, the model trained solely on the experimental dataset shows signs of overfitting, with accuracy plateauing and fluctuating around 0.85 after 10-14 steps. This indicates that while the model initially learns from the limited data, it eventually starts to memorize the training set rather than generalize to new data.  
Figure 6: UNET+ model architecture [52].  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **F1** & **Loss** & **Mean IoU** \\ \hline Experiment & 0.89 & 0.57 & 0.66 \\ \hline FEM + Experiment & 0.90 & 0.26 & 0.87 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Evaluation metrics of UNET+ for each dataset.  
The performance metrics in Table 4 further corroborate these findings. The F1 score for the FEM + experimental dataset is 0.90, compared to 0.89 for the experimental dataset alone. The Mean IoU also saw a substantial increase from 0.66 to 0.87, and the loss for the FEM + experimental dataset was significantly lower at 0.26, compared to 0.57 for the experimental dataset. These results highlight that augmenting the training data with synthetic data from FEM simulations not only improves the overall accuracy and robustness of the model but also mitigates the risk of overfitting, ensuring better applicability to real-world scenarios.