# Smart Detection System of Safety Hazards in Industry 5.0
## 5 Experiments and Results
### Real-World and Synthetic Datasets Description  
To evaluate the efficacy of the synthetic dataset generated following our proposed methodology, it is crucial to contrast it with an appropriate real-image dataset, which can be considered a benchmark. This can be achieved by comparing the performance of trained AI models. The models are trained on both real and synthetic datasets, and they are evaluated on the same test set to achieve comparable results. Therefore, a properly selected open dataset is crucial to ensuring a fair and insightful comparison. The samples in the synthetic dataset represent various scenarios and conditions of industrial setup. On the contrary, we selected a publicly available real dataset consisting of images acquired from real-world site environments. Both of those datasets represent the challenges encountered in PPE detection, ensuring equitable comparison by evaluating the trained AI models' effectiveness. The detailed description of those datasets is vital for providing a basis for the subsequent analysis and evaluation of AI models trained on them.  
Utilizing the methodology for creating a synthetic dataset, as presented in Section 3.1, we generate a dataset of two classes, namely helmets and vests, aimed at PPE detection in industrial environments. For ease of reference throughout this paper, we named our synthetic dataset SYN_HV. SYN_HV comprises 600 images with a total of 7071 instances; among these, 3510 belong to the vest class, while 3561 belong to the helmet class. We choose to have an approximately equal number of instances in each class to prevent class imbalance, ensuring that the trained AI model has an equal opportunity to learn each class, therefore achieving more reliable and generalized performance across all classes.  
Color Helmet and Vest (CHV) [52] is a novel, open-source dataset for PPE detection in construction environments. The dataset consists of 6 PPE classes, including helmets of four different colors, vests, and persons. Specifically, the CHV categorizes helmets based on their colors, namely blue, yellow, white, and red, as different colors indicate different roles on construction sites. It contains 1300 images, considering the site background, different gestures, varied angles and distances, and 9209 instances in total. The CHV dataset is split into training, validation, and test sets, comprising 80% (1064 images), 10% (133 images), and 10% (133 images) of the total dataset, respectively.  
To ensure comparability among the AI models trained on both datasets, some modifications should be performed on the CHV dataset. Specifically, the different color categories of the CHV 'helmet' class might be useful in some situations; however, in our study, we are interested in recognizing the presence of a helmet, regardless of its color. For this reason, we combine the color-specific categories of CHV into a single 'helmet' category. Additionally, the 'person' category of the CHV dataset is removed since it does not align with our core focus. Regarding the'vest' class, it is retained as it is without any modification. After the aforementioned modifications, the 'helmet' category of the CHV dataset contains 2762 instances, while the'vest' class consists of 1396 instances. Figure 7 shows the number of instances per class for both the SYN_HV and CHV datasets.  
It is worth noting that the validation and testing of both AI models, trained on real and synthetic datasets, are performed using the same validation and test set as proposed in the CHV dataset. This approach ensures consistency in our assessment and allows a direct comparison between the performance of the AI models, enabling us to derive valid conclusions about the usage of synthetic datasets during the training of AI models.