## 4 Results
### Experimental Results According to Training Methods and Models  
In this study, the pre-trained backbone model was selected as ResNet50, MobileNetV2, and ShuffleNetV1 in consideration of the number of parameters and model size. The models modified to suit our study were named Safety-ResNet, Safety-MobileNet, and Safety-ShuffleNet, respectively. Later, we tried fine-tuning in two different ways to find the most suitable training method for the task of this study. Table 2 presents the experimental results. When only a part of the upper layer was retrained on new data (hereinafter referred to as method (1)), the F1 scores of Safety-ResNet, Safety-MobileNet, and Safety-ShuffleNet were 49.15%, 80.92%, and 96.59%, respectively. By contrast, when all layers were newly re-trained, (hereinafter referred to as method (2)), the F1 scores were 99.75, 99.79%, and 98.04%, respectively, indicating higher performance compared to method 1. Given that the size of the dataset used in this study is large enough to solve the task, it is judged that re-training the entire layer would have been more efficient. In particular, it seems that Safety-ResNet did not sufficiently learn the features of a new dataset when using method 1. Furthermore, when using method 1, the lighter the model, the better the performance.  
Based on the experimental results by model, Safety-MobileNet, Safety-ResNet, and Safety-ShuffleNet showed high performance, in that order. The performance of Safety-MobileNet and Safety-ResNet is similar, but Safety-MobileNet has 0.1 times fewer parameters and 2.23 times lower CPU inference time than Safety-ResNet (based on method 2). This is because Safety-MobileNet is a lightweight model designed for a mobile environment. Although Safety-ShuffleNet is a lightweight model like Safety-MobileNet, there is a difference in performance in terms of accuracy. Therefore, in this study, Safety-MobileNet was selected as the final backbone model. As mentioned above, Safety-MobileNet is designed to be able to use deep learning models even where computing performance is limited or battery performance is important. Considering actual construction sites where the risk recognition algorithm will be used, it is likely that mobile devices such as cell phones and tablet PCs  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Safety-ResNet** & **Safety-MobileNet** & **Safety-ShuffleNet** \\ \hline Parameters & 23,606,153 & 2,269,513 & 911,601 \\ Inference Time (ms) & 67 & 30 & 0.93 \\ Method 1 F1 score (\%) & 49.15 & 80.92 & 96.59 \\ Method 2 F1 score (\%) & 99.75 & 99.79 & 98.04 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparison of performance according to training methods by model.  
Figure 3: Confusion matrix.  
will be used to find risk factors. Therefore, it is very important to use a lightweight model as a backbone. Through experiments, it was confirmed that Safety-MobileNet was the most suitable for the purpose of this study.  
Table 3 summarizes the experimental results of Safety-MobileNet when method 2 was used. F1 scores for each class are all between 99% and 100%, and high accuracy is obtained even though the dataset is imbalanced between classes. It was judged that this is because the characteristics between the classes are distinct. In particular, the material factor has a large difference in the equipment that appears in the image, so even people who do not have prior knowledge in the domain can distinguish each class to some extent. However, human factors do not cause a large difference in features between images, but it seems that model training was well performed and high performance was achieved.