## 3 Methodology  
### Dataset  
Publicly released benchmark datasets related to existing construction sites are labeled only for equipment or are classified into large categories such as buildings and workers [22; 32]. However, the dataset used in this study is labeled for material and human risk factors that can cause disasters directly. At this time, the human factors discussed in this paper refer to unsafe human behavior and the presence or absence of a device that is attributed to humans and guarantees safety. For example, in the former case, working near an edge with the risk of a fall is representative, and in the latter case, there is the wearing or not wearing of a safety belt or a helmet. The material factors refer to improper installation of a device that guarantees safety for work convenience. A typical example is not installing an outrigger or safety handrail, which increases the likelihood of a fall. Both factors are caused by human carelessness, but they are divided into human factors or material factors depending on whether the responsibility is attributable to a person or an object. Table 1 summarizes the data for risk factor classes at construction sites. The total number of data is 25,859, which consists of 4149 cases of material factors, 15,412 cases of human factors, and 6298 cases of factors defined as normal. As summarized in the table, there is an imbalance between classes in the collected dataset, and insufficient data per label to use deep learning models, which increase representation power by stacking many layers. Therefore, in this study, data augmentation techniques were used to increase the amount of data. Data augmentation is a technique that improves the performance of computer vision. It increases the size of the actual training dataset by applying perturbation (crop, jitter, shear, zoom, etc.) to the image. Data augmentation is conducted only on the training dataset and not on the validation and test sets.  
The image data were collected from devices such as cameras and mobile phones, and are of various sizes, such as 1920 \(\times\) 1080, 789 \(\times\) 927, 992 \(\times\) 925, 854 \(\times\) 929 or 597 \(\times\) 923. When high-resolution data is used as input to the model, the number of parameters in the model increases notably, which increases computation costs and the possibility of overfitting to the training data, so the image size was adjusted. In addition, given that the sizes of images are very different, the work of unifying them into a single size was carried out. Although information loss in the image may occur during the resizing process, it was determined that sufficient information would be transmitted if the data was enough to identify risk factors by the naked eye, so the size of image was modified to 224 \(\times\) 224. This size 224 \(\times\) 224 is also the default input size of the ImageNet pre-trained model. Thereafter, the prepared dataset was divided into training, validation, and test  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Factor** & **Class** & **Training** & **Validation** & **Test** \\ \hline \multirow{4}{*}{Material} & A-type ladder outrigger & 390 & 56 & 112 \\  & Movable scaffolding frame outrigger & 381 & 109 & 55 \\  & Safety handrail & 106 & 30 & 16 \\  & Scissor-type aerial work platform & 2025 & 327 & 542 \\  & Safety vest & 4173 & 838 & 951 \\  & Safety helmet & 4734 & 880 & 1150 \\  & Working in pairs & 622 & 178 & 89 \\  & Working near edges & 1257 & 301 & 239 \\  & Normal & 4408 & 710 & 1180 \\  & Total & 18,096 & 3429 & 4334 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Data distribution by class.  
datasets. During the division of the dataset, it was adjusted so that a specific class could be evenly distributed among all sets without bias toward one set. The data of each class were randomly distributed among the training, validation, and test datasets at a ratio of approximately 7:1:2.