# Automatic Scaffolding Workface Assessment for Activity Analysis through Machine Learning
## 4 Case Study and Results Analysis
The classification results of every video frame were evaluated and validated via 10-fold cross-validation and the performance metrics including accuracy, precision, recall and F1 score indicated that all the classifiers achieved good performances. The confusion matrix of the classifiers are presented in Figure 11.  
To test the capability of the formation of workface assessment, three video clips containing all activity categories were used for validation. As shown in Table 3, Video Clip 1 lasts for 206 s and Clips 2 and 3 last 116 s and 79 s, respectively. In addition, Clip 1 was converted into a sequence of image frames by extracting frames at 2 s intervals while Clips \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Classifier**} & \multirow{2}{*}{**Accuracy**} & **Macroaverage** & **Macroaverage** & **Average\_F1** \\  & & **Recall** & **Precision** & **Score** \\ \hline RF & 96.58\% & 0.9445 & 0.9405 & 0.9422 \\ \hline SVM & 94.24\% & 0.9209 & 0.9349 & 0.9273 \\ \hline DT & 92.08\% & 0.9040 & 0.8749 & 0.8880 \\ \hline KNN & 96.13\% & 0.9492 & 0.9578 & 0.9537 \\ \hline NN & 93.12\% & 0.9339 & 0.9337 & 0.9337 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The performance of the classifiers. Figure 11: Confusion matrix of the classifiers. The shaded boxes indicate the majority of detection results. 2 and 3 were converted with 1 s intervals. 1731 frames were used for training and testing and these frames went through 10-fold cross-validation. Some 314 frames were used for the validation of the case study. For the shot angle, Clip 1 was collected at kneel level, while Clips 2 and 3 were collected at chest level and overhead, respectively, to test the ability of generalization of our model. Since an RF model has the highest mean accuracy and relatively high score in Recall, Precision and F1 score through our training procedure, we chose an RF model for the validation of the workforce assessment. The sequences of image frames from three clips were fed into the OpenPose and 3D extraction model and then 3D pose features were generated and transmitted through our RF classifier for activity classification. Figure 12 shows an example of 3D key  
Based on the comparison between the model outputs and the manual observation in Figure 13, this case study displayed several phenomena. Firstly, our model presented the feasibility and capability of discrimination between the scaffolding activities of erecting and transporting, and generated relatively effective intervals between these two activities. Although there exists a little divergence between the boundary of distinct activities, which may be caused by the transitional procedure during the conduct of scaffolding activities or the occlusion of body parts, the model classification is capable of basically reflecting the activity status. Secondly, the discrimination between the activities erecting and waiting/idling is vulnerable when the participant performed either erecting or waiting/idling activities. In three testing clips, the waiting/idling activity was misclassified as scaffolding erecting. This poor discrimination may be a result of the complexity of the activity of scaffolding erecting or the similarity between these two activities. Figure 12: 3D key joints and skeleton extraction: (**a**) key joint distribution in the image frame of scaffolding erecting; (**b**) the corresponding skeleton model in 3D.