# Occlusion-Aware Worker Detection in Masonry Work: Performance Evaluation of YOLOv8 and SAMURAI
## 3 Experimental Evaluation
### Detection Performance in Brick Transportation  
The detection performance of YOLOv8 and SAMURAI was evaluated for the brick transportation task by analyzing detection results under three occlusion conditions: non-occlusion, partial occlusion, and severe occlusion. The evaluation process involved segmenting video sequences into specific time intervals and measuring detection accuracy at each time-step. Performance assessment was conducted using multiple metrics, including the F1-confidence curve, recall-confidence curve, precision-recall curve, precision-confidence curve, and confusion matrix, as shown in Figure 6.  
Detection results were analyzed over 60 evaluation time-steps, recorded at 0.25 s intervals. Detection accuracy was determined based on the classification of true positives (TPs), true negatives (TNs), false positives (FPs), and false negatives (FNs). The accuracy for each occlusion case was computed using the following equation:  
\[\frac{TP+TN}{TP+TN+FP+FN} \tag{1}\]  
Based on the values in the confusion matrix, the accuracy was calculated to be 0.975.  
Each detection event was recorded at 0.25 s intervals, and detection success was systematically analyzed at each time-step. The evaluation period for each case was set to 15 s, resulting in 60 detection time-steps from T\({}_{1}\) to T\({}_{60}\). The dataset included sample detection images for visualization, with four representative detections per algorithm for each occlusion level, totaling twenty-four images, as shown in Figure 7.  
Detection accuracy varied across occlusion levels and algorithms. The time-series detection status for YOLOv8 and SAMURAI in brick transportation is summarized in Table 2, which provides a structured comparison of detection performance across different  
Figure 6: Performance evaluation of worker detection YOLOv8 model: (**a**) F1–confidence curve; (**b**) recall–confidence curve; (**c**) precision–recall curve; (**d**) precision–confidence curve; and (**e**) confusion matrix.  
Figure 7: Sample YOLOv8 and SAMURAI detection results by different cases involving carrying bricks.  
occlusion conditions, now incorporating accuracy values. The non-occlusion case was evaluated separately for YOLOv8 and SAMURAI. For the partial occlusion case, the analysis focused on frames where the worker was partially obstructed by an obstacle, continuing until the obstruction was fully removed. The severe occlusion case was evaluated for the period when the worker's upper body was significantly obstructed, affecting visibility.  
Table 2 provides a detailed time-step-based evaluation, outlining detection success at each recorded time-step, now including accuracy values. The overall performance was computed using the following equation:  
\[\frac{\text{Correct\ Detections}}{\text{Total\ Time\ Steps}} \tag{2}\]  
YOLOv8 detected 56 out of 60 time-steps in the non-occlusion case, achieving an accuracy of 93.33%. In the partial occlusion case, it detected 49 out of 60 time-steps, achieving an accuracy of 81.67%. In the severe occlusion case, it detected 41 out of 60 time-steps, resulting in an accuracy of 68.33%. Similarly, SAMURAI detected 59 out of 60 time-steps in the non-occlusion case, achieving an accuracy of 98.33%. In the partial occlusion case, it detected 57 out of 60 time-steps, achieving an accuracy of 95.00%. In the severe occlusion case, it detected 56 out of 60 time-steps, resulting in an accuracy of 93.33%.