## 3 Methodology
### Individual Detection  
The patterns to be estimated in conventional object detection-based approaches for the identification of proper PPE use [10; 11] are the \"Individual with PPE\", thus the image samples like \"worker wearing hard hat\" are required to train the object detection model, which are not as easily obtained as the information security of the construction sites. In contrast to these approaches, we employ an individual detection model in this work to specify individual features. This strategy makes the model of this work more robust with viewpoint changes and different individual postures based on the specified human pose by the individual detection model. We characterize the decommissioning workers' postures by extracting the body parts' keypoints of the person detected in on-site images using OpenPose [4]. Instead of the conventional top-down approaches which first detect persons in the image and perform single-person pose estimation for each detected person, OpenPose provides a bottom-up approach by detecting all body parts in the image and associating them with a different person. As illustrated in Figure 2, OpenPose takes the image of size \(w\times h\) as input and processes images through a two-branch multi-stage convolutional neural network (CNN) to predict confidence maps for body part detection and part affinity fields (PAF) for body parts association. First, the backbone (VGG-19 [13] in the original paper) generates a set of feature maps \(F\) from the raw image. Subsequently, the pipeline is divided into multiple similar stages. There are two branches for each stage: (1) the branch to predict a set of confidence maps \(S^{t}\) for the candidates of joint position, and (2) the branch to produce a set of PAFs \(L^{t}\) for joint points relationships correlation. At the first stage, the feature map \(F\) is input to the CNNs to generate a set of confidence maps \(S^{1}\) and a set of PAFs \(L^{1}\). In each subsequent stage \(t(t>1)\), the predictions of the previous stage, \(S^{t-1}\) and \(L^{t-1}\), are concatenated with the feature map \(F\) and fed into the current stage. The loss functions of stage \(t\) are  
\[\left\{ \begin{array}{l}f_{S}^{t}=\sum_{j=1}^{J}\sum_{p}W(p)\cdot\left\|S_{ j}^{t}(p)-S_{j}^{*}(p)\right\|_{2}^{2},\\ f_{L}^{t}=\sum_{c=1}^{C}\sum_{p}W(p)\cdot\left\|L_{c}^{t}(p)-L_{c}^{*}(p)\right\| _{2}^{2}.\end{array} \right. \tag{1}\]where \(S_{j}^{*}\) and \(L_{j}^{*}\) are the ground truth part confidence map and ground truth PAF of the \(j_{th}\) joint point, respectively. \(W\) is a binary mask with \(W(p)=0\) when the annotation is missing at an image location \(p\). The overall loss function is  
\[f=\sum_{t=1}^{T}\left(f_{\text{S}}^{t}+f_{L}^{t}\right) \tag{2}\]  
Finally, the confidence maps and the PAFs are parsed to output the 2D keypoints for all persons in the image by greedy inference (18 keypoints, pretrained using COCO 2016 keypoints challenge dataset [14], see Figure 3). The choice of OpenPose is motivated by its functionality on an RGB image or video taken by on-site surveillance cameras in real-time. This provides a huge benefit in comparison with the skeletal tracking capability of RGB-D devices (e.g., Microsoft Kinect [15]) which depend on depth information. To further improve the speed of estimation, we deploy a lightweight architecture, Mobilenetv2 [16], as the feature extractor instead of VGG-19. To achieve model acceleration, Mobilenetv2 employs a special convolutional filter called depthwise separable convolution as the replacement of the standard convolutional filter together with linear bottleneck (\(1\times 1\) convolutional layer without ReLU) to solve the problem of information loss due to nonlinear activation functions. First, a pointwise (\(1\times 1\)) convolution is deployed to expand the low-dimensional input feature map to a higher-dimensional space suited to nonlinear activations. Next, a depthwise convolution is performed using \(3\times 3\) kernels to achieve spatial filtering of the higher-dimensional tensor.