# Robust Human Pose Estimation Method for Body-to-Body Occlusion Using RGB-D Fusion Neural Network
## 4 Experimental Results
### Performance Metrics in Human Pose Estimation  
Object Keypoint Similarity (OKS) [58] is adopted to evaluate the pose estimations for scenes with multiple people. OKS is a concept similar to IoU in object detection and is used to compute Average Precision (AP) and Average Recall (AR). OKS generalizes body joint scale variations across different human sizes as follows:  
\[OKS=\frac{\sum_{i\in[0,N-1]}exp(\frac{-d_{i}^{2}}{2s^{2}k_{i}^{2}})\delta(v_{i} )}{\sum_{i\in[0,N-1]}\delta(v_{i})}, \tag{14}\]  
where \(i\) represents the index of each joint; \(N\) denotes the total number of joints; \(d_{i}\) is the Euclidean distance between the ground truth and predicted locations for joint \(i\); \(s\) is the square root of the object area; \(k_{i}\) is a constant assigned based on the standard deviation of joint localization for joint \(i\); and \(\delta(v_{i})\) is an indicator function based on a visibility flag \(v_{i}\) in the ground truth annotation. The \(\delta(v_{i})\) is 1 if \(v_{i}>0\) and 0 otherwise. The 17-COCO Pose dataset provides joint-wise standard deviations computed across the entire benchmark dataset, and these values follow the relationship \(k_{i}=2\sigma_{i}\) for joint \(i\). The standard deviations for joint types are presented in Table 4.  
The standard deviation \(\sigma_{i}\) tends to be larger for the shoulders, knees, and hips compared to facial joints such as the nose, eyes, and ears. Since \(s\) in OKS is defined as the object area, segmentation data are required. However, the NTU RGB+D dataset does not provide segmentation information for objects. In the COCO dataset, an empirical approximation method is used when segmentation data are unavailable, where \(s\) is defined by multiplying the bounding box area by 0.53. Therefore, \(s\) is determined using this empirical approximation method.  
To evaluate joint detection performance, a prediction is considered a true positive (TP) for a given threshold (set within the range of 0 to 1) if its OKS score exceeds the threshold. In cases where multiple predictions are associated with a single GT, only the prediction with the highest OKS score is regarded as a TP, while the others are counted as false positives (FPs). Furthermore, if there is no prediction corresponding to a particular GT, it is counted as a false negative (FN). The precision (P) and recall (R) are calculated as follows:  
\[\begin{split}\text{P}&=\text{TP}/\left(\text{TP}+ \text{FP}\right)\\ \text{R}&=\text{TP}/\left(\text{TP}+\text{FN}\right) \end{split}. \tag{15}\]  
The OKS scores were assessed using thresholds of 0.5 (P\({}^{50}\)) and 0.75 (P\({}^{75}\)), respectively. The overall AP is computed by averaging multiple P values over a range of OKS thresholds (0.50:0.05:0.95). To evaluate the impact of object scale variations, performance is also  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Joints Type** & \(\sigma_{i}\) \\ \hline Nose & 0.026 \\ Neck & 0.026 \\ Eyes & 0.025 \\ Ears & 0.035 \\ Shoulders & 0.079 \\ Elbows & 0.072 \\ Wrists & 0.062 \\ Hips & 0.107 \\ Knees & 0.087 \\ Ankles & 0.089 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Standard deviations for joint types.  
measured for medium-scale objects (AP\({}^{\text{M}}\)) with \(32^{2}<s^{2}<96^{2}\) and large-scale objects (AP\({}^{\text{L}}\)) with \(s^{2}>96^{2}\).