# Diverse Humanoid Robot Pose Estimation from Images Using Only Sparse Datasets
## 4 Diverse Humanoid Robot Pose Estimation
### The Evaluation Method  
To evaluate pose estimation methods, we adopted a standard evaluation procedure similar to that used in the COCO dataset [63], utilizing Object Keypoint Similarity (OKS). OKS measures the proximity of predicted keypoints to ground truth keypoints for a given object, with scores ranging from 0 to 1, where a higher OKS score indicates more accurate predictions. OKS is defined as:  
\[OKS\ =\ \sum_{i=1}^{|K|}exp(-d_{i}^{2}/(2s^{2}k_{i}^{2}))\cdot\delta(v_{i}^{gt}>0) /\sum_{i=1}^{|K|}\delta(v_{i}^{gt}>0) \tag{3}\]  
where \(|K|\) denotes the number of keypoints for an object, \(d_{i}\) represents the Euclidean distance between the \(i\)-th predicted keypoint and the corresponding ground truth keypoint, \(s\) is the square root of the object's segment area (defined by its bounding box), and \(k\) is the normalization constant for each keypoint. The term \(v_{i}^{gt}\) indicates the binary visibility in the ground truth, and \(\delta(\cdot)\) is a binary indicator function that evaluates to 1 if the condition inside the parentheses is true, and 0 otherwise. While the COCO dataset employs a normalization constant of \(k_{i}=2\sigma_{i}\) for each keypoint, we use a stricter normalization constant of \(k_{i}=\sigma_{i}\), indicating that our dataset demands higher accuracy in keypoint estimation. For the DHRP dataset, we base the normalization constant values \(k\) on those from the COCO dataset (available at [[https://cocodataset.org/#keypoints-eval](https://cocodataset.org/#keypoints-eval)]([https://cocodataset.org/#keypoints-eval](https://cocodataset.org/#keypoints-eval)), accessed on 2 October 2024). The specific values of \(k\) used are as follows: 0.079 for the nose, 0.079 for the neck, 0.079 for the shoulders, 0.072 for the elbows, 0.062 for the wrists, 0.107 for the hips, 0.087 for the knees, and 0.089 for the ankles. Since most humanoid robots lack distinct facial features, we assigned the same value as the shoulders to the nose and neck. We further evaluate accuracy on a per-joint basis using Joint Keypoint Similarity (JKS), calculated in a manner similar to the OKS:  
\[JKS\ =\ \sum_{i=1}^{|F|}exp(-d_{i}^{2}/(2s^{2}k_{i}^{2}))\cdot\delta(v_{i}^{gt}> 0)/\sum_{i=1}^{|F|}\delta(v_{i}^{gt}>0) \tag{4}\]  
where \(|F|\) denotes the number of frames for a particular joint, and \(d_{i}\) represents the Euclidean distance between the predicted keypoint and the ground truth in the \(i\)-th frame. We evaluate the overall pose estimation performance using Average Precision (AP) and Average Recall (AR), following the methodology from the COCO dataset:  
\[AP^{t}\ =\ \sum_{i=1}^{m}\delta(OKS>t);AR^{t}\ =\ \sum_{i=1}^{n}\delta(OKS>t) \tag{5}\]  
where \(m\) denotes the number of instances where the network's output is positive, \(n\) represents the total number of instances in the test set, and \(t\in T\) denotes a threshold value from the set \(T\). We adopt the threshold range \(T\ =\ (0.50,0.55,\ldots,0.90,0.95)\), following the COCO dataset, which includes a total of 20 thresholds. \begin{table}
\begin{tabular}{c l c c} \hline
**Augment. Type** & **Augment. Method** & **Prob.** & **Range** \\ \hline Motion jitter & Image scale & 0.8 & 0.5–1.6 \\ Motion jitter & Image rotation & 0.8 & \(\leq\)90\({}^{\circ}\) \\ Motion jitter & Image translation & 0.8 & \(\leq\)0.3 \(\times\) image height \\ Motion jitter & Horizontal flip & 0.5 & - \\ Color jitter & Pixel contrast & 0.8 & \(\leq\times\)0.2 \\ Color jitter & Pixel brightness & 0.8 & \(\leq\pm\)30 \\ Noise jitter & Gaussian blur & 0.4 & \(\sigma\leq 2.0\) \\ Noise jitter & Salt-and-pepper noise & 0.3 & \(\leq\pm\)25 \\ \hline \end{tabular}
\end{table} 
Table 4: Random data augmentation during training. The primary evaluation metric for the DHRP dataset is the mean Average Recall (mAR). In contexts such as human-robot interactions, the focus is on reducing false negatives (missed detections) during video tracking.