# Diverse Humanoid Robot Pose Estimation from Images Using Only Sparse Datasets
## 1 Introduction
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Method** & **Keypoints** & **Target Joints** & **Target Robots** \\ \hline (a) Robot Arm & & Arm Joints & Less than 3 \\ Pose Estimation [5; 6; 7] & & & \\ \hline (b) Humanoid Robot & & Partial-Body Joints & Less than 3 \\ Pose Estimation [8; 9] & & & \\ \hline (c) Diverse Humanoid Robot & & Full-body Joints & 11 commercial robots \\ Pose Estimation (Ours) & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of robot pose estimation methods. Additionally, we propose a new benchmark method for real-time, learning-based full-body 2D keypoint estimation from single images, as presented in Table 1(c). Our extensive experiments demonstrate that leveraging our extended pose dataset significantly improves full-body pose estimation accuracy by over 33.9% compared to using only sparse datasets, across 11 types of humanoid robots, requiring approximately 350 training images per robot. Our method also achieves real-time performance, operating at 42 frames per second (FPS), while consistently estimating full-body poses for side and back views, even with varying humanoid robot shapes. These advancements overcome the limitations of traditional pose estimation methods and offer a scalable solution for human-robot interaction applications involving diverse humanoid robots. To further support the research community, we have made our dataset publicly available. It can be accessed upon request at [[https://xrlabku.webflow.io/papers/diverse-humanoid-robot-pose-estimation-using-only-sparse-datasets](https://xrlabku.webflow.io/papers/diverse-humanoid-robot-pose-estimation-using-only-sparse-datasets)]([https://xrlabku.webflow.io/papers/diverse-humanoid-robot-pose-estimation-using-only-sparse-datasets](https://xrlabku.webflow.io/papers/diverse-humanoid-robot-pose-estimation-using-only-sparse-datasets)) (accessed on 2 October 2024). In summary, the primary contributions of this work are as follows:  
* The first pose dataset providing full-body annotations for over ten different types of humanoid robots. * A benchmark method for real-time, learning-based full-body pose estimation from a single image. * An evaluation dataset and standard metrics for assessing pose estimation performance. The remainder of this paper is organized as follows: Section 2 discusses related work. Section 3 presents the diverse humanoid robot pose dataset. Section 4 describes the benchmark method for real-time full-body pose estimation. The results are presented and discussed in Section 5, followed by a discussion of limitations and future work in Section 6. Figure 1: We present a learning-based full-body pose estimation method for various humanoid robots. Our keypoint detector, trained on an extended pose dataset, consistently estimates humanoid robot poses over time from videos, capturing front, side, back, and partial poses, while excluding human bodies (see supplemental video [11]). The following robots are shown: (**a**) Optimus Gen 2 (Tesla) [12]; (**b**) Apollo (Apptronik) [13]; (**c**) Atlas (Boston Dynamics) [14]; (**d**) Darwin-OP (Robotis) [15]; (**e**) EVE (1X Technologies) [16]; (**f**) FIGURE 01 (Figure) [17]; (**g**) H1 (Unitree) [18]; (**h**) Kepler (Kepler Exploration Robot) [19]; (**i**) Phoenix (Sanctuary AI) [20]; (**j**) TALOS (PAL Robotics) [21]; (**k**) Toro (DLR) [22].