# Depth Segmentation Approach for Egocentric 3D Human Pose Estimation with a Fisheye Camera
## 6 Conclusions and Discussion  
In this paper, we propose a novel approach for egocentric 3D human pose estimation using fisheye images captured by a head-mounted display (HMD). We propose the SegDepth module, which uses depth maps and segmentation maps to address the limitations of previous methods based on 2D heatmap-based approaches. Our approach improves the accuracy of 3D pose estimation in ambiguous and occluded scenarios. The SegDepth module combines depth and segmentation functions through a single-decoder architecture, reducing the network size while maintaining or improving performance. The single-decoder architecture reduces the network size by 9.5% compared to a dual-decoder configuration, while achieving a better mean per joint position error (MPJPE). These results demonstrate that the shared representation of depth and segmentation features improves the network's generalization ability and computational efficiency. The adoption of our three-region segmentation strategy, which separates the body and background regions, improves the robustness of the model. Compared to single-body segmentation, three-region segmentation provides finer-grained spatial information, enabling the network to reduce noise and optimize training stability. Qualitative results and optimization curves show the superiority of the three-region approach in reducing noise and maintaining stable convergence. We also demonstrate the generality of the SegDepth module by integrating it into existing pose estimation frameworks, including xR-EgoPose and Mo2 Cap2. In both cases, the addition of depth and segmentation features resulted in improvements across different action types. These results demonstrate the adaptability of the SegDepth module and its potential to enhance other 3D pose estimation systems. Despite its effectiveness, the proposed method has several limitations. First, the evaluations were conducted on the synthetic xR-EgoPose dataset, which, while diverse, may not fully capture the variability of real-world environments. Future work should focus on validating the method using real-world datasets to ensure broader applicability. Second, the method is designed for fisheye camera setups, which may limit its generalizability to other camera types or configurations. Exploring extensions to other lens types or multi-camera systems could broaden its applicability. Finally, the current framework processes each frame independently, without considering temporal information. Incorporating temporal dynamics could further improve performance for actions involving rapid motion or complex transitions. The proposed method provides a solution for egocentric 3D pose estimation in VR. By addressing the limitations of 2D-based methods and introducing depth and segmentation features, the SegDepth module shows potential for improving pose estimation accuracy. Its adaptable architectures and compatibility with VR-specific constraints make it useful for future virtual and augmented reality applications. H.S. : Conceptualization, Methodology, Software, Writing - original draft, Visualization, Writing--review & editing, Investigation. S.K. : Supervision, Methodology, Validation, Funding acquisition, Resources, Writing--review & editing, Project administration. All authors have read and agreed to the published version of the manuscript. This research was supported by a grant from the ICT R&D Project through the Institute for Information & Communication Technology Planning and Evaluation (IITP), funded by the Ministry of Science & ICT, the Republic of Korea (grant number: RS-2022-II20874, grant title: Metaverse content conversion technology development for real-time accessibility support), and the Innovative Human Resource Development for Local Intellectualization program through the Institute of Information &