# Depth Segmentation Approach for Egocentric 3D Human Pose Estimation with a Fisheye Camera
## 4 Experiments
### Implementation Details
#### 4.2.3 Implementation of xR-EgoPose and Mo2 Cap2  
Since the implementation of xR-EgoPose [8] and Mo2 Cap2 [10] is not publicly available, we implemented them based on their paper and dataset. To implement xR-EgoPose, we followed their dual-branch architecture to predict 3D joints from a single RGB image. The image was referenced as a heatmap via ResNet-101, the encoder extracted the 20-lengthvector, and the vector was used as input to two decoder branches. One decoder branch regressed the heatmap and the other estimated the 3D human pose.  
When implementing Mo2 Cap2, we took two images as initial inputs by following their implementations: one image was a full-sized fisheye image that showed a full body, and the other image was a zoomed-in image with a clear view of the lower body (i.e., legs and feet). The implemented Mo2 Cap2 had a dual-branch architecture to predict two heatmaps with the two zoom-level images. By averaging these two heatmaps, the Mo2 Cap2 network generated an averaged heatmap on u,v coordinates. Additionally, the network regressed the distance between the camera and each joint using a distance module consisting of two residual blocks and a fully connected layer. We followed their implementation up to this step, but we were unable to feed the u,v coordinates directly into Mo2 Cap2's calibration function [44] because the coordinates of the dataset that we used were not compatible with Mo2 Cap2. Therefore, instead of adopting Mo2 Cap2's final step, which inputs the 3D coordinate normal vectors and distance values into the fisheye camera calibration function, we implemented a simpler network. This network consists of a fully connected layer that regresses the 3D joint positions using the 2D joint u,v coordinates from the averaged heatmap and the distance values.  
Additionally, we implemented the SegDepth module on top of the xR-EgoPose and Mo2 Cap2 implementations; their SegDepth module has the same architecture as ours: the modified ResNet-50 encoder and the SegDepth decoder, so it uses the same process to extract the depth features. In the xR-EgoPose pipeline, we modified the dual-decoder structure to incorporate the depth features estimated by the SegDepth module. The xR-EgoPose consists of a single encoder, which encodes the heatmap into an embedding vector, and two decoders: one reconstructs the heatmap using the embedding vector, while the other estimates the 3D pose. We kept the embedding vector input to the heatmap reconstruction decoder unchanged but added the depth features estimated by the SegDepth module to the embedding vector input of the 3D pose estimation decoder via concatenation (see Appendix A.1). Similarly, we incorporated the depth features estimated by the SegDepth module into the Mo2 Cap2 pipeline by modifying Mo2 Cap2 as follows: Mo2 Cap2 calculated the average joint coordinates u,v from two heatmaps at different scales and multiplied these coordinates by the camera distance vector to estimate the 3D pose. We added the depth features from the SegDepth module to the camera distance vector by concatenating them (see Appendix A.2).