# Pose Tracking and Object Reconstruction Based on Occlusion Relationships in Complex Environments
## 3 Experiments
### Comparison and Analysis
#### 3.5.2 Frequent Interaction Comparative Experiment  
Different from the previous experiment where the structure was fixed and the motion was uniform, in practical applications such as AR and robotics, interactions with complex structures like human limbs and robotic arms from a first-person perspective are more common. Therefore, in this section, we mainly discuss the effectiveness of our method in such complex scenarios. The data used in this section include _video_switch_ and _video_cup_, both of which contain challenging situations representative of pose estimation tasks. The characteristics of _video_switch_ are as follows:  
1. The object has a smooth black screen on the front, which lacks texture information and also exhibits specular reflection, causing significant interference with feature extraction and matching. 2. Over 90% of the video time involves interactions with human hands, resulting in prolonged and random occlusions. 3. Rapid rotational motion: In the video, the object undergoes full-angle rotations to showcase its entire appearance, including the front and back. Each rotation action is completed within one to two seconds, spanning no more than 50 frames (at a frame rate of 30). The tracking results are shown in Figure 7. Although there is no ground truth bounding box to directly calculate the 3D IoU for evaluating pose prediction results, we can still assess the correctness of the prediction results based on the closeness and alignment between the bounding boxes and the target objects. In terms of closeness, our method achieves seamless alignment between the bounding boxes and the surface of the target objects. However, BundleSDF is affected by frequent interactions and fails to accurately lock onto the target object, resulting in erroneous spatial predictions and causing the bounding boxes to completely deviate from the target object. Regarding alignment, as seen in Figure 7b, the orientation of the bounding boxes predicted by our method consistently matches the current orientation of the objects, indicating that our method can accurately estimate the rotation of the target objects. In contrast, for the comparative method (Figure 7a), while the orientations of the bounding boxes in the first two rows generally represent the orientations of the target objects, the method loses track of the target when the object flips to the other side. The bounding boxes do not adjust their orientation with the rotation of the target object, leading to errors. Figure 8 presents observations of the reconstruction results for the _video_switch_ dataset, comparing the two methods from three different angles. From a frontal view, the mesh reconstructed by our method essentially reproduces the shape of the target object (the switch console) and preserves some details, including the joysticks and the boundary between the controllers and the screen. In contrast, the comparative method exhibits some extraneous structures, and there is a hole near the left joystick, barely discernible as the outline of the object, resulting in poor performance. From side and top views, the deficiencies of BundleSDF are even more pronounced. The reconstructed mesh contains numerous structures that do not belong to the target object. This may be attributed to the reliance of reconstruction on accurate pose estimation. Repeatedly projecting the depth image onto incorrect poses can incorrectly assemble the object surface (for example, the plane extending from the bottom of the model in view 3 is formed by incorrectly stitching the screen of the target). While our method's reconstruction also includes a small amount of extraneous structure, it overall preserves the appearance of the original object, further confirming the accuracy of our method in object tracking. The characteristics of the _video_cup_ dataset are as follows:  
1. The target object is a red cup with a smooth outer surface and no specific texture. The cup is symmetrical. Figure 8: The comparison of reconstruction results for the _video_switch_ dataset.