# Pose Tracking and Object Reconstruction Based on Occlusion Relationships in Complex Environments
###### Abstract  
For the reconstruction of objects during hand-object interactions, accurate pose estimation is indispensable. By improving the precision of pose estimation, the accuracy of the 3D reconstruction results can be enhanced. Recently, pose tracking techniques are no longer limited to individual objects, leading to advancements in the reconstruction of objects interacting with other objects. However, most methods struggle to handle incomplete target information in complex scenes and mutual interference between objects in the environment, leading to a decrease in pose estimation accuracy. We proposed an improved algorithm building upon the existing BundleSDF framework, which enables more robust and accurate tracking by considering the occlusion relationships between objects. First of all, for detecting changes in occlusion relationships, we segment the target and compute dual-layer masks. Secondly, rough pose estimation is performed through feature matching, and a keyframe pool is introduced for pose optimization, which is maintained based on occlusion relationships. Lastly, the estimated results of historical frames are used to train an object neural field to assist in the subsequent pose-tracking process. Experimental verification shows that on the HO-3D dataset, our method can significantly improve the accuracy and robustness of object tracking in frequent interactions, providing new ideas for object pose-tracking tasks in complex scenes. pose tracking; 6 degrees of freedom pose estimation; occlusion relationship +
Footnote †: journal: Medical  
movements, and emphasizing the continuity of estimation results. Although current methods have made good progress in addressing issues such as object motion and drift caused byong-term tracking, these methods are mostly limited to relatively simple scenarios. In complex scenarios, challenges remain for pose tracking, such as difficulties in accurately segmenting targets and occlusions between targets and the environment or other objects. Object pose tracking can be categorized into instance-level and category-level approaches. Instance-level tracking relies on specific 3D models, such as CAD models [8], to track known objects with high precision. In contrast, category-level tracking does not depend on CAD models and aims to generalize across objects within the same category. Because the estimated parameters are unified as 6-DoF poses (the target scale is determined by the first frame), these approaches are discussed together. Unlike single-frame object pose estimation, object pose tracking does not independently estimate the pose of the target in each frame of the image. Instead, it is considered a motion process, where the pose of the new frame is adjusted based on the estimated pose of the previous frame. At the same time, new information is introduced into the system, and historical estimation results are optimized through consistency constraints. [PERSON] et al. [9] proposed a method that takes single-frame RGB-D images as input and estimates the pose of objects by tracking the cumulative changes in relative pose over time. Specifically, the method first employs an anchor-based key-point generation scheme to adaptively generate key points from previous frames and the current frame. Then, using two sets of ordered key points and previously computed instance poses, it derives the current estimated pose of the object, including six degrees of freedom of pose information. [PERSON] et al. [10] utilize Rao-Blackwellized particle filters to sample object poses and estimate the discrete distribution of rotations for each particle using a precomputed codebook. This method can effectively track object poses and is more robust to motion blur and occlusion. However, its efficiency significantly decreases when objects are heavily occluded, or measurement results deviate significantly from synthetically generated training data.