# Pose Tracking and Object Reconstruction Based on Occlusion Relationships in Complex Environments
## 2 Methods
### Optimization  
#### 2.3.1 Two-Stage Pose Optimization  
Pose estimation is approached in two stages, similar to the method used in BundleSDF [12]. The first stage involves estimating a coarse pose using random sample consensus (RANSAC), while the second stage refines this pose through pose graph optimization. In the first stage, feature matching is performed between two frames to obtain an initial pose estimate. Specifically, for video segmentation results, a feature matching network based on transformers [25] is employed. This network establishes feature correspondences between the current frame \(F_{t}\) and the previous frame \(F_{t-1}\) using RGB images. The relative change in object pose between \(F_{t}\) and \(F_{t-1}\) is then approximately calculated. The RGB feature correspondences are filtered through RANSAC [26] and least squares methods [27]. The coarse pose estimate \(\xi_{t}\) for the current frame is determined by selecting the result that maximizes the number of inliers. In the second stage, after obtaining frame \(F_{t}\) and its coarse pose estimate \(\xi_{t}\), \(K\) frames are selected from a memory pool for pose graph optimization. The optimized pose is output as the final pose estimation result. The selection of these \(K\) frames is based on choosing the subset \(P_{gg}\in P\) of \(K\) key-frames that have the maximum observation overlap with \(F_{t}\). In the pose graph \(G=(V,\epsilon)\), the nodes consist of \(F_{t}\) and the selected subset of key frames,  
Figure 3: The images of mask difference between adjacent frames and the original RGB image are shown above. (**a**) The original RGB image of this scene. (**b**) The result of superimposing foreground occlusion masks of adjacent frames is shown above. White indicates overlapping areas, while gray indicates non-overlapping areas. (**c**) The portion showing the change in foreground occlusion masks between adjacent frames (white areas). such that \(V=F_{t}\cup P_{pg}\) and \(|V|=K+1\). The objective is to determine the best pose that minimizes the total loss of the pose graph, given by:  
\[L_{\text{pg}}=\omega_{s}L_{s}(t)+\sum_{i\in V,j\in V,i\
eq j}[\omega_{f}L_{f}(i,j)+\omega_{p}L_{p}(i,j)] \tag{3}\]  
where \(L_{f}\) and \(L_{p}\) denote pairwise edge losses [17], and \(L_{s}\) represents an additional unary loss, which will be detailed in subsequent sections. The loss functions \(L_{f}\) and \(L_{p}\) used in the pose graph optimization are defined as follows. \(L_{f}\) measures the Euclidean distance between RGB-D feature correspondences \(p_{m}\) and \(p_{n}\) in \(\mathbb{R}^{3}\), where \(\xi\) denotes the object pose in frame \(F^{(i)}\), and \(\rho\) represents the robust Huber loss [28]. Specifically:  
\[L_{f}(i,j)=\sum_{(p_{m},p_{n})\in C_{ij}}\rho\Big{(}\Big{\|}\xi_{i}^{-1}p_{m}- \xi_{j}^{-1}p_{n}\Big{\|}_{2}\Big{)} \tag{4}\]  
Here, \(T_{ij}\) denotes the transformation from frame \(F^{(i)}\) to frame \(F^{(j)}\), given by \(\xi_{i}\xi_{j}^{-1}\); \(\pi_{j}\) represents the perspective projection onto image \(I_{j}\) associated with \(F^{(j)}\), \(\pi_{D}^{-1}\) is the inverse projection mapping used to obtain pixel positions in the depth image \(D_{j}\), and \(n_{i}(p)\) is the normal vector of point \(p\) in the point cloud corresponding to image \(I_{j}\). The deformed reprojection loss \(L_{p}\) is given by:  
\[L_{p}(i,j)=\sum_{p\in I_{i}}\rho\Big{(}\Big{|}n_{i}(p)\cdot\Big{(}T_{ij}^{-1} \pi_{D_{j}}^{-1}(\pi_{j}(T_{ij}p))-p\Big{)}\Big{|}\Big{)} \tag{5}\]  
In addition, we integrate object information from the neural object field (NOF) to jointly optimize the pose for improved accuracy. The unary loss \(L_{s}\) associated with the NOF is incorporated into the pose graph optimization process:  
\[L_{s}(t)=\sum_{p\in I_{t}}\rho\Big{(}\Big{|}\Omega(\xi_{t}^{-1}(\pi_{D}^{-1}( p)))\Big{|}\Big{)} \tag{6}\]  
Here, \(\Omega(\cdot)\) denotes the signed distance function of the neural object field. This loss quantifies the alignment between the neural implicit shape and the observed frames.