# Tracking and Registration Technology Based on Panoramic Cameras
## 3 Method
### Panoramic Pose Estimation Sub-Network  
The relative camera pose estimation network is designed based on an encoder-decoder sub-network, aiming to estimate the relative poses between neighboring frames as well as to compute occlusion masks between adjacent images. The network comprises a feature extraction module, a pose prediction module, and an occlusion mask module group, as \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Name** & **Operations** & **Kernel Size** & **Stride** & **Padding** & **Dilation Rate** & **Output Dimensions** \\ \hline T1 & ConvTranspose2d & \(4\times 4\) & 2 & \(1\times 1\) & 1 & \(256\times 128\times 256\) \\  & Concation & & & & & \(384\times 128\times 256\) \\ C8 & Conv2d & \(5\times 5\) & 1 & \(2\times 2\) & 1 & \(256\times 128\times 256\) \\ C9 & Conv2d & \(5\times 5\) & 1 & \(2\times 2\) & 1 & \(1\times 128\times 256\) \\ D\({}_{1}\) & & & & & & \(1\times 128\times 256\) \\  & & & & & & \(1\times 256\times 512\) \\ T2 & ConvTranspose2d & \(4\times 4\) & 2 & \(1\times 1\) & 1 & \(128\times 256\times 512\) \\  & Concation & & & & & \(192\times 256\times 512\) \\ C10 & Conv2d & \(5\times 5\) & 1 & \(2\times 2\) & 1 & \(128\times 256\times 512\) \\  & Concation & & & & & \(129\times 256\times 512\) \\ C11 & Conv2d & \(1\times 1\) & 1 & \(0\times 0\) & 1 & \(64\times 256\times 512\) \\ C12 & Conv2d & \(3\times 3\) & 1 & \(1\times 1\) & 1 & \(1\times 256\times 512\) \\ D\({}_{2}\) & & & & & & \(1\times 256\times 512\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Output shapes and hyperparameters of decoder operations. illustrated in Figure 3. The entire image sequence is provided as input, and the network is trained to predict the global frame \(I_{t}\), along with the relative poses \(T_{t,t-1}\) and \(T_{t,t+1}\) between neighboring frames \(I_{t-1}\) and \(I_{t+1}\), as well as multi-scale occlusion masks \(M_{s}\). To ensure a lightweight architecture, the pose estimation sub-network adopts an encoder design. The feature extraction module is built upon ResNet18 [29], consisting of one convolutional layer followed by four residual blocks. Given that three consecutive frames are input simultaneously, the number of input channels is set to 9, allowing for the extraction of shallow features from the input images. The four residual blocks capture features at multiple scales, with each block composed of standard residual units. Each residual unit contains two 3 \(\times\) 3 convolutional layers and a residual skip connection from input to output. The pose prediction module comprises five convolutional layers and one global average pooling layer. It outputs two 6-DoF relative poses, \((T_{t,t-1},T_{t,t+1})\), for the three input images, where each pose vector includes translation and rotation parameters. Rotations are represented using Euler angles. The occlusion mask module utilizes intermediate features at two different resolutions from the feature extraction module. These are passed through two convolutional layers to produce occlusion masks \(M_{1}\) and \(M_{2}\) at different scales, which represent the occluded regions resulting from camera motion between neighboring frames. The camera pose estimation sub-network exhibits a certain degree of robustness to dynamic scene motions. This is attributed to the use of an unsupervised learning approach based on view reconstruction, which relies on the following assumptions: (1) All objects in the scene are static, and only the camera is moving, i.e., the scene is rigid, and the appearance changes between adjacent frames are primarily caused by camera motion. (2) Objects are assumed to remain visible across adjacent frames, without being occluded or leaving the field of view due to either camera motion or object motion. (3) The illumination of the same object should remain consistent across adjacent frames, implying that no strong reflections are present and all surfaces exhibit Lambertian reflectance.