# Non-Cooperative Spacecraft Pose Measurement with Binocular Camera and TOF Camera Collaboration
## 3 Detailed Description of the Proposed Method
### System Joint Calibration  
Binocular system calibration is first required. Points in the left and right camera coordinate system and the world coordinate system can be represented as \(\textbf{X}_{\text{I}},\;\textbf{X}_{\text{r}}\) and \(\textbf{X}_{\text{w}}\), respectively. Then, the following formula is:  
\[\left\{ \begin{aligned} \textbf{X}_{\text{I}}&=\textbf{R}_{ \text{I}}\textbf{X}_{\text{w}}+\textbf{T}_{\text{I}}\\ \textbf{X}_{\text{r}}&=\textbf{R}_{\text{r}}\textbf{X }_{\text{w}}+\textbf{T}_{\text{r}}\end{aligned} \right. \tag{4}\]  
Combined with the above formula, there are:  
\[\textbf{X}_{\text{r}}=[\textbf{R}_{\text{r}}\textbf{R}_{\text{I}}^{-1}] \textbf{X}_{\text{I}}+[\textbf{T}_{\text{r}}-\textbf{R}_{\text{r}}\textbf{R }_{\text{I}}^{-1}\textbf{T}_{\text{I}}] \tag{5}\]  
\[\left\{ \begin{aligned} \textbf{R}_{\text{r}}&=\textbf{R}_{ \text{r}}\textbf{R}_{\text{I}}^{-1}\\ \textbf{T}_{\text{r}}&=\textbf{T}_{\text{r}}-\textbf{R }_{\text{r}}\textbf{R}_{\text{I}}^{-1}\textbf{T}_{\text{I}}\end{aligned} \right. \tag{6}\]  
where \(\textbf{R}_{\text{I}},\textbf{T}_{\text{I}}\) and \(\textbf{R}_{\text{r}},\textbf{T}_{\text{r}}\) represent the rotation matrix and translation vector from the camera coordinate system to the world coordinate system, respectively. \(\textbf{R}_{\text{r}},\textbf{T}_{\text{r}}\) are the external parameters of the left and right camera that need to be calculated in the binocular calibration. The feasibility of a circular calibration board was verified in our previous work [24]. The circular calibration board is used to replace the traditional checkerboard calibration board. The center coordinates of the circle can be accurately extracted from the image. The circular calibration board is moved to different positions to acquire images that fill the entire camera field of view. Figure 4 shows the extraction results of the circular marker points on the left and right camera calibration boards. Figure 3: Flow chart of the pose measurement algorithm for non-cooperative spacecrafts. After completing the calibration of the binocular system, we must perform the joint calibration of the pose measurement system. The purpose of this step is to obtain the transformation relationship between the camera coordinate system and the point cloud coordinate system. It is impossible to acquire accurate three-dimensional coordinates of the circle center directly in the point cloud. Therefore, the corresponding center coordinates are yielded based on the intensity image of the TOF camera, and we map them to the point cloud coordinate system. Figure 5 illustrates the circle extraction result in the intensity image. For the 2D points \((u_{1},v_{1})\) in the left camera image coordinate system and the corresponding 3D point \((X_{\text{p}},Y_{\text{p}},Z_{\text{p}})\) in the point cloud coordinate system, the following relationship holds true:  
\[Z_{\text{c}} \begin{bmatrix}u_{1}\\ v_{1}\\ 1\end{bmatrix} =\mathbf{R}_{\text{kl}} \begin{bmatrix}\mathbf{R}_{\text{pc}}&\mathbf{T}_{ \text{pc}}\end{bmatrix}\begin{bmatrix}X_{\text{p}}\\ Y_{\text{p}}\\ Z_{\text{p}}\\ 1\end{bmatrix} =\mathbf{A} \begin{bmatrix}X_{\text{p}}\\ Y_{\text{p}}\\ Z_{\text{p}}\\ 1\end{bmatrix}  \tag{7}\]  
Figure 4: The extraction results of the circle center of the calibration plate in the left and right cameras: (**a**) left image; (**b**) right image. Figure 5: The extraction result of the circle center of the calibration plate in the intensity image. where \(\mathbf{R}_{\text{kl}}= \begin{bmatrix}f_{\text{xl}}&0&c_{\text{xl}}\\ 0&f_{\text{yl}}&c_{\text{yl}}\\ 0&0&1\end{bmatrix} \), \(\mathbf{R}_{\text{pc}}\) and \(\mathbf{T}_{\text{pc}}\) represent the joint calibration relationship to be solved. The A matrix can be represented as \(\mathbf{A}= \begin{bmatrix}a_{11}&a_{12}&a_{13}&a_{14}\\ a_{21}&a_{22}&a_{23}&a_{24}\\ a_{31}&a_{32}&a_{33}&a_{34}\end{bmatrix} \). The joint calibration problem becomes the problem of solving matrix A. The initial iteration value of matrix A is yielded based on the direct linear transformation (DLT).