# Steel Roll Eye Pose Detection Based on Binocular Vision and Mask R-CNN
## 5 Positional Deviation Solving  
Any coordinate in the camera coordinate system can be transformed into a coordinate in the world coordinate system through a series of four coordinate transformations. The schematic of the coordinate transformation is shown in Figure 18. First, translate the coordinate system \(O-X^{\prime}Y^{\prime}Z^{\prime}\) so that point \(O^{\prime}\) coincides with point \(O\), with the translation matrix denoted as (\(T\)). Next, rotate the system by angles (\(\theta_{x}\)), (\(\theta_{z}\)), and (\(\theta_{y}\)), align the camera coordinate system with the world coordinate system. The coordinates of any object  
Figure 17: Sample elliptic parameter solution. point \(P(x_{i},y_{i},z_{i})\) in the camera coordinate system can then be transformed into the world coordinate system by leveraging the left and right camera linear imaging models. The translation matrix is:  
\[T=\left[x_{0},y_{0},z_{0}\right]^{T} \tag{19}\]  
The rotation matrix:  
\[R_{X}(\theta_{x})= \begin{bmatrix}1&0&0\\ 0&cos\theta_{x}&-sin\theta_{x}\\ 0&sin\theta_{x}&cos\theta_{x}\end{bmatrix}  \tag{20}\]  
\[R_{Z}(\theta_{z})= \begin{bmatrix}cos\theta_{z}&-sin\theta_{z}&0\\ sin\theta_{z}&cos\theta_{z}&0\\ 0&0&1\end{bmatrix}  \tag{21}\]  
\[R_{Y}(\theta_{y})= \begin{bmatrix}cos\theta_{z}&-sin\theta_{z}&0\\ sin\theta_{z}&cos\theta_{z}&0\\ 0&0&1\end{bmatrix}  \tag{22}\]  
This results in the determination of the camera's external reference matrix [_R T_]:  
\[T=\left[x_{0},y_{0},z_{0}\right]^{T} \tag{23}\]  
\[R= \begin{bmatrix}\cos\theta_{y}\cos\theta_{z}&-\cos\theta_{x}\cos\theta_{y} \sin\theta_{z}+\sin\theta_{x}\sin\theta_{y}&\sin\theta_{x}\cos\theta_{y}\sin \theta_{z}+\cos\theta_{x}\sin\theta_{y}\\ \sin\theta_{z}&\cos\theta_{x}\cos\theta_{z}&-\sin\theta_{x}\cos\theta_{y}\\ -\sin\theta_{y}\cos\theta_{z}&\cos\theta_{x}\sin\theta_{y}\sin\theta_{z}+ \sin\theta_{x}\cos\theta_{y}&-\sin\theta_{x}\sin\theta_{y}\sin\theta_{z}+ \cos\theta_{x}\cos\theta_{y}\end{bmatrix}  \tag{24}\]  
The relationship between the left and right camera pixel coordinate systems and the world coordinate system is as follows:  
\[Z_{1} \begin{bmatrix}u_{1}\\ v_{1}\\ 1\end{bmatrix} =M_{t} \begin{bmatrix}R&T\end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix} = \begin{bmatrix}a_{11}^{\prime}&a_{12}^{\prime}&a_{13}^{\prime}&a_ {14}^{\prime}\\ a_{21}^{\prime}&a_{22}^{\prime}&a_{23}^{\prime}&a_{24}^{\prime}\\ a_{31}^{\prime}&a_{32}^{\prime}&a_{33}^{\prime}&a_{34}^{\prime}\end{bmatrix} \begin{bmatrix}X_{W}\\ Y_{W}\\ Z_{W}\\ 1\end{bmatrix}  \tag{25}\]  
\[Z_{1} \begin{bmatrix}u_{r}\\ v_{r}\\ 1\end{bmatrix} =M_{r} \begin{bmatrix}R&T\end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix} = \begin{bmatrix}a_{11}^{\prime}&a_{12}^{\prime}&a_{13}^{\prime}&a_ {14}^{\prime}\\ a_{21}^{\prime}&a_{22}^{\prime}&a_{23}^{\prime}&a_{24}^{\prime}\\ a_{31}^{\prime}&a_{32}^{\prime}&a_{33}^{\prime}&a_{34}^{\prime}\end{bmatrix} \begin{bmatrix}X_{W}\\ Y_{W}\\ Z_{W}\\ 1\end{bmatrix}  \tag{26}\]  
Figure 18: Schematic diagram of coordinate transformation. The collation results in a mapping from the left and right camera pixel coordinate systems to the world coordinate system:  
\[ \begin{bmatrix}u_{1}a_{31}^{l}&u_{1}a_{32}^{l}&u_{1}a_{33}^{l}\\ v_{1}a_{31}^{3}&v_{1}a_{32}^{l}&v_{1}a_{33}^{l}\\ u_{r}a_{31}^{3}&u_{r}a_{32}^{3}&u_{r}a_{33}^{l}\\ v_{r}a_{31}^{3}&v_{r}a_{32}^{3}&v_{r}a_{33}^{l}\end{bmatrix}\begin{bmatrix}X_{W }\\ Y_{W}\\ Z_{W}\end{bmatrix} = \begin{bmatrix}a_{14}^{l}-u_{1}a_{34}^{l}\\ a_{24}^{l}-v_{1}a_{34}^{l}\\ a_{14}^{r}-u_{r}a_{34}^{l}\\ a_{24}^{r}-v_{r}a_{34}^{l}\end{bmatrix}  \tag{27}\]  
By substituting the camera parameters, the coordinates of the spatial point \(P\) in the world coordinate system can be estimated using the coordinates of the pixel points in the left and right cameras via Equation (27). In this equation, the parameters of the left and right cameras are denoted by the subscripts \(l\) and \(r\), respectively.