# Steel Roll Eye Pose Detection Based on Binocular Vision and Mask R-CNN
## 3 Image Acquisition and Processing
### Visual Imaging Principle  
Visual sensors primarily capture information about objects in three-dimensional space by analyzing the target's light intensity data, the spatial distribution of light intensity, and color identification. This information is subsequently converted into two-dimensional image plane projection data [24]. In this study, a CMOS camera (IMX298) was utilized to capture images of the steel coil's end face. By leveraging camera imaging principles and coordinate translation transformations, the flatness of the end face and the precise coordinates of its mounting holes are accurately determined. A spatial coordinate system was established to derive the camera's imaging model. The machine vision system encompasses four primary coordinate systems: the world coordinate system, which represents the absolute coordinates of three-dimensional points in the scene as (\(X_{w}\), \(Y_{w}\), \(Z_{w}\)); the camera coordinate system, which describes the relative position of objects within the camera, with the three-dimensional coordinates of a scene point P denoted as (\(X_{c}\), \(Y_{c}\), \(Z_{c}\)); the image physical coordinate system, whose origin \(O_{1}\) (\(u_{0}\), \(y_{0}\)) is situated at the center of the image plane, with the x-axis and y-axis aligned parallel to the \(u\)-axis and \(v\)-axis of the image pixel coordinate system; and the image pixel coordinate system, which delineates the coordinates of image pixels, with the coordinates of scene point P expressed as (\(u\), \(v\)) [25]. The relationship between the coordinate systems is shown in Figure 7.  
The transformation relationship between the image coordinate system and the image physical coordinate system can be expressed as:  
\[\left\{ \begin{array}{l}u=\frac{x}{dx}+u_{0}\\ v=\frac{y}{dy}+v_{0}\end{array} \right. \tag{5}\]  
Figure 7: Schematic of the linear imaging model of the camera. It can be represented in the form of its chi-square coordinate matrix as:  
\[ \begin{bmatrix}u\\ v\\ 1\end{bmatrix} = \begin{bmatrix}\frac{1}{d_{x}}&0&u_{0}\\ 0&\frac{1}{d_{y}}&v_{0}\\ 0&0&1\end{bmatrix}\begin{bmatrix}x\\ y\\ 1\end{bmatrix}  \tag{6}\]  
The relationship between the camera coordinate system and the world coordinate system can be represented by a rotation matrix \(\mathbf{R}\) and a translation matrix \(T\), which is expressed as its sub-matrix:  
\[ \begin{bmatrix}X_{c}\\ Y_{c}\\ Z_{c}\end{bmatrix} = \begin{bmatrix}R&T\end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix}  \tag{7}\]  
where \(\mathbf{R}\) is an orthogonal unit rotation matrix of size \(3\times 3\) and \(\mathbf{T}\) is a 3D translation matrix of size \(3\times 1\). (\(X_{c}\), \(Y_{c}\), \(Z_{c}\)) are the coordinates of the point in the camera coordinate system, (\(X_{w}\), \(Y_{w}\), \(Z_{w}\)) are the coordinates of the point in the world coordinate system. For any point \(P\) (\(X_{c}\), \(Y_{c}\), \(Z_{c}\)) in the camera coordinate system, the physical coordinate system (\(x\), \(y\)) in the image plane is:  
\[x=f\frac{x_{c}}{z_{c}}+u_{0},u=\frac{x}{dx}+u_{0} \tag{8}\]  
This can be expressed in terms of the matrix:  
\[Z_{c} \begin{bmatrix}x\\ y\\ 1\end{bmatrix} = \begin{bmatrix}f&0&0\\ 0&f&0\\ 0&0&1\end{bmatrix}\begin{bmatrix}X_{c}\\ Y_{c}\\ Z_{c}\end{bmatrix}  \tag{9}\]  
From the linear camera imaging model [26], we derive the relationship between coordinate points (\(X_{w}\), \(Y_{w}\), \(Z_{w}\)) win the world coordinate system and their projection points (\(u\), \(v\)) in the image pixel coordinate system, as shown in Equation (10). \[Z_{1} \begin{bmatrix}u\\ v\\ 1\end{bmatrix} = \begin{bmatrix}k_{u}&k_{s}&u_{0}\\ 0&k_{v}&v_{0}\\ 0&0&1\end{bmatrix}\begin{bmatrix}R&T\end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix} =M \begin{bmatrix}R&T\end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix}  \tag{10}\]  
In the formula, \(\mathbf{M}\), \(\mathbf{[}R\mathbf{T}\mathbf{]}\) are referred to as the \"inner parameter matrix\" and \"outer parameter matrix\" of the camera, respectively.