# Automated Docking System for LNG Loading Arm Based on Machine Vision and Multi-Sensor Fusion
## 3 Design of a Vision-Based Deep Learning Loading-Arm Positioning System
### Monocular Camera Positioning Model
#### 3.3.1 Camera Imaging Model  
During the early stages of the coarse and fine alignment phases, the system uses a monocular camera to obtain image data of the target flange and uses the transformation relationship between coordinate systems to derive the three-dimensional spatial coordinates of the target point. As shown in Figure 11, the coordinates of the target point P need to undergo multiple transformations between the world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system. The following is the transformation relationship between each coordinate system.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Docking Stage** & **YOLOv8 FPS** & **Overall Computation FPS** \\ \hline Coarse docking & 42 & 26 \\ Fine docking & 39 & 16 \\ \hline \hline \end{tabular}
\end{table}
Table 2: YOLOv8 performance in coarse and fine docking stages (FPSs).  
The transformation from the world coordinate system to the camera coordinate system at the target point \(P\) can be described by a rigid transformation, which is related by:  
\[\begin{bmatrix}X_{c}\\ Y_{c}\\ Z_{c}\\ 1\end{bmatrix}=\begin{bmatrix}R&T\\ \overset{\rightarrow}{\rightarrow}&1\\ \overset{\rightarrow}{\rightarrow}&1\\ \end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix} \tag{18}\]  
where \(R\) is a 3 \(\times\) 3 rotation matrix; \(T\) is a 3D translation vector; and \((X_{w},Y_{w},Z_{w})\) and \((X_{c},Y_{c},Z_{c})\) are the coordinates of the target point in the world coordinate system and camera coordinate system, respectively. The target point, \(P_{c}(X_{c},Y_{c},Z_{c})\), is mapped to the point \(P\) (\(x,y\)) on the imaging plane by optical projection. This mapping relationship is based on the perspective projection principle, as shown in Equation (19):  
\[\frac{Z_{c}}{f}=\frac{X_{c}}{x}=\frac{Y_{c}}{y} \tag{19}\]  
where \(f\) is the focal length of the camera, and (\(x\) and \(y\)) are the projection coordinates of the target point on the imaging plane. Further expanding the perspective projection formula into matrix form, the mapping relationship of the target point from the camera coordinate system to the imaging plane can be expressed as Equation (20):  
\[Z_{c}\begin{bmatrix}x\\ y\\ 1\end{bmatrix}=\begin{bmatrix}f&0&0\\ 0&f&0\\ 0&0&1\end{bmatrix}\begin{bmatrix}X_{c}\\ Y_{c}\\ Z_{c}\end{bmatrix} \tag{20}\]  
The pixel coordinate system and the image coordinate system are in the same plane. The origin of the image coordinate system coincides with the optical center of the camera, while the pixel coordinate system takes the upper-left corner of the image as its origin. The conversion relationship between the two can be described by Equation (21):  
\[\begin{bmatrix}u\\ v\\ 1\end{bmatrix}=\begin{bmatrix}\frac{1}{dx}&0&u_{0}\\ 0&\frac{1}{dy}&v_{0}\\ 0&0&1\end{bmatrix}\begin{bmatrix}x\\ y\\ 1\end{bmatrix} \tag{21}\]  
where \(d_{x}\) and \(d_{y}\) are the physical dimensions of the pixel in the \(x\) and \(y\) directions, respectively; and \((u_{0}\) and \(v_{0})\) are the offset of the origin of the image coordinate system  
Figure 11: Schematic diagram of monocular camera multi-coordinate system mapping.  
relative to the pixel coordinate system. Combining the above relationships, we can obtain the complete mapping formula from the world coordinate system to the pixel coordinate system for the target point, as shown in Equation (22):  
\[Z_{c}\begin{bmatrix}u\\ v\\ 1\end{bmatrix}=\begin{bmatrix}f_{x}&0&u_{0}&0\\ 0&f_{y}&v_{0}&0\\ 0&0&1&0\end{bmatrix}\begin{bmatrix}R&T\\ \overset{\rightarrow}{0}&1\end{bmatrix}\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix} \tag{22}\]  
Among them, \(f_{x}=\frac{f}{d_{x}}\), and \(f_{y}=\frac{f}{d_{y}}\) indicate the relationship between the focal length of the camera and the pixel size.