# Automated Docking System for LNG Loading Arm Based on Machine Vision and Multi-Sensor Fusion
## 3 Design of a Vision-Based Deep Learning Loading-Arm Positioning System
### Monocular Camera Positioning Model
#### 3.3.2 Method for Deriving Spatial Coordinates Based on Geometric Constraints  
In the localization stage, in order to establish a correspondence between the pixels in the image and the points in three-dimensional space, this paper performs an inverse transformation of Equation (22) to derive a conversion formula between pixel coordinates and world coordinates, as shown in Equation (23):  
\[\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\\ 1\end{bmatrix}=\begin{bmatrix}R&T\\ \overset{\rightarrow}{0}&1\end{bmatrix}^{-1}\begin{bmatrix}\frac{(u-u_{0}) \times Z_{c}}{f_{x}}\\ \frac{(y-v_{0})\times Z_{c}}{f_{y}}\\ Z_{c}\end{bmatrix} \tag{23}\]  
Monocular cameras cannot directly obtain depth information, while binocular cameras may experience a decrease in measurement accuracy due to a failure of point matching during stereo ranging. In order to improve positioning accuracy, this paper proposes a geometric constraint method based on known flange features (radius length). The coordinates of the center of the flange and its radius are accurately determined by selecting feature points (a, b, c, and d) on the flange. These reference points can be used to assist in calculating the coordinates of other points in the flange plane.  
Assuming that the camera plane is parallel to the flange plane, the four edge points (a, b, c, and d) of the flange are selected as reference points, as shown in Figure 12. Taking points a and b as an example, if the pixel width between the two points is \(\Delta v\) and the actual distance in the y direction is \(\Delta Y\), then the relationship between the two satisfies, as shown in Equation (24):  
\[\left\{\Delta v=-\frac{f_{y}}{Z_{c}}\Delta Y\right. \tag{24}\]  
Figure 12: Target identification box feature point diagram.  
According to the ratio of pixel width to actual distance, \(\Delta v/\Delta Y=-f_{y}/Z_{c}\), the influence of visual depth on the calculation of the \(y\) coordinate can be eliminated when \(\Delta v\) and \(\Delta Y\) are known. Similarly, the \(x\)-direction coordinate can be calculated. At this point, the three-dimensional coordinates of any point in the flange plane can be expressed as shown in Equation (25):  
\[\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\end{bmatrix}=\begin{bmatrix}\frac{(u-u_{0})\Delta X}{f_{x}\Delta u}\\ \frac{(v-v_{0})\Delta Y}{f_{y}\Delta v}\\ Z_{c}\end{bmatrix} \tag{25}\]  
By transforming Equation (25), the depth, \(Zc\), can be further solved, as shown in Equation (26):  
\[Z_{c}=-\frac{f_{y}}{\Delta v}\Delta Y=\frac{f_{x}}{\Delta u}\Delta X \tag{26}\]  
The internal parameters of the camera, such as \(u_{0}\), \(v_{0}\), \(f_{x}\), and \(f_{y}\), can be obtained through calibration. The complete coordinate transformation formula is shown in Equation (27):  
\[\begin{bmatrix}X_{w}\\ Y_{w}\\ Z_{w}\end{bmatrix}=\begin{bmatrix}\frac{(u-u_{0})\Delta X}{f_{x}\Delta v}\\ \frac{(v-v_{0})\Delta Y}{f_{y}\Delta v}\\ -\frac{f_{y}\Delta Y}{\Delta v}\end{bmatrix} \tag{27}\]  
The feature points a, b, c, and d in Figure 12 are the key feature information extracted by YOLOv8 object recognition. This allows the radius of the circle and the pixel coordinates of the circle center to be determined and the three-dimensional spatial coordinates of the circle center to be calculated using Equation (27).