# Automated Docking System for LNG Loading Arm Based on Machine Vision and Multi-Sensor Fusion
## 3 Design of a Vision-Based Deep Learning Loading-Arm Positioning System
### Monocular Camera Positioning Model
#### 3.3.4 Ultrasonic Sensor Positioning and Adjustment Method  
In the later stages of the fine alignment phase, due to the limitations of the installation position and distance of camera B, it is unable to fully obtain the image information of the flange surface. Therefore, the system uses ultrasonic sensors with higher positioning accuracy to supplement the positioning data of the flange. The advantage of ultrasonic sensors is that they can measure the distance to the surface of an object, rather than being limited to a single point. This ability comes from the characteristics of the sound cone of ultrasonic waves, which cover a small area rather than a single focus. With a reasonable installation position design, when the three-dimensional joint is roughly aligned with the flange, the four ultrasonic sensors can simultaneously provide valid distance data, as shown in Figure 15a,b. By analyzing the measurement data of the sensors in  
Figure 14: Schematic diagram of coordinate conversion between imaging planes. Figure 13: (**a**) Camera coordinate system in real perspective. (**b**) Camera coordinate system in virtual perspective. the up-down and left-right directions, the system can identify the direction of deviation between the flange and the joint, and adjust the position of the joint accordingly to achieve precise docking. The ultrasonic sensor is installed on the outside of the 3D joint and is used to measure the distance information in the four directions of up, down, left, and right of the flange. The definitions \(s_{1}\), \(s_{2}\), \(s_{3}\), \(s_{4}\), represent the ultrasonic ranging values in the left, right, up, and down directions, respectively; and \(d_{max}\) is the maximum range of the ultrasonic sensor. When there is a deviation in the distance within the measurement range of the ultrasonic sensor, the system adjusts the position of the 3D joint according to the following logic:  
1. As shown in Figure 15c, the sensors installed on the upper and lower sides of the three-dimensional joint measure the vertical distance between the flange and the joint. When \(s_{3}\leq d_{max}\) and \(s_{4}>d_{max}\), it means that the joint is lower than the flange, and the system instructs the joint to move upwards. Conversely, when \(s_{3}>d_{max}\) and \(s_{4}\leq d_{max}\), the system instructs the joint to move downwards until both sensors meet the conditions \(s_{3}\leq d_{max}\) and \(s_{4}\leq d_{max}\) at the same time;
2. As shown in Figure 15d, the left and right sensors measure the horizontal offset of the joint. When \(s_{1}\geq d_{max}\) and \(s_{2}<d_{max}\), the joint is biased to the left, and the system commands the joint to adjust to the right. Conversely, when \(s_{2}\geq d_{max}\) and \(s_{1}<d_{max}\), the system commands the joint to adjust to the left. The adjustment process continues until \(s_{1}\leq d_{max}\) and \(s_{2}\leq d_{max}\) are satisfied;  
Figure 15: (**a**) Ultrasonic sensor installation—vertical view. (**b**) Ultrasonic sensor installation—horizontal view; in the second panel. (**c**) Ultrasonic measurement for vertical alignment. (**d**) Flange misalignment and angular deviation. When \(s_{1}\) and \(s_{2}\) are both within the measurement range, the system calculates the deflection angle, \(\beta\), of the flange using the distance difference between the left and right sensors, and it adjusts the attitude of the 3D joint to ensure accurate alignment of the flange and joint. The formula for calculating the deflection angle is shown in Equation (30):  
\[\beta= \begin{cases}\arctan(\frac{5-s_{1}}{l})(s_{2}>s_{1})\\ -\arctan(\frac{s_{1}-s_{2}}{l})(s_{1}>s_{2})\end{cases}  \tag{30}\]  
where \(l\) is the distance between the left and right sensors, and \(\beta\) is the deflection angle of the flange.