# A Review of Autonomous Tugboat Operations for Efficient and Safe Ship Berthing
## 4 Autonomous Tugboat Element Technologies
### Control Technology
Therefore, there is still a need for research on developing control strategies that take into account the actual physical interaction between the tugboats and the mothership and ensure precise control of the mothership's position and attitude during the berthing process. Figure 19: Trajectory tracking framework proposed by [PERSON] et al. [23]. In order to better reflect the actual operation method, [PERSON] et al. [53] developed a ship berthing system that utilizes cooperative control between the damper and tugboat, as well as a subsequent study [54] was conducted to design a berthing system considering the ship characteristics and dynamics of the towline. However, the dynamics of the entire system were not taken into account. In actual tugboats, controlling the mothership accurately to the predefined path using a towline is challenging. If the tugboat and the vessel to be towed follow different paths due to waves and complex maneuvering environments, the risk of collision is high, making precise control of the path essential. To address this challenge, [PERSON] [27] proposed a basic mathematical model of a tugboat equipped with three active rudders (see Figure 16). As tugboats are dependent on the towline to pull the vessel or deal with obstacles, [PERSON] proposed a follow-up control for the leader to enhance the tracking performance of the towed vessel along the path of the tugboat. In the study by [28], the motherships and each tugboat were treated as one system, and the independent variables were controlled by considering multibody dynamics. A pose control method was proposed based on simplified dynamics, excluding the contact force term. Figure 20 shows the simulation result, which was demonstrated using a tracking control technique that can be manipulated to track a straight path while maintaining a desired heading angle. The problem of autonomy of a tugboat when docking a ship is a problem in which the tugboat's positional movement problem and control problem are complex. Therefore, in order to solve these complex problems, recently, research on reinforcement learning-based control methods using AI artificial intelligence is increasing. If reinforcement learning is used, it has the advantage that it can be applied to complex nonlinear problems or problems that are difficult to solve with existing control methods. The study in [29] proposes a hybrid control concept that combines a discrete-time dynamic model accounting for the tugboat's position movement and a continuous-time dynamic model representing the tugboat control. A control approach using the policy gradient-based Proximal Policy Optimization (PPO) algorithm is introduced, which is suitable for continuous action spaces and has shown high performance in reinforcement learning. The PPO algorithm is an artificial intelligence technique used to determine the optimal action to take in the current state. In the study in [29], the error between the current state and the state of the target ship is calculated, and the compensation value is obtained to minimize the error, maximizing the outcome. In Figure 21, the output value of the network consists of a total of five values (\(\triangle f_{1}\), \(\triangle f_{2}\), \(\triangle\alpha_{1}\), \(\triangle\alpha_{2}\), \(k\)) that select the amount of change in the force of each tugboat, the amount of change in the direction of the force, and the tugboat mode. Among these, the k value was discretized into two continuous action values from \(-1\) to \(1\) to represent the discretized mode change in the tugboat. If \(-1\) to \(0\), \(q_{1}\) (towing  
Figure 20: Tracking a given path using two tugboats while maintaining angle between tugboat and mothership [28]. mode), and if 0 to 1, \(q_{2}\) (berthing mode), were used to select the tugboat mode. Network inputs are \(x,y,\psi,u,v\), and \(r\), representing the state of the ship and \(e_{d}\), and \(e_{\psi},e_{\psi}\) (distance error, heading error, and velocity error), and the output value of the previous step was set as the network input value in the current step.