###### Abstract  
Despite receiving much attention from researchers in the field of naval architecture and marine engineering since the early stages of modern shipbuilding, the berthing phase is still one of the biggest challenges in ship maneuvering due to the potential risks involved. Many algorithms have been proposed to solve this problem. This paper proposes a new approach with a path-planning algorithm for automatic berthing tasks using deep reinforcement learning (RL) based on a maneuvering simulation. Unlike the conventional path-planning algorithm using the control theory or an advanced algorithm using deep learning, a state-of-the-art path-planning algorithm based on reinforcement learning automatically learns, explores, and optimizes the path for berthing performance through trial and error. The results of performing the twin delayed deep deterministic policy gradient (TD3) combined with the maneuvering simulation show that the approach can be used to propose a feasible and safe path for high-performing automatic berthing tasks.  
path planning; deep reinforcement learning; TD3; maneuvering simulation; automatic berthing +
Footnote †: journal: Applied sciences  
understand the mathematical model of ships. However, acquiring a substantial number of labeled training data can be time-consuming and costly.  
Unlike the aforementioned methods where the training dataset is not necessary, reinforcement learning techniques, which constitute an area of machine learning, allow the ship to learn and optimize its berthing maneuvers through interactions with a simulated environment. The application of RL in the automatic berthing task has shown good results when the ship can automatically learn the strategy and optimize the control policy to move to the berthing point [14; 15].  
In this paper, initial development of a novel path-planning algorithm for autonomous ship berthing that uses the latest technique in reinforcement learning, called twin delayed deep deterministic policy gradient (TD3), is proposed. The TD3 algorithm was introduced by [PERSON] (2018) and is specifically designed for continuous action spaces. TD3 is an extension of deep deterministic policy gradients (DDPGs) and aims to address certain challenges and improve the stability of learning in complex environments. TD3 exploration algorithms allow the agent to explore the environment and gain new experiences that optimize rewards through trial and error. It employs two distinct value function estimators, which mitigate the overestimation bias and stabilize the learning process. Leveraging these two critics, TD3 provides more accurate value estimates and facilitates better policy updates. High performance and stability compared to other algorithms in the field of RL were shown in [16]. In combination with the MMG model, the solutions for ship-maneuvering motion simulation that were proposed by a research group maneuvering modeling group (MMG) in 1977 [17] suggest a feasible path, resulting in faster convergence and improved accuracy.  
This article is organized into five parts. The first section introduces previous studies conducted in this field. Section 2 presents the equation of motion for the ship based on the MMG model along with hydrodynamic and interaction coefficients. Section 3 outlines the path-planning algorithm based on deep reinforcement learning TD3. Section 4 showcases and discusses simulation results for two berthing cases. Finally, Section 5 concludes this research.