# A DDPG-Based USV Path-Planning Algorithm
## 1 Introduction  
USVs represent a specialized category of compact, multifunctional, and intelligent unmanned marine platforms, which can be remotely piloted or operated autonomously to execute extensive and enduring marine operations in complex and dangerous waters. For proficient path planning in USVs, a system is required to ensure navigation between two designated points (such as start and target locations) and includes obstacle-avoidance capabilities ([PERSON] et al., 2019; [PERSON] et al., 2019; [PERSON] et al., 2020)[1, 2, 3]. The efficiency of path planning has a profound impact on the overall quality of the routes created by the USV within its surroundings. Path-planning strategies can be segmented into two primary types, specifically global and local methods, contingent on environmental visibility ([PERSON] et al., 2011; [PERSON] et al., 2017)[4, 5]. Global methods are designed for conditions with only static hindrances and complete availability of environmental information in advance. They permit the pre-calculation of the briefest trajectory from the starting to the ending point using classical methods such as the A-star and Dijkstra algorithms ([PERSON] et al., 2019; [PERSON] et al., 2018)[6, 7]. They offer strong convergence and consistency, but they are primarily suitable for static surroundings and, therefore, require further refinement for generating USV-compatible paths ([PERSON] et al., 2018)[8]. For instance, by refining the paths planned by the A-star algorithm through spline fitting, the feasibility of unmanned diving vessels can then be enhanced ([PERSON] et al., 2018)[9]. However, these methods formulate trajectories by linking unidirectional points linearly, which are commonly at a low resolution, making them inapplicable in dynamic settings with moving obstructions. They also usually exhibit a high time complexity, making them challenging in vast environments. In contrast, local methods fit dynamic situations better ([PERSON] et al., 2021)[10]. These methods do not rely on prior information about the environment and are capable of finding high-resolution paths even when mobile objects are present. Traditional approaches to USV path planning comprise heuristic search-based methods, minimum spanning tree-based methods, and neural network-based methods. Heuristic search-based strategies utilize algorithms like A* and Dijkstra for optimal USV path discovery, but they may suffer from premature convergence and local optimality. Therefore, it is suggested that algorithms, as well as the introduction of intelligent algorithms such as genetic and particle swarm algorithms, should be enhanced ([PERSON] et al., 2021; [PERSON] et al., 2021) [11; 12]. Minimum spanning tree-based approaches, including [PERSON]'s and Kruskal algorithms ([PERSON], 1957; [PERSON], 1956) [13; 14], utilize graph theory to calculate the USV's optimal path. In contrast, neural network-based methods utilize artificial neural networks for path modeling and calculation ([PERSON] et al., 2019; [PERSON] et al., 2016) [15; 16]. The development of technology has brought increasing attention to neural networks. ML focuses on developing algorithms to enable computers to learn and make their own decisions based on data ([PERSON] et al., 2023) [17]. RL (Reinforcement Learning), a branch of machine learning, focuses on actions derived from environmental interactions that amplify anticipated gains ([PERSON] et al., 1996) [18]. It is influenced by psychological behaviorists, whose theories concern how organisms, motivated by rewards or penalties from their surroundings, develop their expectations and form habits to maximize benefits. RL is a learning approach wherein an agent learns, which correlates states to actions to optimize rewards. The agent should continually test and adapt within the environment, employing feedback (rewards) to refine the state-action alignment. Thus, trial and error, along with delayed rewards, represent RL's key features.