## 1 Introduction  
Autonomous underwater vehicles (AUVs) continue to attract significant attention within the scientific community. They are employed in various fields, such as oceanography, the oil and gas industry, maritime rescue, military applications, underwater archeology, and photogrammetry. Research on the development of AUVs encompasses multiple areas, with a strong focus on enhancing vehicle autonomy. This includes the development of artificial intelligence and machine learning algorithms for improved decision-making capabilities [1, 2] and increased precision in navigation and obstacle avoidance [3, 4], as well as adaptive algorithms that can adjust to changing conditions [1]. Simultaneously, efforts are underway to advance underwater communications and data transmission. These include developing efficient underwater communication systems, such as acoustic and optical data transfer methods [5]; improving the connectivity between surface vessels and other underwater vehicles; and reducing latency while increasing the data throughput [6, 7, 8]. Maritime route-planning systems are generally assessed in terms of their total path length, adherence to safety contours (sometimes referred to as \"safety isobaths\"), and avoidance of restricted or prohibited zones, as well as ensuring path smoothness and sufficient separation from hazards. Vehicle motion models must accurately capture the AUV's maneuvering capabilities so that the path-planning algorithm can account for the proper trajectory-following behavior [9; 10; 11]. External factors like ocean currents, tides, wave conditions, the accuracy of navigation systems, and the presence of obstacles affect path planning both prior to and during vessel operation. Consequently, an AUV should be capable of making autonomous decisions regarding the trajectory it follows. Up to now, the research has predominantly centered on classical route-planning approaches, such as the A* algorithm and its numerous adaptations [12; 13; 14], Rapidly Exploring Random Trees (RRTs) [15; 16], and Probabilistic Roadmaps (PRMs) [17; 18]. Other trajectory planning strategies encompass fuzzy-inference-based methods [19; 20], optimization techniques such as genetic algorithms [21; 22] and particle swarm optimization [23]. Likewise, neural-network-based algorithms have been used to discover efficient paths, as discussed in [24; 25]. Another branch of research focused on control and path planning for underwater vehicles is reinforcement learning (RL). It constitutes a field of machine learning that has been intensively developed in the last decade, both by technology companies [26; 27; 28] and by individual researchers. These methods involve studying and optimizing the interaction of an agent with its environment. During training, the algorithms collect experience from environmental data and the agent's chosen actions. The objective is for the agent to perform a specific task by maximizing a reward function, which depends on the actions taken over the course of its interaction with the environment. Reinforcement learning (RL) encompasses a broad range of algorithms that can be categorized in multiple ways. From the perspective of value functions or policy representation and updating, one can distinguish several groups: (1) _tabular methods_ such as SARSA and Q-learning, typically relying on Q-value tables and temporal-difference updates for relatively low-dimensional state spaces [29; 30]; (2) _policy gradient_ approaches that directly optimize a parameterized policy; (3) _actor-critic_ methods combining value-based and policy-based ideas; (4) _hierarchical RL_, which introduces higher-level structures or subgoals; and (5) _evolutionary_ or genetic-based algorithms. Furthermore, _Deep Reinforcement Learning_ (deep RL) extends these ideas by using deep neural networks to approximate value functions, policies, or even entire environment models, allowing for more effective handling of high-dimensional and complex tasks [31; 32].