# Maximizing UAV Coverage in Maritime Wireless Networks: A Multiagent Reinforcement Learning Approach
## 2 Related Work  
Nowadays, heuristic algorithms are widely used for UAV path planning. For instance, [PERSON] [13] developed an optimal fitness value search method for continuous and discrete data ranges based on particle swarm optimization (PSO) and electromagnetic-like (EML) algorithms, aiming to ensure the best coverage rate of user devices positioned at certain distance intervals. However, heuristic algorithms require reoptimization and solving when ocean environment data change, making them less adaptive. Additionally, due to inherent algorithmic limitations, they often cannot provide corresponding solutions quickly, making it difficult to adapt to scenarios with high real-time requirements. Therefore, combining cooperative exploration among drones and DRL algorithms can address the challenges posed by the real-world environment. Many existing DRL methods focus only on a single-drone scenario. [PERSON] et al. [14] proposed an autonomous drone path-planning framework using the DDPG method, aimed at training drones to avoid obstacles and reach target points. [PERSON] et al. [15] proposed a real-time path-planning approach based on a deep Q network (DQN) to solve the problem of an unknown environment. Unlike quadrotor UAVs, this method provides real-time action planning for fixed-wing UAVs on a feasible path while satisfying the kinematic constraints.  
Recently, several works have been proposed to address the problem of path planning using multiple UAVs. [PERSON] et al. [16] proposed a D3 QN-based learning method for decision-making policies of typical UAVs, which requires no prior knowledge of the environment (such as channel propagation models and obstacle positions) and other UAVs (such as their tasks, movements, and policies). Numerical results show that real-time navigation can be performed efficiently with high success rates and low collision rates. [PERSON] et al. [17] studied a mobile edge computing network installed on UAVs. They formulated the trajectory optimization problem as a Markov decision process and developed a QoS-based action selection policy based on the DDQN algorithm. DTPC [18] was designed in a multiagent reinforcement learning (MARL) framework based on MADDPG, which solves the problem of decentralized joint trajectory and transmission power control for multi-UAV ABS networks. [PERSON] et al. [19] proposed a novel end-to-end reinforcement learning approach for autonomous trajectory planning of UAVs to collect data from IoT devices in urban environments in support of next-generation communication networks. HAS-DQN [20] is an algorithm that combines the hexagonal area search (HAS) approach with a multiagent DQN. It effectively addresses collision issues among drones by constraining the overall coverage area of the UAVs. [PERSON] et al. [9]. designed a locally optimal UAV path-planning algorithm by leveraging 3D antenna radiation patterns and multistage D3 QN. Numerical results substantiate the algorithm's effectiveness in the context of connectivity-aware UAV path planning.  
However, in MARL-based path planning algorithms, each intelligent agent can only observe partial environmental information, overlooking the historical interaction among agents and environmental changes [21]. This may lead to increased difficulties in information propagation and coordination, especially in large-scale multiagent systems. Recurrent Neural networks (RNNs) can memorize and process sequential data [22]. However, RNNs tend to suffer from the vanishing gradient problem, causing them to forget long-term information. LSTM, as a sequence-modeling method, addresses this issue by introducing \"forget gates\" to improve the RNN structure. Therefore, using MATD3 combined with LSTM can consider the temporal correlations in agent path planning, enabling agents to make decisions with a more long-term planning capability rather than solely focusing on the current state.