# DQN-Based Shaped Reward Function Mold for UAV Emergency Communication
## 1 Introduction  
Emergency communication pertains to scenarios where the primary communication infrastructure sustains significant damage, prompting emergency response departments to coordinate efforts across various agencies and address crises efficiently. Given the expansive territories of many countries and the prevalence of natural disasters in certain regions, establishing a scientific and effective emergency communication system is of paramount importance. The mobility and positioning of base stations are further constrained by factors such as road accessibility, terrain undulations, and vegetation cover. Additionally, the placement of these base stations must adhere to specific topographic and geomorphic conditions [1]. Consequently, in numerous instances, large-scale UAV networking for emergency support is impracticable due to terrain, resource, and other limitations. In such circumstances, it becomes crucial to implement dynamic planning strategies for individual or groups of UAVs, with the objective of maximizing coverage and protection for users in need. After the initiation of post-disaster relief operations, commanders require a range of fundamental information, including personnel locations and critical areas, to support their decision-making processes. In such situations, UAVs can serve as nodes, tasked with traversing all sensors within the disaster zone and promptly relaying the gathered information back to the command post [2]. Under emergency conditions, command post nodes play a pivotal role in swiftly integrating information reported from various levels. However, it remains challenging for the command post's guarantee node to receive timely situation updates from the disaster area and make accurate decisions. Therefore, it is worthwhile to consider how to optimize the efficiency of a single UAV, ensuring it can cover and support as many users as possible. To address this, this paper establishes a virtual simulation environment and employs deep reinforcement learning to enable agents to autonomously explore solutions, providing valuable insights for intelligent UAV planning. The UAV node is characterized by its strong mobility and immunity to the terrain environment. In military contexts, UAVs play a crucial role in various operations such as battlefield reconnaissance, surveillance, border patrol, and precision strikes [3]. [PERSON] et al. utilized deep neural networks for the topology planning of emergency communication networks [4], while [PERSON] et al. employed the DQN algorithm for hierarchical design of such networks [5]. [PERSON] et al. proposed a method for determining the optimal deployment location of UAVs [6]. However, the aforementioned research has not yet fully maximized the support efficiency of a single UAV. In this paper, we enhance the UAV system by integrating the DQN algorithm with reward shaping. The simulation environment serves as the input, and through training, the agent UAV is able to automatically adjust its position. Ultimately, the improved efficiency of the trained UAV is effectively demonstrated. Focusing on the design of the reward function, [PERSON] et al. crafted a heuristic reward function that incorporates both distance-based and subgoal-based elements [7]. Experimental results demonstrated its significant impact on reducing training time. Meanwhile, [PERSON] [8] introduced a theoretical optimization framework rooted in reward shaping drive, aiming to enhance the training efficiency and stability of reinforcement learning, addressing the current challenge of low efficiency in reinforcement learning training. In recent years, despite the notable advancements in the application of UAVs in emergency communication, the challenge of swiftly locating all nodes within a given environment remains pressing, given the necessity to ensure timely communication. While existing communication routing methods offer some relief, they fail to fundamentally address the underlying bottleneck.