# AidA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV
## 1 Introduction
(2018) extended their previous research in (2018) to indoor planning with PRM-RL, using a point-to-point policy trained with DDPG as the local planner for PRM (2018). In the present paper, our algorithm creates a map from the sensors to control the velocity in the same way as the approaches developed by [PERSON] et al., but our model is trained  
Figure 1: Challenges related to the development of a successful real-time autonomous mission planning system for a single UAV. with Auto Reinforcement Learning (AutoRL) method, which tunes the agent's reward and network architectures to improve the performance of the model and time needed for manual engineering. [PERSON] et al. [8] applied meta-heuristics to the learning of the embedding network to solve the combinatorial optimization problem based on graphs. Due to the learned greedy policy behavior for the intended problems (i.e., minimum vertex cover, maximum cut, or TSP), the solution search within the network was more efficient than pointer network-based neural combinatorial optimization [3; 9]. [PERSON] et al. [10] also extended the work in [6] by proposing a hybrid path planning method that combined PRM with RL for long-range operation system for a ground robot in an indoor environment. In PRM-RL, an RL agent learns a local point-to-point task, incorporating system dynamics and sensor noise independent long-range environment structure. [PERSON] et al. [11] proposed a kinodynamic motion planning scheme based on the integration of a rapidly exploring randomized tree (RRT), and an efficient kinodynamic motion planner that used RL. The aim of this approach was to find an efficient solution to long-range path planning problems for ground robots. In [1; 2; 5; 6; 11], the proposed methods mainly involved static environments, and did not consider the vehicle constraints as in the RL-RRT approach [10]. The problems in this research is not taken into account in the re-planning phase. When a vehicle carries out planning in a real-world environment, initial path planning is not sufficient, and re-planning should be applied in order to minimize the failure rate, distance, and time. Basically, path planning algorithms which did not include the re-planning feature take a lot of computation time to generate the path in the real-time problem. For the real-world environment problems, the re-planning of the path is a necessary and critical issue for all of the algorithms. The path planning algorithm without considering re-planning can update the path again in real time, but the computation cost of the re-plan path is expensive. The algorithm must recalculate the whole path from the start to the goal destination again to get the obstacle-free path. For a novel anytime incremental search algorithm called improved the Anytime Dynamic A\({}^{\star}\) (iADA\({}^{\star}\)) algorithm, it considered the re-planning feature; in that case, if the obstacles or cost changes occurred in the initial path, the algorithm reused the previous planning information and updated the cost nearly in the affected area only. By doing these steps, the algorithm saves the computation time and can provide the new path very quickly. The proposed research will bridge these gaps by developing an integrated planning system intended for the real-world environment. A hybrid path planning algorithm is developed by integrating a graph-based path planning algorithm with a learning-based algorithm for local planning to allow the UAV to avoid facing obstacles in real time. The global path planning problem is solved in the first stage using iADA\({}^{\star}\). A reinforcement learning method is used to carry out local planning between waypoints, to avoid any collision within the environment. This study aims to overcome a number of the significant challenges of mission planning as well as path planning in complex and dynamically changing environments for a UAV to reach the destination to achieve a successful mission.