# AidA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV
## 1 Introduction  
Real-time mission planning systems for autonomous vehicles such as UAVs have attracted widespread research interest over recent years. In these mission planning systems, path planning forms the main task, and dynamic path planning has been identified as one of the most important tasks for autonomous systems. To achieve a successful outcome from any mission, autonomous systems must have efficient planning mechanisms that are cost-effective, executable, and most importantly safe in terms of the UAV operation system. Many types of optimization, path planning, and machine learning algorithms have been proposed over the years to solve these problems for autonomous UAV systems [1, 2, 3]. However, the available literature shows that researchers have faced certain problems involving high computational times, insecure/inefficient collision avoidance, lack of intelligence,and unreliable or non-optimum allocation of resources. The development of intelligent, computationally efficient planning is a topic that remains significantly challenging for current researchers, meaning that the development of an efficient algorithm that can find a high-quality solution for path planning and re-planning is necessary. Figure 1 illustrates the challenges that need to be overcome to create a successful real-time autonomous mission planning system for a single UAV system. The main challenges for real-time autonomous system are an efficient path planning and re-planning, ensure risk free for planning while avoiding collisions in the consideration of the UAV constraints such as velocity and/or maximum speed. Finally, the system should consider the safety in the environment which included not only static but also dynamic moving obstacles. Numerous researchers have proposed schemes for real-time mission planning or path planning for mission planning system as a guidance and collision avoidance by combining a graph- or sampling-based path planning process with machine learning methods to create hybrid path planning approaches. These take advantage of a path planner to plan obstacle-free paths more rapidly, obtain a globally planned path, and exploit the benefits of reinforcement learning (RL) to allow the UAV to learn about the environment while traveling to the path in the local local destination while avoiding collisions. Most of the research in this area has focused on three degrees of freedom (3 DOF) vehicles such as ground robots and cars. [PERSON] et al. (2018) proposed a system that aimed to achieve adaptive robot mission planning in a human traffic context, in environments crowded with people. This work focused on planning within a dynamic, complex environment by considering the constraints on UAV's velocity. The authors designed a mission planning system in which local navigation and operation were based on deep RL and global mission planning relied on a topological map. [PERSON] et al. (2018) developed a novel, efficient, real-time hybrid path planning method by combining probabilistic road-map (PRM) path planning with RL to allow ground robots to deal with a dynamic environment. The objective of this method was to improve the adaptability of a PRM global planner in a dynamic environment. [PERSON] et al. (2018) proposed a hybrid path planning approach that combined PRM with RL for aerial cargo delivery tasks. In their approach, an RL agent first learned within an obstacle-free environment by employing a specifically designed feature vector to approximate the value function and to permit generalization beyond the training domain. Their method also considered the RL agent with continuous states as input and discrete actions as output. In addition, the research world could be extended to tasks in environments with static obstacles where the load-displacement must be bounded throughout the trajectory. The sampling-based motion planning is used to generate collision-free paths in the research. [PERSON] et al.