# AidA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV
## 3 Case Study: Mission Planning for an Autonomous UAV
### Experiments and Results  
The overall system was tested based on 50 test cases for both environments. A randomized mission was assigned to the UAV, and as it flew along the planned path generated by the iADA* path planner, the DQN agent continually checked for dynamic obstacles. If the DQN detected a dynamic obstacle along the path, the agent generated an action behavior to safely avoid the obstacle. When the UAV had arrived safely at the waypoint, the iADA* path planner re-planned the path to arrive at the overall goal of the mission. This procedure was applied only to dynamic obstacles; unknown static obstacles were detected using the octree, and the octree map was regenerated to allow the path planner to re-plan the path.  
In this research, experiments were considered into two cases for the mission task cases: success and failure cases. Success cases mean when the UAV will start from the start destination and arrive the goal destination successfully without attacking any obstacle on the way, this case considered as a completed mission. Otherwise, it was failure cases which considered that the UAV could not finish its assigned mission. The case which is considered as a failure case is divided into two cases. For the first case, the algorithm could not find a path between the start and the target destination. In that case, the UAV could not start because there is no path to follow and fail to complete the mission. These kinds of constraints can see if the environment is a GPS denied environment so the goal destination can be on the obstacles or no fly zone in the reality. Another case with consideration as a failure is when the UAV starts the mission and hits the obstacle on the way before it finishes the mission.  
Figure 10: Architecture of the SITL system.  
For the static environment, the success rate is 94%, and, for a total of 50 test cases, there are only three instances of failure when running only the iADA* global path planning and re-planning algorithms in known static environments, as shown in Table 1. In the experimental test cases, the start and goal destinations of the UAV mission are randomly assigned. Failures occurred because the algorithm could not find a path between the start and the target points. The obstacle hitting failure case is not observed because the global path planning and re-planning algorithm is already aware of the environmental information beforehand, and the path planned by the algorithm had already considered the static obstacles. The three cases of failure are eliminated when iADA* is used with DQN and DDPG, since the local planners can travel to the nearest goal point if there is no goal point; hence, although the path is not optimal, there is no failure, and the mission could be achieved successfully.  
The path planning for the UAV may fail within the dynamic environment, but the success rate can be acceptable, with a value of over 90%. Table 2 shows an analysis of the results from three methods: the iADA* algorithm alone, iADA* with DQN, and iADA* with DDPG. In total, we evaluated 50 test cases with 12 dynamic obstacles and 10 static obstacles, over an area of 100 \(\times\) 100 m. For the iADA* algorithm alone, we found 13 cases of failure. The failures are mostly that the UAV is attacked by the obstacles because the iADA* algorithm creates an octree grid using octmap in real time, but, due to the computation time required to create the grid map, the UAV could not avoid moving obstacles and hitting the obstacles. Some small cases of this failure occurred because the UAV could not find a path between the start and goal point, and this is mostly due to the presence of a dynamic obstacle along the path. When a failure is observed due to an inability to find a path, this arose due to the random generation of start and goal locations, in which some points are co-located with obstacles, meaning that the path planner could not find them. As a result, the overall success rate is 74% for 50 test cases, and hence the failure rate is rather high for this low number of test cases.