# AidA*-RL: Anytime Graph-Based Path Planning with Deep Reinforcement Learning for an Autonomous UAV
## 3 Case Study: Mission Planning for an Autonomous UAV
### Discussions and Future Work  
Using a complete framework, we ran experiments with a minimum of 50 test cases and analyzed the results in terms of the success rates for static and dynamic environments, using three different methods: iADA* alone, iADA* with DQN, and iADA* with DDPG When iADA* was implemented with DQN, the results were fairly good, with a total success rate of 94% and only three cases of failure. A summary of the results for the dynamic obstacles involved environment can be seen in Table 2. The high success rate was attained because DQN is a local path planner, and the avoidance of collisions allowed the iADA \begin{table}
\begin{tabular}{c c c c} \hline \hline  & \multicolumn{4}{c}{**Algorithms**} \\ \cline{2-4}  & **iADA *** & **iADA * + DQN** & **iADA * + DDPG** \\ \hline Success & 47 & 50 & 50 \\ \hline Failure & 3 & \(-\) & \(-\) \\ \hline Success rate & 94\% & 100\% & 100\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of success rates for different approaches for 50 test cases in a static environment. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Task**} & \multicolumn{4}{c}{**Algorithms**} \\ \cline{2-4}  & **iADA *** & **iADA * + DQN** & **iADA * + DDPG** \\ \hline Success & 37 & 47 & 45 \\ \hline Failure & 13 & 3 & 5 \\ \hline Success rate & 74\% & 94\% & 90\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Comparison of success rates for different approaches for 50 test cases in a dynamic environment. algorithm to find paths more easily. In this approach, each case of failure was due to a collision with a dynamic obstacle. Failure arose because, when the collisions were moving in a direction in which the UAV was not heading, the UAV's sensor could not identify them. Finally, when iADA* was implemented with DDPG, the results were also fairly good, with a success rate of 90% and five cases of failure. All of these failures were due to unavoidable dynamic obstacles. This approach therefore achieved a better result than iADA* alone but was less effective than when DQN was used. This was because the controller used for the simulator also needs to be considered. DQN is more effective when the system has a low-level controller in comparison with DDPG, and this may be the reason that DQN outperformed DDPG. By comparing one of the highest results and recent papers from the research which used PRM-RL [10] applied for the ground robot, in the dynamic obstacle cases, it obtained 82% accuracy per 20 test cases and, for this current paper's research, the experiment achieved a success rate of greater than 90% in the 50 test cases. Based on this fact, for the iADA*-RL algorithm based on iADA* with DQN and DDPG learning, the results were acceptable for a complex, dynamic indoor environment. The results from RL learning also depend on the parameter setup and training time, and the results presented here could be improved by setting up the parameters carefully and enhancing the currently trained model. Simulations and experiments were carried out to test our scheme using a real-time simulation platform, and it was shown that the proposed mission planning system was effective in an environment with complex obstacles. The autonomous mission planning system was successfully implemented and tested and was shown to avoid obstacles in a real-time simulation environment. Since we have achieved many successes and improvements on this research problem in the simulation environment, we could extend our proposed research system in the real-world environment with practical UAVs. For the future work of this research, there are still several problems that remain with our approach. Firstly, the collision avoidance scheme for moving objects needs to be improved to include not only a single sensor but also a 360\({}^{\circ}\)-view sensor. For the moving obstacles, the environment used here for testing contained dynamic obstacles that moved with constant directions and speeds and should be updated to consider more realistic objects.