A Feasibility Evaluation of a Reinforcement Learning Based Path Planning Algorithm for UAV in Indoor Environment
Creators: name: Cochachin, Andres
Publisher: name: Torrens University Australia
Year: 2024
Subjects: classificationCode: 461105
subject: Reinforcement learning

description: The aerial robot navigation research area is challenging due to the many elements and tasks involved. Path planning is one key element of robot navigation that searches an optimum and safe pathway without colliding with static and dynamic obstacles. In case of path planning algorithm for unmanned aerial vehicles (UAV), many authors have studied and proposed a hybrid approach: local path planning for short-term planning with a Reinforcement Learning (RL)-based algorithm, and global path planning that considers the whole area of the operational environment and mostly achieved with a classical method. Recent studies have indicated that this approach can be successful to be implemented in real scenarios. Indeed, many authors have evaluated their proposals by simulations, showing promising results. Though, only few studies have been experimented it in real scenarios. In that regard, this thesis has the aim of designing of a computer simulation and evaluating the practical implementation feasibility of a specific path planning algorithm based on the classical A star and Reinforcement Learning. To this end, ninety different evaluations were performed in python scripts followed by their corresponding realistic simulations by employing the Gazebo simulator and three SITL drone models from the PX4 Framework. From these, nine numerical indicators are obtained for comparison and evaluation through linear regression analysis and hypothesis testing. The results confirm the existence of a positive linear trend between the map complexity and the training time, path length, flight time, and state collision percentage metrics. On the other hand, a negative linear trend was detected between the map complexity and the traversability and average avoidance length metrics. The other metrics related to the RL-training stage did not show a linear relationship. Likewise, multiple regression analysis revealed the importance of drone radius and braking distance in the calculation of the three metrics related to the operational environment. These results are used to determine the feasibility of the path planning algorithm applied to indoor scenarios of different complexity levels. The results of this thesis aim to contribute to the reality gap reduction between the theory and real implementation outcomes with respect to RL-based path planning solutions for UAVs. Particularly, this work presents an evaluation not only from the RL algorithm performance point of view as performed by many studies, but also Page 3 considering some real drone features and relating them to the structure of the operational environment. It is expected that the results can help to companies, engineers, and researchers to take better decisions when it comes to selecting a drone to operate safely in a customized indoor environment, thus encourage them to take the challenge of operating drones inside closed areas. For further research, the use of Gazebo simulator on the training phase with more powerful computing resources will provide a higher accuracy in the results and more insights can be obtained from its implementation. In addition to that, the assessing of the global path planning with other improved methods is expected to reduce the path length which in turn can reduce the chance of collision.