## 6 Results  
The project was in Python, allowing full integration with the StableBaselines3, OpenAI Gym, and PyBullet libraries. The documentation and clarity of the code allowed for quick implementation of changes and understanding of the algorithms' operation. Thanks to free libraries and programs, the scheme of action developed in this work provides an accessible introduction to reinforcement learning, enabling the acquisition of knowledge and practical skills. The low entry threshold allows for modifying maps and algorithms and testing different configurations, which favors independent experimentation. This work allows for quick observation of effects, such as drone flights controlled by algorithms, and easy modification of maps and parameters, facilitating independent experiments and testing various configurations. This paper presents numerous simulations in different environmental configurations, gradually introducing elements of increasing complexity, such as obstacles and narrow passages. The main conclusions from the analysis of the experiments are as follows:  
Effectiveness of the algorithms in environments without obstacles:  
* The best-performing algorithms in the environment without obstacles were PPO and A2C, which showed high stability and relatively short training times. * DQN also proved effective, but it tended to have longer training times and more significant variations in results, which means that initial parameters and random simulation conditions may have influenced its effectiveness. Impact of obstacles on the performance of the algorithms:  
* When obstacles were added to the simulation, algorithms operating in a continuous set of actions, such as SAC and TD3, performed better. These algorithms avoided obstacles well and were more resistant to environmental changes. * The average number of actions of the PPO algorithm to reach the goal indicates that the algorithm may need more time to optimize the bypass mechanism. Complex environments (e.g., slalom, four rooms with narrow passages):  
Figure 28: The effect of distance-to-goal reward on training time. * PPO and SAC proved more efficient, especially in complex scenarios like crossing narrow corridors between rooms. These algorithms optimized the drone's path, minimizing time-to-goal and obstacle avoidance. * TD3, while performing well in simpler environments, began to show some limitations, which may be due to difficulties in stably processing environmental changes, especially in situations requiring precise, continuous actions. * In more complex environments, algorithms based on discrete action sets (such as DQN) lost their effectiveness. The high number of time steps needed to complete training indicates that they were less effective in planning a route in a complex environment. Stability and training time:
* PPO and SAC were the most stable, maintaining minor deviations between the minimum and maximum values of the number of steps. They are, therefore, more reliable when trained in different environmental conditions. * A2C showed good stability but required more time to achieve results comparable to PPO and SAC. To sum up, we can say the following:
* PPO and SAC are the most versatile algorithms that can cope with simple and complex environments. They are effective in avoiding obstacles and planning optimal routes. * TD3 works well in environments of moderate complexity but may have difficulties in more complex scenarios. * DQN and A2C are effective in simple tasks but show lower reliability in environments with many obstacles. * Thanks to their stability and adaptability, PPO and SAC algorithms are optimal for drone navigation in environments that may contain obstacles and narrow passages. Testing the algorithms in a simulation environment revealed limitations in discrete action sets. They required longer training time and were susceptible to the problem of local maxima--the agent often fell into a cycle of repeating the same movements.