# Autonomous Navigation of Robots: Optimization with DQN
## 6 Discussion of Results
### DQN Scenarios Validation  
To evaluate the effectiveness of the DQN algorithm for path determination, a series of experiments were conducted using a Turtlebot3 Burger in a Gazebo simulation environment. In each scenario, the robot was placed in the same starting position, while a cube represented the goal to be reached. The Lidar sensor obtained information about the surrounding obstacles and walls. The first scenario was a simple one without any obstacles. The DQN algorithm was run for a total of 200 episodes, during which the robot was able to reach the goal successfully. Initially, the robot collided with the environment, resulting in a negative reward. However, as the algorithm learned, the reward level increased to almost 800 points, showing a clear increasing trend. This indicates that the DQN algorithm is capable of learning from past experiences, resulting in a robot that can autonomously navigate toward the goal while avoiding collisions. See Figure 19.  
The success of the DQN algorithm in the first scenario highlights its ability to learn from past experiences to improve future decision making. During each episode, the DQN algorithm collects sensory inputs from the environment, such as Lidar scans, and generates actions to control the robot's movements towards the goal. As the algorithm interacts with the environment, it receives rewards that are used to update the Q-value function, which is used to select the best actions in a given state. In the case of the first scenario, the negative reward signal received from the robot's initial collision with the environment helped the algorithm learn to avoid collisions in the future shown in Figure 20. The increasing trend in the reward level throughout the episodes shows that the algorithm was able to learn an effective policy for navigating toward the goal while avoiding collisions. Figure 18: Comparison between the optimizers Adam, RMSprop, and SGD. In the second scenario (See Figure 21), the environment was made more complex by adding four static obstacles, which made navigation for the robot more challenging. However, after 300 episodes, it can be concluded that the DQN algorithm enables the robot to learn effectively. Throughout the training process, the robot learned to avoid collisions and choose the correct path toward the goal. Figure 19: Learning and performance improvement in the initial scenario: validation results. Figure 20: DQN algorithm reward evolution for the first validation scenario. As with any training process, the robot initially received negative rewards when it chose the wrong path or collided with an obstacle. However, the trend was positive, and the robot's score increased to nearly 1500 positive points. This validates the successful performance of the DQN algorithm in the second scenario shown in Figure 22.  
In the third scenario, shown in Figure 23, the same obstacles as in the previous scenario were added, but now they are mobile and follow a circular trajectory around the center of the environment. These conditions make it more challenging for the robot to learn, as there is a higher probability of collision, and it must choose the correct path to avoid low scores. After 400 episodes, it can be concluded that the DQN algorithm allowed the robot to learn effectively in this scenario as well. The robot was able to avoid collisions, resulting in a score of over 3000 points, and the trend was positive throughout the training process. Initially, the robot received negative rewards (See Figure 24), indicating collisions in the early stages of training. However, as the robot learned, the trend became increasingly positive. The DQN algorithm was still able to learn from past experiences and adapt to the changing environment, resulting in successful path determination. The fact that the robot was able to achieve a higher score in the third scenario, despite the added difficulty, shows the potential of the DQN algorithm to handle complex and dynamic environments.