# Position Control of a Mobile Robot through Deep Reinforcement Learning
## 5 Experiments and Results
### Results
#### 5.3.1 One Target Point  
For the case without obstacles, the initial position of the robot is represented by a black circle at \((-0.4,\,0)\), while the target point is represented by a red circle at position \((0.8,\,0)\). The orientation of the robot is indicated by a black arrow with a value of \(-178^{\circ}\) (the values extracted from the experiments are presented in [52]).  
Figure 9 shows a graph of the trajectory followed by the robot for each algorithm. Please observe how the RL, Villela and IPC agents manage to fulfill the task, reaching the desired target. Figure 10 shows the robot trajectory for the RL agents and the control algorithms for the environment with obstacles. A blue circle symbolizes the presence of an obstacle in the path of the robot to the target point. All the algorithms can reach the target, dodging the obstacle by using their sensors. In addition, the control algorithms result in a wider curve than the RL agents, maintaining a greater distance from the obstacle.  
Figure 10: Graph of the trajectory of the experiment with obstacles.  
Figure 9: Graph of the trajectory of the experiment without obstacles.  
To analyze how the angular velocity of the robot varies, Figures 11 and 12 are presented for the environment without and with obstacles, respectively. Figure 11 shows that the control algorithms have similar behaviors, taking approximately 10 s to rotate and orient themselves to the target point. The RL agents are oriented to the target faster, but they continue to maneuver the robot throughout the experiment. These algorithms reach the target point first. For the case with obstacles, Figure 12 shows how the algorithms in the first instance maneuver to orient themselves toward the target and then again to avoid the obstacle. The control algorithms show a slower behavior than the RL agents.  
To perform an analysis of the arrival times at the target point of each algorithm, Tables 2 and 3 are presented, where the best values obtained are marked in bold. In Table 2, the arrival times of the different algorithms at the target point can be observed. Notice how DDPG has the best time (27.4 s), followed by DQN, IPC and, finally, Villela (33.95 s). For the experiment with obstacles, in Table 3, the DDPG reaches the target point very quickly (30.2 s), avoiding the obstacle in a better way than DQN, Villela and IPC Braitenberg. In both experiments, the RL agents perform this task the fastest. All of the algorithms tested in this work are capable of maneuvering the robot by manipulating its linear and angular velocities. The difference in the time it takes for the robot to reach the TP is mainly due to the trajectory but also to the linear velocity that the Khepera robot has. RL agents learn to set the linear velocity to the maximum allowed value almost all of the time, in contrast to control methods such as IPC, which reduce the linear velocity when the angular velocity is non-zero, i.e., when the robot is turning.  
Finally, a comparative table of the control and RL algorithms is made using the performance indices in both environments (with and without obstacles). The indices to be evaluated correspond to the integral of the absolute magnitude (IAE), the integral of the square of the error (ISE), the integral of time multiplied by the absolute error (ITAE) and the integral of time multiplied by the squared error (ITSE). From Table 4, it is observed that the RL agents obtain better results for all the indices, with DDPG having the best performance. Table 5 shows that the DDPG agent again obtains better results for all the indices, followed by DQN, IPC-Braitenberg and Villela-Braitenberg. To summarize the improvement of the RL agents, they have a faster response at the beginning of the experiment, decreasing their distance to the target in the first steps of time, adjusting much better than the control algorithm and reducing the error. DQN and DDPG obtain the best results.