# Position Control of a Mobile Robot through Deep Reinforcement Learning
## 1 Introduction  
Mobile robots are widely used today and allow for the performance of tasks in environments dangerous for humans--for example, minefields, radioactive areas, deep-sea exploration, etc.--and for access to complex places, such as extraterrestrial exploration and nanorobotics in medicine [1]. The main problem lies in the need for these robots to be able to move around the environment with some autonomy such that they do not require human interaction--that is, they can be controlled automatically by their own intelligent systems to allow them to achieve certain objectives [2]. By combining control and robotics, different strategies can be provided that allow for the efficient control of mobile robots. The foregoing is represented by the use of control algorithms, such as Villela [3] and IPC [4], which seek to manipulate the speeds of the robot by controlling its linear and angular speed. To these algorithms, other systems can be added to strengthen their tasks--for example, obstacle avoidance systems through the use of sensor arrays arranged in the robot [5] and line trackers to improve position control [6,7]. In the previous cases, models of the feedback system are used that allow for the modification of a control law through inputs to calculate the next action and gradually decrease the error associated with its measurements. Additionally, different approaches are available, such as finite-time adaptive fault-tolerant control [8] and adaptive fuzzy control algorithms [9], which have been used for nonlinear systems in mobile robotics, as in [10]. Reinforcement learning (RL) is an area of machine learning that has its origins in the psychological and neuroscientific perspectives of animal behavior [11], where an agent learns to behave in an environment by performing actions and seeing the results of these actions in order to obtain the best expected result. Reinforcement learning has been used to solve different tasks--for example, in games obtaining high performance, as in Go [12], Starcraft [13] and Dota [14]. In the pursuit of making a general-purpose algorithm,DeepMind presented the MuZero [15] agent, which is capable of solving different games such as Go, Chess, Shogi and Atari. RL has been implemented in robotic systems for the precise manipulation of objects [16; 17], and it has also been used to study how to transfer the training learned in simulation to reality (Sim-to-Real) [18; 19; 20]. Currently, RL is a widely used approach for robot control, and, regarding position control in mobile robots, it has been analyzed in [21; 22], where they compare the results with control algorithms. Obstacle evasion has also been studied using different techniques from RL. For example, in [23], the Q-learning algorithm [24] is used, which bases its operation on the use of matrices (Q tables); thus, according to the state in which one is within the environment, observe which action has the best quality, that is, which one delivers the best reward. In [25; 26], the neural Q-learning algorithm [27] is used, an approach quite similar to the previous one but which includes the use of neural networks as a nonlinear approximation to obtain the Q-values. In [28; 29; 30], different approaches of path planning with reinforcement learning are used, and in [31], a mixture of Q-learning and a neural network planner is used to solve route path planning problems. In [32], a version of the deep deterministic policy gradient (DDPG) [33] is implemented, which is used when the environment has a continuous action space. The indicated methods attempt to improve classical path-planning algorithms such as Dijkstra [34], BUG1 and BUG2 [35], and A* [36] has also been used for the navigation task. In this work, we propose the development of an environment to control the position of the simulated mobile robot Khepera IV [37] by using two algorithms of the state of the art of RL--the deep Q-network (DQN) [38] and the deep deterministic policy gradient (DDPG)--and two control algorithms: Villela and IPC.