Endtoend sensorimotor control problems of AUVs with deep reinforcement learning This paper studies on sensorimotor control problems of Autonomous Underwater Vehicles (AUVs) using deep reinforcement learning. We design an endtoend learning architecture mapping original sensor input to continuous control output without referring to the dynamics of vehicles. To avoid difficult and noisy underwater localization, we implement the learning without knowing the positions of AUVs by proposing novel state encoder and reward shaping strategies. Two distinct underwater tasks, obstacle avoidance with sonar sensor and pipeline following with visual sensor, are simulated to validate the effectiveness of proposed architecture and strategies. For the latter, we test the learned policy on realistic images of underwater pipelines to check its generalization ability.

On Training Flexible Robots using Deep Reinforcement Learning The use of robotics in controlled environments has flourished over the last several decades and training robots to perform tasks using control strategies developed from dynamical models of their hardware have proven very effective. However, in many realworld settings, the uncertainties of the environment, the safety requirements and generalized capabilities that are expected of robots make rigid industrial robots unsuitable. This created great research interest in developing control strategies for flexible robot hardware for which building dynamical models are challenging. In this paper, inspired by the success of deep reinforcement learning (DRL), we systematically study the efficacy of policy search methods using DRL in training flexible robots. Our results indicate that DRL is successfully able to learn efficient and robust policies for complex tasks at various degrees of flexibility. We also note that DRL using Deep Deterministic Policy Gradients can be sensitive to the choice of sensors and adding more informative sensors does not necessarily make the task easier to learn.