Motion Coordination of Multiple Robots Based on Deep Reinforcement Learning Multirobot motion coordination is a sequential decision problem that controls each robotu0027s action through a coordination mechanism to avoid collisions and reach their respective destinations. This problem can be viewed as a Markov Decision Process. Therefore, under determinate conditions of the motion path, we make the attempt of applying deep reinforcement learning (DRL) to solve the motion coordination problems. In this paper, in order to make this problem solvable through RL methods, we design a Path Checkerboard Diagram (PCD) to represent the collision relationship among different paths and determine whether collisions occur when the agent takes action. Technically, we design the states, actions and reward function. However, the traditional Deep QNetworks (DQN) algorithm has the problem of overestimation of stateaction values and high approximation error variance of the target values. To fit the motion coordination problem scenario, we further propose a new algorithm MultiLoss Double DQN (MLDDQN) based on the Double DQN (DDQN), which minimizes the weighted sum of loss of different target networks we have learned before. We evaluate the performance of our algorithm through the different motion coordination tasks. Results suggest that our MLDDQN method yields remarkable improvements in terms of the success rate, success steps and the stability of training process compared with the original DDQN algorithm.

Adaptive LeaderFollower Formation Control and Obstacle Avoidance via Deep Reinforcement Learning We propose a deep reinforcement learning (DRL) methodology for the tracking, obstacle avoidance, and formation control of nonholonomic robots. By separating visionbased control into a perception module and a controller module, we can train a DRL agent without sophisticated physics or 3D modeling. In addition, the modular framework averts daunting retrains of an imagetoaction endtoend neural network, and provides flexibility in transferring the controller to different robots. First, we train a convolutional neural network (CNN) to accurately localize in an indoor setting with dynamic foregroundbackground. Then, we design a new DRL algorithm named Momentum Policy Gradient (MPG) for continuous control tasks and prove its convergence. We also show that MPG is robust at tracking varying leader movements and can naturally be extended to problems of formation control. Leveraging reward shaping, features such as collision and obstacle avoidance can be easily integrated into a DRL controller.