  In this paper, the circle formation control problem is addressed for a group
of cooperative underactuated fish-like robots involving unknown nonlinear
dynamics and disturbances. Based on the reinforcement learning and cognitive
consistency theory, we propose a decentralized controller without the knowledge
of the dynamics of the fish-like robots. The proposed controller can be
transferred from simulation to reality. It is only trained in our established
simulation environment, and the trained controller can be deployed to real
robots without any manual tuning. Simulation results confirm that the proposed
model-free robust formation control method is scalable with respect to the
group size of the robots and outperforms other representative RL algorithms.
Several experiments in the real world verify the effectiveness of our RL-based
approach for circle formation control.
