# Research on Multi-Robot Formation Control Based on MATD3 Algorithm
## 1 Introduction  
The fields of formation control, cooperative control, and swarm control of multi-agent systems have been rapidly developing in recent decades [1]. The research on formation control focuses on making multi-agent systems generate and maintain a predetermined formation structure to perform the corresponding scenario tasks when moving toward a target area. Multi-robot formation control has been widely used in military and firefighting applications, such as cruise reconnaissance, collaborative search and rescue, and transportation [2; 3; 4; 5; 6]. In the early stages of formation control research, researchers applied traditional control methods such as predictive control [7] and sliding mode variable structure control [8] to solve formation problems and developed formation control strategies based on the leader-following method [9], the virtual structure method [10], and the behavior-based method [11]. All of the above traditional control methods require accurate system modeling to calculate the control law, which significantly increases the complexity of the system design. Obstacle avoidance in formation navigation for multiple robots is a fundamental and important capability. The work [12] proposed a distributed control method for the formation navigation task of robot teams that combines constrained nonlinear optimization with consistency to achieve obstacle avoidance in formations. However, this approach needs to avoid the risk of deadlock. The work [13] combined state transition techniques with model predictive control (MPC) to propose a decentralized controller for the formation reconfiguration of a UAV system in an obstacle environment. The experimental results showed that, based on this method, the UAV system can successfully learn an unknown dynamics model and solve the optimal control problem using MPC. Over the years, reinforcement-learning-based solutions to formation control problems have received much attention, with the advantages of avoiding cumbersome dynamics modeling and coping well with changes in unknown environments through self-learning capabilities [14]. The work [15] used the GQ(\(\lambda\)) algorithm to learn leader-following control, which improved the path-planning capability of the followers and was experimented on a physical robot platform. However, only a linear formation structure could be considered. The work [16] improved the actor-critic algorithm by adding an adaptive discriminator to estimate the unknown system dynamics and performed stability analysis. The work [17] improved the DDPG algorithm by applying the advantage function to a multi-USV formation generation and maintenance strategy. The work [18] used the DDPG algorithm to solve the USV formation path-tracking problem by adding a random braking mechanism after action selection, which allowed the USV team to correctly control the velocity when it approached the reference position and prevented it from falling into a local optimum. The work [19] improved the VDN algorithm by proposing a C2 VDN algorithm with cognitive consistency based on the fish-like robot formation control problem, where all robots share state-value network parameters and achieved circular formation control based on a decentralized controller. The work [20] used the DDPG algorithm for formation motion planning and control tasks for TWODMR robot teams and performed comparison experiments with the artificial potential field algorithm. We also refer to [21; 22; 23; 24] for more related research on learning-based methods for different applications. All of the literature above studied and improved the formation control problem in their respective scenarios using reinforcement learning algorithms. However, most of the literature does not consider the detection capability of the agents for obstacles, and there is less research on multi-agent formation navigation strategies in environments with obstacles.