## 5 Simulation and Analysis  
### Experimental Setup  
The experiment was performed on a desktop machine with 256 GB RAM, one 64-core CPU and one GeForce RTX 3090 GPU (ThinkStation P-, Hefei, China), and the whole framework of the proposed algorithm is shown in Figure 9. The whole experiment was on a two-dimensional plane as \([5000\ \mathrm{m}\ \times\ 5000\ \mathrm{m}]\). There were \(N=4-7\) agents in the plane as a comparison experiment, each with a radius of sensing range \(d_{son}=30\ \mathrm{m}\). The initial formation configurations were set with different numbers of agents \(N=4-7\) as Figure 9 shows; \(N_{obs}=4,6,8\) obstacles were set in Pygame library in PYTHON (3.12.2 AMD64), with the constant velocity (\(v_{obs-x}=5\ \mathrm{m/s}\), \(v_{obs-y}=3\ \mathrm{m/s}\)) motives in four random directions (up, down, left and right) in the two-dimensional plane. In addition, the direction of any obstacles was reset randomly with the detection of the experimental boundaries for obstacles' positions in each time step, which can ensure the obstacles stay within the experimental field. The agents' positions were randomly selected from the box \([-50\ \mathrm{m},\ 50\ \mathrm{m}]\times[-50\ \mathrm{m},\ 50\ \mathrm{m}]\) at the initial stage. The success of configuration change was set as the distance between the center of agent and the obstacle and other agent being less than \(10\ \mathrm{m}\), when the formation still remains the complete structure. Table 1 lists the main parameters for the environment and agents.  
It can be seen from Figure 10 that the whole experimental process contains three major parts, the initial multi-agent formation in the partially known environment, the DMPC module and the MARL process.  
In the MARL process, the actor network utilizes an RNN and target two-layer network. To observe configuration change more effectively, the environment is set not to terminate upon collision. Instead, it only resets when the agent exceeds the maximum number of steps allowed. The Adam optimizer was used to update the neural network parameters  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Agent** & **Symbol** & **Value** \\ \hline Initial linear velocity & \(v_{0}\) & \(2\ \mathrm{m/s}\) \\ Desired linear velocity & \(v_{d}\) & \(45\ \mathrm{m/s}\) \\ Linear velocity & \(v_{\mathrm{max}},\ v_{\mathrm{min}}\) & \(80\ \mathrm{m/s},\ 5\ \mathrm{m/s}\) \\ Acceleration & \(u_{\mathrm{max}},\ u_{\mathrm{min}}\) & \(3\ \mathrm{m/s^{2}},\ -3\ \mathrm{m/s^{2}}\) \\ Maximum angular velocity & \(\omega_{\mathrm{max}}\) & \(10\ \mathrm{rad/s}\) \\ Predictive horizon & \(N_{p}\) & \(10\) \\ Minimum distance & \(R_{s},\ R_{a}\) & \(15\ \mathrm{m},\ 20\ \mathrm{m}\) \\ Obstacle sensing range & \(R_{obs}\) & \(20\ \mathrm{m}\) \\ Obstacle constant velocity & \(v_{x},\ v_{y}\) & \(5\ \mathrm{m/s},\ 3\ \mathrm{m/s}\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: The parameter settings of multi-agent system.  
Figure 9: Illustration of the different numbers of agents 4–7. (**a**) 4 agents (**b**) 5 agents (**c**) 6 agents (**d**) 7 agents.  
after the replay circular buffer accumulated the designated batch size. For a comprehensive overview of the network hyperparameters, refer to Table 2.