## 5 Experimental Results and Analyses
The switching strategy of the training environment is based on the number of iterations of the interaction with the environment to actually switch to the next new training environment by the sequence of environmental complexity. The training rounds for Environment 1 fall in the range of 0-1000, Environment 2 in the range of 1001-2000, Environment 3 in the range of 2001-3000, Environment 4 in the range of 3001-4000, and Environment 5 in the range of 4001-5000. In addition, it is designed to ensure that the learning process is accelerated, while the training radius is selected within the specific distribution of the target, whose training range is gradually expanded from small to large. In our experiment, the duration of each training round is controlled to less than 5000 times for a quick convergence test in this case. Figure 11: Expanded 6-agent mirror experiment. The different color dotted-line indicate different trajectories of agents, solid circles represent initial positions of different agents and different semi-transparent circles represent targets. A total of 100 rounds were run for each environment, and the time and number of collisions to reach the target point for each round were calculated. The data results are presented in terms of the average time spent and average number of collisions for 100 rounds. The experimental numerical analysis is shown in Tables 3 and 4. It can be seen in Table 3 that the time taken by all the agents to reach their respective destinations varied depending on the environment, and they were all able to reach the target. The time in Table 3 is denoted as the simulated time. Based on the analysis, it was found that the agent operated very efficiently from the start point to the target in this simulated environment. The results show that the more complex the environment, the longer the time that is required. As shown in Table 4, all had a great deal of interaction and training accuracy. In Environments 1 and 2, no collision occurred due to the relative simplicity of the environment. In Environment 3, Environment 4, and Environment 5, because the agent needed to pass through an intermediate region, the chance for collision increased. The worst total collision ratio was 38% percent, i.e., its worst success ratio was 62%. This indicates that the experiment result had a better success rate within than in the model-free multi-agent soft actor-critic (25.9%) and multi-agent model-based policy optimization (MAMBPO) (37.1%) [25]. According to the analysis of the collision probability after learning, more different environments can be redesigned for specific requirements in the future. Figure 12: Training environment with 5 different complexities. The solid circles represent initial positions of different agents and different semi-transparent circles represent targets. * **Case 3: Motion Approximation Experiment**  
The final experimental result of this paper is presented in a video where the operation of each agent's execution state and must-be-allowed policies learned between agents can be observed. The experimental video is divided into two parts; the first one is the experimental video of the four agents ([[https://youtu.be/30](https://youtu.be/30) EO3 bLNnNM]([https://youtu.be/30](https://youtu.be/30) EO3 bLNnNM); accessed on 3 July 2023), and the second part is the experimental video of six agents ([[https://youtu.be/OOachwbKgCI](https://youtu.be/OOachwbKgCI)]([https://youtu.be/OOachwbKgCI](https://youtu.be/OOachwbKgCI)); accessed on 3 July 2023 ). As shown in the videos, one of the training environments was used randomly in each round, and the agents and targets in the environment were randomly generated. The agent is presented by the solid circle color code, and the target points are indicated by the corresponding colors of the translucent circles. The interaction of the four agents in Environment 4 is shown in Figure 13. Figure 13a shows the randomly generated agent and target locations.