## 5 Experimental Results and Analyses
The training results are shown in Figure 8. The x-axis is the number of training iterations; the y-axis is the loss value. The loss function is the amount of error calculated by the central critic on the three agents during the centralized training, where the smoothing parameter for the data graph was 0.8. The dotted lines are the original values and the solid lines are the experimental results of loss value with smoothing method. Experimentally, it can be seen that the loss function of the central reward function. The random generation of locations can help the agent to learn different experiences during training. Figure 9 shows the average value of rewards received by all agents in each round, where the x-axis is the number of training iterations, and the y-axis is the reward value. It can be observed that as the number of rounds increased, the number of times the agent interacted with the environment also increased. The agent also learned from the reward values obtained by the environment how to achieve better results from the designed training environment. The simulation results of the actual interaction with the environment are shown in Figure 10, where the red, black, and green agent locations and their target locations are randomly generated in the environment. The target locations are represented as semi-transparent dots of the corresponding colors. Figure 10a shows the initial locations of the randomly generated agents and targets. Figure 10b shows that each agent starts to move towards the target point. Figure 10c shows the black agent moving to the target point and waiting for other agents to reach the target point. Figure 10d shows the target point for all agents. The method of expanding the proxy experiment is to add three more proxies from the previous experiment of three proxies, so there are six proxies in total. The experimental results are shown in the split-mirror diagram in Figure 11. Figure 11a shows the movement of the random generator agent towards the target position. Figure 11b shows the waiting for the end of the round for the agent that has reached the target point and confirming if there is an avoidance action to be performed. Figure 11c shows that the red agent path passes through the blue agent, so Figure 11d shows that the blue agent avoids the red agent  
Figure 10: Mirror image of 3 agents in training environment. The different color dotted-line indicate different trajectories of agents, solid circles represent initial positions of different agents and different semi-transparent circles represent targets. Figure 9: Average round reward value of agents. The dotted and solid line lines present the original and smoothing values, respectively. path. In the left half of Figure 10(e), it can also be observed that the black and blue-green agents are avoiding each other when they meet because they want to reach their respective targets. Figure 10(f) shows the final state of the round. These results show that most of the agents can reach their respective targets, and the red and blue-green agents will terminate the round because they have reached the maximum number of steps in the round. * **Case 2: Experimental analysis of different environments** The aim of this example in analyzing different environments was to determine the effectiveness of the overall system development and operation. The collision rate and average operating time were used as the criteria for evaluating the overall system operation. Figure 12 shows the proposed experiment setup consisting of five training environments of varying environmental complexity. The focus of this paper on training neural networks for DRL is developed in a progressive learning approach. In the training procedure, its sequence is designed to start from the low-complexity Environments 1 and 2 of Figure 12, move into the medium-complexity Environments 3 and 4, and then to the high-complexity Environment 5.