## 4 Simulation Experiment
### Experiment 2: Random Initial Position  
To show the robustness of the proposed MADDPG-E algorithm, here, the random initial position experiments are performed, which means for every episode of training, the positions of the agents, the obstacles, and the target are randomly generated. Figure 8 shows three groups in the multi-agent target search process. It can be seen that although the system starts to learn from different situations, it can still search the target, and the agents learn a good collaborative target search strategy, which proves the strong learning ability of the MADDPG-E algorithm.  
Figure 9 shows the average reward during the training process. It can be seen that in the early stage (before about 6000 episodes of training), the reward curves of all three algorithms fluctuate heavily because of the excessive exploration. However, the MADDPG-E algorithm begins to converge after about 10,000 episodes of training, which is faster than the other two algorithms. Because MADDPG-E adds emotional intrinsic motivation, it can increase the reward that the multi-agent system obtains and can accelerate the learning speed as well as the convergence speed. What is more, compared with MADDPG-ICM and MADDPG, the reward line of MADDPG-E seems smoother, which means its learning result is more stable and less oscillation happens. Figure 10 shows the mean reward value and standard deviation of the system after 5000 episodes of training in experiment 2. What can be seen is that after convergence, the average standard deviation of the reward of the MADDPG-E algorithm is smaller than that of the other two algorithms, indicating that the MADDPG-E model has better stability. The curves of the average score of the three algorithms in experiment 2 are shown in Figure 11. It shows that the MADDPG-E algorithm has a significantly higher average score than the other two algorithms, and the agents successfully search for the target more times. What is more, in the early stage of exploration, MADDPG-E is more stable. Table 5 shows the values of the agent's average reward, average score, and average target search time in 30,000 rounds of training. Compared with the other two algorithms, the MADDPG-E algorithm in this paper searches for the target in a  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Average Reward** & **Average Score** & **Target Search Time** \\ \hline MADDPG-E & 8.94 & 3.67 & 3.84 \\ MADDPG-ICM & 8.55 & 3.59 & 4.28 \\ MADDPG & 6.45 & 2.68 & 4.73 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Average reward value, average score, and average target search time in 30,000 episodes.  
Figure 7: Average score of MADDPG-E, MADDPG, and MADDPG-ICM in experiment 1.  
shorter time and more times, indicating that the MADDPG-E algorithm is more efficient in task learning.  
Compared with the experimental results of the fixed initial position, the MADDPG-E algorithm proposed in this paper has more obvious advantages over the other two algorithms in the complex random initial position experiments, whether it is the size of the reward value after convergence or the convergence speed and stationarity, which indicates that the proposed MADDPG-E algorithm can better adapt to complex and unknown situations.  
Figure 8: Multi-agent target search experiment with randomized initial positions.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Algorithm** & **Average Reward** & **Average Score** & **Target Search Time** \\ \hline MADDPG-E & 9.26 & 3.79 & 3.68 \\ MADDPG-ICM & 8.63 & 3.46 & 4.12 \\ MADDPG & 6.83 & 2.54 & 4.57 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Average reward value, average score, and average target search time in 30,000 episodes.