# Design of Swarm Intelligence Control Based on Double-Layer Deep Reinforcement Learning
## 3 Double-Layer Deep Reinforcement Learning Design for Swarm Intelligence Control
### Bi-Directional Dynamic Interaction Process and Evolutionary Game Mechanism
Subfigures (**a**–**d**,**g**) use a smoothing factor of 0.8, and subfigures (**e**,**f**,**h**) use a smoothing factor of 0.6.

income gap, with ABC-DRL showing the smallest gap and task income following a normal distribution; (4) Under D-DRL control, most UAVs have a task completion efficiency greater than zero, significantly enhancing the swarm task completion efficiency, enhancing internal collaboration, and task income for individual UAVs are normally distributed, with the highest swarm task income. (2) At the group level, Figure 6a-f analyze the UAV swarm trajectory heatmaps under different control algorithms, the following conclusions can be drawn: (1) Under RBCA control, the UAV swarm trajectory heatmap mainly shows light colors, indicating strong randomness, high exploration, and no significant aggregation effect; (2) Under GA, ABC, and DRL control, UAVs exhibit a clear aggregation effect in areas with higher task generation probability, reducing random exploration and improving efficiency, especially in the ABC algorithm, where trajectory overlaps and aggregation effect are most significant; and (3) Under ABC-DRL and D-DRL control, aggregation areas are dispersed, trajectories are relatively independent, exploration is high, and the trajectories overlap with task distribution. Under D-DRL, the overlap between trajectories and task distribution is more significant, improving the task completion efficiency. Figure 5: Distribution of UAV income and efficiency under different methods (individual level). (**a**) shows the distribution of income and efficiency in RBCA, (**b**) shows the distribution of income and efficiency in GA, (**c**) shows the distribution of income and efficiency in ABC, (**d**) shows the distribution of income and efficiency in DRL, (**e**) shows the distribution of income and efficiency in ABC-DRL, (**f**) shows the distribution of income and efficiency in D-DRL. (3) Figure 7a-f show the mean values of task income, energy cost, task completion amount, task completion efficiency, task throughput, and comprehensive task effectiveness. Figure 6: Heatmap of UAV motion trajectories under different control methods (group level). (**a**) shows the RBCA trajectory heatmap, (**b**) shows the GA trajectory heatmap, (**c**) shows the ABC trajectory heatmap, (**d**) shows the DRL trajectory heatmap, (**e**) shows the ABC-DRL trajectory heatmap, (**f**) shows the D-DRL trajectory heatmap. Figure 7: Comparative analysis chart of indicators of the UAV swarm control system (system level). (**a**) shows the task income, (**b**) shows the energy cost, (**c**) shows the task throughout, (**d**) shows the task completion, (**e**) shows the task completion efficiency, (**f**) shows the comprehensive task effectiveness. under different swarm control algorithms. From the system-level analysis, the following conclusions can be drawn: (1) For task income and energy cost, a higher task income does not necessarily accompany a higher energy cost. The D-DRL and DRL methods exhibit strong learning capabilities and achieve a higher income with a low energy cost. RBCA initially has a high income with a higher energy cost but fails later due to the inability to adjust the strategy. GA and ABC show relatively stable but poor results. ABC-DRL outperforms ABC in terms of learning ability. (2) For task completion amount and task throughput, D-DRL and DRL achieve higher task throughput values compared to other methods, indicating stronger task search and completion capabilities. ABC-DRL improves the task completion efficiency by reducing high-energy tasks and ineffective competition within the swarm. The RBCA method is effective initially, but its performance weakens over time. (3) For task completion efficiency, RBCA outperforms GA and ABC, suggesting that simpler models avoid undesirable evolutionary directions to some extent. ABC alleviates the conflict between energy efficiency and task completion amount in the double-layer learning framework, validating the effectiveness of the double-layer learning architecture.