# Design of Swarm Intelligence Control Based on Double-Layer Deep Reinforcement Learning
## 3 Double-Layer Deep Reinforcement Learning Design for Swarm Intelligence Control
### Bi-Directional Dynamic Interaction Process and Evolutionary Game Mechanism
Specifically, the UAVs' movement and task-related energy costs follow [27; 28]; the task value distribution is drawn from [28; 29]; the task respawn probability is based on [29; 30]; and environmental dynamics (e.g., task generation rule) are modeled using sinusoidal functions inspired by [30]. The parameters listed in Table 4 can be categorized into two types: the first comprises core parameters (e.g., the number of UAVs, UAVs' performance, UAVs' vision, running time, single-period revenue, and the nonlinear parameter of efficiency), which are directly derived from the actual operational data of the UAV swarm to ensure consistency with real-world mission environments; the second consists of secondary parameters, which are set based on the existing literature to prevent the disclosure of sensitive data. Since the core parameters remain unchanged and the replacement of secondary parameters does not affect the overall modeling approach or experimental design, this adjustment has a limited impact on the validity and applicability of this study's conclusions while ensuring the scientific rigor and verifiability of the model. To enhance the realism and traceability of the simulation environment, this study adopts a hybrid parameterization approach that combines empirical data with established modeling principles. Compared to the existing works that often utilize simplified or static formulations, the proposed model introduces dynamic and context-aware mechanisms. This \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Algorithm** & **Architecture** & **Optimization Goal** & \begin{tabular}{c} **Double-Layer** \\ **Structure** \\ \end{tabular} & \begin{tabular}{c} **Characteristics/Performance** \\ **Comparison** \\ \end{tabular} \\ \hline RBCA [15] & Centralized & \begin{tabular}{c} Comprehensive \\ task efficiency \\ \end{tabular} & Single layer & \begin{tabular}{c} Suitable for coordinated tasks \\ but lacks flexibility \\ \end{tabular} \\ GA [19] & Distributed & \begin{tabular}{c} Comprehensive \\ task efficiency \\ \end{tabular} & Single layer & \begin{tabular}{c} High randomness, unsuitable \\ for complex searches \\ \end{tabular} \\ ABC [20] & Distributed & \begin{tabular}{c} Comprehensive \\ task efficiency \\ \end{tabular} & Single layer & \begin{tabular}{c} Efficient local search, weaker \\ global optimization \\ Strong learning ability, but \\ incomplete separation of \\ individual and \\ global optimization \\ \end{tabular} \\ ABC-DRL [20; 26] & Hybrid & \begin{tabular}{c} Comprehensive \\ task efficiency \\ \end{tabular} & Single layer & \begin{tabular}{c} Simple learning ability, but \\ incomplete separation of \\ individual and \\ global optimization \\ \end{tabular} \\ ABC-DRL & Hybrid & \begin{tabular}{c} Inner: task completion \\ amount; Outer: \\ comprehensive \\ task efficiency \\ \end{tabular} & Double layer & \begin{tabular}{c} Good for individual \\ optimization, but weak in \\ global optimization \\ \end{tabular} \\ D-DRL & Hybrid & \
\begin{tabular}{c} amount; Outer: \\ comprehensive \\ task efficiency \\ \end{tabular} & Double layer &
\begin{tabular}{c} Combines individual and \\ global optimization, \\ superior performance \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparative summary of UAV swarm control algorithms. integration of real-system-informed and literature-guided components enables a parameter setting framework that is both practically grounded and academically rigorous. Here, U denotes Uniform Distribution and N denotes Normal Distribution. Specifically, the PPO algorithm in this project was implemented using the RLlib library. The environment for simulating the UAV swarm control system is set up using the Gym library. The detailed parameters of D-DRL training are shown in Table 5.