## 3 Bio-Inspired Fission-Fusion Control and Planning via Reinforcement Learning Algorithm
### Sub-Swarm Confrontation via Reinforcement Learning
```
Input:\(\mathbf{t}_{t}^{parent-swarm}\), \(\mathbf{c}_{t}^{sub-swarm}\), \(\mathbf{c}_{t}^{ enemy}\), \(\mathbf{c}_{t}^{target}\) Output:\(u_{lur}\) Function: sub-swarm Confrontation with dynamic obstacles  Initialize parameter vectors \(\mathbf{\psi}\), \(\overline{\mathbf{\psi}}\), \(\mathbf{\theta}\), \(\mathbf{\phi}\)  Initialize replay buffer \(\mathbf{\mathcal{D}}\) for each iteration do  for each time step do  Sample \(\mathbf{a}_{t}\) according to \(\mathbf{a}_{t}\sim\pi_{\mathbf{\phi}}(\mathbf{a}_{t}|\mathbf{s}_{t})\) from target network  Observe \(\mathbf{s}_{t+1}\) according to \(\mathbf{s}_{t+1}\sim p(\mathbf{s}_{t+1}|\mathbf{s}_{t}\), \(\mathbf{a}_{t})\) from transition  probability distribution  Obtain \(\mathbf{r}_{t}=\mathbf{r}_{s2e}+\mathbf{s}_{2t}+\mathbf{r}_{e2p}\) from the environment  Store transitions to replay buffer \(\mathbf{\mathcal{D}}\) endfor for each gradient step do  Update \(\mathbf{\psi}\), \(\mathbf{\theta}\), \(\mathbf{\overline{\mathbf{\psi}}}\) in turn endfor endfor endfunction
```  
**Algorithm 3**Sub-swarm confrontation algorithm  
``` Input:\(\mathbf{c}_{t}^{parent-swarm}\), \(\mathbf{c}_{t}^{sub-swarm}\), \(\mathbf{c}_{t}^{ enemy}\), \(\mathbf{c}_{t}^{target}\) Output:\(u_{lur}\) Function: sub-swarm Confrontation with dynamic obstacles  Initialize parameter vectors \(\mathbf{\psi}\), \(\overline{\mathbf{\psi}}\), \(\mathbf{\theta}\), \(\mathbf{\phi}\)  Initialize replay buffer \(\mathbf{\mathcal{D}}\) for each iteration do  for each time step do  Sample \(\mathbf{a}_{t}\) according to \(\mathbf{a}_{t}\sim\pi_{\mathbf{\phi}}(\mathbf{a}_{t}|\mathbf{s}_{t})\) from target network  Observe \(\mathbf{s}_{t+1}\) according to \(\mathbf{s}_{t+1}\sim p(\mathbf{s}_{t+1}|\mathbf{s}_{t

## 4 Simulation Studies

### Swarm Campaign Evaluation Metrics

#### 4.1.1 Order Parameters

The concept of swarm motion order proposed by [PERSON] [38] is typically assessed using an order parameter. In this section, we employ a set of quantitative metrics as order parameters to analyze the fission-fusion motion of the UAV swarm. Polarization Index. \(\varphi\in[0,1]\) denotes the degree to which all drones tend to move in the same direction at this moment. Given the introduction of stochastic disturbances, we have established a benchmark (\(\varphi_{flock}\)) for the polarization index. When surpassing \(\varphi_{flock}\), it indicates the formation of a stable swarm. \[\varphi=||\sum_{i=1}^{N_{i}^{\omega}}\frac{\varphi_{i}^{\omega}}{||v_{i}^{ \omega}||}||\frac{1}{N_{i}^{\omega}} \tag{19}\]

where \(v_{i}^{\omega}\in R^{3}\) is the velocity vector. Differentiation Index. The single polarization index does not adequately capture the movement characteristics of the swarm during sub-swarm movements. To address this limitation, we incorporate the differentiation index [17] to assess the velocity variation among agents within the swarm, defined as follows:

\[\lambda=\frac{\Gamma^{2}+1}{\hbar} \tag{20}\]

\[\Gamma=\frac{1}{N}\sum_{i=1}^{N}{(v_{i}-\overline{v})^{3}}/{(\frac{1}{N}\sum_ {i=1}^{N}{(v_{i}-\overline{v})^{3}})^{-\frac{3}{2}}} \tag{21}\]

\[\hbar=\frac{1}{N}\sum_{i=1}^{N}{(v_{i}-\overline{v})^{4}}/{(\frac{1}{N}\sum_ {i=1}^{N}{(v_{i}-\overline{v})^{2}})^{-2}} \tag{22}\]

where \(\Gamma\) and \(\hbar\) indicate the skewness and kurtosis of the velocity distribution of the UAV swarm, respectively, for differentiation index \(\lambda\in[0,1]\), when \(\lambda=1\) denotes two independent Renoulli distributions signifies the complete fission of velocities; In this study, when \(\lambda>0.9\) we determine that two independent swarms have been formed. #### 4.1.2 Performance Evaluation Index

To assess the effectiveness of the swarm in resisting the interference from dynamic obstacles, this section proposes the precision of stimuli as an indicator of the swarm's resilience to interference. Additionally, we establish a communication load to quantify the communication pressure. Precision of Stimuli. The precision of stimuli denotes the proximity of the swarm's motion direction to that of dynamic obstacles.