# Joint Optimization of Relay Communication Rates in Clustered Drones under Interference Conditions
## 4 Intelligent Channel Selection Scheme
### Bayesian Optimization Algorithm  
Bayesian Optimization is an efficient global optimization algorithm that can optimally extract hyperparameters with minimal evaluations using a statistical approach. The fundamental principle is based on [PERSON] theorem [26], and its expression is as follows:  
\[p(\mathrm{X}|\Theta)=\frac{p(\Theta|\mathrm{X})p(\mathrm{X})}{p(\Theta)} \tag{47}\]  
In this context, \(\mathrm{X}\) represents the distribution model to be specified, \(\Theta=\{(x_{i},y_{i})|y_{i}=\mathrm{X}(x_{i})\}\) is the set of observed data, \(p(\Theta|\mathrm{X})\) denotes the likelihood distribution, and \(p(\mathrm{X})\) is the prior probability. In this paper, the decision variables \(\{x_{i}\}\) include the discount factor \(\gamma^{u}\), learning rate \(lr^{u}\), and target entropy \(\bar{H}\). The cumulative reward function is the target value, \(y_{i}\). The prior and posterior models use Gaussian distributions for simulation. The acquisition function, based on Expected Improvement function, is used to select the next evaluation point, \(x_{i+1}\), from the posterior distribution, and its expression is as follows:  
\[ \begin{array}{l}x_{i+1}=argmaxEI(x_{i},\Theta)\\ EI(x_{i})=\mathbb{E}\big{[}(max(X(x_{i})-\mathrm{X}^{+}),0)\big{|}x_{i}\big{]} \end{array}  \tag{48}\]  
In this context, \(\mathrm{X}^{+}\) represents the current known maximum value simulated by the Gaussian process. This establishes a probabilistic optimization model concerning the discount factor, learning rate, and target entropy. By continuously selecting the next point through the acquisition function, the process iterates until the optimal solution is found. Summarizing, the implementation process of the BO_DSAC intelligent channel selection scheme is detailed as follows in Algorithm 1. ```
Inputs: discount factor \(\gamma^{u}\), learning rate \(lr^{u}\), and target entropy \(\bar{H}\), each with their respective optimization ranges. Additionally, inputs include the entropy coefficient \(\alpha\), maximum number of training iterations \(num\_episodes\), Markov chain length \(MDPL\), soft update parameter \(\tau^{ls}\), capacity of the replay buffer \(p_{size}\), batch size for data processing \(batch\_size\), and the number of iterations for Bayesian Optimization \(BSN\). These parameters collectively set the framework for optimizing the intelligent channel selection strategy. Output: policy function \(\pi^{B}(\cdot)\) and value function \(Q^{B}(\cdot)\). 1: Inputs discount factor \(\gamma^{u}\), learning rate \(lr^{u}\), and target entropy \(\bar{H}\), each with their respective optimization ranges. Additionally, inputs include the entropy coefficient \(\alpha\), maximum number of training iterations \(num\_episodes\), Markov chain length \(MDPL\), soft update parameter \(\tau^{ls}\), capacity of the replay buffer \(p_{size}\), batch size for data processing \(batch\_size\), and the number of iterations for Bayesian optimization, \(BSN\). These parameters collectively set the framework for optimizing the intelligent channel selection strategy. 2: Initialize the replay buffer \(D_{I}\
eq\varnothing\) and the network parameters of the policy and evaluation networks \(\{\theta^{u},\omega^{u}_{i},i=1,2,3,4\}\). 3:\(for=1\) : \(num\_episodes\)
4: Initial state: \(s=s^{0}\). 5:\(forj=1\) : \(MDPL\)
6: Choose action: \(a\sim\pi(\cdot)\). 7: Perform the action, receive the reward, and form the data tuple:\(<s,a,r,s^{\prime}>\). 8: Update State: \(s=s^{\prime}\). 9: Store the data tuple in the replay buffer: \(D=D\cup<s,a,r,s^{\prime}>\). 10:\(if\ |D_{I}|>p_{size}\)
11: Delete the earliest data tuple from the replay buffer. 12:\(end\) if
```  
**Algorithm 1** BO_DSAC intelligent channel selection scheme The computational overhead of the Bayesian optimized Discrete Flexible Actor-Critic algorithm comprises the following two parts: the Discrete Flexible Actor-Critic algorithm and the Bayesian optimization process.