## 4 Interference-Aware Cooperative Anti-Jamming Distributed Channel Selection Algorithm
### Algorithm Description  
As is shown in Figure 3, the anti-jamming distributed channel selection framework under different cases is depicted. In the left part of Figure 3, the anti-jamming distributed channel selection framework under the influence of co-channel interference is designed. Users in the UAV group adopt a \"joint action-feedback-adjustment\" idea and realize cooperative anti-jamming using multi-agent Q-learning. In this framework, users ought to cooperate with each other to share the joint Q table so that they can take joint actions. In the right part of Figure 3, the anti-jamming framework under the case where users are not influenced by co-channel interference is shown. In this framework, users adopt the single Q-learning mechanism, which uses an \"independent action-feedback-adjustment\" idea. After receiving the immediate reward, each user adjusts its strategy independently. All users in the UAV group ought to sense co-channel interference signal energy to judge whether they are influenced by co-channel interference. The details of interference-aware cooperative anti-jamming distributed channel selection algorithm are shown in Algorithm 1. In addition, the example of the transmission slot structure of users is shown in Figure 4. The heading UAV of each user firstly chooses a channel to transmit to its receivers. After that, the user obtains the feedback from their receivers to judge whether the transmission is successful. In the following part of the slot, a process of wideband spectrum sensing (WBSS) and co-channel interference sensing is conducted for the purpose of acquiring the currently available channel state and sensing the received co-channel interference signal energy. Later, users compare the signal energy to the threshold and make a judgment whether they are influenced by co-channel interference in the current time slot. If users are influenced by co-channel interference, they start cooperating to share the joint Q table with each other via a reliable control channel and take joint channel selection at the beginning of next slot to avoid co-channel interference, if users are not influenced by co-channel interference, they do not share their Q tables and take actions independently. Then, each user starts multi-agent Q-learning or single Q-learning to update their Q tables. During the last process of the slot, users broadcast the following transmission channel to their receivers. **Initialization:**  
Initialize the starting time, ending time and relative learning parameters of the simulation. Initialize every user \(n\)'s joint action Q table \(\widetilde{Q}^{n}\) and single Q table \(Q^{n}\). Set the initial locations and states of all users. **Repeat Iterations:**  
Each user senses and observes the current environment state and then makes a judgment about the co-channel interference according to co-channel interference sensing. If users are under the influence of co-channel interference, go to multi-agent Q-learning. **Multi-agent Q-learning:**  
(1) Each user observes and chooses one transmission channel, using the following rules:  
\(\bullet\) Randomly choose a joint action combination \(\mathbf{a}\) with probability \(\varepsilon\). \(\bullet\) Choosing the best joint action \(\mathbf{a}^{*}\) according to Equation (16), with probability \(1-\varepsilon\). (2) Each user calculates its immediate reward \(r_{t}^{n}\) via joint action and then transfers the environment state. (3) The Q table \(\widetilde{Q}^{n}\) is updated according to Equation (13). Otherwise, go to single Q-learning. **Single Q-learning:**  
(1) Each user observes and chooses one transmission channel, using the following rules:  
\(\bullet\) Randomly choose an independent action \(a^{n}\) with probability \(\varepsilon\). \(\bullet\) Choosing the best action \(a^{n*}\) with probability \(1-\varepsilon\), which realizes the highest Q value in the current state. (2) Each user calculates its own immediate reward \(r_{t}^{n}\) and then transfers the environment state.