## 5 Simulation Results and Discussions  
### Simulation Setting  
In the simulation part, a UAV communication network, which consists of three users and one jammer, was investigated. The available channel number for users to access was four. Moreover, we assumed that there existed one dedicated control channel that was reliable for users to share information. The jammer sent a sweeping jamming signal to the available channels, and the jamming signal stayed at one channel for about 2.28 ms. Referring to [24], the transmission time in each user's slot was set to be \(T_{tr}=0.98\) ms, and the time for obtaining feedback, sensing, information sharing, learning and broadcasting were in total \(T_{of}+T_{se}+T_{is}+T_{le}+T_{br}=0.2\) ms in each slot. Other simulation settings are shown as follows. We assumed that the transmission power of each user was 0.1 W, and the initial locations of three users were (100 m, 100 m), (300 m, 800 m) and (150 m, 0 m), respectively. The trajectories of the UAVs are shown in Figure 5, and the flying time was divided into 10 epochs (Note that we designed the trajectories of users for the purpose of supplying a simulation environment, but this does not mean that the location information of users was known in advance. The proposed algorithm had good universality and could work well in high mobility UAV communication networks.). UAVs moved 150 m per epoch, and the duration time of each epoch was set to be 3 s. Furthermore, the pass-loss factor \(\alpha=2\), co-channel interference threshold distance was set to be 400 m; hence, the co-channel interference threshold was \(6.25\times 10^{-7}\) W. We assumed that each user consisted of one heading UAV and one receiver, and the distance between the transmitter and receiver was 20 m. The channel noise power was assumed to be \(-110\) dBm. The total simulation time was equal to the flying time (approximately 30 s). Motivated by [24], \(\lambda_{1}=...=\lambda_{n}=\tilde{\lambda}_{1}=...=\tilde{\lambda}_{n}=0.8\), \(\gamma_{1}=...=\gamma_{n}=\tilde{\gamma}_{1}=...=\tilde{\gamma}_{n}=0.6\), \(\varepsilon=0.1\). In multi-agent Q-learning, we assumed that the joint Q table of each user was a matrix with 64 rows and 16 columns, whereas in single Q-learning, the independent Q table of each user was a matrix with 16 rows and four columns. Each user had to maintain two joint Q tables and one single Q table. When users cooperated with each other to share joint Q table, the data of Q table sharing in every slot were 1024 bytes. If \(T_{is}=0.1\) ms, then the required transmission rate for Q table sharing \begin{table}
\begin{tabular}{c c} \hline \hline
**Computation** & \(\sum\limits_{n=1}^{N}\left\{Ts_{n}\left[O\left(C_{1}\right)+O\left(C_{2} \right)\right]+Tm_{n}\left[O\left(C_{1}\right)+O\left(C_{2}\right)\right]\right\}\) \\ \hline
**Storage Size** & \(N\left(O\left(S_{1}\right)+\left(N-1\right)O\left(S_{2}\right)\right)\) \\ \hline
**Information Sharing** & \(\sum\limits_{n=1}^{N}Tm_{n}O\left(S_{1}\right)\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Complexity analysis of the UAV group. was 9.76 MB/s, which was realistic. Moreover, the normalized switching cost unit \(w_{s}\) and cooperation cost unit \(w_{c}\) varied from zero to 0.3, respectively. Each user's immediate reward can be viewed as the normalized utility, which can be calculated by Equations (10) and (14). In addition, Figure 6 depicts the interference distance of UAVs. In detail, the flying process is divided into four stages. During flying time 0 s to 6 s (the first stage), the distance between User 1 and User 3 was less than 400 m, and they would be influenced by co-channel interference as the received signal energy was higher than threshold. During 6 s to 15 s (the second stage), User 1 and User 3 were influenced by co-channel interference. From 15 s to 21 s (the third stage), all users kept relatively far from each other, so there existed no co-channel interference, while from 21 s to 30 s (the fourth stage), User 2 and User 3 were influenced by co-channel interference.