## 2 System Model and Problem Formulation
\tag{3}\]  
where \(P_{x}\) is the transmission power of user \(x,x\in\mathcal{N}/n\) and \(d_{x,n}^{-\alpha}\left(t\right)\) denotes the interference distance from user \(x\) to user \(n\), then \(P_{x}d_{x,n}^{-\alpha}\left(t\right)\) can be viewed as the received signal energy from user \(x\) to user \(n\). \(f\left(a_{n}\left(t\right),a_{x}\left(t\right)\right)\) is an indicator function, which depicts the channel occupation of user \(n\), shown as:  
\[f\left(x,y\right)=\left\{ \begin{array}{l}1,\quad x=y,\\ 0,\quad x\
eq y.\end{array} \right. \tag{4}\]  
As is shown in Equation (2), \(Tr_{n}\left(a_{n}\left(t\right),a_{-n}\left(t\right),a_{j}\left(t\right)\right)\) depicts the user \(n\)'s throughput under the threat of malicious jamming and co-channel interference, and in Equation (3), the congestion degree \(I_{n}\left(c_{n},t\right)\) reflects the number of users who are influenced by co-channel interference. Different from [25; 31], this work consider the channel switching of users and introduces the channel switching cost unit \(W_{s}\) to evaluate the performance loss. Moreover, if users cooperate with each other and take actions jointly, a cooperation cost unit \(W_{c}\) is also brought in. The introduction of channel switching cost makes it meaningful to investigate the variation of channel switching number, whereas the introduction of cooperation cost makes it interesting to decide whether to cooperate. Then, as a tradeoff between throughput and its cost, the utility of user \(n\) in one time slot is defined as:  
\[u_{n}\left(a_{n}\left(t\right),a_{-n}\left(t\right),a_{j}\left(t\right)\right) =Tr_{n}\left(a_{n}\left(t\right),a_{-n}\left(t\right),a_{j}\left(t \right)\right)-W_{s}\delta_{s}\left(a_{n}\left(t\right),a_{n}\left(t-1 \right)\right)-W_{c}\delta_{c}\left(I_{n}\left(c_{n},t\right)\right), \tag{5}\]  
where \(\delta_{s}\) and \(\delta_{c}\) are indicator functions for channel switching and cooperation and can be expressed as:  
Figure 1: Cooperative anti-jamming UAV communication networks. \[\delta_{s}\left(a_{n}\left(t\right),a_{n}\left(t-1\right)\right)=\left\{ \begin{array} []{l}1,\ \ \ \ a_{n}\left(t\right)\
eq a_{n}\left(t-1\right),\\ 0,\ \ \ \ a_{n}\left(t\right)=a_{n}\left(t-1\right).\end{array} \right. \tag{6}\]  
\(\delta_{c}\left(I_{n}\left(c_{n},t\right)\right)=\left\{ \begin{array}{l}1, \ \ \ \ I_{n}\left(c_{n},t\right)>1,\\ 0,\ \ \ \ otherwise.\end{array} \right.\)  
\(\delta_{s}=1\) indicates that channel switching occurs at the beginning of the current slot, whereas \(\delta_{s}=0\) means that the user keeps its channel strategy. \(\delta_{c}=1\) indicates that the users are influenced by co-channel interference, cooperate with each other and take joint channel actions, whereas \(\delta_{c}=0\) means users choose channels independently. The optimization object of user \(n\) is:  
\[a_{n}\left(t\right)=\underset{a_{n}\left(t\right)\in\mathcal{M}}{\arg\max}u_{ n}\left(a_{n}\left(t\right),a_{-n}\left(t\right),a_{j}\left(t\right)\right). \tag{7}\]  
Every user in the UAV group wants to employ an optimal anti-jamming channel selection strategy for the purpose of maximizing its utility. However, due to the dynamic feature of the jamming channel and the time-varying locations of UAVs, solving the optimization problem is challenging. Therefore, in the next section, we combine MDP, the Markov game and Q-learning to investigate and solve the anti-jamming channel selection problem in UAV communication networks. **3. Interference-Aware Cooperative Anti-Jamming Mechanism in the UAV Group**  
In this part, the interference-aware cooperative anti-jamming mechanism in the UAV group is designed and analyzed. According to the wideband spectrum sensing and co-channel interference sensing of users, the process of the designed cooperative anti-jamming mechanism is shown in Figure 2.  
The details are shown as follows. After users accomplish transmissions with their receivers, they conduct the step of wideband spectrum sensing (WBSS) and co-channel interference sensing.