## 2 System Model
\begin{array}{l}\frac{T-\tau_{c}}{T}C_{0}\left(1-P_{f}\right)\Pr \left(J\right),~{}~{}~{}~{}(10a)\\ \frac{T-\tau_{c}-\tau_{SR}}{T}C_{1}P_{d}\Pr\left(J\right),~{}~{}~{}~{}~{}~{}~{ }(10b)\\ \frac{T-\tau_{c}-\tau_{c}-\tau_{SR}}{T}C_{2}P_{f}\Pr\left(J\right).~{}~{}~{}~{ }~{}~{}~{}(10c)\end{array} \right. \tag{10}\]  
where  
\[C_{0}=\log_{2}\left(1+\varphi_{0}\right), \tag{11}\]  
\[C_{1}=\log_{2}\left(1+\varphi_{1}\right), \tag{12}\]  
\[C_{2}=\log_{2}\left(1+\varphi_{2}\right). \tag{13}\]  
\(P_{f}\) and \(P_{d}\) are the probability of false alarm and the probability of detection of the sensing mechanism, respectively, according to sensing time duration \(\tau_{s}\). \(Pr\left(\bar{J}\right)\) and \(Pr\left(J\right)\) denote the probability of no jamming and the probability of jamming in the network, respectively. \(C_{0}\), \(C_{1}\), and \(C_{2}\) represent achievable throughput at the destination under the different cases in (10), i.e., direct transmission without jamming (10a), relay-assisted transmission with jamming (10b), and relay-assisted transmission without jamming (10c). The probability of detection and false alarm can be estimated as follows [28]:  
\[P_{d}=Q\left(\frac{\vartheta-M\times(\gamma+1)}{\sqrt{2M\times(2 \gamma+1)}}\right), \tag{14}\]  
and  
\[P_{f}=Q\left(\frac{\vartheta-M}{\sqrt{2M}}\right), \tag{15}\]  
where \(\vartheta\) denotes energy threshold, \(M\) represents for number of sensing samples and can be calculated as \(M=2\tau_{s}f_{s}\) ( \(f_{s}\) is sensing bandwidth), \(\gamma\) is signal-to-noise ratio (SNR) of the sensing channel (i.e., the channel between source and jammer). There are some available researches that propose methods to estimate the SNR value. Therefore, in this paper we assume that the value is available at the source. The probability of false alarm also can be achieved by  
\[P_{f}=Q\left(\sqrt{2\gamma+1}Q^{-1}\left(P_{d}\right)+\sqrt{ \tau_{s}f_{s}}\gamma\right). \tag{16}\]  
In this paper, we assume that the states of the jammer follow a Markov chain model. The states of the jammer changes between the two states, presence (\(J\)) and absence (\(\overline{J}\)), shown in Figure 3. The transition probabilities of the jammer from state \(J\) to state \(\overline{J}\) and from state \(J\) to itself are denoted as \(P_{J\overline{J}}\) and \(P_{J\overline{J}}\), respectively [29]. We assume that the source always has a data packet to transmit to the destination. At the beginning of a time slot, the information about remaining energy of the relay ( \(e^{re}\), \(0\leq e^{re}\leq E_{ca}\)) is assumed to be available at the source. Figure 4 shows the operation process of the system. First of all, the source performs sensing to identify the states ( _\"presence\"_ or _\"absence\"_) of the jammer. If the sensing engine provides the result \"_absence_\", i.e., there is no jamming signal in the current time slot (not always true due to the imperfect sensing), the source will trust the result and then transmits its data directly to the destination. If the source receives an acknowledge (ACK) message after the transmission phase, then the reward is calculated as  
\[R=\frac{T-\tau_{s}}{T}C_{0}. \tag{17}\]  
Figure 3: A Markov chain model of the jammer. In order to formulate the framework of POMDP, we define the state space of the system as \(s=\left\{e_{t}^{re},p_{t}^{b}\right\}\) where \(e_{t}^{re}\) and \(p_{t}^{b}\) are the remaining energy of the relay and the probability of the presence of the jammer in time slot \(t\)th, respectively. Value function \(V_{\left(e^{re},p^{b}\right)}\) represents the maximum total discounted throughput of the system, which is given by  
\[V_{\left(e^{re},p^{b}\right)}=\max_{a_{k}}\left(\sum_{t=k}^{\infty}a^{t-k}R \left(e_{t}^{re},p_{t}^{b},a_{t}\right)\left|e_{k}^{re}=e^{re},p_{k}^{b}=p^{b} \right.\right). \tag{22}\]  
where \(0<\alpha<1\) denotes the discount factor and it is chosen to adjust the impact of future action to current action.