## 3 Optimal Transmission Policy Based on POMDP  
In this section, we derive the optimal policy for the transmitter that knows the CSI feedback at the end of each slot as shown in Figure 1. The necessary conditions that tell if the action of _betting opportunistically_ should be used under certain SIN communication scenarios was derived, and then the key thresholds for selecting the optimal transmission actions for the SIN communications are derived. From the above discussion, we understand that an optimal policy exists for our POMDP problem as shown in Equation (2), and the expected reward is \(R(X_{i},a_{i})\); if the aggressive action \(A\) is selected, since the probability of the channel in a good state at the next time is \(X_{i}\), then the expected number of successfully transmitted data bits is \(X_{i}R_{g}\); if the conservative action \(C\) is selected, \(R_{b}\) data bits will be transmitted without error; at last, if opportunistically action \(O\) is selected, the expected transmitted data bits are \((1-\tau)[(1-X_{i})R_{b}+X_{i}R_{g}]\). Then, we define the value function \(V_{\beta}(p)\) as \(V_{\beta}(p)=\max\limits_{\pi}V_{\beta}^{\pi}(p)\), for all \(p\in[0,1]\), and \(\pi\) denotes the map from the belief at a particular time to an action in the action space \(\{A,C,O\}\). The value function \(V_{\beta}(p)\) satisfies the Bellman equation  
\[V_{\beta}(X_{i})=\max\limits_{a_{i}\in\{A,C,O\}}\{V_{\beta,a_{i}}(X_{i})\}, \tag{3}\]  
Figure 1: Adaptive data transmission based on delayed CSI. where \(V_{\beta,a_{i}}(X_{i})\) is the value acquired by taking action \(a_{i}\) when the belief is \(X_{i}\). By using the delayed feedback belief probability \(X_{i}=p_{i}\), the transmitter selects the optimal transmission action \(a_{i}\in\{A,C,O\}\) to maximize the throughput with the feedback belief \(X_{i}=p_{i}\) being given by  
\[V_{\beta,a_{i}}(X_{i})=R(p_{i},a_{i})+\beta E[V_{\beta}(X)|X_{i}=p_{i},a_{i}=A], \tag{4}\]  
where \(X_{i}\) is the channel belief of the optimal action which selected at the beginning of the next time slot, and \(X=T(p)=\lambda_{0}(1-p)+\lambda_{1}p=\alpha p+\lambda_{0}\) in which \(\alpha=\lambda_{1}-\lambda_{0}\). Then, \(V_{\beta,a_{i}}(p_{i})\) can be explained for the three possible actions:  
1. _Betting aggressively_: If aggressive action \(A\) is taken, then, the value function evolves as \(V_{\beta,A}(X_{i}=p_{i})=p_{i}R_{g}+\beta V_{\beta}(T(p_{i}))\);
2. _Betting conservatively_: If the conservative action \(C\) is selected, the value function evolves as \(V_{\beta,C}(X_{i}=p_{i})=R_{b}+\beta V_{\beta}(T(p_{i}))\);
3. _Betting opportunistically_: If opportunistic action \(O\) is selected, the value function evolves as \(V_{\beta,O}(X_{i}=p_{i})=(1-\tau)[p_{i}R_{g}+(1-p_{i})R_{b}]+\beta V_{\beta}(T (p_{i}))\). Hence, if the channel belief \(X_{i}=p_{i}\) is probability of the slot \(w_{i}\) in a good state, then the channel belief of next slot \(w_{i+1}\) is \(\lambda_{1}\). Similarly, if the CSI in a bad state is \((1-p_{i})\) in \(w_{i}\), then the channel belief of \(w_{i+1}\) is \(\lambda_{0}\), i.e., \(V_{\beta}(T(p_{i}))=(1-p_{i})V_{\beta}(\lambda_{0})+p_{i}V_{\beta}(\lambda_{1})\), and then we can rewrite the above equations as follows  
\[\left\{ \begin{array}{l}V_{\beta,A}(X_{i}=p_{i})=p_{i}R_{g}+\beta(p_{i}V_{ \beta}(\lambda_{1})+(1-p_{i})V_{\beta}(\lambda_{0}))\\ V_{\beta,C}(X_{i}=p_{i})=R_{b}+\beta(p_{i}V_{\beta}(\lambda_{1})+(1-p_{i})V_{ \beta}(\lambda_{0}))\\ V_{\beta,O}(X_{i}=p_{i})=(1-\tau)[p_{i}R_{g}+(1-p_{i})R_{b}]+\beta(p_{i}V_{ \beta}(\lambda_{1})+(1-p_{i})V_{\beta}(\lambda_{0}))\end{array} \right.. \tag{5}\]  
Finally, the Bellman equation for our POMDP-based transmission policy over Ka-Band channels for SINs can be expressed as  
\[V_{\beta}(X_{i}=p_{i})=\max_{a_{i}\in\{A,C,O\}}\{V_{\beta,A}(p_{i}),V_{\beta,C }(p_{i}),V_{\beta,O}(p_{i})\}. \tag{6}\]  
Moreover, [PERSON] et al. [27] proved that \(V_{\beta}(X_{i})\) is convex and nondecreasing, and there exist three thresholds \(0\leq\rho_{1}\leq\rho_{2}\leq\rho_{3}\leq 1\).