## 3 Optimal Transmission Policy Based on POMDP
Next, we detailed the optimal transmission action in the one-threshold policy region and the two-thresholds policy region in Figure 3, and gave a complete characterization of the thresholds for each policy in the following, respectively. Assume that the one-threshold policy has one threshold \(0<\rho<1\), and the transition probability matrix \(\mathbf{G}\) of the two-state GE channel is \(\mathbf{G}= \begin{bmatrix}\lambda_{1}&1-\lambda_{1}\\ \lambda_{0}&1-\lambda_{0}\end{bmatrix} \). Then, the optimal transmission action \(a_{i}\) is introduced in the following Theorem 2. **Theorem 2**. : _Let \(a_{i}=\{A,C\}\) denote the action space in the one-threshold policy region, \(R_{g}\) and \(R_{b}\) denote the transmitted numbers of data bits that corresponding to the action \(A\) and \(C\), respectively, then \(a_{i}\) is determined as follows:_  
1. _If_ \(R_{b}/R_{g}<\lambda_{0}\)_, then the optimal transmission action is_ \(a_{i}=A\) _regardless of the delayed feedback CSI is_ \(s_{i-1}=1\) _or_ \(s_{i-1}=0\)_;_
2. _If_ \(R_{b}/R_{g}>\lambda_{1}\)_, then the optimal transmission action is_ \(a_{i}=C\) _regardless of the delayed feedback CSI is_ \(s_{i-1}=1\) _or_ \(s_{i-1}=0\)_;_
3. _Finally if_ \(\lambda_{0}\leq R_{b}/R_{g}\leq\lambda_{1}\)_, then the optimal transmission action_ \(a_{i}=A\) _when the delayed feedback CSI is_ \(s_{i-1}=1\)_, and the optimal transmission action is_ \(a_{i}=C\) _when the delayed feedback CSI is_ \(s_{i-1}=0\)_._  
Proof of Theorem 2.: Recall in our POMDP model that any general value function \(V_{\beta}(\cdot)\) is convex. Hence, (1) if \(R_{b}/R_{g}<\lambda_{0}\), when the delayed feedback CSI is \(s_{i-1}=1\), and the channel belief is \(X_{i}=\lambda_{1}\), then we have \(V_{\beta,A}(X_{i}=\lambda_{1})>V_{\beta,C}(X_{i}=\lambda_{1})\) since  
\[\left\{ \begin{array}{l}V_{\beta,A}(X_{i}=\lambda_{1})=\lambda_{1}R_{g}+\beta (\lambda_{1}V_{\beta}(\lambda_{1})+(1-\lambda_{1})V_{\beta}(\lambda_{0})),\\ V_{\beta,C}(X_{i}=\lambda_{1})=R_{b}+\beta(\lambda_{1}V_{\beta}(\lambda_{1})+( 1-\lambda_{1})V_{\beta}(\lambda_{0})),\end{array} \right. \tag{11}\]  
and \(\lambda_{1}>\lambda_{0}>R_{b}/R_{g}\). Similarly, when the delayed feedback CSI is \(s_{i-1}=0\), we still have \(V_{\beta,A}(X_{i}=\lambda_{0})>V_{\beta,C}(X_{i}=\lambda_{0})\) since  
\[\left\{ \begin{array}{l}V_{\beta,A}(X_{i}=\lambda_{0})=\lambda_{0}R_{g}+\beta (\lambda_{0}V_{\beta}(\lambda_{0})+(1-\lambda_{0})V_{\beta}(\lambda_{0}))\\ V_{\beta,C}(X_{i}=\lambda_{0})=R_{b}+\beta(\lambda_{0}V_{\beta}(\lambda_{0})+( 1-\lambda_{0})V_{\beta}(\lambda_{0}))\end{array} \right.. \tag{12}\]  
Hence, the optimal transmission action in this case is \(a_{i}=A\). (2) If \(R_{b}/R_{g}>\lambda_{1}\), similar to the previous case, regardless of the delayed feedback CSI is \(s_{i-1}=1\) or \(s_{i-1}=0\) (i.e., the channel belief is \(X_{i}=\lambda_{1}\) or \(X_{i}=\lambda_{0}\), respectively), we have \(V_{\beta,A}(p_{i})<V_{\beta,C}(p_{i})\), therefore, by substituting \(p_{i}\) into Equations (11) and (12) directly. Therefore, the action \(a_{i}=C\) is optimal in this case. (3) If \(\lambda_{0}\leq R_{b}/R_{g}\leq\lambda_{1}\), the approach here is similar to the previous cases, i.e., when the delayed feedback CSI is \(s_{i-1}=1\) and \(p_{i}=\lambda_{1}\), we will have \(V_{\beta,A}(X_{i}=\lambda_{1})>V_{\beta,C}(X_{i}=\lambda_{1})\) by substituting \(p_{i}=\lambda_{1}\) into Equation (11), and the action \(a_{i}=A\) is the optimal strategy here; otherwise, when the delayed feedback CSI is \(s_{i-1}=0\) and \(p_{i}=\lambda_{0}\), we consequently obtain \(V_{\beta,A}(X_{i}=\lambda_{0})<V_{\beta,C}(X_{i}=\lambda_{0})\) by substituting \(p_{i}=\lambda_{0}\) into Equation (12) and the optimal transmission action in this case is \(a_{i}=C\). The complete characterization of the optimal transmission action of the one-threshold policy is given in Table 1.  
Furthermore, it is worth noting that Theorem 2 proves that the value function is totally determined by finding \(V_{\beta}(\lambda_{1})\) and \(V_{\beta}(\lambda_{0})\).