## 3 Optimal Transmission Policy Based on POMDP
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Conditions** & **Corresponding Value Functions** & **Closed Form Computation Expressions** \\ \hline \multirow{2}{*}{\(\rho<\lambda_{0}\)} & \(V_{\beta}(\lambda_{0})=V_{\beta,A}(X_{i}=\lambda_{0})\) & \(V_{\beta}(\lambda_{0})=\frac{\lambda_{0}R_{g}}{(1-\beta)(1-\alpha\beta)}\) \\  & \(V_{\beta}(\lambda_{1})=V_{\beta,A}(X_{i}=\lambda_{1})\) & \(V_{\beta}(\lambda_{1})=\frac{(\lambda_{1}-\alpha\beta)R_{g}}{(1-\beta)(1-\alpha \beta)}\) \\ \hline \multirow{2}{*}{\(\lambda_{0}\leq\rho\leq\lambda_{1}\)} & \(V_{\beta}(\lambda_{0})=V_{\beta,C}(X_{i}=\lambda_{0})\) & \(V_{\beta}(\lambda_{0})=\frac{((1-\beta_{1})R_{b}+\beta\lambda_{0}\lambda_{1}R_{g} }{(1-\beta)(1-\alpha\beta)}\) \\  & \(V_{\beta}(\lambda_{1})=V_{\beta,A}(X_{i}=\lambda_{1})\) & \(V_{\beta}(\lambda_{1})=\frac{\beta(1-\lambda_{1})R_{b}+(1-\beta+\beta_{0})R_{g} }{(1-\beta)(1-\alpha\beta)}\) \\ \hline \multirow{2}{*}{\(\lambda_{1}<\rho\)} & \(V_{\beta}(\lambda_{0})=V_{\beta,C}(X_{i}=\lambda_{0})\) & \(V_{\beta}(\lambda_{0})=\frac{R_{g}}{(1-\beta)}\) \\  & \(V_{\beta}(\lambda_{1})=V_{\beta,C}(X_{i}=\lambda_{1})\) & \(V_{\beta}(\lambda_{1})=\frac{R_{g}}{(1-\beta)}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Closed form computation expressions of the one-threshold policy. _If_ \(R_{b}/R_{g}<\lambda_{0}\)_, then two cases can be distinguished: if_ \(R_{b}/R_{g}<\tau X_{i}/((1-\tau)(1-X_{i}))\)_, the optimal transmission action is_ \(a_{i}=A\)_, regardless of the delayed feedback CSI being_ \(s_{i-1}=1\) _or_ \(s_{i-1}=0\)_; else, if_ \(R_{b}/R_{g}\geq\tau X_{i}/\big{(}(1-\tau)(1-X_{i})\big{)}\)_, the optimal transmission action is_ \(a_{i}=O\)_, regardless of the delayed feedback CSI being_ \(s_{i-1}=1\) _or_ \(s_{i-1}=0\)_._
2. _If_ \(R_{b}/R_{g}>\lambda_{1}\)_, then two cases can be distinguished: if_ \(R_{b}/R_{g}<(1-\tau)X_{i}/(\tau+X_{i}-\tau X_{i})\)_, the optimal transmission action is_ \(a_{i}=O\)_, regardless of the delayed feedback CSI being_ \(s_{i-1}=1\) _or_ \(s_{i-1}=0\)_; else, if_ \(R_{b}/R_{g}\geq(1-\tau)X_{i}/(\tau+X_{i}-\tau X_{i})\)_, the optimal transmission action is_ \(a_{i}=C\) _regardless of the delayed feedback CSI being_ \(s_{i-1}=1\) _or_ \(s_{i-1}=0\)_._
3. _Finally if_ \(\lambda_{0}\leq R_{b}/R_{g}\leq\lambda_{1}\)_, when the delayed feedback CSI is_ \(s_{i-1}=1\) _and_ \(X_{i}=\lambda_{1}\)_, then the optimal transmission action is_ \(a_{i}=A\)_; when the delayed feedback CSI is_ \(s_{i-1}=0\) _and_ \(X_{i}=\lambda_{0}\)_, then the optimal transmission action is_ \(a_{i}=C\)_._  
Proof of Theorem 3.: The proof here needs to utilize the previous case in Theorem 1; recall in our POMDP model that any general value function \(V_{\beta}(\cdot)\) is convex, the interval [0, 1] of \(R_{b}/R_{g}\) is separate in three regions by the thresholds \(\rho_{1}\) and \(\rho_{2}\), all the six possible optimal policy structures of the two-thresholds policy are illustrated in Figure 4, and, then, we can distinguish them into three possible scenarios. If \(R_{b}/R_{g}<\lambda_{0}\), we can distinguish three subcases:  
If \(\rho_{2}<\lambda_{0}\) as shown in Figure 4a, then we have that the optimal action is \(a_{i}=A\) for \(s_{i}=0/1\) where \(V_{\beta,A}(X_{i}=\lambda_{0})>V_{\beta,O}(X_{i}=\lambda_{0})\) and \(V_{\beta,A}(X_{i}=\lambda_{1})>V_{\beta,O}(X_{i}=\lambda_{1})\), and since  
\[\left\{ \begin{array}{l}V_{\beta,A}(X_{i}=p_{i})=p_{i}R_{g}+\beta(p_{i}V_{ \beta}(\lambda_{1})+(1-p_{i})V_{\beta}(\lambda_{0}))\\ V_{\beta,O}(X_{i}=p_{i})=(1-\tau)[p_{i}R_{g}+(1-p_{i})R_{b}]+\beta(p_{i}V_{ \beta}(\lambda_{1})+(1-p_{i})V_{\beta}(\lambda_{0}))\end{array} \right.. \tag{17}\]  
Hence, the optimal transmission action in this case is \(a_{i}=A\) for \(R_{b}/R_{g}<\tau\lambda_{0}/((1-\tau)(1-\lambda_{0}))\) and \(R_{b}/R_{g}<\tau\lambda_{1}/((1-\tau)(1-\lambda_{1}))\).