## 4 Optimization and Analysis
Theorem 1: _There exists a unique solution of the optimization problem (Equation (25)), i.e., \(\exists!\eta^{*}\in\Omega\), such that \(MG(\eta^{*})-\Gamma(\eta^{*})P(\eta^{*})>MG(\eta)-\Gamma(\eta^{*})P(\eta), \forall\eta\
eq\eta^{*}\). Additionally, the average transmission probability \(P(\eta^{*})\leq min\left\{\beta(B\geq l),P(\eta_{\mathit{PLR}_{\mathit{H}}})\right\}\)._  
Proof of Theorem 1: See Appendix C.  
Generally, the symmetric NE may be a suboptimal solution of the original optimization problem (Equation (22)), because it is optimal only for the case that EH-ST deviates from the network unilaterally, not symmetrically. Contrarily, if all the EH-STs change the policy by the same quantity (the policy is still symmetric), then it may improve the throughput of the system. Therefore, in this situation, the obtained policy holds for the symmetric NE may not be globally or locally optimal. In the following theorem, we show that the obtained symmetric NE is a locally optimal solution of the optimization problem proposed in this paper. Theorem 2: _The obtained symmetric NE in Equation (25) is locally optimal for the original optimization problem (Equation (22))._  
Proof of Theorem 2: See Appendix D.  
In this paper, we also present an algorithm to determine the optimal policy \(\eta^{*}\). From the analysis above, we need to determine the unique \(\gamma^{*}\), s.t. \(f(\gamma^{*})=0\), where we have previously defined \(f(\gamma)=\Gamma(\eta^{(\gamma)})-\gamma\). Then, according to \(\gamma^{*}\), we can obtain the optimal policy \(\eta^{*}\) as \(\eta^{*}=\eta^{(\gamma^{*})}\). Since \(f(\gamma)\) is a continuous decreasing function of \(\gamma\), \(f(0)>0\), and \(f(\infty)\rightarrow-\infty\), we can use the bisection method [38] to search the unique \(\gamma^{*}\) which makes \(f(\gamma^{*})=0\). Thus, upper bounds \(\gamma_{up}\) and lower bounds \(\gamma_{low}\) are needed to approach \(\gamma^{*}\), where \(\gamma_{low}<\gamma^{*}<\gamma_{up}\). By computing the value of \(f(\gamma)\) for the updated \(\gamma=(\gamma_{up}+\gamma_{low})/2\), the upper and lower bounds are updated and refined recursively until \(f(\gamma)\) satisfies the expected accuracy. In order to compute \(f(\gamma)\), we also need to determine \(\Gamma(\eta^{(\gamma)})\), which can be computed efficiently by a policy iteration algorithm [39]. The lower bound is initialized by \(\gamma_{low}=0\). For the upper bound, note that \(M\approx\mathit{UP}(\eta)\), and the average throughput gain \(G(\eta)\) is measured by packet/slot which is less than 1. Therefore, the upper bound is initialized by \(\gamma_{up}=U\). The algorithm is described in detail in Algorithm 1. The policy iteration algorithm employed in Algorithm 1 is able to determine the optimal policy \(\eta^{(\gamma)}\) when \(\delta_{PIA}\to 0\). The optimality and convergence of the policy iteration algorithm is proven in Reference [39]. Particularly, in the policy improvement step of Algorithm 1, the optimization function has a unique optimal solution, since \(d_{\gamma}(\eta(e))\) is a concave function of \(\eta(e)\). The optimal solution can be obtained by taking the derivative of the optimization function w.r.t \(\eta(e)\). **Remark 3**: _Algorithm 1 is combined policy iteration algorithm with a bisection method, which can be operated efficiently. For the policy iteration part, we need to calculate the Markov steady state probability distribution \(\pi_{\eta|i}(e)\), which can be solved by linear programming using the transition probability (Equation (13)), and its computing complexity is \(O(e_{max})\). Furthermore, in order to obtain the value function \(v(e)\), we also need to solve the linear system, whose computing complexity is \(O(e_{max})\). For the policy improvement step, it needs to take the derivative of the optimization function and then solve the linear function. Thus, the complexity of this step scales as \(O(e_{max})\). Finally, these steps are needed to iterated \(N_{iter}(\delta_{PIA})\) times until it converges.