SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Assumption 3. In Proposition 3, we verified the geometric ergodicity of the discrete-time chain governed by parameter  $\theta = (\theta_{1},\theta_{2})\in \Theta$  and following policy  $\pi_{\omega}\in \tilde{\Pi}$  and thus, it only remains to verify the uniform model conditions. We define the normalized rates as  $\tilde{\lambda} = \frac{\lambda}{\lambda + \theta_1 + \theta_2}$  and  $\tilde{\theta}_i = \frac{\theta_i}{\lambda + \theta_1 + \theta_2}$ , for  $i = 1,2$ . From the choice of parameter space  $\Theta$ , we have  $\tilde{\lambda}\leq 0.5 - 0.5\delta$ ,  $\tilde{\theta}_{1} + \tilde{\theta}_{2}\geq 0.5 + 0.5\delta$ , and  $\tilde{\theta}_{1}\geq 0.25 + 0.25\delta$ .
1. We first argue that  $\zeta_{1,\theta ,\omega}$  is bounded away from 1 as follows
$$
\begin{array}{l} 1 - \zeta_ {1, \theta , \omega} = 1 - \frac {\frac {\lambda}{\theta_ {1} + \lambda}}{1 - \exp \left(- \frac {a _ {\theta , \omega} ^ {g}}{\omega}\right) \frac {\theta_ {1}}{\theta_ {1} + \lambda}} = \frac {\frac {\theta_ {1}}{\theta_ {1} + \lambda} \left(1 - \exp \left(- \frac {a _ {\theta , \omega} ^ {g}}{\omega}\right)\right)}{1 - \exp \left(- \frac {a _ {\theta , \omega} ^ {g}}{\omega}\right) \frac {\theta_ {1}}{\theta_ {1} + \lambda}} \\ \geq \frac {\theta_ {1}}{\theta_ {1} + \lambda} \left(1 - \exp \left(- \frac {a _ {\theta , (c _ {R} R) ^ {- 1}} ^ {g}}{c _ {R} R}\right)\right) > \tilde {\theta} _ {1} \left(1 - \exp \left(- \frac {a _ {\theta , (c _ {R} R) ^ {- 1}} ^ {g}}{c _ {R} R}\right)\right) \\ \left. > (0. 2 5 + 0. 2 5 \delta) \left(1 - \exp \left(- \frac {a _ {\theta , (c _ {R} R) ^ {- 1}} ^ {g}}{c _ {R} R}\right)\right), \right. \\ \end{array}
$$
where the first line follows from the definition of  $\zeta_{1,\theta,\omega}$  in Appendix F.3, the second line from (71) and the definition of policy class  $\tilde{\Pi}$ . As  $a_{\theta,\omega}^{g}$  does not depend on  $\theta$ ,  $\sup_{\theta \in \Theta, \omega \in [\frac{1}{c_R R}, c_R R]}\zeta_{1,\theta,\omega} < 1$ . Furthermore, by similar arguments it can be shown that  $\zeta_{2,\theta,\omega}$  is bounded away from 1. We next argue that  $\frac{\zeta_{1,\theta,\omega}\omega}{1+\omega}\exp\left(\frac{a_{\theta,\omega}^{g}}{\omega}\right) + \frac{\zeta_{2,\theta,\omega}}{1+\omega}$  is bounded away from 1 using an upper bound found in
Appendix F.3 as below,
$$
\begin{array}{l} 1 - \frac {\zeta_ {1 , \theta , \omega \omega}}{1 + \omega} \exp \left(\frac {a _ {\theta , \omega} ^ {g}}{\omega}\right) - \frac {\zeta_ {2 , \theta , \omega}}{1 + \omega} \\ \geq 1 - \frac {\frac {\lambda}{1 + \omega} \left(\omega + a _ {\theta , \omega} ^ {g} \zeta_ {4}\right)}{\lambda + \frac {\theta_ {1} a _ {\theta , \omega} ^ {g} \zeta_ {3}}{\omega}} - \frac {\frac {\lambda}{1 + \omega}}{\lambda + \theta_ {2} a _ {\theta , \omega} ^ {g} \zeta_ {3}} \\ = \frac {a _ {\theta , \omega} ^ {g} \left(- a _ {\theta , \omega} ^ {g} \zeta_ {3} \theta_ {2} \left(\lambda \zeta_ {4} - \frac {\zeta_ {3} \theta_ {1} (1 + \omega)}{\omega}\right) + \lambda \zeta_ {3} \left(\theta_ {1} + \theta_ {2}\right) - \lambda^ {2} \zeta_ {4}\right)}{(1 + \omega) (\lambda + \theta_ {1} a _ {\theta , \omega} ^ {g} \zeta_ {3} \omega^ {- 1}) (\lambda + \theta_ {2} a _ {\theta , \omega} ^ {g} \zeta_ {3})} \\ > \frac {(\zeta_ {3} a _ {\theta , \omega} ^ {g}) ^ {2} \tilde {\theta} _ {1} \tilde {\theta} _ {2}}{\omega (\tilde {\lambda} + \tilde {\theta} _ {1} a _ {\theta , \omega} ^ {g} \zeta_ {3} \omega^ {- 1}) (\tilde {\lambda} + \tilde {\theta} _ {2} a _ {\theta , \omega} ^ {g} \zeta_ {3})} \\ > \frac {\left(\zeta_ {3} a _ {\theta , \left(c _ {R} R\right) ^ {- 1}} ^ {g}\right) ^ {2} \left(0 . 2 5 + 0 . 2 5 \delta\right) ^ {2}}{c _ {R} R ^ {2} \left(1 + c _ {R} R \zeta_ {3} a _ {\theta , c _ {R} R} ^ {g}\right) ^ {2}}, \tag {75} \\ \end{array}
$$
where  $\zeta_3 = (1 + \delta)^{-1}$ ,  $\zeta_4 = \frac{1 - 0.5\delta}{1 - \delta}$ , and we have used the arguments of Appendix F.3 and the definition of  $\Theta$ . Using a similar argument, we can show that  $\frac{\zeta_{1,\theta,\omega}\omega}{1 + \omega} +\frac{\zeta_{2,\theta,\omega}}{1 + \omega}\exp \left(a_{\theta ,\omega}^{g}\right)$  is bounded away from one, and finally, we conclude that  $\sup_{\theta \in \Theta ,\omega \in [\frac{1}{c_R R},c_R R]}\gamma_{\theta ,\omega}^{g} < 1$ .
2. From (72), we can see that state  $(0,0)$  belongs to  $C_{\theta,\omega}^{g}$  for all  $\theta \in \Theta$  and  $\omega \in [\frac{1}{c_R R}, c_R R]$ . In order for  $C_*^g$  to be a finite set, the supremum of  $x_{i,\theta,\omega}^{g_j}$  over  $\Theta$  and  $\tilde{\Pi}$  should be finite. From the definition of  $x_{1,\theta,\omega}^{g_1}$  in Appendix F.3,
$$
\begin{array}{l} x _ {1, \theta , \omega} ^ {g _ {1}} = \frac {\omega}{a _ {\theta , \omega} ^ {g}} \log \frac {(c _ {R} R + 1) \exp (c _ {R} R a _ {\theta , \omega} ^ {g})}{(\omega + 1) \gamma_ {\theta , \omega} ^ {g} - \omega \zeta_ {1 , \theta , \omega} \exp \left(\frac {a _ {\theta , \omega} ^ {g}}{\omega}\right) - \zeta_ {2 , \theta , \omega}} \\ \leq \frac {c _ {R} R}{a _ {\theta , (c _ {R} R) ^ {- 1}} ^ {g}} \log \frac {(c _ {R} R + 1) \exp (c _ {R} R a _ {\theta , c _ {R} R} ^ {g})}{(\omega + 1) \gamma_ {\theta , \omega} ^ {g} - \omega \zeta_ {1 , \theta , \omega} \exp \left(\frac {a _ {\theta , \omega} ^ {g}}{\omega}\right) - \zeta_ {2 , \theta , \omega}}, \\ \end{array}
$$
and we can derive a lower bound for the denominator from (75). Similarly, we can show that  $\sup_{\theta \in \Theta ,\omega \in [\frac{1}{c_R R},c_R R]}x_{2,\theta ,\omega}^{g_2}$  is finite. We next find a uniform upper bound for  $x_{2,\theta ,\omega}^{g_1}$  from Appendix F.3,
$$
\begin{array}{l} x _ {2, \theta , \omega} ^ {g _ {1}} \\ = \frac {1}{a _ {\theta , \omega} ^ {g}} \log \frac {\left(c _ {R} R + 1\right) \exp \left(c _ {R} R a _ {\theta , \omega} ^ {g}\right) + \omega \exp \left(a _ {\theta , \omega} ^ {g} \frac {x _ {1 , \theta , \omega} ^ {g _ {1}} + 1}{\omega}\right) \left(\zeta_ {1 , \theta , \omega} \exp \left(\frac {a _ {\theta , \omega} ^ {g}}{\omega}\right) - \gamma_ {\theta , \omega} ^ {g}\right)}{\gamma_ {\theta , \omega} ^ {g} - \zeta_ {2 , \theta , \omega}} \\ \leq \frac {1}{a _ {\theta , (c _ {R} R) ^ {- 1}} ^ {g}} \log \frac {(2 c _ {R} R + 1) \exp \left(c _ {R} R a _ {\theta , c _ {R} R} ^ {g} \left(x _ {1 , \theta , \omega} ^ {g _ {1}} + 2\right)\right)}{1 - \gamma_ {\theta , \omega} ^ {g}}, \\ \end{array}
$$
which is uniformly bounded as  $\gamma_{\theta, \omega}^{g}$  is uniformly bounded away from 1 and the second line follows from (74) and the fact that  $\gamma_{\theta, \omega}^{g} - \zeta_{2, \theta, \omega} \geq 1 - \gamma_{\theta, \omega}^{g}$ . Arguments verifying the finiteness of the supremum of  $x_{1, \theta, \omega}^{g_2}$  follow similarly, and we conclude that  $|C_*^g| < \infty$ . To confirm a uniform upper bound for  $b_{\theta, \omega}^{g}$ , we note that from (73),
$$
\sup_{\theta \in \Theta ,\omega \in [ \frac{1}{c_{R}R},c_{R}R]}b_{\theta ,\omega}^{g}\leq \max_{x\in C_{*}^{g}}\Big(2\exp \Big(c_{R}Ra_{\theta ,c_{R}R}^{g}(x_{1} + 2)\Big) + 2\exp \Big(a_{\theta ,c_{R}R}^{g}(x_{2} + 2)\Big)\Big),
$$
which is finite as  $a_{\theta, c_R R}^g$  is independent of the choice of  $\theta$  and  $|C_*^g| < \infty$ .
Assumption 4. We next verify Assumption 4 and show that there exists a finite set  $C_{\theta, \omega}^p$ , constants  $\beta_{\theta, \omega}^p, b_{\theta, \omega}^p > 0, r / (r + 1) \leq \alpha_{\theta, \omega}^p < 1$ , and a function  $V_{\theta, \omega}^p: \mathcal{X} \to [1, +\infty)$  satisfying
$$
\Delta V _ {\theta , \omega} ^ {p} (\boldsymbol {x}) \leq - \beta_ {\theta , \omega} ^ {p} \left(V _ {\theta , \omega} ^ {p} (\boldsymbol {x})\right) ^ {\alpha_ {\theta , \omega} ^ {p}} + b _ {\theta , t} ^ {p} \mathbb {I} _ {C _ {\theta , \omega} ^ {p}} (\boldsymbol {x}), \quad \boldsymbol {x} \in \mathcal {X}. \tag {76}
$$
Proposition 4. The discrete-time Markov process obtained from the queueing system governed by parameter  $\theta = (\theta_{1},\theta_{2})\in \Theta$  and following policy  $\pi_{\omega}\in \tilde{\Pi}$  is polynomially ergodic. This follows because (76) holds for
$$
V _ {\theta , \omega} ^ {p} (\boldsymbol {x}) = \frac {x _ {1} ^ {2}}{\omega} + x _ {2} ^ {2}, \tag {77}
$$
$$
C _ {\theta , \omega} ^ {p} = \left\{\left(x _ {1}, x _ {2}\right) \in \mathcal {X}: x _ {i} \leq \left(1 6 c _ {R} ^ {2} R ^ {3 - i} + 1 0 1 c _ {R} R\right) \frac {\lambda + \theta_ {i}}{\theta_ {i}}, i = 1, 2 \right\}, \tag {78}
$$
$$
\beta_ {\theta , \omega} ^ {p} = \min  \left(\frac {\theta_ {2}}{2 \left(\theta_ {2} + \lambda\right) \sqrt {\omega + 1}}, \frac {\theta_ {1} + \theta_ {2} - \lambda}{\left(\theta_ {1} + \theta_ {2} + \lambda\right) \sqrt {\omega + 1}}, \frac {\theta_ {2}}{2 \left(\theta_ {2} + \lambda\right)}, \frac {\theta_ {1}}{2 \left(\theta_ {1} + \lambda\right) \sqrt {\omega}}\right), \tag {79}
$$
$$
b _ {\theta , \omega} ^ {p} = \left(\beta_ {\theta , \omega} ^ {p} + 1\right) \max  _ {\boldsymbol {x} \in C _ {\theta , \omega} ^ {p}} \left(\frac {(x _ {1} + 1) ^ {2}}{\omega} + (x _ {2} + 1) ^ {2}\right), \tag {80}
$$
$$
\alpha_ {\theta , \omega} ^ {p} = \frac {1}{2}. \tag {81}
$$
Proof of Proposition 4 is given in Appendix F.4. Next, we verify the remaining conditions of Assumption 4.
1. From (77) and the fact that  $\omega \in [\frac{1}{c_R R}, c_R R]$ , the first condition holds with  $r_*^p = 2$  and  $s_*^p = \sup_{\theta \in \Theta, \omega \in [\frac{1}{c_R R}, c_R R]} s_\theta, \omega = c_R R + 1$ .
2. From (78), state  $(0,0)$  belongs to  $C_{\theta ,\omega}^{p}$  for all  $\theta \in \Theta$  and  $\omega \in [\frac{1}{c_R R},c_R R]$ . Furthermore, for  $i = 1,2$ ,
$$
\sup  _ {\theta \in \Theta , \omega \in \left[ \frac {1}{c _ {R} R}, c _ {R} R \right]} \frac {\lambda + \theta_ {i}}{\theta_ {i}} \leq \sup  _ {\theta \in \Theta} \frac {1}{\tilde {\theta} _ {2}} \leq \sup  _ {\theta \in \Theta} \frac {R}{\tilde {\theta} _ {1}} \leq \frac {4 R}{1 + \delta}, \tag {82}
$$
which follows from the fact that  $\theta_{1} \leq R\theta_{2}$  and  $\tilde{\theta}_{1} \geq 0.25 + 0.25\delta$ . Thus, from the definition of  $C_{\theta,\omega}^{p}$  in (78),  $C_{*}^{p} = \cup_{\theta \in \Theta, \omega \in [\frac{1}{c_{R}R}, c_{R}R]} C_{\theta,\omega}^{p}$  is a finite set. We next verify that the infimum of  $\beta_{\theta,\omega}^{p}$ , found in (79), is positive. In (82), we showed that infimum of  $\frac{\lambda + \theta_{i}}{\theta_{i}}$  over  $\Theta$  is lower bounded by  $\frac{1 + \delta}{4}$ . From this, the fact that  $\omega$  belongs to a compact set, and  $\theta_{1} + \theta_{2} + \lambda \geq \delta$ , it follows that  $\inf_{\theta \in \Theta, \omega \in [\frac{1}{c_{R}R}, c_{R}R]} \beta_{\theta,\omega}^{p} > 0$ . Furthermore, it is easy to see that  $\beta_{\theta,\omega}^{p} \leq \sqrt{c_{R}R}$ . Hence, from (80),
$$
\begin{array}{l} \sup  _ {\theta \in \Theta , \omega \in [ \frac {1}{c _ {R} R}, c _ {R} R ]} b _ {\theta , \omega} ^ {p} = \sup  _ {\theta \in \Theta , \omega \in [ \frac {1}{c _ {R} R}, c _ {R} R ]} (\beta_ {\theta , \omega} ^ {p} + 1) \max  _ {\boldsymbol {x} \in C _ {\theta , \omega} ^ {p}} \left(\frac {(x _ {1} + 1) ^ {2}}{\omega} + (x _ {2} + 1) ^ {2}\right) \\ \leq \left(\sqrt {c _ {R} R} + 1\right) \max _ {\boldsymbol {x} \in C _ {*} ^ {p}} \left(C _ {R} R (x _ {1} + 1) ^ {2} + (x _ {2} + 1) ^ {2}\right), \\ \end{array}
$$
which is finite as  $|C_{*}^{p}| < \infty$ .
3. We need to show that  $K_{\theta, \omega}(\pmb{x}) \coloneqq \sum_{n=0}^{\infty} 2^{-n-2} (P_{\theta}^{\pi_{\omega}})^{n} (\pmb{x}, 0^{d})$  is strictly bounded away from zero. We show this using the fact that from any state  $\pmb{x}$ , the queueing system hits  $(0, 0)$  in one step with positive probability. Take  $x_{i, \theta, \omega} = \max_{\pmb{x} \in C_{\theta, \omega}} x_{i}$  for  $i = 1, 2$ . We have
$$
\begin{array}{l} \inf  _ {\theta \in \Theta , \omega \in [ \frac {1}{c _ {R} R}, c _ {R} R ]} \min  _ {\boldsymbol {x} \in C _ {\theta , \omega}} K (\boldsymbol {x}) \geq \inf  _ {\theta \in \Theta , \omega \in [ \frac {1}{c _ {R} R}, c _ {R} R ]} \min  _ {\boldsymbol {x} \in C _ {\theta , \omega}} P (\boldsymbol {x}, 0 ^ {d}) \\ \geq \inf_{\theta \in \Theta ,\omega \in [ \frac{1}{c_{R}R},c_{R}R]}P\left((x_{1,\theta ,\omega},x_{2,\theta ,\omega}),0^{d}\right). \\ \end{array}
$$
The infimum in the right-hand side of the above equation is attained for the minimum normalized service rates possible for each server, or  $\tilde{\theta}_1 = \frac{1 + \delta}{4}$  and  $\tilde{\theta}_2 = \frac{1 + \delta}{4R}$ . Therefore, the infimum of  $K_{\theta,\omega}(\boldsymbol{x})$  over the finite set  $C_*^p$ ,  $\Theta$ , and interval  $[\frac{1}{c_R R}, c_R R]$  is strictly greater than zero.
Assumption 5. We finally verify that  $\sup_{\theta \in \Theta}J(\theta)$  is finite. We first note that for  $\pmb {x} = (x_{1},x_{2})$
$$
(x _ {1} + x _ {2}) ^ {2} \leq 2 \max  (\omega^ {*} (\theta), 1) \left(\frac {x _ {1} ^ {2}}{\omega^ {*} (\theta)} + x _ {2} ^ {2}\right) = 2 \max  (\omega^ {*} (\theta), 1) V _ {\theta , \omega^ {*} (\theta)} ^ {p} (\boldsymbol {x}).
$$
From the above equation,
$$
\begin{array}{l} J (\theta) = \mathbb {E} _ {\boldsymbol {X} \sim \mu_ {\theta , \omega^ {*} (\theta)}} [ c (\boldsymbol {X}) ] \\ = \mathbb {E} _ {\boldsymbol {X} \sim \mu_ {\theta , \omega^ {*} (\theta)}} [ \| \boldsymbol {X} \| _ {1} ] \\ \leq \sqrt {2 \max  (\omega^ {*} (\theta) , 1)} \mathbb {E} _ {\boldsymbol {X} \sim \mu_ {\theta , \omega^ {*} (\theta)}} \left[ \sqrt {V _ {\theta , \omega^ {*} (\theta)} ^ {p} (\boldsymbol {X})} \right], \\ \end{array}
$$
where  $\mu_{\theta, \omega^{*}(\theta)}$  is the stationary distribution of the discrete-time process governed by parameter  $\theta$  and following the best in-class policy according to  $\theta$ , shown by  $\pi_{\omega^{*}(\theta)}$ . From [43], Theorem 14.3.7,
$$
\mu_ {\theta , \omega^ {*} (\theta)} \left(\sqrt {V _ {\theta , \omega^ {*} (\theta)} ^ {p} (\boldsymbol {X})}\right) \leq \frac {\sup _ {\theta \in \Theta , \omega \in [ \frac {1}{c _ {R} R} , c _ {R} R ]} b _ {\theta , \omega} ^ {p}}{\beta_ {*} ^ {p}},
$$
which is finite from the previous verified assumption. Thus,
$$
\sup _ {\theta \in \Theta} J (\theta) \leq \frac {\sqrt {2 c _ {R} R} \left(\sup _ {\theta \in \Theta , \omega \in [ \frac {1}{c _ {R} R} , c _ {R} R ]} b _ {\theta , \omega} ^ {p}\right)}{\beta_ {*} ^ {p}} <   \infty .
$$
F Proofs related to the queueing model examples
F.1 Proof of Proposition 1
Proof. We define the normalized rates as
$$
\tilde {\lambda} = \frac {\lambda}{\lambda + \theta_ {1} + \theta_ {2}}, \quad \tilde {\theta} _ {i} = \frac {\theta_ {i}}{\lambda + \theta_ {1} + \theta_ {2}}, \tag {83}
$$
for  $i = 1,2$ . From the choice of parameter space  $\Theta$ , we have  $\tilde{\lambda} \leq 0.5 - 0.5\delta$ ,  $\theta_{1} + \theta_{2} \geq 0.5 + 0.5\delta$ , and  $\theta_{1} \geq 0.25 + 0.25\delta$ . To prove geometric ergodicity, from the discussions of Section 2, it suffices to show that there exists a finite set  $C_{\theta,t}^{g}$ , constants  $b_{\theta,t}^{g} > 0$ ,  $\gamma_{\theta,t}^{g} \in (0,1)$ , and a function  $V_{\theta,t}^{g}: \mathcal{X}_{t} \to [1, +\infty)$  satisfying
$$
\Delta V _ {\theta , t} ^ {g} (\boldsymbol {x}) \leq - \left(1 - \gamma_ {\theta , t} ^ {g}\right) V _ {\theta , t} ^ {g} (\boldsymbol {x}) + b _ {\theta , t} ^ {g} \mathbb {I} _ {C _ {\theta , t} ^ {g}} (\boldsymbol {x}), \quad \boldsymbol {x} \in \mathcal {X} _ {t}. \tag {84}
$$
Take  $V_{\theta ,t}^{g}(\pmb {x}) = \exp (a_{\theta ,t}^{g}\| \pmb {x}\|_{1})$  for some  $a_{\theta ,t}^g >0$  . For  $i\geq 1$  and  $\pmb {x} = (i,1,1)$
$$
P _ {\theta} ^ {t} V _ {\theta , t} ^ {g} (i, 1, 1) = \tilde {\lambda} V _ {\theta , t} ^ {g} (i + 1, 1, 1) + \tilde {\theta} _ {1} V _ {\theta , t} ^ {g} (i, 0, 1) + \tilde {\theta} _ {2} V _ {\theta , t} ^ {g} (i, 1, 0),
$$
where  $P_{\theta}^{t}$  is the corresponding transition kernel. Thus,
$$
\begin{array}{l} P _ {\theta} ^ {t} V _ {\theta , t} ^ {g} (i, 1, 1) - \left(1 - \gamma_ {\theta , t} ^ {g}\right) V _ {\theta , t} ^ {g} (i, 1, 1) \\ = \tilde {\lambda} \exp \left(a _ {\theta , t} ^ {g} (i + 3)\right) + (\tilde {\theta} _ {1} + \tilde {\theta} _ {2}) \exp \left(a _ {\theta , t} ^ {g} (i + 1)\right) - (1 - \gamma_ {\theta , t} ^ {g}) \exp \left(a _ {\theta , t} ^ {g} (i + 2)\right) \\ = \exp \left(a _ {\theta , t} ^ {g} (i + 1)\right) \left(\tilde {\lambda} \exp \left(2 a _ {\theta , t} ^ {g}\right) + \tilde {\theta} _ {1} + \tilde {\theta} _ {2} - \left(1 - \gamma_ {\theta , t} ^ {g}\right) \exp \left(a _ {\theta , t} ^ {g}\right)\right). \\ \end{array}
$$
Take  $\tilde{a}_{\theta ,t} = \exp (a_{\theta ,t}^{g})$  . We need to find  $\tilde{a}_{\theta ,t} > 1$  and  $0 <   \gamma_{\theta ,t}^{g} <   1$  such that
$$
\tilde {\lambda} \tilde {a} _ {\theta , t} ^ {2} - \left(1 - \gamma_ {\theta , t} ^ {g}\right) \tilde {a} _ {\theta , t} + \tilde {\theta} _ {1} + \tilde {\theta} _ {2} <   0. \tag {85}
$$
Take  $\tilde{a}_{\theta ,t} = (1 - \delta)^{-1} > 1$  and
$$
\tilde {\gamma} _ {\theta , t} := 1 - \gamma_ {\theta , t} ^ {g} = \frac {1}{2} \left(1 + (1 - \tilde {\lambda}) (1 - \delta) + \tilde {\lambda} (1 - \delta) ^ {- 1}\right).
$$
We need to have  $\tilde{\gamma}_{\theta ,t} < 1$  which follows from the stability condition  $\tilde{\lambda}\leq 0.5 - 0.5\delta$  as below:
$$
\begin{array}{l} \tilde {\gamma} _ {\theta , t} = \frac {1}{2} + \frac {1}{2} \left((1 - \tilde {\lambda}) (1 - \delta) + \frac {\tilde {\lambda}}{1 - \delta}\right) = \frac {1}{2} + \frac {1}{2} \left(1 - \delta - \tilde {\lambda} (1 - \delta) + \frac {\tilde {\lambda}}{1 - \delta}\right) \\ = \frac {1}{2} + \frac {1}{2} \left(1 - \delta + \tilde {\lambda} \frac {1 - (1 - \delta) ^ {2}}{1 - \delta}\right) = \frac {1}{2} + \frac {1}{2} \left(1 - \delta + \tilde {\lambda} \frac {\delta (2 - \delta)}{1 - \delta}\right) \\ \leq \frac {1}{2} + \frac {1}{2} \left(1 - \delta + \frac {\delta (2 - \delta)}{2}\right) = 1 - \frac {\delta^ {2}}{4} <   1. \\ \end{array}
$$
We now verify (85):
$$
\begin{array}{l} \tilde {\lambda} \tilde {a} _ {\theta , t} ^ {2} - (1 - \gamma_ {\theta , t} ^ {g}) \tilde {a} _ {\theta , t} + \tilde {\theta} _ {1} + \tilde {\theta} _ {2} = \frac {\tilde {\lambda}}{(1 - \delta) ^ {2}} - \frac {1}{2 (1 - \delta)} - \frac {1 - \tilde {\lambda}}{2} - \frac {\tilde {\lambda}}{2 (1 - \delta) ^ {2}} + 1 - \tilde {\lambda} \\ = \frac {\tilde {\lambda}}{2 (1 - \delta) ^ {2}} + \frac {1 - \tilde {\lambda}}{2} - \frac {1}{2 (1 - \delta)} \\ = \frac {1}{2 (1 - \delta) ^ {2}} \left(\tilde {\lambda} + (1 - \tilde {\lambda}) (1 - \delta) ^ {2} - (1 - \delta)\right) \\ = \frac {\delta}{2 (1 - \delta) ^ {2}} \left(\delta - 1 - \tilde {\lambda} \delta + 2 \tilde {\lambda}\right) \\ = \frac {\delta}{2 (1 - \delta) ^ {2}} \left(\tilde {\lambda} (2 - \delta) + \delta - 1\right) \\ <   0, \\ \end{array}
$$
where the last line follows from  $\tilde{\lambda} \leq 0.5 - 0.5\delta < (1 - \delta) / (2 - \delta)$ .
For  $\pmb{x} = (i,0,1)$  and  $i \geq 1$ , we have
$$
P _ {\theta} ^ {t} V _ {\theta , t} ^ {g} (i, 0, 1) = \tilde {\lambda} V _ {\theta , t} ^ {g} (i, 1, 1) + \tilde {\theta} _ {1} V _ {\theta , t} ^ {g} (i - 1, 0, 1) + \tilde {\theta} _ {2} V _ {\theta , t} ^ {g} (i - 1, 1, 0),
$$
$$
\begin{array}{l} P _ {\theta} ^ {t} V _ {\theta , t} ^ {g} (i, 0, 1) - \left(1 - \gamma_ {\theta , t} ^ {g}\right) V _ {\theta , t} ^ {g} (i, 0, 1) \\ = \tilde {\lambda} \exp \left(a _ {\theta , t} ^ {g} (i + 2)\right) + (\tilde {\theta} _ {1} + \tilde {\theta} _ {2}) \exp \left(a _ {\theta , t} ^ {g} i\right) - (1 - \gamma_ {\theta , t} ^ {g}) \exp \left(a _ {\theta , t} ^ {g} (i + 1)\right) \\ = \exp \left(a _ {\theta , t} ^ {g} i\right) \left(\tilde {\lambda} \exp (2 a _ {\theta , t} ^ {g}) + \tilde {\theta} _ {1} + \tilde {\theta} _ {2} - (1 - \gamma_ {\theta , t} ^ {g}) \exp (a _ {\theta , t} ^ {g})\right), \\ \end{array}
$$
which results in the same conditions as previously discussed. When  $\pmb{x} = (i,1,0)$  and  $i \geq t$  also same argument holds.
Finally, (84) holds for
$$
C _ {\theta , t} ^ {g} = \left\{\left(x _ {0}, x _ {1}, 0\right): x _ {0} <   t \right\} \cup \{(0, 0, 1) \},
$$
$$
a _ {\theta , t} ^ {g} = - \log (1 - \delta),
$$
$$
\gamma_ {\theta , t} ^ {g} = \frac {1}{2} - \frac {1}{2} \left((1 - \tilde {\lambda}) (1 - \delta) + \tilde {\lambda} (1 - \delta) ^ {- 1}\right),
$$
$$
V _ {\theta , t} ^ {g} (\boldsymbol {x}) = \exp \left(a _ {\theta , t} ^ {g} \| \boldsymbol {x} \| _ {1}\right),
$$
$$
b _ {\theta , t} ^ {g} = \max  _ {\boldsymbol {x} \in C _ {\theta , t} ^ {g}} \exp (a _ {\theta , t} ^ {g} \| \boldsymbol {x} \| _ {1}) \left(\exp (a _ {\theta , t} ^ {g}) + 1\right),
$$
where the last line holds because  $PV_{\theta ,t}^{g}(\pmb {x})\leq V_{\theta ,t}^{g}(\pmb {y})$  for  $\pmb{y}$  such that  $\| \pmb {y}\| _1 = \| \pmb {x}\| _1 + 1$ .
F.2 Proof of Proposition 2
Proof. In order to show polynomially ergodicity, we will verify (62). We define  $V_{\theta,t}^p(\boldsymbol{x}) = \|\boldsymbol{x}\|_1^2$  and  $\alpha_{\theta,t}^p = 1/2$ , which is equal to  $r/(r+1)$  for  $r = 1$ ;  $r$  is defined in Assumption 1. For  $\boldsymbol{x} = (i,0,1)$  and  $i \geq 1$ ,
$$
P _ {\theta} ^ {t} V _ {\theta , t} ^ {p} (i, 0, 1) = \tilde {\lambda} V _ {\theta , t} ^ {p} (i, 1, 1) + \tilde {\theta} _ {1} V _ {\theta , t} ^ {p} (i - 1, 0, 1) + \tilde {\theta} _ {2} V _ {\theta , t} ^ {p} (i - 1, 1, 0),
$$
in which  $\tilde{\lambda},\tilde{\theta}_1$  , and  $\tilde{\theta}_{2}$  are the normalized rates defined in (83). Thus,
$$
\begin{array}{l} P _ {\theta} ^ {t} V _ {\theta , t} ^ {p} (i, 0, 1) - V _ {\theta , t} ^ {p} (i, 0, 1) + \beta_ {\theta , t} ^ {p} \sqrt {V _ {\theta , t} ^ {p} (i , 0 , 1)} \\ = \tilde {\lambda} (i + 2) ^ {2} + \left(\tilde {\theta} _ {1} + \tilde {\theta} _ {2}\right) i ^ {2} - (i + 1) ^ {2} + \beta_ {\theta , t} ^ {p} (i + 1) \\ = i \left(4 \tilde {\lambda} - 2 + \beta_ {\theta , t} ^ {p}\right) + 4 \tilde {\lambda} - 1 + \beta_ {\theta , t} ^ {p}. \\ \end{array}
$$
For  $\beta_{\theta ,t}^p = 1 - 2\tilde{\lambda}$ , the right-hand side of above equation is non-positive for  $i\geq \frac{2\tilde{\lambda}}{1 - 2\tilde{\lambda}}$ . For  $\boldsymbol {x} = (i,1,0)$  and  $i\geq t$
$$
P _ {\theta} ^ {t} V _ {\theta , t} ^ {p} (i, 1, 0) = \tilde {\lambda} V _ {\theta , t} ^ {p} (i, 1, 1) + \tilde {\theta} _ {1} V _ {\theta , t} ^ {p} (i - 1, 0, 1) + \tilde {\theta} _ {2} V _ {\theta , t} ^ {p} (i - 1, 1, 0).
$$
$$
\begin{array}{l} P _ {\theta} ^ {t} V _ {\theta , t} ^ {p} (i, 1, 0) - V _ {\theta , t} ^ {p} (i, 1, 0) + \beta_ {\theta , t} ^ {p} \sqrt {V _ {\theta , t} ^ {p} (i , 1 , 0)} \\ = \tilde {\lambda} (i + 2) ^ {2} + (\tilde {\theta} _ {1} + \tilde {\theta} _ {2}) i ^ {2} - (i + 1) ^ {2} + \beta_ {\theta , t} ^ {p} (i + 1) \\ = i (4 \tilde {\lambda} - 2 + \beta_ {\theta , t} ^ {p}) + 4 \tilde {\lambda} - 1 + \beta_ {\theta , t} ^ {p}, \\ \end{array}
$$
which is also non-positive under the same conditions as the previous case. For  $i \geq 1$  and  $x = (i, 1, 1)$ ,
$$
P _ {\theta} ^ {t} V _ {\theta , t} ^ {p} (i, 1, 1) = \tilde {\lambda} V _ {\theta , t} ^ {p} (i + 1, 1, 1) + \tilde {\theta} _ {1} V _ {\theta , t} ^ {p} (i, 0, 1) + \tilde {\theta} _ {2} V _ {\theta , t} ^ {p} (i, 1, 0).
$$
$$
\begin{array}{l} P _ {\theta} ^ {t} V _ {\theta , t} ^ {p} (i, 1, 1) - V _ {\theta , t} ^ {p} (i, 1, 1) + \beta_ {\theta , t} ^ {p} \sqrt {V _ {\theta , t} ^ {p} (i , 1 , 1)} \\ = \tilde {\lambda} (i + 3) ^ {2} + \left(\tilde {\theta} _ {1} + \tilde {\theta} _ {2}\right) (i + 1) ^ {2} - (i + 2) ^ {2} + \beta_ {\theta , t} ^ {p} (i + 2) \\ = i (4 \tilde {\lambda} - 2 + \beta_ {\theta , t} ^ {p}) + 8 \tilde {\lambda} - 3 + 2 \beta_ {\theta , t} ^ {p}, \\ \end{array}
$$
which is non-positive under the same conditions as the first case. Finally, (62) holds for