SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \sup  _ {a \in \mathcal {A}} \left| \int R (a, \theta) d Q _ {\mathbf {a} _ {\mathrm {R S}} ^ {*}, \gamma} ^ {*} (\theta | \tilde {X} _ {n}) - R (a, \theta_ {0}) \right| \right] \leq \left[ M (\gamma) \epsilon_ {n} ^ {2} + M \eta_ {n} ^ {R} (\gamma) \right] ^ {\frac {1}{2}}. \tag {42}
$$
It follows from above that the  $P_0^n$  - probability of the following event is at least  $1 - \tau^{-1}$
$$
\left\{\tilde {X} _ {n}: R \left(\mathsf {a} _ {\mathrm {R S}} ^ {*}, \theta_ {0}\right) - \inf  _ {z \in \mathcal {A}} R (z, \theta_ {0}) \leq 2 \tau \left[ M (\gamma) \epsilon_ {n} ^ {2} + M \eta_ {n} ^ {R} (\gamma) \right] ^ {\frac {1}{2}} \right\}. \tag {43}
$$
C.4 Proofs in Section 3.1
Proof of Proposition 3.1. Using the definition of  $\eta_n^R (\gamma)$  and the posterior distribution  $\Pi (\theta |\tilde{X}_n)$ , observe that
$$
\begin{array}{l} n \eta_ {n} ^ {R} (\gamma) = \inf  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q \| \Pi_ {n}) - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \right] \\ = \inf  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q \| \Pi) + \int_ {\Theta} d Q (\theta) \log \left(\frac {\int d \Pi (\theta) p (\tilde {X} _ {n} | \theta)}{p (\tilde {X} _ {n} | \theta)}\right) - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \right] \\ = \inf  _ {Q \in \mathcal {Q}} \left[ \mathrm {K L} (Q \| \Pi) - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] + \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathbb {E} _ {Q} \left[ \log \left(\frac {\int d \Pi (\theta) p (\tilde {X} _ {n} | \theta)}{p (\tilde {X} _ {n} | \theta)}\right) \right] \right] \right]. \\ \end{array}
$$
Now, using Fubini's in the last term of the equation above, we obtain
$$
\begin{array}{l} n \eta_ {n} ^ {R} (\gamma) = \inf  _ {Q \in \mathcal {Q}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta)) - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \right. \\ \left. \right. + \mathbb {E} _ {Q} \left[ \right. \mathrm {K L} \left(d P _ {0} ^ {n} \| p (\tilde {X} _ {n} | \theta)\right) - \mathrm {K L} \left( \right.d P _ {0} ^ {n} \left\| \right. \int d \Pi (\theta) p (\tilde {X} _ {n} | \theta)\left. \right)\left. \right]\left. \right]. \tag {44} \\ \end{array}
$$
Observe that,  $\int_{\mathcal{X}^n}\int d\Pi (\theta)p(\tilde{X}_n|\theta)d\tilde{X}_n = 1$ . Since, KL is always non-negative, it follows from the equation above that
$$
\begin{array}{l} \eta_ {n} ^ {R} (\gamma) \\ \leq \frac {1}{n} \inf  _ {Q \in \mathcal {Q}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta)) - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] + \mathbb {E} _ {Q} \left[ \mathrm {K L} \left(d P _ {0} ^ {n} \| p (\tilde {X} _ {n} | \theta)\right) \right] \right] \\ \leq \frac {1}{n} \inf  _ {Q \in \mathcal {Q}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta)) + \mathbb {E} _ {Q} \left[ \mathrm {K L} \left(d P _ {0} ^ {n} \| p (\tilde {X} _ {n} | \theta)\right) \right] \right] - \frac {\gamma}{n} \inf  _ {Q \in \mathcal {Q}} \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ], \tag {45} \\ \end{array}
$$
where the last inequality follows from the following fact, for any functions  $f(\cdot)$  and  $g(\cdot)$ ,
$$
\inf  (f - g) \leq \inf  f - \inf  g.
$$
Recall  $\epsilon_n^\prime \geq \frac{1}{\sqrt{n}}$ . Now, using Assumption 3.1, it is straightforward to observe that the first term in (45),
$$
\frac {1}{n} \inf  _ {Q \in \mathcal {Q}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta)) + \mathbb {E} _ {Q} \left[ \mathrm {K L} \left(d P _ {0} ^ {n} \| p (\tilde {X} _ {n} | \theta)\right) \right] \right] \leq C _ {9} \epsilon_ {n} ^ {\prime 2}. \tag {46}
$$
Now consider the last term in (45). Notice that the coefficient of  $\frac{1}{n}$  is independent of  $n$  and is bounded from below. Therefore, there exist a constant  $C_8 = -\inf_{Q \in \mathcal{Q}} \inf_{a \in \mathcal{A}} \mathbb{E}_Q[R(a,\theta)]$ , such that with equation (46) it follows that  $\eta_n^R(\gamma) \leq \gamma n^{-1}C_8 + C_9\epsilon_n'^2$  and the result follows.
Proof of Proposition 3.2. First recall that
$$
\begin{array}{l} n \eta_ {n} ^ {R} (\gamma) = \inf  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta | \tilde {X} _ {n})) - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \right] \\ = \inf  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta | \tilde {X} _ {n})) \right] - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ]. \tag {47} \\ \end{array}
$$
Observe that the optimization problem is equivalent to solving :
$$
\min  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta | \tilde {X} _ {n})) \right] \text {s . t .} - \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \leq 0. \tag {48}
$$
Now for any  $\gamma > 0$ ,  $Q_{\gamma}^{*}(\theta) \in \mathcal{Q}$  that minimizes the objective in (47) is primal feasible if
$$
- \inf  _ {a \in \mathcal {A}} \int_ {\Theta} d Q _ {\gamma} ^ {*} (\theta) R (a, \theta) \leq 0.
$$
Therefore, it is straightforward to observe that as  $\gamma$  increases  $n\eta_n^R (\gamma)$  decreases that is
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} d Q _ {\gamma} ^ {*} (\theta) \log \frac {d Q _ {\gamma} ^ {*} (\theta)}{d \Pi (\theta | \tilde {X} _ {n})} - \gamma \inf  _ {a \in \mathcal {A}} \int_ {\Theta} d Q _ {\gamma} ^ {*} (\theta) R (a, \theta) \right].
$$
C.5 Sufficient conditions on  $R(a, \theta)$  for existence of tests
To show the existence of test functions, as required in Assumption 2.1, we will use the following result from [11, Theorem 7.1], that is applicable only to distance measures that are bounded above by the Hellinger distance.
Lemma C.7 (Theorem 7.1 of [11]). Suppose that for some non-increasing function  $D(\epsilon)$ , some  $\epsilon_{n} > 0$  and for every  $\epsilon > \epsilon_{n}$ ,
$$
N \left(\frac {\epsilon}{2}, \left\{P _ {\theta}: \epsilon \leq m \left(\theta , \theta_ {0}\right) \leq 2 \epsilon \right\}, m\right) \leq D (\epsilon),
$$
where  $m(\cdot, \cdot)$  is any distance measure bounded above by Hellinger distance. Then for every  $\epsilon > \epsilon_{n}$ , there exists a test  $\phi_{n}$  (depending on  $\epsilon > 0$ ) such that, for every  $j \geq 1$ ,
$$
\mathbb {E} _ {P _ {0} ^ {n}} [ \phi_ {n} ] \leq D (\epsilon) \exp \left(- \frac {1}{2} n \epsilon^ {2}\right) \frac {1}{1 - \exp \left(- \frac {1}{2} n \epsilon^ {2}\right)}, a n d
$$
$$
\sup _ {\{\theta \in \Theta_ {n} (\epsilon): m (\theta , \theta_ {0}) > j \epsilon \}} \mathbb {E} _ {P _ {\theta} ^ {n}} [ 1 - \phi_ {n} ] \leq \exp \left(- \frac {1}{2} n \epsilon^ {2} j\right).
$$
Proof of Lemma C.7: Refer Theorem 7.1 of [11].
For the remaining part of this subsection we assume that  $\Theta \subseteq \mathbb{R}^d$ . In the subsequent paragraph, we state further assumptions on the risk function to show  $L_{n}(\cdot ,\cdot)$  as defined in (6) satisfies Assumption 2.1. For brevity we denote  $n^{-1 / 2}\sqrt{L_n(\theta,\theta_0)}$  by  $d_{L}(\theta ,\theta_{0})$ , that is
$$
d _ {L} \left(\theta_ {1}, \theta_ {2}\right) := \sup  _ {a \in \mathcal {A}} | R (a, \theta_ {1}) - R (a, \theta_ {2}) |, \forall \left\{\theta_ {1}, \theta_ {2} \right\} \in \Theta \tag {49}
$$
and the covering number of the set  $T(\epsilon) \coloneqq \{P_{\theta} : d_L(\theta, \theta_0) < \epsilon\}$  as  $N(\delta, T(\epsilon), d_L)$ , where  $\delta > 0$  is the radius of each ball in the cover. We assume that the risk function  $R(a, \cdot)$  satisfies the following bound.
Assumption C.1. The model risk satisfies
$$
d _ {L} (\theta_ {1}, \theta_ {2}) | \leq K _ {1} d _ {H} (\theta , \theta_ {0}),
$$
where  $d_H(\theta_1,\theta_2)$  is the Hellinger distance between two models  $P_{\theta_1}$  and  $P_{\theta_2}$ .
For instance, suppose the definition of model risk is  $R(a,\theta) = \int_{\mathcal{X}}\ell (x,a)p(y|\theta)dx$ , where  $\ell (x,a)$  is an underlying loss function. Then, observe that Assumption C.1 is trivially satisfied if  $\ell (x,a)$  is bounded in  $x$  for a given  $a\in \mathcal{A}$  and  $\mathcal{A}$  is compact, since  $d_{L}(\theta_{1},\theta_{2})$  can be bounded by the total variation distance  $d_{TV}(\theta_1,\theta_2) = \frac{1}{2}\int |dP_{\theta_1}(x) - dP_{\theta_2}(x)|$  and total variation distance is bounded above by the Hellinger distance [12]. Under the assumption above it also follows that we can apply Lemma C.7 to the metric  $d_{L}(\cdot ,\cdot)$  defined in (49). Now, we will also assume an additional regularity condition on the risk function.
Assumption C.2. For every  $\{\theta_1,\theta_2\} \in \Theta$  , there exists a constant  $K_{2} > 0$  such that
$$
d _ {L} \left(\theta_ {1}, \theta_ {2}\right) \leq K _ {2} \| \theta_ {1} - \theta_ {2} \|,
$$
We can now show that the covering number of the set  $T(\epsilon)$  satisfies
Lemma C.8. Given  $\epsilon >\delta >0$  , and under Assumption C.2,
$$
N (\delta , T (\epsilon), d _ {L}) <   \left(\frac {2 \epsilon}{\delta} + 2\right) ^ {d}. \tag {50}
$$
Proof of Lemma C.8: For any positive  $k$  and  $\epsilon$ , let  $\theta \in [\theta_0 - k\epsilon, \theta_0 + k\epsilon]^d \subset \Theta \subset \mathbb{R}^d$ . Now consider a set  $H_i = \{\theta_i^0, \theta_i^1, \ldots, \theta_i^J, \theta_i^{J+1}\}$  and  $H = \bigotimes_d H_i$  with  $J = \left\lfloor \frac{2k\epsilon}{\delta'} \right\rfloor$ , where  $\theta_i^j = \theta_0 - k\epsilon + i\delta'$  for  $j = \{0, 1, \ldots, J\}$  and  $\theta_i^{J+1} = \theta_0 + k\epsilon$ . Observe that for any  $\theta \in [\theta_0 - k\epsilon, \theta_0 + k\epsilon]^d$ , there exists a  $\theta^j \in H$  such that  $\| \theta - \theta^j \| < \delta'$ . Hence, union of the  $\delta'$ -balls for each element in set  $H$  covers  $[\theta_0 - k\epsilon, \theta_0 + k\epsilon]^d$ , therefore  $N(\delta', [\theta_0 - k\epsilon, \theta_0 + k\epsilon]^d, \| \cdot \|) = (J + 2)^d$ .
Now, due to Assumption C.2, for any  $\theta \in [\theta_0 - k\epsilon ,\theta_0 + k\epsilon ]^d$
$$
d _ {L} (\theta , \theta_ {0}) \leq K _ {2} \| \theta - \theta^ {j} \| \leq K _ {2} \delta^ {\prime},
$$
For brevity, we denote  $n^{-1}L_n(\theta ,\theta_0)$  by  $d_{L}(\theta ,\theta_{0})$  , that is
$$
d _ {L} \left(\theta_ {1}, \theta_ {2}\right) := \sup  _ {a \in \mathcal {A}} | R (a, \theta_ {1}) - R (a, \theta_ {2}) |, \forall \left\{\theta_ {1}, \theta_ {2} \right\} \in \Theta , \tag {51}
$$
and the covering number of the set  $T(\epsilon) \coloneqq \{P_{\theta} : d_L(\theta, \theta_0) < \epsilon\}$  as  $N(\delta, T(\epsilon), d_L)$ , where  $\delta > 0$  is the radius of each ball in the cover.
Hence,  $\delta^\prime$ -cover of set  $[\theta_0 - k\epsilon, \theta_0 + k\epsilon]^d$  is  $K_{1}\delta^{\prime}$  cover of set  $T(\epsilon)$  with  $k = 1 / K_{2}$ . Finally,
$$
N \left(K _ {2} \delta^ {\prime}, T (\epsilon), d _ {L}\right) \leq (J + 2) ^ {d} \leq \left(\frac {2 k \epsilon}{\delta^ {\prime}} + 2\right) ^ {d} = \left(\frac {2 \epsilon}{K _ {2} \delta^ {\prime}} + 2\right) ^ {d}
$$
which implies for  $\delta = K_2\delta'$ ,
$$
N (\delta , T (\epsilon), d _ {L}) \leq \left(\frac {2 \epsilon}{\delta} + 2\right) ^ {s}.
$$
Observe that the RHS in (50) is a decreasing function of  $\delta$ , infact for  $\delta = \epsilon / 2$ , it is a constant in  $\epsilon$ . Therefore, using Lemmas C.7 and C.8, we show in the following result that  $L_{n}(\theta, \theta_{0})$  in (6) satisfies Assumption 2.1.
Lemma C.9. Fix  $n \geq 1$ . For a given  $\epsilon_n > 0$  and every  $\epsilon > \epsilon_n$ , such that  $n\epsilon_n^2 \geq 1$ . Under Assumption C.1 and C.2,  $L_{n}(\theta ,\theta_{0}) = n\left(\sup_{a\in \mathcal{A}}|R(a,\theta) - R(a,\theta_{0})|\right)^{2}$  satisfies
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \phi_ {n} \right] \leq C _ {0} \exp (- C n \epsilon^ {2}), \tag {52}
$$
$$
\sup  _ {\left\{\theta \in \Theta : L _ {n} \left(\theta , \theta_ {0}\right) \geq C _ {1} n \epsilon^ {2} \right\}} \mathbb {E} _ {P _ {\theta} ^ {n}} \left[ 1 - \phi_ {n} \right] \leq \exp (- C n \epsilon^ {2}), \tag {53}
$$
where  $C_0 = 2*10^s$  and  $C = \frac{C_1}{2K_1^2}$  for a constant  $C_1 > 0$ .
Proof of Lemma C.9: Recall  $d_{L}(\theta, \theta_{0}) = (\sup_{a \in \mathcal{A}} |R(a, \theta) - R(a, \theta_{0})|)$  and  $T(\epsilon) = \{P_{\theta} : d_{L}(\theta, \theta_{0}) < \epsilon\}$ . Using Lemma C.8, observe that for every  $\epsilon > \epsilon_{n} > 0$ ,
$$
N \left(\frac {\epsilon}{2}, \{\theta : \epsilon \leq d _ {L} (\theta , \theta_ {0}) \leq 2 \epsilon \}, d _ {L}\right) \leq N \left(\frac {\epsilon}{2}, \{\theta : d _ {L} (\theta , \theta_ {0}) \leq 2 \epsilon \}, d _ {L}\right) <   1 0 ^ {d}.
$$
Next, using Assumption C.1 we have
$$
d _ {L} (\theta , \theta_ {0}) \leq K _ {1} d _ {H} (\theta , \theta_ {0}).
$$
It follows from the above two observations and Lemma 2 that, for every  $\epsilon >\epsilon_{n} > 0$  , there exist tests  $\{\phi_{n,\epsilon}\}$  such that
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \phi_ {n, \epsilon} \right] \leq 1 0 ^ {d} \frac {\exp \left(- C ^ {\prime} n \epsilon^ {2}\right)}{1 - \exp \left(- C ^ {\prime} n \epsilon^ {2}\right)}, \tag {54}
$$
$$
\sup  _ {\left\{\theta \in \Theta : d _ {L} \left(\theta , \theta_ {0}\right) \geq \epsilon \right\}} \mathbb {E} _ {P _ {\theta} ^ {n}} \left[ 1 - \phi_ {n, \epsilon} \right] \leq \exp \left(- C ^ {\prime} n \epsilon^ {2}\right), \tag {55}
$$
where  $C' = \frac{1}{2K_1^2}$ . Since the above two conditions hold for every  $\epsilon > \epsilon_n$ , we can choose a constant  $K > 0$  such that for every  $\epsilon > \epsilon_n$
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \phi_ {n, \epsilon} \right] \leq 1 0 ^ {d} \frac {\exp \left(- C ^ {\prime} K ^ {2} n \epsilon^ {2}\right)}{1 - \exp \left(- C ^ {\prime} K ^ {2} n \epsilon^ {2}\right)} \leq 2 \left(1 0 ^ {d}\right) e ^ {- C ^ {\prime} K ^ {2} n \epsilon^ {2}}, \tag {56}
$$
$$
\sup  _ {\left\{\theta \in \Theta : L _ {n} \left(\theta , \theta_ {0}\right) \geq K ^ {2} n \epsilon^ {2} \right\}} \mathbb {E} _ {P _ {\theta} ^ {n}} \left[ 1 - \phi_ {n, \epsilon} \right] = \sup  _ {\left\{\theta \in \Theta : d _ {L} \left(\theta , \theta_ {0}\right) \geq K \epsilon \right\}} \mathbb {E} _ {P _ {\theta} ^ {n}} \left[ 1 - \phi_ {n, \epsilon} \right] \leq e ^ {- C ^ {\prime} K ^ {2} n \epsilon^ {2}}, \tag {57}
$$
where the second inequality in (56) holds  $\forall n\geq n_0$  , where  $n_0\coloneqq \min \{n\geq 1:C'K^2 n\epsilon^2\geq \log (2)\}$  Hence, the result follows for  $C_1 = K^2$  and  $C = C^{\prime}K^{2}$
Since  $L_{n}(\theta ,\theta_{0}) = \frac{1}{n} d_{L}^{2}$  satisfies Assumption 2.1, Theorem 3.1 implies the following bound.
Corollary C.1. Fix  $a^\prime \in \mathcal{A}$  and  $\gamma >0$  . Let  $\epsilon_{n}$  be a sequence such that  $\epsilon_{n}\rightarrow 0$  as  $n\to \infty$ $n\epsilon_n^2\geq 1$  and
$$
L _ {n} (\theta , \theta_ {0}) = n \left(\sup  _ {a \in \mathcal {A}} | R (a, \theta) - R (a, \theta_ {0}) |\right) ^ {2}.
$$
Then under the Assumptions of Theorem 3.1 and Lemma C.9; for  $C = \frac{C_1}{2K_1^2}$ ,  $C_0 = 2*10^s$ ,  $C_1 > 0$  such that  $\min(C, C_4(\gamma) + C_5(\gamma)) > C_2 + C_3 + C_4(\gamma) + 2$ , and for  $\eta_n^R(\gamma)$  as defined in Theorem 3.1, the RSVB approximator of the true posterior  $Q_{a',\gamma}^*(\theta|\tilde{X}_n)$  satisfies,
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} L _ {n} \left(\theta , \theta_ {0}\right) Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n}) d \theta \right] \leq n (M (\gamma) \epsilon_ {n} ^ {2} + M \eta_ {n} ^ {R} (\gamma)), \tag {58}
$$
for sufficiently large  $n$  and for a function  $M(\gamma) = 2\left(C_1 + MC_4(\gamma)\right)$ , where  $M = \frac{2C_1}{\min(C, \lambda, 1)}$ .
Proof of Corollary C.1: Using Lemma C.9 observe that for any  $\Theta_n(\epsilon) \subseteq \Theta$ ,  $L_n(\theta, \theta_0)$  satisfies Assumption 2.1 with  $C_0 = 2 * 10^s$ ,  $C = \frac{C_1}{2K_1^2}$  and for any  $C_1 > 0$ , since
$$
\sup_{\{\theta \in \Theta_{n}(\epsilon):L_{n}(\theta ,\theta_{0})\geq C_{1}n\epsilon_{n}^{2}\}}\mathbb{E}_{P_{\theta}^{n}}\bigl[1 - \phi_{n,\epsilon}\bigr ]\leq \sup_{\{\theta \in \Theta :L_{n}(\theta ,\theta_{0})\geq C_{1}n\epsilon_{n}^{2}\}}\mathbb{E}_{P_{\theta}^{n}}\bigl[1 - \phi_{n,\epsilon}\bigr ]\leq e^{-C n\epsilon_{n}^{2}}.
$$
Hence, applying Theorem 3.1 the proof follows.
C.6 Newsvendor Problem
We fix  $n^{-1/2} \sqrt{L_n^{NV}(\theta, \theta_0)} = (\sup_{a \in \mathcal{A}} |R(a, \theta) - R(a, \theta_0)|)$ . Next, we aim to show that the exponentially distributed model  $P_\theta$  satisfies Assumption 2.1, for distance function  $L_n^{NV}(\theta, \theta_0)$ . To show this, in the next result we first prove that  $d_L^{NV}(\theta, \theta_0) = n^{-1/2} \sqrt{L_n^{NV}(\theta, \theta_0)}$  satisfy Assumption C.1. Also, recall that the square of Hellinger distance between two exponential distributions with rate parameter  $\theta$  and  $\theta_0$  is  $d_H^2(\theta, \theta_0) = 1 - 2\frac{\sqrt{\theta \theta_0}}{\theta + \theta_0} = 1 - 2\frac{\sqrt{\theta_0 / \theta}}{1 + \theta_0 / \theta}$ .
Lemma C.10. For any  $\theta \in \Theta = [T,\infty)$ , and  $a\in \mathcal{A}$
$$
d _ {L} ^ {N V} (\theta , \theta_ {0}) \leq \left[ \frac {\left(\frac {h}{\theta_ {0}} - \frac {h}{T}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- \underline {{a}} T}}{T} - \frac {e ^ {- \underline {{a}} \theta_ {0}}}{\theta_ {0}}\right) ^ {2}}{d _ {H} ^ {2} (T , \theta_ {0})} \right] ^ {1 / 2} d _ {H} (\theta , \theta_ {0})
$$
where  $\underline{a} \coloneqq \min \{a \in \mathcal{A}\}$  and  $\underline{a} > 0$  and  $\theta_0$  lies in the interior of  $\Theta$ .
Proof. Observe that for any  $a \in \mathcal{A}$ ,
$$
\begin{array}{l} | R (a, \theta) - R (a, \theta_ {0}) | ^ {2} \\ = \left| \frac {h}{\theta_ {0}} - \frac {h}{\theta} + (b + h) \left(\frac {e ^ {- a \theta}}{\theta} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) \right| ^ {2} \\ = \left(\frac {h}{\theta_ {0}} - \frac {h}{\theta}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- a \theta}}{\theta} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) ^ {2} + 2 \left(\frac {h}{\theta_ {0}} - \frac {h}{\theta}\right) (b + h) \left(\frac {e ^ {- a \theta}}{\theta} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) \\ \leq \left(\frac {h}{\theta_ {0}} - \frac {h}{\theta}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- a \theta}}{\theta} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) ^ {2}, \tag {59} \\ \end{array}
$$
where the last inequality follows since for  $\theta \geq \theta_0$ ,  $\left(\frac{h}{\theta_0} - \frac{h}{\theta}\right) \geq 0$  and  $\left(\frac{e^{-a\theta}}{\theta} - \frac{e^{-a\theta_0}}{\theta_0}\right) < 0$  and vice versa if  $\theta < \theta_0$  that together makes the last term in the penultimate equality negative for all  $\theta \in \Theta$ . Moreover, the first derivative of the upperbound with respect to  $\theta$  is
$$
2 \left(\frac {h}{\theta_ {0}} - \frac {h}{\theta}\right) \frac {h}{\theta^ {2}} - 2 (b + h) ^ {2} \left(\frac {e ^ {- a \theta}}{\theta} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) e ^ {- a \theta} \left[ \frac {1}{\theta^ {2}} + \frac {a}{\theta} \right],
$$
and it is negative when  $\theta \leq \theta_0$  and positive when  $\theta >\theta_0$  for all  $b > 0,h > 0$  , and  $a\in \mathcal{A}$  . Therefore, the upperbound in (59) above is decreasing function of  $\theta$  for all  $\theta \leq \theta_0$  and increasing function of  $\theta$  for all  $\theta >\theta_0$  . The upperbound is tight at  $\theta = \theta_0$
Now recall that the squared Hellinger distance between two exponential distributions with rate parameter  $\theta$  and  $\theta_0$  is
$$
d _ {H} ^ {2} (\theta , \theta_ {0}) = 1 - 2 \frac {\sqrt {\theta \theta_ {0}}}{\theta + \theta_ {0}} = 1 - 2 \frac {\sqrt {\theta_ {0} / \theta}}{1 + \theta_ {0} / \theta} = \frac {(1 - \sqrt {\theta_ {0} / \theta}) ^ {2}}{1 + (\sqrt {\theta_ {0} / \theta}) ^ {2}}.
$$
Note that for  $\theta \leq \theta_0$ ,  $d_H^2 (\theta ,\theta_0)$  is a decreasing function of  $\theta$  and for all  $\theta >\theta_0$  it is an increasing function of  $\theta$ . Also, note that as  $\theta \to \infty$ , the squared Hellinger distance as well as the upperbound computed in (59) converges to a constant for a given  $h,b,\theta_0$  and  $a$ . However, as  $\theta \rightarrow 0$ , the  $d_H^2 (\theta ,\theta_0)\rightarrow 1$  but the upperbound computed in (59) diverges.
Since,  $\Theta = [T,\infty)$  for some  $T > 0$  and  $T\leq \theta_0$ , observe that if we scale  $d_H^2 (\theta ,\theta_0)$  by factor by which the upperbound computed in (59) is greater than  $d_{H}$  at  $\theta = T$ , then
$$
\begin{array}{l} \left(\frac {h}{\theta_ {0}} - \frac {h}{\theta}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- a \theta}}{\theta} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) ^ {2} \\ \leq \frac {\left(\frac {h}{\theta_ {0}} - \frac {h}{T}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- a T}}{T} - \frac {e ^ {- a \theta_ {0}}}{\theta_ {0}}\right) ^ {2}}{d _ {H} ^ {2} (T , \theta_ {0})} d _ {H} ^ {2} (\theta , \theta_ {0}) \\ \leq \frac {\left(\frac {h}{\theta_ {0}} - \frac {h}{T}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- \underline {{a}} T}}{T} - \frac {e ^ {- \underline {{a}} \theta_ {0}}}{\theta_ {0}}\right) ^ {2}}{d _ {H} ^ {2} (T , \theta_ {0})} d _ {H} ^ {2} (\theta , \theta_ {0}), \\ \end{array}
$$
where  $\underline{a} = \inf \{a : a \in \mathcal{A}\}$  and in the last inequality we used the fact that  $\left(\frac{e^{-aT}}{T} - \frac{e^{-a\theta_0}}{\theta_0}\right)^2$  is a decreasing function of  $a$  for any  $b, h, T$ , and  $\theta_0$ . Since, the RHS in the equation above does not depend on  $a$ , it follows from the result in (59) and the definition of  $L_n^{NV}(\theta, \theta_0)$  that
$$
d _ {L} ^ {N V} (\theta , \theta_ {0}) \leq \left[ \frac {\left(\frac {h}{\theta_ {0}} - \frac {h}{T}\right) ^ {2} + (b + h) ^ {2} \left(\frac {e ^ {- \underline {{a}} T}}{T} - \frac {e ^ {- \underline {{a}} \theta_ {0}}}{\theta_ {0}}\right) ^ {2}}{d _ {H} ^ {2} (T , \theta_ {0})} \right] ^ {1 / 2} d _ {H} (\theta , \theta_ {0}).
$$
Lemma C.11. For any  $\theta \in \Theta = [T,\infty)$ , for sufficiently small  $T > 0$ , and  $\theta_0$  lying in the interior of  $\Theta$ , we have
$$
d _ {H} ^ {2} (\theta , \theta_ {0}) = 1 - 2 \frac {\sqrt {\theta \theta_ {0}}}{\theta + \theta_ {0}} \leq \left(\frac {\theta_ {0}}{(T + \theta_ {0}) ^ {2}} \left(\sqrt {\frac {\theta_ {0}}{T}} - \sqrt {\frac {T}{\theta_ {0}}}\right)\right) | \theta - \theta_ {0} |.
$$
$$
\frac {\partial d _ {H} ^ {2} (\theta , \theta_ {0})}{\partial \theta} = - 2 \frac {(\theta + \theta_ {0}) \frac {\sqrt {\theta_ {0}}}{2 \sqrt {\theta}} - \sqrt {\theta \theta_ {0}}}{(\theta + \theta_ {0}) ^ {2}} = \frac {\theta_ {0}}{(\theta + \theta_ {0}) ^ {2}} \left(\sqrt {\frac {\theta}{\theta_ {0}}} - \sqrt {\frac {\theta_ {0}}{\theta}}\right).
$$
Observe that  $\theta \to 0$ ,  $\frac{\partial d_H^2(\theta,\theta_0)}{\partial\theta} \to \infty$ . Since,  $\theta \in \Theta = [T,\infty)$ , therefore the  $\sup_{\theta \in \Theta}\left|\frac{\partial d_H^2(\theta,\theta_0)}{\partial\theta}\right| < \infty$ . In fact, for sufficiently small  $T > 0$ ,  $\sup_{\theta \in \Theta}\left|\frac{\partial d_H^2(\theta,\theta_0)}{\partial\theta}\right| = \left|\frac{\theta_0}{(T + \theta_0)^2}\left(\sqrt{\frac{T}{\theta_0}} - \sqrt{\frac{\theta_0}{T}}\right)\right| = \left(\frac{\theta_0}{(T + \theta_0)^2}\left(\sqrt{\frac{\theta_0}{T}} - \sqrt{\frac{T}{\theta_0}}\right)\right)$ . Now the result follows immediately since the derivative of  $d_H^2 (\theta ,\theta_0)$  is bounded on  $\Theta$ , which implies that  $d_H^2 (\theta ,\theta_0)$  is Lipschitz on  $\Theta$ .
Lemma C.12. For any  $\theta \in \Theta = [T,\infty)$ , and  $a\in \mathcal{A}$
$$
d _ {L} ^ {N V} (\theta , \theta_ {0}) \leq \frac {h}{T ^ {2}} | \theta - \theta_ {0} |.
$$
Proof: Recall,
$$
R (a, \theta) = h a - \frac {h}{\theta} + (b + h) \frac {e ^ {- a \theta}}{\theta}.
$$
First, observe that for any  $a \in \mathcal{A}$ ,