SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Lemma B.7. Fix  $n_2 \geq 2$  and any  $\lambda > 1$ . Let  $A_n := \{ \theta \in \Theta : D_{1 + \lambda}(P_0^n \| P_\theta^n) \leq C_3 n \epsilon_n^2 \}$ , where  $D_{1 + \lambda}(P_0^n \| P_\theta^n)$  is the Rényi Divergence between  $P_0^n$  and  $P_\theta^n$ . Then for  $\epsilon_n^2 = \frac{\log n}{n}$  and any  $C_3 > 0$  such that  $C_2 = \frac{4d}{\overline{\Lambda}(\lambda + 1)\left( \prod_{i=1}^d m(\Theta_i) \right)^{2/d}} C_3 \geq 2$  and for large enough  $n$ , the uncorrelated uniform prior restricted to  $\Theta$  satisfies  $\Pi(A_n) \geq \exp(-nC_2 \epsilon_n^2)$ .
Next, it is straightforward to see that the multi-product newsvendor model risk  $R(a,\theta)$  is bounded below for a given  $a\in \mathcal{A}$  on a compact set  $\Theta$  and thus it satisfies Assumption 2.5. Finally, we also show that the newsvendor model risk satisfies Assumption 2.4.
Lemma B.8. Fix  $n \geq 1$  and  $\gamma > 0$ . For any  $\epsilon > \epsilon_n$  and  $a \in \mathcal{A}$ ,  $R(a, \theta)$  satisfies  $\mathbb{E}_{\Pi}[\mathbb{1}_{\{G(a, \theta) \gamma > C_4(\gamma) n \epsilon^2\}} e^{\gamma G(a, \theta)}] \leq \exp(-C_5(\gamma) n \epsilon_n^2)$ , for any  $C_4(\gamma) > 2\gamma \sup_{\{a, \theta\} \in \mathcal{A} \otimes \Theta} G(a, \theta)$  and  $C_5(\gamma) = C_4(\gamma) - 2\gamma \sup_{\{a, \theta\} \in \mathcal{A} \otimes \Theta} G(a, \theta)$ .
Similar to single product example, in our next result, we show that in the multi-product newsvendor setting, we can construct a sequence  $\{Q_n(\theta)\} \in \mathcal{Q}$  that satisfies Assumption 3.1, and thus identify  $\epsilon_{n}^{\prime}$  and constant  $C_9$ . We fix  $\mathcal{Q}$  to be the family of uncorrelated Gaussian distributions restricted to  $\Theta$ .
Lemma B.9. Let  $\{Q_n(\theta)\}$  be a sequence of product of  $d$  univariate Gaussian distribution defined as  $q_{n}^{i}(\theta)\propto \frac{1}{\sqrt{2\pi\sigma_{i,n}^{2}}} e^{-\frac{1}{2\sigma_{i,n}^{2}} (\theta -\mu_{i,n})^{2}}\mathbb{1}_{\Theta_{i}} = \frac{\mathcal{N}(\theta_{i}|\mu_{i,n},\sigma_{i,n})\mathbb{1}_{\Theta_{i}}}{\mathcal{N}(\Theta_{i}|\mu_{i,n},\sigma_{i,n})}$  and fix  $\sigma_{i,n} = 1 / \sqrt{n}$  and  $\theta_i = \theta_0^i$  for all  $i\in \{1,2,\ldots ,d\}$ . Then for uncorrelated uniform distribution restricted to  $\Theta$  and multivariate normal likelihood model  $\frac{1}{n}\left[\mathrm{KL}\left(Q_n(\theta)\| \Pi (\theta)\right) + \mathbb{E}_{Q_n(\theta)}\left[\mathrm{KL}\left(dP_n^n (\tilde{X}_n)\| dP_\theta^n (\tilde{X}_n)\right)\right]\right]\leq C_9\epsilon_n'^2$ , where  $\epsilon_{n}^{\prime 2} = \frac{\log n}{n}$  and  $C_9\coloneqq \frac{d}{2} +\max \Big(0, - \sum_{i = 1}^{d}[\log (\sqrt{2\pi e}) - \log (m(\Theta_i))] + \frac{d}{2}\underline{\Lambda}^{-1}\Big)$ .
Now, using the result established in lemmas above, we bound the optimality gap in values for the multi-product newsvendor model risk.
Theorem B.1. Fix  $\gamma > 0$ . Suppose that the set  $\mathcal{A}$  is compact. Then, for the multi-product newsvendor model with multivariate Gaussian distributed demand with known covariance matrix  $\Sigma$  and unknown mean vector  $\theta$  lying in a compact subset  $\Theta \subset \mathbb{R}^d$ , prior  $\Pi(\cdot) = \prod_{i=1}^{d} \frac{m(\{\cdot\} \cap \Theta_i)}{m(\Theta_i)}$ , and the variational family fixed to uncorrelated Gaussian distribution restricted to  $\Theta$ , and for any  $\tau > 0$ , the  $P_0^n-$  probability of the following event  $\left\{\tilde{X}_n : R(\mathsf{a}_{\mathbb{R}\mathbb{S}}^*, \theta_0) - \inf_{z \in \mathcal{A}} R(z, \theta_0) \leq 2\tau M'(\gamma)\left(\frac{\log n}{n}\right)^{1/2}\right\}$  is at least  $1 - \tau^{-1}$  for sufficiently large  $n$  and for some mapping  $M' : \mathbb{R}^+ \to \mathbb{R}^+$ , where  $R(\cdot, \theta)$  is the multi-product newsvendor model risk.
Proof. The proof is a direct consequence of Theorem 3.2, Lemmas B.6, B.7, B.8, B.9, and Proposition 3.2.  $\square$
B.3 Gaussian process classification (cont.)
We define the distance function as  $L_{n}^{GP}(\theta, \theta_{0}) = n\left(\sup_{a \in \mathcal{A}} |R(a, \theta) - R(a, \theta_{0})|\right)^{2}$ . In anticipation of demonstrating that the binary classification model with GP prior and distance function  $L_{n}^{GP}$  satisfy the desired set of assumptions, we recall the following result, from [30], which will be central in establishing Assumptions 2.1, 2.2, and 2.3.
Lemma B.10. [Theorem 2.1 [30]] Let  $\theta (\cdot)$  be a Borel measurable, zero-mean Gaussian random element in a separable Banach space  $(\Theta ,\| \cdot \|)$  with reproducing kernel Hilbert space (RKHS)  $(\mathbb{H},\| \cdot \|_{\mathbb{H}})$  and let  $\theta_0$  be contained in the closure of  $\mathbb{H}$  in  $\Theta$ . For any  $\epsilon >\epsilon_{n}$  satisfying  $\varphi_{\theta_0}(\epsilon)\leq n\epsilon^2$  where
$$
\varphi_ {\theta_ {0}} (\epsilon) = \inf  _ {h \in \mathbb {H}: \| h - \theta_ {0} \| <   \epsilon} \| h \| _ {\mathbb {H}} ^ {2} - \log \Pi (\| \theta \| <   \epsilon) \tag {12}
$$
and any  $C_{10} > 1$  with  $e^{-C_{10}n\epsilon_n^2} < 1/2$ , there exists a measurable set  $\Theta_n(\epsilon) \subset \Theta$  such that
$$
\log N \left(3 \epsilon , \Theta_ {n} (\epsilon), \| \cdot \|\right) \leq 6 C _ {1 0} n \epsilon^ {2}, \tag {13}
$$
$$
\Pi (\theta \notin \Theta_ {n} (\epsilon)) \leq e ^ {- C _ {1 0} n \epsilon^ {2}}, \tag {14}
$$
$$
\Pi \left(\left\| \theta - \theta_ {0} \right\| <   4 \epsilon_ {n}\right) \geq e ^ {- n \epsilon_ {n} ^ {2}}. \tag {15}
$$
The proof of the lemma above can be easily adapted from the proof of [30, Theorem 2.1], which is specifically for  $\epsilon = \epsilon_{n}$ . Notice that the result above is true for any norm  $\| \cdot \|$  on the Banach space if that satisfies  $\varphi_{\theta_0}(\epsilon)\leq n\epsilon^2$ . Moreover, if  $\varphi_{\theta_0}(\epsilon_n)\leq n\epsilon_n^2$  is true, then it also holds for any  $\epsilon >\epsilon_{n}$  since by definition  $\varphi_{\theta_0}(\epsilon)$  is a decreasing function of  $\epsilon$ .
All the results in the previous lemma depend on  $\varphi_{\theta_0}(\epsilon)$  being less than  $n\epsilon^2$ . In particular, observe that the second term in the definition of  $\varphi_{\theta_0}(\epsilon)$  depends on the prior distribution on  $\Theta$ . Therefore,
[30, Theorem 4.5] show that  $\varphi_{\theta_0}(\epsilon_n) \leq n\epsilon_n^2$  (with  $\|\cdot\|$  as supremum norm and for  $\epsilon_n$  as defined later in (9)) is satisfied by the Gaussian prior of type
$$
W (\cdot) = \sum_ {j = 1} ^ {\bar {J} _ {\alpha}} \sum_ {k = 1} ^ {2 ^ {j d}} \mu_ {j} Z _ {j, k} \vartheta_ {j, k} (\cdot), \tag {16}
$$
where  $\{\mu_j\}$  is a sequence that decreases with  $j$ ,  $\{Z_{i,j}\}$  are i.i.d. standard Gaussian random variables and  $\{\vartheta_{j,k}\}$  form a double-indexed orthonormal basis (with respect to measure  $\nu$ ), that is  $\mathbb{E}_{\nu}[\vartheta_{j,k}\vartheta_{l,m}] = \mathbb{1}_{\{j = l,k = m\}}$ .  $\bar{J}_{\alpha}$  is the smallest integer satisfying  $2^{\bar{J}_{\alpha}d} = n^{d / (2\alpha +d)}$  for a given  $\alpha >0$ . In particular, the GP above is constructed using the function class that is supported on  $[0,1]^d$  and has a wavelet expansion,  $w(\cdot) = \sum_{j = 1}^{\infty}\sum_{k = 1}^{2^{jd}}w_{j,k}\vartheta_{j,k}(\cdot)$ . The wavelet function space is equipped with the  $L_{2}$ -norm:  $\| w\| _2 = \sum_{j = 1}^{\infty}\left(\sum_{k = 1}^{2^{jd}}|w_{j,k}|^2\right)^{1 / 2}$ ; the supremum norm:  $\| w\|_{\infty} = \sum_{j = 1}^{\infty}2^{jd}\max_{1\leq k\leq 2^{jd}}|w_{j,k}|$ ; and the Besov  $(\beta ,\infty ,\infty)$ -norm:  $\| w\|_{\beta ;\infty ,\infty} = \sup_{1\leq j < \infty}2^{j\beta}2^{jd}\max_{1\leq k\leq 2^{jd}}|w_{j,k}|$ . Note that  $W$  induces a measure over the RKHS  $\mathbb{H}$ , defined as a collection of truncated wavelet functions  $w(\cdot) = \sum_{j = 1}^{\bar{J}_{\alpha}}\sum_{k = 1}^{2^{jd}}w_{j,k}\vartheta_{j,k}(\cdot)$ , with norm induced by the inner-product on  $\mathbb{H}$  as  $\| w\|_{\mathbb{H}}^{2} = \sum_{j = 1}^{\bar{J}_{\alpha}}\sum_{k = 1}^{2^{jd}}\frac{w_{j,k}^{2}}{\mu_{j}^{2}}$ . The RKHS kernel  $K:[0,1]^d\times [0,1]^d\mapsto \mathbb{R}$  can be easily derived as
$$
\begin{array}{l} K (x, y) = \mathbb {E} [ W (x) W (y) ] = \mathbb {E} \left[ \left(\sum_ {j = 1} ^ {\bar {J} _ {\alpha}} \sum_ {k = 1} ^ {2 ^ {j d}} \mu_ {j} Z _ {j, k} \vartheta_ {j, k} (y)\right) \left(\sum_ {j = 1} ^ {\bar {J} _ {\alpha}} \sum_ {k = 1} ^ {2 ^ {j d}} \mu_ {j} Z _ {j, k} \vartheta_ {j, k} (x)\right) \right] \\ = \sum_ {j = 1} ^ {\bar {J} _ {\alpha}} \sum_ {k = 1} ^ {2 ^ {j d}} \mu_ {j} ^ {2} \vartheta_ {j, k} (y) \vartheta_ {j, k} (x). \\ \end{array}
$$
Indeed, by the definition of this kernel and inner product, observe that  $\langle K(x,\cdot),w(\cdot)\rangle = \sum_{j = 1}^{\bar{J}_{\alpha}}\sum_{k = 1}^{2^{jd}}w_{j,k}\mu_j^2\vartheta_{j,k}(x)\frac{1}{\mu_j^2} = w(x)$ . Moreover,  $\langle K(x,\cdot),K(y,\cdot)\rangle = \sum_{j = 1}^{\bar{J}_{\alpha}}\sum_{k = 1}^{2^{jd}}\mu_j^2\vartheta_{j,k}(x)\mu_j^2\vartheta_{j,k}(y)\frac{1}{\mu_j^2} = K(x,y)$ . It is clear from its definition that  $W$  is a centered Gaussian random field on the RKHS.
Next, using the definition of the kernel, we derive the covariance operator of the Gaussian random field  $W$ . Recall that  $Y \sim \nu$ , which enables us to define the covariance operator  $\mathcal{C}$ , following [27, (6.19)] as  $(\mathcal{C}h_{\nu})(x) = \int_{[0,1]^d} K(x,y)h_{\nu}(y)d\nu(y)$ . Also, observe that  $\{\mu_j^2, \varphi_{j,k}\}$  is the eigenvalue and eigen function pair of the covariance operator  $\mathcal{C}$ . Consequently, using Karhunen Loève expansion [27, Theorem 6.19] the prior induced by  $W$  on  $\mathbb{H}$  is a Gaussian distribution denoted as  $\mathcal{N}(0,\mathcal{C})$ . We also recall the Cameron-Martin space denoted as  $\mathrm{Im}(\mathcal{C}^{1/2})$  associated with a Gaussian measure  $\mathcal{N}(0,\mathcal{C})$  on  $\mathbb{H}$  to be the intersection of all linear spaces of full measure under  $\mathcal{N}(0,\mathcal{C})$  [27, (page 530)]. In particular,  $\mathrm{Im}(\mathcal{C}^{1/2})$  is the Hilbert space with inner product  $\langle \cdot, \cdot \rangle_{\mathcal{C}} = \langle \mathcal{C}^{-1/2}, \mathcal{C}^{-1/2} \cdot \rangle$ .
Next, we show the existence of test functions in the following result.
Lemma B.11. For any  $\epsilon >\epsilon_{n}$  with  $\epsilon_{n}\rightarrow 0$ $n\epsilon_n^2\geq 2\log 2$  , and  $\varphi_{\theta_0}(\epsilon)\leq n\epsilon^2$  , there exists a test function  $\phi_{n}$  (depending on  $\epsilon$  ) such that  $L_{n}^{GP}(\theta ,\theta_{0}) = n\left(\sup_{a\in \mathcal{A}}|R(a,\theta) - R(a,\theta_{0})|\right)^{2}$  satisfies Assumption 2.1 with  $C = 1 / 6$ $C_0 = 2$  and  $C_1 = (\max (c_+,\bar{c_-}))^2$
Assumption 2.2 is a direct consequence of (14) in Lemma B.10. Next, we prove that prior distribution and the likelihood model satisfy Assumption 2.3 using (15) of Lemma B.10.
Lemma B.12. For any  $\lambda > 1$ , let  $A_{n} := \left\{\theta \in \Theta : D_{1 + \lambda}(P_{0}^{n}\| P_{\theta}^{n}) \leq C_{3}n\epsilon_{n}^{2}\right\}$ , where  $D_{1 + \lambda}(P_0^n \| P_\theta^n)$  is the Rényi Divergence between  $P_0^n$  and  $P_{\theta}^{n}$ . Then for any  $\epsilon > \epsilon_{n}$  satisfying  $\varphi_{\theta_0}(\epsilon) \leq n\epsilon^2$  and  $C_3 = 16(\lambda + 1)$  and  $C_2 = 1$ , the GP prior satisfies  $\Pi(A_n) \geq \exp(-nC_2\epsilon_n^2)$ .
Assumption 2.4 and 2.5 are straightforward to satisfy since the model risk function  $R(a, \theta)$  is bounded from above and below.
Now, suppose the variational family  $\mathcal{Q}_{GP}$  is a class of Gaussian distributions on  $\Theta$ , defined as  $\mathcal{N}(m_q,\mathcal{C}_q)$ ,  $m_{q}$  belongs to  $\Theta$  and  $\mathcal{C}_q$  is the covariance operator defined as  $\mathcal{C}_q = \mathcal{C}^{1 / 2}(I - S)\mathcal{C}^{1 / 2}$ ,
for any  $S$  which is a symmetric and Hilbert-Schmidt (HS) operator on  $\Theta$  (eigenvalues of HS operator are square summable). Note that  $S$  and  $m_q$  span the distributions in  $\mathcal{Q}_{GP}$ .
The following lemma verifies Assumption 3.1, for a specific sequence of distributions in  $\mathcal{Q}$ .
Lemma B.13. For a given  $J \in \mathbb{N}$ , let  $\{Q_n\}$  be a sequence variational distribution such that  $Q_n$  is the measure induced by a GP,  $W_Q(\cdot) = \theta_0^J(y) + \sum_{j=1}^{J} \sum_{k=1}^{2^{jd}} \zeta_j^2 Z_{j,k} \vartheta_{j,k}(\cdot)$ , where  $\theta_0^J(\cdot) = \sum_{j=1}^{J} \sum_{k=1}^{2^{jd}} \theta_{0:j,k} \vartheta_{j,k}(\cdot)$  and  $\zeta_j^2 = \frac{\mu_j^2}{1 + n\epsilon_n^2\tau_j^2}$ . Then for GP prior induced by  $W = \sum_{j=1}^{J} \sum_{k=1}^{2^{jd}} \mu_j Z_{j,k} \vartheta_{j,k}$  and  $\mu_j = 2^{-jd/2 - ja}$  for some  $a > 0$ ,  $\| \theta_0\|_{\beta; \infty, \infty} < \infty$ , and  $\theta_0^J(y)$  lie in the Cameron-Martin space  $Im(\mathcal{C}^{1/2})$ , we have  $\frac{1}{n} \mathrm{KL}(\mathcal{N}(\bar{\theta}_0^J, \mathcal{C}_q) \| \mathcal{N}(0, \mathcal{C})) + \frac{1}{n} \mathbb{E}_{Q_n} \mathrm{KL}(P_0^n \| P_\theta^n) \leq C_9 \epsilon_n^2$ , where  $\epsilon_n$  is defined in 9 and  $C_9 := \max \left( \| \theta_0\|_{\beta, \infty, \infty}^2, \frac{2^{-2a} - 2^{-2Ja - 2a}}{1 - 2^{-2a}}, 2^d / (2^d - 1), C' \right)$ , where  $C'$  is a positive constant satisfying  $\| \theta_0(y) - \theta_0^J(y)\|_\infty^2 \leq C' 2^{-2J\beta}$ .
Using the result above together with Proposition 3.2 implies that the RSVB posterior converges at the same rate as the true posterior, where the convergence rate of the true posterior is derived in [30, Theorem 4.5] for the binary GP classification problem with truncated wavelet GP prior. Finally, we use the results above to obtain bound on the optimality gap in values of the binary GP classification problem.
C.1 Alternative derivation of LCVB
We present the alternative derivation of LCVB. Consider the logarithm of the Bayes posterior risk,
$$
\begin{array}{l} \log \mathbb {E} _ {\Pi (\theta | \tilde {X} _ {n})} [ \exp (R (a, \theta)) ] = \log \int_ {\Theta} \exp (R (a, \theta)) d \Pi (\theta | \tilde {X} _ {n}) \\ = \log \int_ {\Theta} \frac {d Q (\theta)}{d Q (\theta)} \exp (R (a, \theta)) d \Pi (\theta | \tilde {X} _ {n}) \\ \geq - \int_ {\Theta} d Q (\theta) \log \frac {d Q (\theta)}{\exp (R (a , \theta)) d \Pi (\theta | \tilde {X} _ {n})} =: \mathcal {F} (a; Q (\cdot), \tilde {X} _ {n}) \tag {17} \\ \end{array}
$$
where the inequality follows from an application of Jensen's inequality (since, without loss of generality,  $\exp(R(a,\theta)) > 0$  for all  $a \in \mathcal{A}$  and  $\theta \in \Theta$ ), and  $Q \in \mathcal{Q}$ . Then, it follows that
$$
\begin{array}{l} \min  _ {a \in \mathcal {A}} \log \mathbb {E} _ {\Pi (\theta | \tilde {X} _ {n})} [ \exp (R (a, \theta)) ] \geq \min  _ {a \in \mathcal {A}} \max  _ {q \in \mathcal {Q}} \mathcal {F} (a; Q (\theta), \tilde {X} _ {n}) \\ = \min  _ {a \in \mathcal {A}} \max  _ {q \in \mathcal {Q}} - \operatorname {K L} \left(Q (\theta) \mid \mid \Pi (\theta | \tilde {X} _ {n})\right) + \int_ {\Theta} R (a, \theta) d Q (\theta). \tag {18} \\ \end{array}
$$
C.2 Proof of Theorem 3.1
We prove our main result after a series of important lemmas. For brevity we denote  $\mathcal{LR}_n(\theta, \theta_0) = \frac{p(\tilde{X}_n|\theta)}{p(\tilde{X}_n|\theta_0)}$ .
Lemma C.1. For any  $a' \in \mathcal{A}$ ,  $\gamma > 0$ , and  $\zeta > 0$
$$
\begin{array}{l} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \zeta \int_ {\Theta} L _ {n} (\theta , \theta_ {0}) d Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n}) \right] \\ \leq \log \mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} e ^ {\zeta L _ {n} (\theta , \theta_ {0})} \frac {e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L} \mathcal {R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)}{\int_ {\Theta} e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L} \mathcal {R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)} \right] + \inf  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q (\theta) \| \Pi (\theta | \tilde {X} _ {n})) \right. \\ \left. - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \right] + \log \mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} e ^ {\gamma R \left(a ^ {\prime}, \theta\right)} \frac {\mathcal {L R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta)}{\int_ {\Theta} \mathcal {L R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta)} \right]. \tag {19} \\ \end{array}
$$
Proof. For any fixed  $a' \in \mathcal{A}, \gamma > 0$ , and  $\zeta > 0$ , and using the fact that KL is non-negative, observe that the integral in the LHS of equation (19) satisfies,
$$
\begin{array}{l} \zeta \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} [ L _ {n} (\theta , \theta_ {0}) ] \leq \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} \left[ \log e ^ {\zeta L _ {n} (\theta , \theta_ {0})} \right] \\ + \operatorname {K L} \left( \right.d Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n}) \left\| \right. \frac {e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})}{\int_ {\Theta} e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})}\left. \right) \\ = \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} \left[ \log e ^ {\zeta L _ {n} (\theta , \theta_ {0})} \right] + \log \mathbb {E} _ {\Pi_ {n}} \left[ e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime}, \theta)} \right] \\ + \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} \left[ \log \frac {d Q _ {a ^ {\prime} , \gamma} ^ {*} (\theta | \tilde {X} _ {n})}{e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})} \right] \\ = \log \mathbb {E} _ {\Pi_ {n}} \left[ e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime}, \theta)} \right] + \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} \left[ \log \frac {d Q _ {a ^ {\prime} , \gamma} ^ {*} (\theta | \tilde {X} _ {n})}{e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})} \right]. \\ \end{array}
$$
Next, using the definition of  $Q_{a',\gamma}^{*}(\theta|\tilde{X}_{n})$  in the second term of last equality, for any other  $Q(\cdot) \in \mathcal{Q}$
$$
\zeta \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} \left[ L _ {n} (\theta , \theta_ {0}) \right] \leq \log \mathbb {E} _ {\Pi_ {n}} \left[ e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime}, \theta)} \right] + \mathbb {E} _ {Q} \left[ \log \frac {d Q (\theta)}{e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})} \right].
$$
Finally, it follows from the definition of the posterior distribution that
$$
\begin{array}{l} \zeta \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*}} (\theta | \tilde {X} _ {n}) [ L _ {n} (\theta , \theta_ {0}) ] \\ \leq \log \int_ {\Theta} e ^ {\zeta L _ {n} (\theta , \theta_ {0})} e ^ {\gamma R (a ^ {\prime}, \theta)} \frac {\mathcal {L R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)}{\int_ {\Theta} \mathcal {L R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)} + \mathbb {E} _ {Q} \left[ \log \frac {d Q (\theta)}{e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})} \right], \\ = \log \int_ {\Theta} e ^ {\zeta L _ {n} (\theta , \theta_ {0})} \frac {e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L} \mathcal {R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)}{\int_ {\Theta} e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L} \mathcal {R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)} + \mathbb {E} _ {Q} \left[ \log \frac {d Q (\theta)}{e ^ {\gamma R (a ^ {\prime} , \theta)} d \Pi (\theta | \tilde {X} _ {n})} \right] \\ + \log \int_ {\Theta} e ^ {\gamma R \left(a ^ {\prime}, \theta\right)} \frac {\mathcal {L} \mathcal {R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta)}{\int_ {\Theta} \mathcal {L} \mathcal {R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta)}, \tag {20} \\ \end{array}
$$
where the last equality follows from adding and subtracting  $\log \mathbb{E}_{\Pi}\left[e^{\gamma R(a',\theta)}\mathcal{LR}_n(\theta ,\theta_0)\right]$ . Now taking expectation on either side of equation (20) and using Jensen's inequality on the first and the last term in the RHS yields
$$
\begin{array}{l} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \zeta \mathbb {E} _ {Q _ {a ^ {\prime}, \gamma} ^ {*} (\theta | \tilde {X} _ {n})} \left[ L _ {n} (\theta , \theta_ {0}) \right] \right] \\ \leq \log \mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} e ^ {\zeta L _ {n} (\theta , \theta_ {0})} \frac {e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)}{\int_ {\Theta} e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)} \right] + \inf  _ {Q \in \mathcal {Q}} \mathbb {E} _ {P _ {0} ^ {n}} \left[ \mathrm {K L} (Q \| \Pi_ {n}) \right. \\ \left. - \gamma \inf  _ {a \in \mathcal {A}} \mathbb {E} _ {Q} [ R (a, \theta) ] \right] + \log \mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} e ^ {\gamma R \left(a ^ {\prime}, \theta\right)} \frac {\mathcal {L R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta)}{\int_ {\Theta} \mathcal {L R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta)} \right], \tag {21} \\ \end{array}
$$
where in the second term in RHS of (20), we first take infimum over all  $a \in \mathcal{A}$  which upper bounds the second term in (20) and then take infimum over all  $Q \in \mathcal{Q}$ , since the LHS does not depend on  $Q$ .
Next, we state a technical result that is important in proving our next lemma.
Lemma C.2 (Lemma 6.4 of [34]). Suppose random variable  $X$  satisfies
$$
\mathbb {P} (X \geq t) \leq c _ {1} \exp (- c _ {2} t),
$$
for all  $t \geq t_0 > 0$ . Then for any  $0 < \beta \leq c_2 / 2$
$$
\mathbb {E} [ \exp (\beta X) ] \leq \exp (\beta t _ {0}) + c _ {1}.
$$
Proof. Refer Lemma 6.4 of [34].
In the following result, we bound the first term on the RHS of equation (19). The arguments in the proof are essentially similar to Lemma 6.3 in [34]
Lemma C.3. Under Assumptions 2.1, 2.2, 2.3, 2.4, and 2.5 and for  $\min(C, C_4(\gamma) + C_5(\gamma)) > C_2 + C_3 + C_4(\gamma) + 2$  and any  $\epsilon \geq \epsilon_n$ ,
$$
\mathbb {E} _ {P _ {0} ^ {n}} \left[ \int_ {\Theta} e ^ {\zeta L _ {n} (\theta , \theta_ {0})} \frac {e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L} \mathcal {R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)}{\int_ {\Theta} e ^ {\gamma R (a ^ {\prime} , \theta)} \mathcal {L} \mathcal {R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta)} \right] \leq e ^ {\zeta C _ {1} n \epsilon^ {2}} + (1 + C _ {0} + 3 W ^ {- \gamma}), \tag {22}
$$
for  $0 < \zeta \leq C_{10} / 2$ , where  $C_{10} = \min \{\lambda, C, 1\} / C_1$  for any  $\lambda > 0$ .
Proof. First define the set
$$
B _ {n} := \left\{\tilde {X} _ {n}: \int_ {\Theta} \mathcal {L R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta) \geq e ^ {- \left(1 + C _ {3}\right) n \epsilon^ {2}} \Pi \left(A _ {n}\right) \right\}, \tag {23}
$$
where set  $A_{n}$  is defined in Assumption 2.3. We demonstrate that, under Assumption 2.3,  $P_0^n (B_n^c)$  is bounded above by an exponentially decreasing(in  $n$ ) term. Note that for  $A_{n}$  as defined in Assumption 2.3:
$$
\begin{array}{l} \mathbb {P} _ {0} ^ {n} \left(\frac {1}{\Pi (A _ {n})} \int_ {\Theta} \mathcal {L R} _ {n} (\theta , \theta_ {0}) d \Pi (\theta) \leq e ^ {- (1 + C _ {3}) n \epsilon^ {2}}\right) \\ \leq \mathbb {P} _ {0} ^ {n} \left(\frac {1}{\Pi \left(A _ {n}\right)} \int_ {\Theta \cap A _ {n}} \mathcal {L R} _ {n} \left(\theta , \theta_ {0}\right) d \Pi (\theta) \leq e ^ {- \left(1 + C _ {3}\right) n \epsilon^ {2}}\right). \tag {24} \\ \end{array}
$$
Let  $d\tilde{\Pi}(\theta) \coloneqq \frac{\mathbb{1}_{\{\Theta \cap A_n\}}(\theta)}{\Pi(A_n)} d\Pi(\theta)$ , and use this in (24) for any  $\lambda > 0$  to obtain,