SlowGuess/NeurIPS_2023_OCR huggingface dataset:
where  $\pmb{\theta} \in \Theta$  and  $A(\pmb{\theta}) = \log(Z(\pmb{\theta})) = \log\left(\int_{\mathcal{T}(\mathcal{X})} h(\mathbf{t}) \exp(\pmb{\theta}^\top \mathbf{t}) d\mathbf{t}\right)$  is the log-partition function. Note that  $\nabla^2 A(\pmb{\theta}) = \mathbf{Cov}_{\mathbf{t} \sim p_\theta(\mathbf{t})}[\mathbf{t}] = \mathbf{Cov}_{\mathbf{x} \sim p_\theta(\mathbf{x})}[T(\mathbf{x})]$ , and by A1,  $A(\pmb{\theta})$  is a  $\lambda$ -strongly convex and  $L$ -smooth function in  $\pmb{\theta}$ .
To show that  $p_{\theta}(\mathbf{t})$  is sub-exponential with parameters  $(\nu^2,\beta)$  we need to show that its moment generating function satisfies  $\mathbb{E}[e^{\gamma \mathbf{u}^\top (\mathbf{t} - \boldsymbol {\mu})}]\leq e^{\gamma^2\nu^2 /2}$ , where  $\pmb {\mu} = \mathbb{E}_{p_{\theta}}[\mathbf{t}]$ ,  $\mathbf{u}$  is a unit vector, for  $|\gamma | < 1 / \beta$ .
$$
\begin{array}{l} \mathbb {E} \left[ e ^ {\gamma \mathbf {u} ^ {\top} (\mathbf {t} - \boldsymbol {\mu})} \right] = \int \left(e ^ {\gamma \mathbf {u} ^ {\top} \mathbf {t} - \gamma \mathbf {u} ^ {\top} \boldsymbol {\mu}}\right) h (\mathbf {t}) e ^ {\boldsymbol {\theta} ^ {\top} \mathbf {t} - A (\boldsymbol {\theta})} d \mathbf {t} \\ = \frac {\exp (- \gamma \mathbf {u} ^ {\top} \boldsymbol {\mu})}{Z (\boldsymbol {\theta})} \int h (\mathbf {t}) \exp ((\gamma \mathbf {u} + \boldsymbol {\theta}) ^ {\top} \mathbf {t}) d \mathbf {t} \\ = \frac {Z (\gamma \mathbf {u} + \boldsymbol {\theta})}{Z (\boldsymbol {\theta})} \cdot \exp (- \gamma \mathbf {u} ^ {\top} \boldsymbol {\mu}) \\ \end{array}
$$
The inequality we need to show is equivalent to proving
$$
\begin{array}{l} \mathbb {E} [ e ^ {\gamma \mathbf {u} ^ {\top} (\mathbf {t} - \boldsymbol {\mu})} ] \leq e ^ {\gamma^ {2} \nu^ {2} / 2} \\ \Longleftrightarrow \frac {Z (\gamma \mathbf {u} + \boldsymbol {\theta})}{Z (\boldsymbol {\theta})} \cdot e ^ {- \gamma \mathbf {u} ^ {\top} \boldsymbol {\mu}} \leq e ^ {\gamma^ {2} \nu^ {2} / 2} \\ \Longleftrightarrow \frac {Z (\gamma \mathbf {u} + \boldsymbol {\theta})}{Z (\boldsymbol {\theta})} \leq e ^ {\gamma \mathbf {u} ^ {\top} \boldsymbol {\mu}} \cdot e ^ {\gamma^ {2} \nu^ {2} / 2} \\ \Longleftrightarrow A (\gamma \mathbf {u} + \boldsymbol {\theta}) - A (\boldsymbol {\theta}) \leq \gamma \mathbf {u} ^ {\top} \boldsymbol {\mu} + \frac {\gamma^ {2} \nu^ {2}}{2} \\ \end{array}
$$
Since  $A(\pmb {\theta})$  is  $L$  -smooth, we have that
$$
A (\gamma \mathbf {u} + \boldsymbol {\theta}) - A (\boldsymbol {\theta}) \leq \langle \underbrace {\nabla A (\boldsymbol {\theta})} _ {= \boldsymbol {\mu}}, \gamma \mathbf {u} \rangle + \frac {L}{2} \| \gamma \mathbf {u} \| ^ {2} = \gamma \mathbf {u} ^ {\top} \boldsymbol {\mu} + \frac {\gamma^ {2} L}{2}
$$
where we've used the property of exponential families that the gradient of the log partition function is the mean sufficient statistic. Now we can see that the appropriate parameter for  $\nu^2$  is  $L$  and  $\gamma$  must be small enough so that  $\gamma u + \theta \in \Theta$ , i.e.,  $|\gamma| < \frac{1}{\beta}$  for some  $\beta > 0$ . This is possible if  $\theta$  is bounded away from the boundary of  $\Theta$ .
Remark. In the above, we only needed to use that  $p_{\theta}$  is an exponential family distribution and that its log-partition function  $A(\pmb{\theta})$  is smooth. It is also possible to show that  $p_{\theta}$  has exponentially decreasing tails (in quantities involving  $\mathbf{x}$  rather than  $T(\mathbf{x})$ ) if it is log-concave in  $\mathbf{x}$  (assumption A2), e.g., by [39].
C.3 Proof of Lemma 3.4
Let  $p_{\theta}(\mathbf{x}) = h(\mathbf{x})\exp (\langle \theta ,T(\mathbf{x})\rangle -A(\pmb {\theta}))$  and  $A\colon \Theta \to \mathbb{R}$  is the log-partition function:
$$
A (\boldsymbol {\theta}) = \int_ {\mathcal {X}} h (\mathbf {x}) \exp (\langle \theta , T (\mathbf {x}) \rangle) d \mathbf {x}.
$$
Lemma C.3. For any  $q > 1, \theta, \theta' \in \Theta$ :
$$
\mathbb {E} _ {p _ {\boldsymbol {\theta}}} \left[ \left(\frac {p _ {\boldsymbol {\theta} ^ {\prime}}}{p _ {\boldsymbol {\theta}}}\right) ^ {q} \right] = \exp \left((q - 1) A (\boldsymbol {\theta}) - q A (\boldsymbol {\theta} ^ {\prime}) + A \left(q \boldsymbol {\theta} ^ {\prime} - (q - 1) \boldsymbol {\theta}\right)\right).
$$
$$
\begin{array}{l} \mathbb {E} _ {p _ {\boldsymbol {\theta}}} \left[ \left(\frac {p _ {\boldsymbol {\theta} ^ {\prime}}}{p _ {\boldsymbol {\theta}}}\right) ^ {q} \right] = \int_ {\mathcal {X}} h (x) \exp \left(\langle \boldsymbol {\theta}, T (x) \rangle - A (\boldsymbol {\theta})\right) \cdot \exp \left(q \langle \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta}, T (x) \rangle - q A (\boldsymbol {\theta} ^ {\prime}) + q A (\boldsymbol {\theta})\right) d x \\ = \exp \left((q - 1) A (\boldsymbol {\theta}) - q A \left(\boldsymbol {\theta} ^ {\prime}\right) + A \left(q \boldsymbol {\theta} ^ {\prime} - (q - 1) \boldsymbol {\theta}\right)\right). \\ \end{array}
$$
Lemma C.4. Assume  $A$  is convex and  $L$ -smooth on  $\Theta$ . For any  $S \subseteq \mathcal{X}$ , and  $\theta, \theta' \in \Theta$ :
Proof. By Cauchy-Schwarz,
$$
\begin{array}{l} p _ {\boldsymbol {\theta} ^ {\prime}} (S) ^ {2} = \mathbb {E} _ {p _ {\boldsymbol {\theta}}} \left[ \frac {p _ {\boldsymbol {\theta} ^ {\prime}}}{p _ {\boldsymbol {\theta}}} \mathbf {1} _ {S} \right] ^ {2} \\ \leq p _ {\boldsymbol {\theta}} (S) \cdot \mathbb {E} _ {p _ {\boldsymbol {\theta}}} \left[ \left(\frac {p _ {\boldsymbol {\theta} ^ {\prime}}}{p _ {\boldsymbol {\theta}}}\right) ^ {2} \right] \\ = p _ {\pmb {\theta}} (S) \cdot \exp \left(A (\pmb {\theta}) - 2 A (\pmb {\theta} ^ {\prime}) + A \left(2 \pmb {\theta} ^ {\prime} - \pmb {\theta}\right)\right). \\ \end{array}
$$
Since  $A$  is convex and  $L$ -smooth,
$$
A (\boldsymbol {\theta}) \leq A \left(\boldsymbol {\theta} ^ {\prime}\right) + \langle \nabla A (\boldsymbol {\theta}), \boldsymbol {\theta} - \boldsymbol {\theta} ^ {\prime} \rangle
$$
$$
A (2 \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta}) \leq A (\boldsymbol {\theta} ^ {\prime}) + \langle \nabla A (\boldsymbol {\theta} ^ {\prime}), \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta} \rangle + \frac {L}{2} \| \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta} \| ^ {2}
$$
$$
\begin{array}{l} A (\boldsymbol {\theta}) - 2 A \left(\boldsymbol {\theta} ^ {\prime}\right) + A \left(2 \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta}\right) \leq \langle \nabla A \left(\boldsymbol {\theta} ^ {\prime}\right) - \nabla A (\boldsymbol {\theta}), \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta} \rangle + \frac {L}{2} \| \boldsymbol {\theta} ^ {\prime} - \boldsymbol {\theta} \| ^ {2} \\ \leq \frac {3 L}{2} \| \pmb {\theta} ^ {\prime} - \pmb {\theta} \| ^ {2}. \\ \end{array}
$$
Compare this to the Gaussian case (e.g., see H.8 of [37]) where this was  $p_{\pmb{\theta}}(S) \geq \frac{\alpha}{2} \exp \left(-r \cdot \sqrt{2 \log 1 / \alpha} - \frac{1}{2} r^2\right)$  for  $\| \pmb{\theta} - \pmb{\theta}' \| < r$ .
C.4 Proof of Lemma 3.7
Let  $\overline{T} = \frac{1}{n}\sum_{i=1}^{n}T(\mathbf{x}_i)$  be the empirical mean sufficient statistics given our samples  $\{\mathbf{x}_i\}_{i=1}^n$  each  $\mathbf{x}_i \sim p_{\boldsymbol{\theta}^*}^S$ .
Let  $\epsilon_S > 0$ . For  $n \geq \Omega \left(\frac{2\beta}{\epsilon_S} \log \left(\frac{1}{\delta}\right)\right)$ ,
$$
\| \overline {{T}} - \mathbb {E} _ {p _ {\boldsymbol {\theta} ^ {*}}} [ T (\mathbf {x}) ] \| \leq \epsilon_ {S} + \mathcal {O} (\log 1 / \alpha)
$$
Proof. Let  $\pmb{\nu}^{*} = \mathbb{E}_{\mathbf{x}\sim p_{\theta^{*}}}[T(\mathbf{x})]$ .
For any event  $A$ , we have that
$$
\mathbb {P} _ {p _ {\pmb {\theta} ^ {*}} ^ {S}} [ A ] = \int \mathbb {1} \{\omega \in A \} d p _ {\pmb {\theta} ^ {*}} ^ {S} (\omega) = \frac {1}{\alpha} \int \mathbb {1} \{\omega \in A \} \mathbb {1} \{\omega \in S \} d p _ {\pmb {\theta} ^ {*}} (\omega) \leq \frac {1}{\alpha} \mathbb {P} _ {p _ {\pmb {\theta} ^ {*}}} [ A ]
$$
and for the product measure with  $n$  independent components  $\mathbb{P}_{\Pi_{i\in [n]}p_{\theta^*}^S}[A]\leq \left(\frac{1}{\alpha}\right)^n\mathbb{P}_{\Pi_{i\in [n]}p_{\theta^*}}[A]$ . So we can bound the probability of events on  $p_{\theta^*}^S$  with those on  $p_{\theta^*}$ . In particular, by Claim 1 and by the composition property of independent sub-exponential random variables, we have that
$$
\begin{array}{l} \mathbb {P} _ {p _ {\theta^ {*}}} \left(\frac {1}{n} \left| \mathbf {u} ^ {\top} \left(\sum_ {i} T (\mathbf {x} _ {i}) - \boldsymbol {\nu} ^ {*}\right) \right| \geq t\right) \leq \exp \left(- \frac {n t}{2 \beta}\right) \quad \text {f o r a n y u n i t v e c t o r u} \\ \Rightarrow \mathbb {P} _ {p _ {\boldsymbol {\theta} ^ {*}}} \left(\left\| \frac {1}{n} \sum_ {i} T (\mathbf {x} _ {i}) - \boldsymbol {\nu} ^ {*} \right\| \geq t\right) \leq \exp \left(- \frac {n t}{2 \beta}\right). \\ \end{array}
$$
To translate this to the probability of the same event on  $p_{\theta^*}^S$ , note that
$$
\left(\frac {1}{\alpha}\right) ^ {n} \exp \left(- \frac {n t}{2 \beta}\right) \leq \delta \Longleftrightarrow \exp \left(n \cdot \left(\log 1 / \alpha - \frac {t}{2 \beta}\right)\right) \leq \delta
$$
which holds when  $t = 2\beta \left( \log 1 / \alpha + \frac{1}{n} \log 1 / \delta \right)$ . Thus for  $n > \frac{2\beta}{\epsilon_S} \log 1 / \delta$  samples from the truncated  $p_{\pmb{\theta}^*}^S$  we have that with probability at least  $1 - \delta$ , the quantity  $\| \overline{T} - \pmb{\nu}^* \| \leq 2\beta (\log 1 / \alpha) + \epsilon_S$ .
D Additional Proofs for Algorithm Analysis
Algorithm 3 Stochastic Gradient Descent
Initialize some  $\theta_0\in K$
for iteration  $t = 1,2,\dots ,T$  do
Compute  $\mathbf{v}_t$  such that  $\mathbb{E}[\mathbf{v}_t\mid \boldsymbol {\theta}_t] = \nabla f(\boldsymbol {\theta}_t)$
$\widetilde{\pmb{\theta}}_{t + 1}\gets \pmb {\theta}_t - \eta \mathbf{v}_t$
$\widetilde{\pmb{\theta}}_{t + 1} = \Pi_K(\widetilde{\pmb{\theta}}_{t + 1})$  (Project onto  $K$
Return  $\theta_T$
D.1 SGD Algorithm and its Analysis
Although the setting of Theorem 5.7 of [20] is when the objective is a sum of many functions, the proof and its result can be easily adapted to our setting.
Theorem. Let  $f$  be a  $\lambda$ -strongly convex function. Let  $\pmb{\theta}^{*}\in \arg \min_{\pmb{\theta}\in K}f(\pmb{\theta})$ . Consider the sequence  $\{\pmb{\theta}_t\}_{t = 1}^N$  generated by SGD (Algorithm 3) and  $\{\mathbf{v}_t\}_{t = 1}^N$  the sequence random vectors satisfying  $\mathbb{E}[\mathbf{v}_t|\pmb {\theta}_t] = \nabla f(\pmb {\theta}_t)$  and  $\mathbb{E}[[\| \mathbf{v}_t\|^2 |\pmb {\theta}_t] < \rho^2$  for all  $t$ , with a constant step size  $\eta$  satisfying  $0 < \eta < \frac{1}{\lambda}$ . It follows that for  $t\geq 0$
Proof. At any iteration  $i$ ,
$$
\widetilde {\boldsymbol {\theta}} _ {i + 1} = \boldsymbol {\theta} _ {i} - \eta \mathbf {v} _ {i}
$$
$$
\widetilde {\boldsymbol {\theta}} _ {i + 1} - \boldsymbol {\theta} ^ {*} = \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} - \eta \mathbf {v} _ {i} \tag {1}
$$
$$
\left\| \widetilde {\boldsymbol {\theta}} _ {i + 1} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} = \left\| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} - 2 \eta \langle \mathbf {v} _ {i}, \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \rangle + \eta^ {2} \| \mathbf {v} _ {i} \| ^ {2}
$$
where the last line comes from multiplying the line (1) with the transpose of the same equation on either side. After projecting to the set  $K$  to obtain  $\theta_{i + 1} = \arg \min_{\pmb {\theta}\in K}\| \widetilde{\pmb{\theta}}_{i + 1} - \pmb {\theta}\| ^2$  and given that  $\pmb {\theta}^{*}\in K$ , we have that  $\| \widetilde{\pmb{\theta}}_{i + 1} - \pmb {\theta}^{*}\|^{2}\geq \| \pmb{\theta}_{i + 1} - \pmb{\theta}^{*}\|^{2}$ , so
$$
\left\| \boldsymbol {\theta} _ {i + 1} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} \leq \left\| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} - 2 \eta \langle \mathbf {v} _ {i}, \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \rangle + \eta^ {2} \| \mathbf {v} _ {i} \| ^ {2} \tag {2}
$$
Fixing  $\theta_{i}$  in the  $i^{th}$  iteration and taking the conditional expectation in (2) gives
$$
\begin{array}{l} \mathbb {E} [ \| \boldsymbol {\theta} _ {i + 1} - \boldsymbol {\theta} ^ {*} \| ^ {2} \mid \boldsymbol {\theta} _ {i} ] \leq \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \| ^ {2} - 2 \eta \langle \nabla f (\boldsymbol {\theta} _ {i}), \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \rangle + \eta^ {2} \mathbb {E} [ \| \mathbf {v} _ {t} \| ^ {2} \mid \boldsymbol {\theta} _ {i} ] \\ \leq (1 - 2 \eta \lambda) \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \| ^ {2} + \eta^ {2} \mathbb {E} [ \| \mathbf {v} _ {t} \| ^ {2} | \boldsymbol {\theta} _ {i} ] \\ \end{array}
$$
where the last line is due to strong convexity,  $\langle \nabla f(\pmb{\theta}_i), \pmb{\theta}_i - \pmb{\theta}^* \rangle \geq \lambda \| \pmb{\theta}_i - \pmb{\theta}^* \|^2$ . By taking iterated expectations and recursively applying the above, we get that
$$
\begin{array}{l} \mathbb {E} \left[ \left\| \boldsymbol {\theta} _ {T} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} \right] \leq (1 - 2 \eta \lambda) ^ {T} \left\| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} + \eta^ {2} \rho^ {2} \sum_ {i = 0} ^ {T - 1} (1 - 2 \eta \lambda) ^ {i} \\ \leq \left(1 - 2 \eta \lambda\right) ^ {T} \left\| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} + \eta^ {2} \frac {1}{\eta \lambda} \rho^ {2} \\ = (1 - 2 \eta \lambda) ^ {T} \| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \| ^ {2} + \frac {\eta}{\lambda} \rho^ {2} \\ \end{array}
$$
where in the second line we used that  $\sum_{i=0}^{T-1}(1 - 2\eta \lambda)^{i} = \frac{1 - (1 - 2\eta\lambda)^{T}}{1 - (1 - 2\eta\lambda)} < \frac{2}{2\eta\lambda}$  provided  $\eta < 1/\lambda$ .
We can derive the complexity (number of iterations) to get  $\mathbb{E}[\|\pmb{\theta}_T - \pmb{\theta}\|^2] < \epsilon$  using the following Lemma from [20].
Lemma D.1 (Lemma A.2 of [20]). Consider the recurrence given by
$$
\alpha_ {k} \leq (1 - \eta \mu) ^ {t} \alpha_ {0} + A \eta ,
$$
where  $\mu > 0$ , and  $A, C \geq 0$  are given constants and  $\eta < 1 / C$ . If
$$
\eta = \min  \left\{\frac {\epsilon}{2 A}, \frac {1}{C} \right\}
$$
$$
t \geq \max  \left\{\frac {1}{\epsilon} \frac {2 A}{\mu}, \frac {C}{\mu} \right\} \log \left(\frac {2 \alpha_ {0}}{\epsilon}\right) \Rightarrow \alpha_ {k} \leq \epsilon .
$$
Note that to get bounds on  $\| \pmb{\theta}_T - \pmb{\theta}^* \|$  rather than  $\| \pmb{\theta}_T - \pmb{\theta}^* \| ^2$ , we can solve the number of iterations we need to get  $\epsilon^2$  on the right hand side, and we will get the number of iterations for  $\| \pmb{\theta}_T - \pmb{\theta}^* \| < \epsilon$ . Then the resulting complexity bounds will replace  $1 / \epsilon$  with  $1 / \epsilon^2$ .
D.2 Approximate Sampling of Non-Truncated Distribution
Many analyses of stochastic gradient descent assume unbiased directions at every iteration of the algorithm, but since we need to be able to sample from  $p_{\theta_i}$  multiple times at each iteration  $i$  until we get a sample in  $S$ , our directions are only unbiased if we can indeed sample exactly from  $p_{\theta_i}$  each time for all  $i$ .
However if  $p_{\theta_i}$  is complicated, exact sampling can be difficult or take too long. Since we assume that  $p_{\theta}$  is log-concave in  $x$  for all  $\theta \in \Theta$ , we can at least approximately sample from it efficiently via Langevin Monte Carlo, MALA, or other algorithms with convergence guarantees for log-concave densities.
Lemma D.2 (SGD Analysis with Biased Directions). Let  $f$  be a  $\lambda$ -strongly convex function. Let  $\pmb{\theta}^{*}\in \arg \min_{\pmb {\theta}\in K\subseteq B(\pmb{\theta}^{*},D)}f(\pmb {\theta})$ . Consider the sequence  $\{\pmb {\theta}_t\}_{t = 1}^N$  generated by SGD but with  $\{\widetilde{\mathbf{v}}_t\}_{t = 1}^N$  the sequence random vectors satisfying  $\mathbb{E}[\widetilde{\mathbf{v}}_t|\pmb {\theta}_t] = \mathbf{b}_t - \nabla f(\pmb {\theta}_t)$  with  $\| \mathbf{b}_t\| < B$  and  $\mathbb{E}[\| \widetilde{\mathbf{v}}_t\|^2 |\pmb {\theta}_t] < \rho^{\prime 2}$  for all  $t$ , with a constant step size  $\eta$  satisfying  $0 < \eta < \frac{1}{\lambda}$ . It follows that for  $t\geq 0$
$$
\mathbb {E} \| \boldsymbol {\theta} _ {t} - \boldsymbol {\theta} ^ {*} \| ^ {2} \leq (1 - 2 \eta \lambda) ^ {t} \| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \| ^ {2} + \frac {\eta}{\lambda} \rho^ {2} + \frac {2 B D}{\lambda}.
$$
Proof. Define  $\mathbf{b}_t \coloneqq \mathbb{E}[\mathbf{b}_t \mid \boldsymbol{\theta}_t] - \nabla f(\boldsymbol{\theta}_t)$  the bias for each  $t \geq 0$ . The analysis of Theorem 3.10 can be applied generically to get Eq. (2):
$$
\left\| \boldsymbol {\theta} _ {i + 1} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} \leq \left\| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} - 2 \eta \langle \widetilde {\mathbf {v}} _ {i}, \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \rangle + \eta^ {2} \| \widetilde {\mathbf {v}} _ {i} \| ^ {2}
$$
Now when taking the conditional expectation, we get
$$
\begin{array}{l} \mathbb {E} \left[ \| \boldsymbol {\theta} _ {i + 1} - \boldsymbol {\theta} ^ {*} \| ^ {2} \mid \boldsymbol {\theta} _ {t} \right] \leq \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \| ^ {2} - 2 \eta \langle \mathbf {b} _ {t}, \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \rangle - 2 \eta \langle \nabla f (\boldsymbol {\theta} _ {i}), \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \rangle - \eta^ {2} \mathbb {E} \left[ \| \widetilde {\mathbf {v}} _ {i} \| ^ {2} \mid \boldsymbol {\theta} _ {i} \right] \\ \leq (1 - 2 \eta \lambda) \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \| ^ {2} + \eta^ {2} \rho^ {\prime 2} + 2 \eta \| \mathbf {b} _ {i} \| \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \| \\ \end{array}
$$
Taking iterated expectations and recursively applying this gives now
$$
\begin{array}{l} \mathbb {E} [ \| \boldsymbol {\theta} _ {T} - \boldsymbol {\theta} ^ {*} \| ^ {2} ] \leq (1 - 2 \eta \lambda) ^ {T} \| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \| ^ {2} + \sum_ {i = 0} ^ {T - 1} (1 - 2 \eta \lambda) ^ {i} \cdot \left(\eta^ {2} \rho^ {\prime 2} + 2 \eta \| \mathbf {b} _ {i} \| \| \boldsymbol {\theta} _ {i} - \boldsymbol {\theta} ^ {*} \|\right) \\ \leq \left(1 - 2 \eta \lambda\right) ^ {T} \left\| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \right\| ^ {2} + \frac {\eta^ {2} \rho^ {\prime 2} + 2 \eta B D}{\eta \lambda} \\ = (1 - 2 \eta \lambda) ^ {T} \| \boldsymbol {\theta} _ {0} - \boldsymbol {\theta} ^ {*} \| ^ {2} + \frac {\eta \rho^ {\prime 2}}{\lambda} + \frac {2 B D}{\lambda} \\ \end{array}
$$
where the second line holds if  $\| \mathbf{b}_i\| \leq B,\| \pmb {\theta}_i - \pmb{\theta}^*\| <  D,\forall i$  (which holds under the assumptions).
Note that in our Algorithm 1,  $D$  here is simply  $\frac{2}{\lambda} d(\alpha)$  by construction. We can also control  $B$  through the following.
Bounding the bias. Fix some  $t$ . Let  $\widetilde{\mathbf{v}}_t \coloneqq T(\mathbf{z}) - T(\mathbf{x})$  where  $\mathbf{z} \sim \widetilde{p}_{\boldsymbol{\theta}_i}^S$  and  $\mathbf{x} \sim p_{\boldsymbol{\theta}^*}^S$ . Then we can write
$$
\begin{array}{l} \mathbf {b} _ {t} = \mathbb {E} _ {\mathbf {z} \sim \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S}} \left[ T (\mathbf {z}) \right] - \mathbb {E} _ {\mathbf {x} \sim p _ {\boldsymbol {\theta} ^ {*}} ^ {S}} \left[ T (\mathbf {x}) \right] + \mathbb {E} _ {\mathbf {z} \sim p _ {\boldsymbol {\theta} _ {i}} ^ {S}} \left[ T (\mathbf {z}) \right] - \mathbb {E} _ {\mathbf {x} \sim p _ {\boldsymbol {\theta} ^ {*}} ^ {S}} \left[ T (\mathbf {x}) \right] \\ = \mathbb {E} _ {\mathbf {z} \sim \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S}} [ T (\mathbf {z}) ] - \mathbb {E} _ {\mathbf {z} \sim p _ {\boldsymbol {\theta} _ {i}} ^ {S}} [ T (\mathbf {z}) ] \\ = \int_ {\mathcal {X}} T (\mathbf {x}) \cdot \left(\widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} (\mathbf {x}) - p _ {\boldsymbol {\theta} _ {i}} ^ {S} (\mathbf {x})\right) d \mathbf {x} \tag {3} \\ \end{array}
$$
If we know that  $T(\mathbf{x})$  is bounded over  $S$ , we can upper-bound this given the TV distance between  $\widetilde{p}_{\boldsymbol{\theta}_i}^S$  and  $p_{\boldsymbol{\theta}_i}^S$ :
$$
\left\| \mathbf {b} _ {t} \right\| \leq \sup  _ {\mathbf {x} \in S} \left\| T (\mathbf {x}) \right\| \left\| \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} - p _ {\boldsymbol {\theta} _ {i}} ^ {S} \right\| _ {T V}.
$$
Since we assume that  $\mathbb{E}_{p_{\theta_i}}[T(\mathbf{x})]$  is finite, it should be the case that  $T(\mathbf{x})$  is bounded over its support except potentially on some negligible sets. In that case, we can replace  $T(\mathbf{x})$  with  $\widetilde{T}(\mathbf{x})$  which replaces those potentially infinite values on negligible sets with 0 and the integral expression in (3) would be equal to one which uses  $\widetilde{T}(\mathbf{x})$  instead of  $T(\mathbf{x})$ , and the bound on its norm holds given that  $\widetilde{T}(\mathbf{x})$  is bounded.
Otherwise we can use bounds from Lemma C.1 to bound this as
$$
\left\| \mathbf {b} _ {t} \right\| \leq \sqrt {\chi^ {2} \left(\widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} \mid \mid p _ {\boldsymbol {\theta} _ {i}} ^ {S}\right)} \cdot \sqrt {\mathbf {V a r} _ {p _ {\boldsymbol {\theta} _ {i}} ^ {S}} (T (\mathbf {x}))}
$$
if we have control over the chi-square divergence (see Section C.1 for definitions).
If we know that  $T(x)$  is a 1-Lipschitz, real-valued function (e.g., when  $T(x) = x$ ), we can use the dual representation of  $W_{1}$  distance to bound this as
$$
\mathbf {b} _ {t} = \int_ {S} T (x) d \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} - \int_ {S} T (x) d p _ {\boldsymbol {\theta} _ {i}} ^ {S} \leq \sup _ {f \in \mathcal {F} _ {1 \mathrm {L i p}}} \int f (x) d \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} - \int f (x) d p _ {\boldsymbol {\theta} _ {i}} ^ {S} = W _ {1} (\widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S}, p _ {\boldsymbol {\theta} _ {i}} ^ {S}).
$$
Proposition D.3 (Bounds on truncated total variation, given bounds on non-truncated). Suppose  $\| \widetilde{p}_{\theta_i} - p_{\theta_i}\|_{TV}\leq \epsilon_{TV}$  for some  $\epsilon_{TV} > 0$ . Then
$$
\| \widetilde {p _ {\boldsymbol {\theta} _ {i}} ^ {S}} - p _ {\boldsymbol {\theta} _ {i}} ^ {S} \| _ {T V} \leq \frac {\epsilon_ {T V} ^ {2}}{p _ {\boldsymbol {\theta} _ {i}} (S) - \epsilon_ {T V}}.
$$
Proof. First, given that  $\| \widetilde{p}_{\boldsymbol{\theta}_i} - p_{\boldsymbol{\theta}_i}\|_{TV} \leq \epsilon_{TV}$ , we have by one characterization of the total variation distance (the supremum of the difference in mass over all measurable sets)
$$
\widetilde {p} _ {\boldsymbol {\theta} _ {i}} (S) \geq p _ {\boldsymbol {\theta} _ {i}} (S) - \epsilon_ {T V}.
$$
Now for the truncated densities,
$$
\begin{array}{l} \| \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} - p _ {\boldsymbol {\theta} _ {i}} ^ {S} \| _ {T V} = \frac {1}{2} \int | \widetilde {p} _ {\boldsymbol {\theta} _ {i}} ^ {S} (\mathbf {x}) - p _ {\boldsymbol {\theta} _ {i}} ^ {s} (\mathbf {x}) | d \mathbf {x} \\ = \frac {1}{2} \int \mathbb {1} \left\{\mathbf {x} \in S \right\} \cdot \left| \frac {\widetilde {p} _ {\boldsymbol {\theta} _ {i}} (\mathbf {x})}{\widetilde {p} _ {\boldsymbol {\theta} _ {i}} (S)} - \frac {p _ {\boldsymbol {\theta} _ {i}} (\mathbf {x})}{p _ {\boldsymbol {\theta} _ {i}} (S)} \right| d \mathbf {x} \\ \leq \frac {\epsilon_ {T V}}{p _ {\boldsymbol {\theta} _ {i}} (S) - \epsilon_ {T V}} \cdot \frac {1}{2} \int | \widetilde {p} _ {\boldsymbol {\theta} _ {i}} (\mathbf {x}) - p _ {\boldsymbol {\theta} _ {i}} (\mathbf {x}) | d \mathbf {x} \\ \leq \frac {\epsilon_ {T V} ^ {2}}{p _ {\boldsymbol {\theta} _ {i}} (S) - \epsilon_ {T V}} \\ \end{array}
$$