SlowGuess/NeurIPS_2023_OCR huggingface dataset:
[1] Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. arXiv preprint arXiv:2302.07121, 2023.
[2] Gabriel Cardoso, Yazid Janati El Idrissi, Sylvain Le Corff, and Eric Moulines. Monte Carlo guided diffusion for Bayesian linear inverse problems. arXiv preprint arXiv:2308.07983, 2023.
[3] Sourav Chatterjee and Persi Diaconis. The sample size required in importance sampling. The Annals of Applied Probability, 2018.
[4] Gregory Chirikjian and Marin Kobilarov. Gaussian approximation of non-linear measurement models on Lie groups. In 53rd IEEE Conference on Decision and Control. IEEE, 2014.
[5] Nicolas Chopin and Omiros Papaspiliopoulos. An introduction to sequential Monte Carlo. Springer, 2020.
[6] Hyungjin Chung, Jeongsol Kim, Michael T McCann, Marc L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In International Conference on Learning Representations, 2023.
[7] Justas Dauparas, Ivan Anishchenko, Nathaniel Bennett, Hua Bai, Robert J Ragotte, Lukas F Milles, Basile IM Wicky, Alexis Courbet, Rob J de Haas, Neville Bethel, Philip J Leung, Timothy Huddy, Sam Pellock, Doug Tischer, F Chan, Brian Koepnick, H Nguyen, Alex Kang, B Sankaran, Asim K. Bera, Neil P. King, and David Baker. Robust deep learning-based protein sequence design using ProteinMPNN. Science, 2022.
[8] Valentin De Bortoli, Emile Mathieu, Michael John Hutchinson, James Thornton, Yee Whye Teh, and Arnaud Doucet. Riemannian score-based generative modelling. In Advances in Neural Information Processing Systems, 2022.
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 2021.
[10] Arnaud Doucet, Nando De Freitas, and Neil James Gordon. Sequential Monte Carlo methods in practice. Springer, 2001.
[11] Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and MCMC. In International Conference on Machine Learning, 2023.
[12] Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association, 2011.
[13] Samuel Gershman and Noah Goodman. Amortized inference in probabilistic reasoning. In Proceedings of the Annual Meeting of the Cognitive Science Society, 2014.
[14] Neil J Gordon, David J Salmond, and Adrian FM Smith. Novel approach to nonlinear/non-Gaussian Bayesian state estimation. In *IEE Proceedings F (Radar and Signal Processing)*. IET, 1993.
[15] Pieralberto Guarniero, Adam M Johansen, and Anthony Lee. The iterated auxiliary particle filter. Journal of the American Statistical Association, 2017.
[16] Jeremy Heng, Adrian N Bishop, George Deligiannidis, and Arnaud Doucet. Controlled sequential Monte Carlo. Annals of Statistics, 2020.
[34] Brian L Trippe, Jason Yim, Doug Tischer, Tamara Broderick, David Baker, Regina Barzilay, and Tommi Jaakkola. Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem. In International Conference on Learning Representations, 2023.
[35] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Computation, 2011.
[36] Jue Wang, Sidney Lisanza, David Juergens, Doug Tischer, Joseph L Watson, Karla M Castro, Robert Ragotte, Amijai Saragovi, Lukas F Milles, Minkyung Baek, Ivan Anishchenko, Wei Yang, Derrick R Hicks, Marc Exposit, Thomas Schlichthaele, Jung-Ho Chun, Nathaniel Dauparas, Justas Bennett, Basile I M Wicky, Andrew Muenks, Frank DiMaio, Bruno Correia, Sergey Ovchinnikov, and David Baker. Scaffolding protein functional sites using deep learning. Science, 2022.
[37] Joseph L. Watson, David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, Andrew J. Borst, Robert J. Ragotte, Lukas F. Milles, Basile I. M. Wicky, Nikita Hanikel, Samuel J. Pellock, Alexis Courbet, William Sheffler, Jue Wang, Preetham Venkatesh, Isaac Sappington, Susana Vázquez Torres, Anna Lauko, Valentin De Bortoli, Emile Mathieu, Sergey Ovchinnikov, Regina Barzilay, Tommi S. Jaakkola, Frank DiMaio, Minkyung Baek, and David Baker. De novo design of protein structure and function with RFdiffusion. Nature, 2023.
[38] Nick Whiteley and Anthony Lee. Twisted particle filters. The Annals of Statistics, 2014.
[39] Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathieu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. SE(3) diffusion model with application to protein backbone generation. In International Conference on Machine Learning, 2023.
[40] Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, and Shiyu Chang. Towards coherent image inpainting using denoising diffusion implicit models. In International Conference on Machine Learning, 2023.
A Twisted Diffusion Sampler additional details 14
B Riemannian Twisted Diffusion Sampler 18
C Additional Related Work 20
D Empirical results additional details. 20
E Motif-scaffolding application details. 28
A Twisted Diffusion Sampler additional details
In this section we provide additional details on TDS. Appendix A.1 describes its generalization to alternative common formulations of diffusion models. Appendix A.2 describes the choices of proposal variance and resampling strategy. Appendix A.3 provides a derivation of Equation (12). Appendix A.4 describes modifications to the final step of TDS in inpainting and inpainting with degrees of freedom applications. And Appendix A.5 provides a full statement and proof of Theorem 1.
A.1 Diffusion models with non-constant noise variance and variance-preserving scaling
TDS as developed in Section 3 assumed noise with constant variance  $\sigma^2$  added at each timestep. This choice corresponds to a particular discretization of a "variance exploding" (VE) diffusion model [33]. Here we describe generalizations of TDS to non-constant variance schedules, and to variance preserving (VP) diffusion models [18]. We adopt the VP formulation in our experiments; though it introduces additional notation, it is known to work well in practice [33].
TDS algorithm for variance exploding diffusion models. VE diffusion models define the forward process  $q$  by
$$
q \left(x ^ {1: T} \mid x ^ {0}\right) := \prod_ {t = 1} ^ {T} q \left(x ^ {t} \mid x ^ {t - 1}\right), \quad q \left(x ^ {t} \mid x ^ {t - 1}\right) := \mathcal {N} \left(x ^ {t}; x ^ {t - 1}, \sigma_ {t} ^ {2}\right). \tag {16}
$$
where  $\sigma_t^2$  is an increasing sequence of variances such that  $q(x^T) \approx \mathcal{N}(0, \bar{\sigma}_T^2)$ , with  $\bar{\sigma}_t^2 \coloneqq \sum_{t'=1}^{t} \sigma_{t'}^2$  for  $t = 1, \dots, T$ . And so one can set  $p(x^T) = \mathcal{N}(0, \bar{\sigma}_T^2)$  to match  $q(x^T)$ . Notably, Equation (16) implies the conditional  $q(x^t \mid x^0) = \mathcal{N}(x^t; x^0, \bar{\sigma}_t^2)$ .
The reverse diffusion process  $p_{\theta}$  is parameterized as
$$
p _ {\theta} (x ^ {0: T}) := p (x ^ {T}) \prod_ {t = T} ^ {1} p _ {\theta} (x ^ {t - 1} \mid x ^ {t}), \qquad p _ {\theta} (x ^ {t - 1} \mid x ^ {t}) := \mathcal {N} (x ^ {t - 1}; x ^ {t} + \sigma_ {t} ^ {2} s _ {\theta} (x ^ {t}, t), \sigma_ {t} ^ {2})
$$
where the score network  $s_{\theta}$  is modeled through a denoiser network  $\hat{x}_{\theta}$  by  $s_{\theta}(x^{t}, t) \coloneqq (\hat{x}_{\theta}(x^{t}, t; \theta) - x^{t}) / \bar{\sigma}_{t}^{2}$ . Note that the constant schedule is a special case where  $\sigma_{t}^{2} = \sigma^{2}$ , and  $\bar{\sigma}_{t}^{2} = t\sigma^{2}$  for all  $t$ .
The TDS algorithm for general VE models is described in Algorithm 1, where  $t\sigma^2$  in Line 6 is replaced by  $\bar{\sigma}_t^2$  and  $\sigma^2$  in Line 7 is replaced by  $\sigma_t^2$ .
Extension to variance preserving diffusion models. Another widely used diffusion framework is variance preserving (VP) diffusion models [18]. VP models define the forward process  $q$  by
$$
q (x ^ {1: T} \mid x ^ {0}) := \prod_ {t = 1} ^ {T} q (x ^ {t} \mid x ^ {t - 1}), \qquad q (x ^ {t} \mid x ^ {t - 1}) := \mathcal {N} (x ^ {t}; \sqrt {1 - \sigma_ {t} ^ {2}} x ^ {t - 1}, \sigma_ {t} ^ {2})
$$
where  $\sigma_t^2$  is a sequence of increasing variances chosen such that  $q(x^{T})\approx \mathcal{N}(0,1)$ , and so one can set  $p(x^{T}) = \mathcal{N}(0,1)$ . Define  $\alpha_{t}\coloneqq 1 - \sigma_{t}^{2}$ ,  $\bar{\alpha}_{t}\coloneqq \prod_{t^{\prime} = 1}^{t}\alpha_{t^{\prime}}$ , and  $\bar{\sigma}_t^2\coloneqq 1 - \bar{\alpha}_t$ . Then the marginal conditional of eq. (16) is  $q(x^{t}\mid x^{0}) = \mathcal{N}(x^{t};\sqrt{\bar{\alpha}_{t}} x^{0},\bar{\sigma}_{t}^{2})$ . The reverse diffusion process  $p_{\theta}$  is parameterized as
$$
p _ {\theta} (x ^ {0: T}) := p (x ^ {T}) \prod_ {t = T} ^ {1} p _ {\theta} (x ^ {t - 1} \mid x ^ {t}), \quad p _ {\theta} (x ^ {t - 1} \mid x ^ {t}) := \mathcal {N} (x ^ {t - 1}; \frac {1}{\sqrt {\alpha_ {t}}} x ^ {t} + \frac {\sigma_ {t} ^ {2}}{\sqrt {\alpha_ {t}}} s _ {\theta} (x ^ {t}, t), \sigma_ {t} ^ {2})
$$
where  $s_{\theta}$  is now defined through the denoiser  $\hat{x}_{\theta}$  by  $s_{\theta}(x^{t},t)\coloneqq (\sqrt{\overline{\alpha_{t}}}\hat{x}_{\theta}(x^{t},t;\theta) - x^{t}) / \bar{\sigma}_{t}^{2}$ .
TDS in Algorithm 1 extends to VP models as well, where the conditional score approximation in Line 6 is changed to  $\tilde{s}_k\gets (\sqrt{\bar{\alpha}_{t + 1}}\hat{x}_\theta (x_k^{t + 1}) - x_k^{t + 1}) / \bar{\sigma}_{t + 1}^2 +\nabla_{x_k^{t + 1}}\log \tilde{p}_k^{t + 1}$ , and the proposal in Line 7 is changed to  $x_{k}^{t}\sim \tilde{p}_{\theta}(\cdot \mid x_{k}^{t + 1},y)\coloneqq \mathcal{N}\left(\frac{1}{\sqrt{\alpha_{t + 1}}} x_{k}^{t + 1} + \frac{\sigma_{t + 1}^{2}}{\sqrt{\alpha_{t + 1}}}\tilde{s}_{k},\tilde{\sigma}^{2}\right)$ .
A.2 TDS parameters
**Proposal variance.** The proposal distribution in Line 7 of Algorithm 1 is associated with a variance parameter  $\tilde{\sigma}^2$ . In general, this parameter can be dependent on the time step, i.e. replacing  $\tilde{\sigma}^2$  by some  $\tilde{\sigma}_{t+1}^2$  in Line 7. Unless otherwise specified, we set  $\tilde{\sigma}_{t+1}^2 \coloneqq \mathrm{Var}_{p_\theta}[x^t \mid x^{t+1}]$  the variance of the unconditional diffusion model (typically learned along with the score network during training).
Resampling strategy. The multinomial resampling strategy in Line 4 of Algorithm 1 can be replaced by other strategies, see [25, Chapter 2] for an overview. In our experiments, we use the systematic resampling strategy.
In addition, one can consider setting an effective sample size (ESS) threshold (between 0 and 1), and only when the ESS is smaller than this threshold, the resampling step is triggered. ESS thresholds for resampling are commonly used to improve efficiency of SMC algorithms [see e.g. 25, Chapter 2.2.2], but for simplicity we use TDS with resampling at every step unless otherwise specified.
A.3 Derivation of Equation (12)
Equation (12) illustrated how the extended intermediate targets  $\nu_{t}(x^{0:T})$  provide approximations to the final target  $p_{\theta}(x^{0:T} \mid y)$  that become increasingly accurate as  $t$  approaches 0. We obtain Equation (12) by substituting the proposal distributions in Equation (9) and weights in Equation (11) into Equation (4) and simplifying.
$$
\nu_ {t} \left(x ^ {0: T}\right) \propto \left[ p \left(x ^ {T}\right) \prod_ {t ^ {\prime} = 0} ^ {T - 1} \tilde {p} _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}, y\right) \right] \left[ \tilde {p} (y \mid x ^ {T}) \prod_ {t ^ {\prime} = t} ^ {T - 1} \frac {p _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}\right) \tilde {p} _ {\theta} \left(y \mid x ^ {t ^ {\prime}}\right)}{\tilde {p} _ {\theta} \left(y \mid x ^ {t ^ {\prime} + 1}\right) \tilde {p} _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}, y\right)} \right]
$$
Rearrange and cancel  $p_{\theta}$  and  $\tilde{p}_{\theta}$  terms.
$$
= \left[ p \left(x ^ {T}\right) \prod_ {t ^ {\prime} = t} ^ {T - 1} p _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}\right) \right] \left[ \tilde {p} _ {\theta} \left(y \mid x ^ {t}\right) \prod_ {t ^ {\prime} = 0} ^ {t - 1} \tilde {p} _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}, y\right) \right]
$$
Group  $p_{\theta}$  terms by chain rule of probability.
$$
= p _ {\theta} \left(x ^ {t: T}\right) \tilde {p} _ {\theta} (y \mid x ^ {t}) \prod_ {t ^ {\prime} = 0} ^ {t - 1} \tilde {p} _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}, y\right)
$$
Apply Bayes' rule and note that  $p_{\theta}(y \mid x^{t:T}) = p_{\theta}(y \mid x^t)$ .
$$
\propto p _ {\theta} \left(x ^ {t: T} \mid y\right) \frac {\tilde {p} _ {\theta} \left(y \mid x ^ {t}\right)}{p \left(y \mid x ^ {t}\right)} \prod_ {t ^ {\prime} = 0} ^ {t - 1} \tilde {p} _ {\theta} \left(x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1}, y\right)
$$
Note that  $p_{\theta}(x^{0:T} \mid y) = p_{\theta}(x^{t:T} \mid y) \prod_{t=0}^{t' - 1} p_{\theta}(x^{t'} \mid x^{t' + 1}, y)$ .
$$
= p _ {\theta} (x ^ {0: T} \mid y) \left[ \frac {\tilde {p} _ {\theta} (y \mid x ^ {t})}{p (y \mid x ^ {t})} \prod_ {t ^ {\prime} = 0} ^ {t - 1} \frac {\tilde {p} _ {\theta} (x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1} , y)}{p _ {\theta} (x ^ {t ^ {\prime}} \mid x ^ {t ^ {\prime} + 1} , y)} \right].
$$
The final line is the desired expression in Equation (12).
A.4 Inpainting and degrees of freedom final steps for asymptotically exact target
The final  $(t = 0)$  twisting function for inpainting and inpainting with degrees of freedom described in Section 3.3 do not satisfy the assumption of Theorem 2 that  $\tilde{p}_{\theta}(y \mid x^0) = p_{y|x^0}(y|x^0)$ . This choice introduces error in the final target of TDS relative to the exact conditional  $p_{\theta}(x^{0:T} \mid y)$ .
For inpainting, to maintain asymptotic exactness one may instead choose the final proposal and weights as
$$
\tilde {p} _ {\theta} (x ^ {0} \mid x ^ {1}, y; \mathbf {M}) := \delta_ {y} (x _ {\mathbf {M}} ^ {0}) p _ {\theta} (x _ {\mathbf {M}} ^ {0} \mid x ^ {1}) \quad \text {a n d} \quad w _ {0} (x ^ {0}, x ^ {1}) = 1.
$$
One can verify the resulting final target is  $p_{\theta}(x^0 \mid x_{\mathbf{M}}^0 = y)$  according to eq. (4).
Similarly, for inpainting with degrees of freedom, one may define the final proposal and weight as
$$
\tilde {p} _ {\theta} (x ^ {0} \mid x ^ {1}, y, \mathcal {M}) := \sum_ {\mathbf {M} \in \mathcal {M}} \frac {p _ {\theta} (x _ {\mathbf {M}} ^ {0} \mid x ^ {1} , \mathbf {M})}{\sum_ {\mathbf {M} ^ {\prime} \in \mathcal {M}} p _ {\theta} (x _ {\mathbf {M} ^ {\prime}} ^ {0} \mid x ^ {1} , \mathbf {M} ^ {\prime})} \delta_ {y} (x _ {\mathbf {M}} ^ {0}) p _ {\theta} (x _ {\mathbf {M}} ^ {0} \mid x ^ {1}) \quad \mathrm {a n d} \quad w _ {0} (x ^ {0}, x ^ {1}) = 1.
$$
A.5 Asymptotic accuracy of TDS - additional details and full theorem statement
In this section we (i) characterize sufficient conditions on the model and twisting functions under which TDS provides arbitrarily accurate estimates as the number of particles is increased and (ii) discuss when these conditions will hold in practice for the twisting function  $\tilde{p}_{\theta}(y\mid x^{t})$  introduced in Section 3.
We begin with a theorem providing sufficient conditions for asymptotic accuracy of TDS.
Theorem 2. Let  $p_{\theta}(x^{0:T})$  be a diffusion generative model (defined by eqs. (1) and (3)) with
$$
p _ {\theta} \left(x ^ {t} \mid x ^ {t + 1}\right) = \mathcal {N} \left(x ^ {t} \mid x ^ {t + 1} + \sigma_ {t + 1} ^ {2} s _ {\theta} \left(x ^ {t + 1}\right), \sigma_ {t + 1} ^ {2} I\right),
$$
with variances  $\sigma_1^2, \ldots, \sigma_T^2$ . Let  $\tilde{p}_{\theta}(y \mid x^t)$  be twisting functions, and
$$
r _ {t} \left(x ^ {t} \mid x ^ {t + 1}\right) = \mathcal {N} \left(x ^ {t} \mid x ^ {t + 1} + \sigma_ {t + 1} ^ {2} \left[ s _ {\theta} \left(x ^ {t + 1}\right) + \nabla_ {x ^ {t + 1}} \log \tilde {p} _ {\theta} (y \mid x ^ {t + 1}) \right], \tilde {\sigma} _ {t + 1} ^ {2}\right)
$$
be proposals distributions for  $t = 0, \dots, T - 1$ , and let  $\mathbb{P}_K = \sum_{k=1}^K w_k^0 \delta_{x_k^0}$  for weighted particles  $\{(x_k^0, w_k^0)\}_{k=1}^K$  returned by Algorithm 1 with  $K$  particles. Assume
(a) the final twisting function is the likelihood,  $\tilde{p}_{\theta}(y\mid x^0) = p_{y|x^0}(y|x^0)$
(b) the first twisting function  $\tilde{p}_{\theta}(y \mid x^T)$ , and the ratios of subsequent twisting functions  $\tilde{p}_{\theta}(y \mid x^t) / \tilde{p}_{\theta}(y \mid x^{t+1})$  are positive and bounded,
(c) each  $\log \tilde{p}_{\theta}(y \mid x^t)$  with  $t > 0$  is continuous and has bounded gradients in  $x^t$ , and
(d) the proposal variances are larger than the model variances, i.e. for each  $t$ ,  $\tilde{\sigma}_t^2 > \sigma_t^2$ .
Then  $\mathbb{P}_K$  converges setwise to  $p_{\theta}(x^0 \mid y)$  with probability one, that is for every set  $A$ ,  $\lim_{K \to \infty} \mathbb{P}_K(A) = \int_A p_{\theta}(x^0 \mid y) dx^0$ .
The assumptions of Theorem 2 are readily satisfied in common applications.
- Assumption (a) may be satisfied by construction by choosing  $\tilde{p}_{\theta}(y \mid x^0) = p_{y|x^0}(y|x^0)$  by defining  $\hat{x}_{\theta}(x^0, t = 0) = x^0$ .
- Assumption (b) is satisfied if (i)  $p_{y|x^0}(y|x)$  is smooth in  $x$  and everywhere positive and (ii)  $\tilde{p}_{\theta}(y \mid x^t) = p_{y|x^0}(y|\hat{x}_\theta(x^t))$  where  $\hat{x}_\theta(x^t)$  takes values in some compact domain. An alternative sufficient condition for Assumption (b) is for  $p_{y|x^0}(y|x)$  to be positive and bounded away from zero; this latter condition will hold when, for example,  $p_{y|x^0}(y|x)$  is a classifier fit with regularization.
- Assumption (c) is the strongest assumption. It will be satisfied, for example, if (i)  $\tilde{p}_{\theta}(y \mid x^t) = p_{y|x^0}(y|\hat{x}_{\theta}(x^t))$ , and (ii)  $p_{y|x^0}(y|x)$  and  $\hat{x}_{\theta}(x^{t})$  are smooth in  $x$  and  $x^t$ , with uniformly bounded gradients. While smoothness of  $\hat{x}_{\theta}(\cdot, t)$  can be encouraged by the use of skip-connections and regularization,  $\hat{x}_{\theta}(\cdot, t)$  may present sharp transitions, particularly for  $t$  close to zero.
- Assumption (d), that the proposal variances satisfy  $\tilde{\sigma}_t^2 > \sigma_t^2$  is likely not needed for the result to hold. However, this assumption permits usage of existing SMC theoretical results in the proof; in practice, our experiments use  $\tilde{\sigma}_t^2 = \sigma_t^2$ , but alternatively the assumption could be met by inflating each  $\tilde{\sigma}_t^2$  by some arbitrarily small  $\delta$  without markedly impacting the behavior of the sampler.
Proof of Theorem 2: Theorem 2 characterizes a set of conditions under which SMC algorithms converge. We restate this result below in our own notation.
Theorem 3 (Chopin and Papaspiliopoulos [5] - Proposition 11.4). Let  $\{(x_k^0, w_k^0)\}_{k=1}^K$  be the particles and weights returned at the last iteration of a sequential Monte Carlo algorithm with  $K$  particles using multinomial resampling. If each weighting function  $w_t(x^t, x^{t+1})$  is positive and bounded, then for every bounded,  $\nu_0$ -measurable function  $\phi$  of  $x^t$
$$
\lim  _ {K \to \infty} \sum_ {k = 1} ^ {K} w _ {k} ^ {0} \phi \left(x _ {k} ^ {0}\right) = \int \phi \left(x ^ {0}\right) \nu_ {0} \left(x ^ {0}\right) d x ^ {0}.
$$
with probability one.
An immediate consequence of Theorem 3 is the setwise convergence of the discrete measures,  $\hat{P}_K = \sum_{k=1}^K w_k^0 \delta_{x_k^0}$ . This can be seen by taking for each  $\phi(x) = \mathbb{I}[x \in A]$  for any  $\nu_0$ -measurable set  $A$ . The theorem applies both in the Euclidean setting, where each  $x_k^t \in \mathbb{R}^D$ , as well as the Riemannian setting.
We now proceed to prove Theorem 2.
Proof. To prove the theorem we show (i) the  $x^0$  marginal final target  $\nu_0$  is  $p_{\theta}(x^0 \mid y)$  and then (ii)  $\mathbb{P}_K$  converges setwise to  $\nu_0$ .
We first show (i) by manipulating  $\nu_{0}$  in Equation (4) to obtain  $p_{\theta}(x^{0:T} \mid y)$ . From Equation (4) we first have
$$
\nu_ {0} (x ^ {0: T}) = \frac {1}{\mathcal {L} _ {0}} \left[ r (x ^ {T}) \prod_ {t = 0} ^ {T - 1} r _ {t} (x ^ {t} \mid x ^ {t + 1}) \right] \left[ w _ {T} (x ^ {T}) \prod_ {t = 0} ^ {T - 1} w _ {t ^ {\prime}} (x ^ {t}, x ^ {t + 1}) \right]
$$
Substitute in weights from eq. (11).
$$
= \frac {1}{\mathcal {L} _ {0}} \left[ p (x ^ {T}) \prod_ {t = 0} ^ {T - 1} r _ {t} (x ^ {t} \mid x ^ {t + 1}) \right] \left[ \tilde {p} _ {\theta} (y \mid x ^ {T}) \prod_ {t = 0} ^ {T - 1} \frac {p _ {\theta} (x ^ {t} \mid x ^ {t + 1}) \tilde {p} _ {\theta} (y \mid x ^ {t})}{\tilde {p} _ {\theta} (y \mid x ^ {t + 1}) r _ {t} (x ^ {t} \mid x ^ {t + 1})} \right]
$$
Rearrange  $p_{\theta}$  and  $r_t$  terms, and cancel out  $r_t$  terms.
$$
= \frac {1}{\mathcal {L} _ {0}} \left[ p (x ^ {T}) \prod_ {t = 0} ^ {T - 1} p _ {\theta} (x ^ {t} \mid x ^ {t + 1}) \right] \left[ \tilde {p} _ {\theta} (y \mid x ^ {T}) \prod_ {t = 0} ^ {T - 1} \frac {r _ {t} (x ^ {t} \mid x ^ {t + 1}) \tilde {p} _ {\theta} (y \mid x ^ {t})}{\tilde {p} _ {\theta} (y \mid x ^ {t + 1}) r _ {t} (x ^ {t} \mid x ^ {t + 1})} \right]
$$
Collapse  $p_{\theta}$  terms.
$$
= \frac {1}{\mathcal {L} _ {0}} p _ {\theta} \left(x ^ {0: T}\right) \left[ \prod_ {t = 0} ^ {T - 1} \frac {\tilde {p} _ {\theta} \left(y \mid x ^ {t}\right)}{\tilde {p} _ {\theta} \left(y \mid x ^ {t + 1}\right)} \right] \tilde {p} _ {\theta} \left(y \mid x ^ {T}\right)
$$
Cancel out  $\tilde{p}_{\theta}$  terms.
$$
= \frac {1}{\mathcal {L} _ {0}} p _ {\theta} \left(x ^ {0: T}\right) \tilde {p} _ {\theta} \left(y \mid x ^ {0}\right)
$$
Recognize  $\tilde{p}_{\theta}(y|x^0) = p_{y|x^0}(y|x^0)$  by Assm.(a) and apply Bayes' rule with  $\mathcal{L}_0 = p_\theta (y)$ .
$$
= p _ {\theta} (x ^ {0: T} \mid y).
$$
The final line reveals that once we marginalize out  $x^{1:T}$  we obtain  $\nu_0(x^0) = p(x^0 \mid y)$  as desired.
We next show that  $\mathbb{P}_K$  converges to  $\nu_0$  with probability one by applying Theorem 3. To apply Theorem 3 it is sufficient to show that the weights at each step are upper bounded, as they are defined through (ratios of) probabilities and hence are positive. Since there are a finite number of steps  $T$ , it is enough to show that each  $w_{t}$  is bounded. The initial weight is the initial twisting function,  $w_{T}(x^{T}) = \tilde{p}_{\theta}(y\mid x^{T})$ , which is bounded by Assumption (b). So we proceed to intermediate weights.
To show that the weighting functions at subsequent steps are bounded, we decompose the log-weighting functions as
$$
\log w _ {t} (x ^ {t}, x ^ {t + 1}) = \log \frac {\tilde {p} _ {\theta} (y \mid x ^ {t})}{\tilde {p} _ {\theta} (y \mid x ^ {t + 1})} + \log \frac {p _ {\theta} (x ^ {t} \mid x ^ {t + 1})}{r _ {t} (x ^ {t} \mid x ^ {t + 1})},
$$
and show independently that  $\log \tilde{p}_{\theta}(y \mid x^t) / \tilde{p}_{\theta}(y \mid x^{t+1})$  and  $\log p_{\theta}(x^t \mid x^{t+1}) / r_t(x^t \mid x^{t+1})$  are bounded. The first term  $\log \tilde{p}_{\theta}(y \mid x^t) / \tilde{p}_{\theta}(y \mid x^{t+1})$  is again bounded by Assumption (b), and we proceed to the second.
That  $\log p_{\theta}(x^t\mid x^{t + 1}) / r_t(x^t\mid x^{t + 1})$  is bounded follows from Assumptions (c) and (d). First write
$$
p _ {\theta} \left(x ^ {t} \mid x ^ {t + 1}\right) = \mathcal {N} \left(x ^ {t} \mid \hat {\mu}, \sigma_ {t + 1} ^ {2} I\right)
$$
with  $\hat{\mu} = x^{t + 1} + \sigma_{t + 1}^2 s_\theta (x^{t + 1})$ , and
$$
r _ {t} (x ^ {t} \mid x ^ {t + 1}) = \mathcal {N} \left(x ^ {t} \mid \hat {\mu} _ {\psi}, \tilde {\sigma} _ {t + 1} ^ {2} I\right),
$$
for  $\hat{\mu}_{\psi} = \hat{\mu} + \sigma_{t+1}^2 \nabla_{x^{t+1}} \log \tilde{p}_{\theta}(y \mid x^{t+1})$ . The log-ratio then simplifies as
$$
\log \frac {p _ {\theta} (x ^ {t} | x ^ {t + 1})}{r _ {t} (x ^ {t} | x ^ {t + 1})} = \log \frac {| 2 \pi \sigma_ {t + 1} ^ {2} I | ^ {- 1 / 2} \exp \{- (2 \sigma_ {t + 1} ^ {2}) ^ {- 1} \| \hat {\mu} - x ^ {t} \| ^ {2} \}}{| 2 \pi \tilde {\sigma} _ {t + 1} ^ {2} I | ^ {- 1 / 2} \exp \{- (2 \tilde {\sigma} _ {t + 1} ^ {2}) ^ {- 1} \| \hat {\mu} _ {\psi} - x ^ {t} \| ^ {2} \}}
$$