Preconditioned Langevin Dynamics with Score-based Generative Models for Infinite-Dimensional Linear Bayesian Inverse Problems
Authors: Lorenzo Baldassari
Josselin Garnier
Knut Solna
Maarten V. de Hoop
Venue: NeurIPS 2025 spotlight ()
Keywords: theory
score-based generative models
error analysis
hilbert space
langevin dynamics
preconditioning
Primary area: theory

Abstract: Designing algorithms for solving high-dimensional Bayesian inverse problems directly in infinite‑dimensional function spaces – where such problems are naturally formulated – is crucial to ensure stability and  convergence as the discretization of the underlying  problem is refined. In this paper, we contribute to this line of work by analyzing a widely used sampler for linear inverse problems: Langevin dynamics driven by score‑based generative models (SGMs) acting as priors, formulated directly in function space. Building on the  theoretical framework for SGMs in Hilbert spaces, we give a rigorous definition of this sampler in the infinite-dimensional setting and derive, for the first time, error estimates that explicitly depend on the approximation error of the score. As a consequence, we obtain sufficient conditions for global convergence in Kullback–Leibler divergence on the underlying function space. Preventing numerical instabilities requires preconditioning of the Langevin algorithm and we prove the existence  and form of an optimal preconditioner. The preconditioner depends on both the score error and the forward operator and guarantees a uniform convergence rate across all posterior modes. Our analysis applies to both Gaussian and a general class of non‑Gaussian priors. Finally, we present examples that illustrate and validate our theoretical findings.

Preconditioned Langevin Dynamies with Score-Based Generative Models for Infinite-Dimensions Linear Bayesian Inverse Problems
Lorenzo Baldassari University of Basel
Josselin Garnier Ecole Polytechnique, IP Paris http::/iggaaieeppoloreeccieeieee edu
aeddasssriuuiiaas.....a lorenzo. baldassari @unibas ch
Knut Sølna University of California Irvine ksolna@uci.. edu
Maarten V. de Hoop Rice University
mvd2@rice. edu
Abstract
Designing algorithms for solving high- ligh-dimensional Bayesian inverse problems di- rectly in diiiite-dimeniionll function phere such problems are naturally formulated-1 is crucial to ensure stability and convergence as the discretization of the underlying problem is refined. this paper, we contribute to this line of work by analyzing a widely used sampler for linear inverse problems: Langevin dynamics driven by score-based generative models (SGMs) acting as priors, formulated di- rectly in function space. Building on the theoretical framework for SGMs in Hilbert spaces, we give a rigorous definition of this sampler in the infiniteddimensional setting and derive, for the first time, етror estimates that explicitly depend on the approximation error of the score. As a consequence, we obtain sufficient conditions for global convergence in Kullback-Leibler divergence the underlying function space. Preventing numerical instabilities requires preconditioning of the Langevin algorithm and we prove the existence and the form of optimal preconditioner. The preconditioner depends on both the score error and the forward operator and guarantees uniform convergence rate across all posterior modes. Our analysis applies to both Gaussian and a general class of non-Gaussian priors. Finally, we present examples that illustrate and validate theoretical findings.
Introduction
Inverse problems arise in many challenging applications, such X-ray computed tomography. seismic tomography, inverse heat conduction, and inverse scattering. These problems share common goal: to estimate unknown parameters from noisy observations or measurements []] What makes them difficult is that they are often ill- ill-posed in the sense of Hadamard (2): they may have multiple solutions, no solutions at all, or solutions that are highly sensitive small perturbations in the data. A possible probabilistic framework known as
approach address these difficulties is to cast the problem Bayesian inference. In the Bayesian approach, one first specifies a prior distribution that encodes knowledge about the unknown before any data is observed, along with model for the observational noise. Bayes' rule then used to update this prior knowledge in light of the measurements, yielding the so-called posterior distribution, which describes the distribution of the unknown conditioned on the data. By sampling from the posterior one can extract statistical information and quantify uncertainty in the solution 3
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

A central challenge in applying Bayesian inference to inverse problems is that in many cases- especially those governed by partial differential equations (PDEs) the unknowns to be estimated are functions that lie in suitable function space, typically an infrnitedimmnnional Hilbert space. It therefore crucial to design Bayesian inference algorithms that are both theoretically sound and computationally effective in arbitrarily high dimensions. A way to achieve this is by lifting these problems to an This approach, sometimes referred to as Cupply-algoritm--teee -hent-ttiieenieieee"" -or, in the context of Bayesian inference, "Bayesianize-then-eee then-discretize" daiicretizeeallllva -allows for the development of algorithms that inherently discretization invarrinn,, as the Bayes formula and algorithms are properly defined on Hilbert spaces [ 77 In contrast, the opposite approach- ppproach""disceeee rdiscretize-ther then-Bayysiinize" can lead to several issues, such instability the discretization of the underlying problem is refined, or worse, methods that seem stable but whose results are theoretically implausible g 9].
space and designing inference methods directly in that setting.
These considerations manifest clearly even in simple scenarios. In Figure 1 we consider two examples involving a vanilla diffusion Langevin sampler. the first one, we sample from a Gaussian posterior. While the method appears numerically stable and produces samples with seemingly reasonable behavior, a closer inspection shows that the samples carry infinite energy-they do not belong to the anfinieedimmensioall Hilbert space. That is, the algorithm is producing objects that are not valid functions in the limit of refined discretization. In the second example, we attempt to fix this by choosing trace-class prior, which ensures that samples have finite energy and are well-defined in Hilbert space. This theoretically-mnn motivvted structure, however, comes at a cost: without adjustments, the drift of the vanilla Langevin sampler may diverge at fine scales.
Longesio 105
Sample putli andlyi poszeikr
Poccsiar sa larce
12
80-
Lungeu Lees
Sorvcie posfee STRVEE poiterlo
Pesteror verlece
190
-20
2500
2000
ny in the basis (ey) of the Hilbert space H, with A, = e -0.1 and n, ~ (0, 0.053), for 100. In the top row, sample the posterior using an identity prior covariance on Xo. The Langevin diffusion seems stable, but the cigenvalues of the posterior covariance do not decay infinity and therefore the samples do not belong to the Hilbert space H. In the bottom row, we use a trace-class covariance prior with diagonal terms 1/52; the drift of the vanilla Langevin sampler starts diverging at fine scales.
Figure 1: We consider the toy linear inverse problem y An x())
These types of challenges, intrinsic to the infinite-dimensionall setting, have long been studied in the Bayesian inverse problems community, but are now receiving renewed attention with the rise of deep learning methods for posterior sampling. One popular class of methods that still lacks a
complete theoretical understanding this context is score-based generative models (SGMs), which
generate samples from complex distributions by first learning the (Stein) score-the gradient of the log-density [10] and then using it in various sampling algorithms [2] For example, [T3] employs the learned score in a Langevin-based based sampler, while [14] unified SGMs and diffusion- based
methods [6] through stochastic differential equation (SDE) framework, known as score-based diffusion models. After their introduction, SGMs have been applied successfully to Bayesian inverse problems, either by learning the score conditioned on data 17-19, or by using the score of the prior distribution- the unconditional score within I Langevin-type samplers. Crucially, with few exceptions (20-23), these approaches assume that the posterior is supported a fimite-dimensional space, leaving the challenges of infinite dimensions to heuristics and ad hoc solutions.

---

In this work, we present a detailed analysis of SGMs for Bayesian inference of linear inverse problems, going beyond the common assumption that the posterior is supported on a finiteddimensional space. We focus on widely used posterior sampling technique that combines SGMs used as powerful learned priors to capture complex features-wi problem directly to function spaces is not a mere technicality: we show that, to provably sample the posterior, the Langevin diffusion must be modified by a preconditioning operator C' acting on the Hilbert space. This preconditioner is not ad hoc fix but rather built into the fabric of the infinite- dimensional setting: it first appears in the forward diffusion process ( whose time-reversal is used to learn the prior, and must then be carried through into the Langevin sampler to ensure convergence to the correct posterior (Section 3). Crucially, C' cannot be the identity: for the time-reversed diffusion to remain Hilbert- weaeettssppp-allaaaee valuee, C. must be trace class. Setting C' = I leads to the same theoretical and numerical issues as highlighted in Figure @ ahove.
a Langevin-based sampler 24/26)], Lifting the
The importance of preconditioning in function spaces has been well established in the context of posterior sampling + о (7--3)), but its implications have not yet fully explored for infinite- dimensional SGMs. In this setting, we characterize the interplay between the preconditioner C, the trace-class prior, the score approximation error, and the linear forward map of the inverse problem. In particular, we analyze the impact of the score approximation error at small times-where the score is learned in practice- and identify a theoretically optimal preconditioner that ensures uniform convergence rates across posterior modes (Section 4). We carry out the analysis by focusing two cases: a Gaussian prior measure, and a more general class of priors which are absolutely continuous with respect to a Gaussian measure (Section 5). Illustrations are provided in Section 6
MCMC algorithms 8 40, which include a variety of preconditioning strategies for posterior sampling. However, these works precede the recent wave of papers on SGMs and therefore do not address the central focus of our analysis: the interplay between the score approximation error, the preconditioning operator, the trace-class prior, and the sampler convergence, which we study in detail in both the Gaussian and non-Gaussian settings.
Related Work. There exists a large body of literature on unfnite-dimensionn
Closest to our work are the papers that use SGMs for posterior sampling, such as (2 26). which employ SGMs as learned priors in a Langevin-type diffusion algorithm. Among these works, the theoretical analysis of [25 the most directly related to ours. However, there are key differences. 25 analyze Langevin dynamics with SGMs for posterior sampling in finite dimensions, as their results provide convergence error estimates that explicitly depend on the score approximation error but diverge the dimension of the problem increases. In contrast, our error analysis, since it is formu- lated directly in infinite dimensions, provides conditions to ensure global boundedness (Theorem 3.1). Moreover, the finite-dimensional setting of (25) does not address the role of preconditioning, which becomes essential in infinite dimensions. Other related works include 41 42] which investigate pre- conditioning in Langevin dynamics with SGMs. However, these analyses
also minte-aimensional and do not account for the score approximation error. As a result, they do not capture the critical role of preconditioning, which-an as show in Section 4 becomes crucial in function spaces.
As we have pointed out several times, the learned score plays a key role in our analysis. Among the theoretical frameworks defining SGMs in infinite dimensions (3 43 48 we follow those of (2) 49 for continuous- -time diffusions. An important contribution of our work to show that the convergence bound depends explicitly on the accuracy of the approximated score, and that controlling this error is key to designing a preconditioner that ensures convergence in function spaces (Theorem 4.1).
Finally, we note that [21 also explores the role of preconditioning to ensure convergence in infinite dimensions in the context of SGMs. Their analysis conducted in a more complex setting- nonlineea inverse problems. Their argument builds on the proof of 225, but the difficulties of the nonlinear setting prevent them from identifying an optimal preconditioner. In contrast, our work takes full advantage of the linear setting, where the distributions at play admit explicit formulas. This allows us to derive detailed error estimates in the small diffusion time regime, where the score is typically learned, and discuss the impact of the score approximation error on posterior sampling sanpling-including including the effects of preconditioning on the bias error. Furthermore, their analysis focuses only on Gaussian priors, while we generalize and consider non-Gaussian priors (Section 5).
3

---

2 Langevin Posterior Sampling with Score-Based Generative Priors
We work in the setting of a linear Bayesian inverse problem formulated in infinite dimension. Let H be a separable Hilbert space with inner product (-,)), and let C, C. H - H be trace-class, positive-definit, symmetric covariance operators. The unknown quantity of interest is H-valued random variable Xy ~ where the prior measure is assumed to be absolutely continuous with respect Gaussian reference measure (0, C,). . with density
du d0 C.))
(X) exp ((X)).
The observations y E R are modeled as
AXu n,
where A H + -R is a linear operator, and ~ N(0,02Ix is Gaussian observational noise independent of Xo- Since we consider an observational model corresponding to observing finite- dimensional subspace of H, there exists an orthonormal basis (1z) of H such that Au = 0 for all j > N. Let (e,) denote the standard basis of R.V. Then the observation model can be written as V/ Σラ= A.y x + n4, where Aij = (es: uz)). yi = (y, e,), x( (Xo, 05), and n, (n, e.). The posterior distribution Ny of Xo conditioned the observations y is absolutely continuous with respect to (0, C₁):
diy (X) & exp (X( dN Cj)
20 |AX v1f²)
1
The goal of fiinite-dimensiona. Bayesian inference is design sampling methods for ny whose performance remains stable as the underlying discretization refined. To this end, we study a widely used sampler-a Langevin-type diffusion driven by score-based generative priors-this time formulated directly in infinite dimensions rather than in the usual finite-dimensional setting. In particular, we consider the continuous-time SDE
dx, = So T; ptdt + CVx log ply - AX,)dt + V2CdW,
(3)
where p is the noise density, C acts preconditioner, V Vx denotes the Fréchet derivative with respect to X, W, is a Wiener process on H, and (X,, t; μ1) is a neural network approximation of the score function
S(X, r: (1) = - - - (X - / =X),
(4)
which corresponds to the drift term in the time-reversed SDE of the Hilbert-space-vallee valued forward diffusion
1 ---=- dx, = X + VCdw,,
Xo ~
(5)
There are two important aspects to note here. First, both the Langevin SDE (3) and the forward diffusion 5 are driven by C-Wiener process, where C' is trace-class, which is crucial for ensuring that the samples are supported on the Hilbert space. Most of the technical difficulties in infinite dimensions arise from this. Second, although the score is often expressed
log Pr in finite- dimensional settings, the density pr is not defined in infinite dimensions, since a Lebesgue reference measure does not exist. For this reason, in the following we adopt the conditional expectation representation of the score in the next proposition, which was first proved in [49].
more precisely, an equivalent formulation derived from it, as stated
Proposition 2.1. The score (4) can be written
S(X, Ti = -IC
X, =X]-
(6)
where
The idea behind samplers like 33 simple yet powerful. By training So(X, T; T; to approximate S(X, 7; μ1), one can effectively learn potentially complex j-nince, once Sp(X, r; is known, one can sample from by simulating the backward-in-timm aynamics-and then incorporate such priors within Langevin sampling scheme. What remains less understood, however, is how this

---

approach extends to the anfinit-ddmeeseee by approximating the score and whether the sampler remains stable. In the sections that follow, we address this gap- prove convergence of (3) to the correct posterior and derive error bounds, along with conditions ensuring globally bounded convergence error. We also elucidate the role of the preconditioner C. Our analysis is divided into two parts: one addressing the case of Gaussian priors, and the other the non-Gaussian case.
setting, particularly in relation to the error introduced
3 Error Analysis in the Gaussian Setting
We begin our analysis of the continous time Langevin SDE 3 in the fiimnte-dimensiioall setting by examining the case where the prior of Xo Gaussian measure. While this case may seem to defeat the purpose of using a score-based generative model to learn a simple prior, it provides it allows us detail the impact of the score approximation error the stationary distribution of (3), offers a clear interpretation of the dimeetteadd difficulties, and paves the way for the derivation of an explicit form of the optimal preconditioner (Section
illuminating insights,
We assume in this section that & 0. The posterior
Gaussian:
Ky = W (I0 +o 7 2A -
(qp + to +
We also assume that both C and Cy, are diagonal in the basis (1z), with eigenvalues (Ay) and (15). respectively. We can make a few remarks:
the (0y) basis, the posterior decomposes into Gaussian π. on the span of the first N observed modes and product of marginal Gaussian over the unobserved modes N.
For the observed modes i.e., those N influenced by the data through the forward operator the distribution is
N =N +
σ --PANy, to =A-AAAAN]] with CA,N Diag (u²)
my
15$5N
For the unobserved modes j > N, which lie in the nullspace of A, the posterior coincides with the prior: πy = N pz).
(5)
The score function is S(X,T; n) E, (T(T:))X)0 uj. with sy(T;)) - (-
The block diagonalization of the system by (uz) justifies the following assumption on the form of the score approximation error.
Assumption 1. We consider an approximate score So p)) such that
(S(X, 1) So(X, 14), U5) = xx)
Define X = Σn X vj and similarly let W.N denote the projection of W, onto the first N
modes. By Assumption 1 become
for the observed modes j s N, the preconditioned Langevin dynamics
d.N.
Diag (s)(Tin)) + Diag (eG(T)) + CCNO--2NN XiNet ISJEN
ISiSN
+
ONa ANN
Diag (c)(T)) dt V2CxdWN.
ISiSN
with ON Diag (Az). For the unobserved modes j > N, we have ISJSN
ax( = [s,(T:1) (s(15()))) x
()) + 22)
We are now ready to derive the stationary distribution of the continuous-time SDE 3 The following proposition makes explicit the dependence on the score approximation error; its proof is included in Appendix A.I
5

---

Proposition 3.1. The stationary distribution Ny of the preconditioned Langevin diffusion with approx- imate score in the drift term is the Gaussian measure with mean mm(T) = (m (T), (mng(T)) (2))
and covariance ö(T) = の. (T) + Diag (ūG(T)). For the observed modes j S N, we have
12N+1
8 (7) ON
Diag (85(T:14)) to 3ANAN + CN ISiSN
Diag ISSEN
(c(())
(T) = (T) (T) 2 AAyy CN'
Diag ISJSN
(2)(T))
(8)
while for the unobserved modes j >> N, we have
- =
+ (y sy(T;)) + [
( (1)))==(((-))66000
(9)
Based on Proposition 3.1 we make few comments:
If we have access to the perfect score, that is, ej 5 0 for all j. then
mT() C + - AAT A
2 y [ o
A
--
b(T) -o [ = [ +o ATA A
That is, we recover the posterior Ty given the data. does not depend on the preconditioner C.
The error eg can have an impact on the stationary distribution of x( but as long as is smaller than Aạ/My (i.e., the relative error in the approximation of the score small), the impact is small.
The error e/ can induce a bias. The bias can be large because the mean of the j-th mode marginal of #(()) is amplified by X5 The preconditioner cannot prevent from this bias.
We can make our analysis more quantitative by presenting mode-by-mode and global convergence error estimates for the preconditioned Langevin sampler in the Gaussian setting. To simplify the discussion, the following theorem is stated by assuming that A AN Av is diagonal.
Theorem 3.1. We define P5 >g/Pg for all j. Let Ht. and πy denote the j-th mode marginals of the approximate and true posterior distributions, Ry and Ty. respectively. Suppose that Pj =T( O(T). X3 (T) O(1). Then, for j S. N, the Sellback-Leibles divergence satisfies
DKL (16))
x 2
>j AT=5(())
(0 (ATT() x7)))(7)) ((D - 1)7 -7 + +0(7). (10)
11 2 (AT AN)is
nT (1)) (7) (5 + (p. - 1 + 0(72).
For j > N, we have D
Proof. The proof relies on Proposition 3.1 and the fact that the j-th mode marginals # and x.
are Gaussian, (m) (r)) and A ((m,, vj), respectively, hence the Kullback-Leibler divergence has an explicit form and standard perturbation arguments lead to the desired estimates. Full details are provided in Appendix A.2
Remark 3.1. Note that Theorem 3.1 can provide set of sufficient conditions that ensure that the global convergence error of the sampler is bounded in infinite dimensions: E, |^, ( < oo,
P7 (T) < C1. and (ANg)s) < <C2, where C1, C2 do not depend j.
4 The Essence of Preconditioning
We now elucidate the role of the preconditioner C' in the infinite-dimensionee Gaussian setting introduced in the previous section. We begin with two preliminary remarks:

---

In our analysis, C' first appears in the forward diffusion ( whose time-reversal learss the prior, and must be carried through the Langevin sampler (3) to target the correct posterior.
C cannot be the identity: it must be trace-class to keep the diffusion well-posee and to stabilize the Langevin updates across all modes. Indeed, if C = Digg(>g), the drift in the j-th mode contains the factor Xyle_Tu + (1 -e -) which, unless X, decays sufficiently fast, blows up like 14 as j + oo, making the sampler unstable at fine scales. This is a consequence of the infinite-dimessinna setting, where C, must be trace-class.
Since the preconditioner plays a role in the rate of convergence across all posterior modes, it is natural to ask whether there exists a C that ensures a uniform convergence rate for the Langevin sampler. To this aim, in the next propositionwe derive the mean reversion rate K of the preconditioned Langevin dynamics 3 the proof is given in Appendix B.I
Proposition 4.1. Assume that AN is diagonal. For the observed modes j S N, the mean reversion rate is
x - >3 ((e + (1 (1- to (ANAAN) + +
(II)
while for the unobserved modes j > N, k00 = A35 [ + (1 - ) + 17 c7()))
We make a few comments:
For the unobserved modes j >> N, the convergence rate is Xyle +1y+(0 (1- X for small 7) when the error 03 is negligible, and therefore we should choose X, By for all j to get convergence uniform in j. that is to say, C = Cj..
For the observed modes j $ N, the convergence rates for those modes such that jy (2/(AAANN), (or (A) = 0) are >//Ug, whereas for modes such that Hy ( the convergence rates are > AN We should then choose ) [u, + o AN)) or equivalently C'= [C= to AA
We now refine our analysis of the preconditioner by incorporating first-order correction that accounts for the score approximation error at small т, the regime in which the score is typically learned. The proof of the following theorem relies on straightforward perturbation argument; full details given in Appendix [B.2]
Theorem 4.1. In addition to Assumption we further suppose that A A is diagonal, and that (T) = EGT + O(r2). Under these condition, the optimal preconditioner C is also diagonal in the basis (1z), with eigenvalues that admit the expansion >3- = =>, + > T O(+2). For the observed
modes j S N, we have
- + - (ATAN)s
100 11 11 -2 - -x,0 43 66
(0)
(12)
For the unobserved modes j > N, we have A!o = Ms. 11 -nz5J.
Based on Theorem
we make a few comments:
To compute the preconditioner C, one would need information on A A, a, Cu. and the score approximation error. Our analysis, however, suggests a simple and practical choice: take as close as possible to the prior covariance C,.. For higher-order modes, the deading-order term of the preconditioner coincides with Cy., and this approximation is particularly justified when the prior decays quickly, so that μ4;" > o 2 2(A (ANAAN AN) for the low-order modes. Any available knowledge of the posterior covariance score error can then be used to retine this first approximation.
This is not the first occurrence in the literature of an optimal preconditioner for diffusion models in infinite dimensions. For example, while analyzing the convergence error of time-reversed SDE dynamics in infinite dimensions, Pidstrigach et al. [49] derived a similar result for the optimal C' by minimizing the Wasserstein-2 2 distance between the true data distribution and the learned sample distribution. Interestingly, assuming no data model and perfect score, our framework yields the same optimal C, with a few caveats: our case, the preconditioner arises directly from the mean-reversion rate of the Langevin dynamics. Hence, the optimal covariance we identify does not merely minimize an upper bound: represents, under the stated assumptions, the best achievable choice in practice for ensuring uniform rate of convergence across all modes.
7

---

5 Non-Gaussian Sampling
We can generalize the results of Sections and 4 by considering the case of general class of prior measures 14 assumed to be absolutely continuous with respect to a Gaussian reference measure N( C₁) with density proportional to exp(-$), We present the main ideas here, relegating the more quantitative results and proofs in Appendix
To reproduce the approach of the Gaussian setting, one first needs to diagonalize the Langevin SDE system, which in turn requires diagonalizing the score function S T; (4).
Proposition 5.1. We assume that C and C₁ have the same basis of eigenfunctions (1z)) and we define x = (X, uz). We assume that (X) = E, (x)())), The score function (6) can be written as S(X, T: μ4) = Ej (X()) 1)g where
(t (X
( iT 1)) = -X,3,0,(x(1,, 5,(
(13)
with 7)())))77) log El exp( - 05 |x,00 = x and
(()) ~N 0, (0 e + (1- ---))))))))
(14)
x
We assume a more general form of the score approximation error. Assumption 2. We consider an approximate score So(X, T; pt) such that
(5(X, p) So(X,T; p), 02) = =x) + رررة + t)(().
With the learned score, the preconditioned Langevin SDE for the observed modes j S N becomes
dx, - -Mx X; di + bydl V2CxdW.N
where
MN Diag (8)(T;1)) ONa -2 AANAN + Diag IS/5N
(ef(T))
15I5N
by =NNo -2 AA - CN Diag (0)Øz) - Diag ISigN
Diag (s)(7))
ISiSN
I5igN
The SDE for the unobserved modes j > N can be obtained by taking AN above.
The stationary distribution the preconditioned Langevin SDE derived in the following proposition, which makes explicit the dependence on the score approximation error.
Proposition 5.2. Let Assumption 2 hold true. Under the hypotheses of the previous proposition, the preconditioned Langevin with approximate score in the drift term has ãy as its stationary distribution. It is absolutely continuous with respect to N (mi(T), and is given by the density
dig
(N(m())) B(T)) (X,T) x exp (-0(X,
(15)
For the observed modes j S N, the negative lon-density is &N (X~N, T) En (T)=)(T)0,(X) (x{")) the covariance EN (N(r) and mean m. (r) are given by 18. For the unobserved modes j > N, the negative log densityit is $00) (X()), T) = (x(6) 7) + (T()))(()((()) the covariance ((5) (r) and mean mn (7) are given by g
j=1 (x()
The interested reader can find in Appendix Cla quantitative analysis of this general case, including Theorem C.I which provides an error analysis analogous to Theorem 3.1 for the Gaussian case. Here we make few qualitative comments:
If eg = c = 0 for all j. then .(X,7) 3
Σj (x ((r) 3 [G] [C-1+=-AATA]
m(T) 0 [G,] - AA AAT y, that is say, get the posterior given the data.
8

---

The preconditioner influences the convergence rate. Consider the case in which Øj is convex, i.e., 0g 2 Cop > 0. In this case, for the unobserved modes j > N, the convergence rate of the j-th mode is Xylu5² C2y1 for small т, assuming the error ej is negligible. To achieve a convergence rate that is uniform in j. we should then choose X, = ln5 + Ca,l For the observed modes j - NN, and assuming A AN is diagonal for simplicity, should instead choose > luj + Ax))s + 1=,1-
If the error leğl < Ag. its impact on the stationary distribution of x! is small. Like the Gaussian case, the error o) can induce a bias. It can be large since ,; + +00 as j + +00.
6 Illustrations
We verify our theory by applying the preconditioned Langevin dynamics with SGM (3) two linear inverse problems: one based on the Karhunen-Loève (KL) expansion of the Brownian sheet [50], and the other one on an inverse source problem for the heat equation [3. Both examples are consistent with the theory of the paper. Implementation and further details are provided in Appendix [D]
property of approach the Brownian sheet, represented by its truncated KL expansion B (x) Ej (z x [o, 1]2, where Tjj,k ~ (O0, ((,kk) and (65,A, (bj,e) are the KL eigenpairs. For M S N, the inverse problem consists of reconstructing the KL coefficients and Brownian sheet from noisy data yj.k Mi. + E.A= S M, with ī NO (y,k), jj (0, 0.013) (i.e. Ayk if j k: S M and O otherwise). Figure 2 shows the robustness of our approach with respect to the number of modes M
Brownian sheet. We illustrate the discretization-inaarine
True Stret
Piecond, posterior mesn IN 751
10
Precond, postericr mean (M 200
12
0 mesn 75 N 200
nc
00
66
0.0
050
-00
000
458
1.0
10
Figure 2: Discretization invariance in reconstructing the KL expansion of the Brownian sheet for increasing number of observed modes M2, with M = 75 and M = 200. Here N = 200.
Heat equation. We verify the benefits of the optimal preconditioner from Theorem by considering the ill-posed inverse problem of recovering the initial condition u(x, 0), E {0. of the heat equation from noisy observations of the solution u(z T) at time T 0.1. Expanding the eigenpairs (Wy,s: Sy,s) of the Dirichlet Laplacian, one finds that u(x, t) = Σ,, f Ci.at 9y, (
where Ij.k- = (u(- 0), vyjx). ) The inverse problem diagonalizes: we observe Vj.k e - +Ej.A. j,k & M, with Ij.k ~
B = 0.1, εjAA (O, 0.0052). In Figure 3 we compare reconstructions using Langevin dynamies preconditioned with the optimal C (3rd column) and vanilla Langevin (4th column). Both samplers use a score perturbed by a relative error eÿ ~ (0, 0.1²) scaled by small T, as assumed in Theorem 4.1 The results support our theory: (i) the preconditioned sampler is robust to score approximation error, as expected from the design of C' (Theorem 4.1); ii) as shown in the autocorrelation plot in Figure 4 (corresponding to the top row of Figure 3 the modes converge faster and more uniformly than with the vanilla dynamics, since C' targets the optimal mean reversion rates (Proposition 4.1 : iii) vanilla Langevin deteriorates as the score eтror increases (Figure 3) due to amplification at fine scales, reproducing the pathological behavior seen in Figure
7 Discussion and Future Work
We studied a popular sampler-ap Langevin-type diffusion driven by score-based generative priors- directly in the infinite-dimensional Bayesian setting, rather than in the usual finite-dimensiona one. We showed that naïvely applying standard techniques in infinite dimensions leads to several issues. To ensure provable posterior sampling and discretization innretiizatiooiinvrrinnnee our analysis shows that

---

Observet ons
Tre
Procord. posterior mean
Unproc, posseror monn
605
31
630
10.5
-332
-33
04
100
00
24
24
08
1G
0.
03
64
62
02
08
1.0
Coservatioss
23
SOUTCM
Trocand ptalarie mean
14
Urprec. prasericr trean
13
-0553
08
0613
60
022
00
Figure 3: Effects of preconditioning in Langevin sampling for the inverse heat source problem. M
15; top row: T 10 bottom row: T 10 The 4th column shows the issues of Figure
Preconditiored ACF
Langevin ACF
1.0
1.000
0.5
m-
m1
05-
mP5
mB8
m--
mm2
64-
mp7
c.00
m2
62-
m-3
m--10
CB75
60
10
60
60
200
20
60
200
Log
Figure 4: Mode autocorrelation. Left: Preconditioned Langevin with SGM. Right: Vanilla Langevin.
preconditioning the vanilla Langevin is necessary. We prove detailed convergence error estimates and the existence (and form) of an optimal preconditioner-depeee depending on both the forward map A and the score error-that yields uniform convergence across all modes.
simplifying assumptions: the prior and the diffusion's 's noise covariance. In some parts, also assumed that A A is diagonal, common assumption in the theory of linear Bayesian inverse problems [3] 4. This is not merely a technical convenience: in many classical linear inverse problems, such as the heat equation, tomography, or inverse scattering for Schrödinger-type operators under the Born approximation, the forward operator is compact, and hence one can always find basis in which A A is diagonal.
As is standard in dimensinail analysis, results rely inite-dimensional data, and co-diagonalizability
Nevertheless, the main conclusions of our analysis remain valid even without the diagonalization assumption. For example, the asymptotic expansion in Eq. (TO) of Theorem 3.1 can be extended to non-diagonal by replacing scalar expansions with their corresponding matrix-series counterparts, while the arguments for the higher-order modes remains the same. Likewise, in Section 4 one can verify that under perfect score function the optimal preconditioner still takes the form = - AT AA
Several open questions remain. In particular, how do these results extend to nonlinear inverse problems? Extending the analysis of [21] to determine an optimal preconditioner for nonlinear inverse problems represents an important direction for future work.
10

---

Acknowledgments
JG was supported by Agence de l'Innovation de Défense AID via Centre Interdisciplinaire d' Etudes pour la Défense et la Sécurité CIEDS (project 2021 PRODIPO). KS was supported by Air Force Office of Scientific Research under grant TA9550-22-1-0117 and the National Science Foundation under grant DMS-2308389, MVdH acknowledges support from the Department of Energy under grant DE-SC0020345, Oxy, the corporate members of the Geo-Mathematical Imaging Group at Rice University, and the Simons Foundation under the MATH + X program.
References
[1] A. Tarantola. Inverse problem theory and methods for model parameter estimation. SIAM, 2005.
[2] J. Hadamard. Lectures on Cauchy's problem in linear partial differential equations. Yale University Press, 1923.
[3] A. M. Stuart. Inverse problems: bayesian perspective. Acta numerica, 19:451-559, 2010.
[4] B. T. Knapik, A. W. van der Vaart, and J. H. van Zanten. Bayesian inverse problems with Gaussian priors. The Annuls of Statistics, 3965):226-265777 2657, 2011.
[5] M. Dashti and A. M. Stuart. Uncertainty quantification and weak approximation an elliptic inverse problem. SIAM Journal on Numerical Analysis, 49(6):252425544 2011.
[6] A. M. Stuart. Uncertainty quantification in bayesian inversion. /CM2014. Invited Lecture, 1279, 2014.
[7] M. Dashti and A. M. Stuart. The bayesian approach to inverse problems. arXiv preprint 13Xiv:1302.690 6989, 2013.
[8] S. L. Cotter, G. O. Roberts, A. M. Stuart, and D. White. MCMC methods for functions: Modifying old algorithms to make them faster. Statistical Science, 28(3):424 446, 2013.
[9] M. Lassas and S. Siltanen. Can one total variation prior for edge-preserving bayesian inversion? Inverse problems, 20(5):1537, 2004.
[10] Q. Liu, J. Lee, and M. Jordan. A kernliized stein discrepancy for goodness-of-fi tests. In International conference on machine learning, pages 276-284. PMLR, 2016.
[11) P. Vincent. A connection between score matching and denoising autoencoders. Neural compu- tation, 23(7):1661-1674 2011.
[12] Y. Song and S. Ermon. Improved techniques for training score-based generative models. Advances neural information processing systems, 33:12438-12248. 2020.
[13] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[14] Y. Song, J. Sohl-Dickstein, D. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.134 2020.
[15] J. Sohl-Dicksteri E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pages 2256-2263 PMLR, 2015.
[16] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:5140-6851, 2020.
[17] G. Batzolis, Stanezuk, C.-B. Schönlieh, and C. Etmann. Conditional image generation with score-based diffusion models. arXiv preprint arXiv:2111.13666 2021.

---

[18] B. Kawar, G. Vaksman, and M. Elad. Snips: Solving noisy inverse problems stochastically. Advances in Neural Information Processing Systems, 34:21757-21769, 2021.
[19] A. Jalal, M. Arvinte, G. Daras, E. Price, A. G. Dimakis, and Tamir. Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems, 11114493894444 2021.
[20] L. Baldassari, A. Siahkoohi, J. Garnier, K. Solna, and M. V. de Hoop. Conditional score-based diffusion models for bayesian inference in infinite dimensions. Advances in Neural Information Processing Systems, 36, 2024.
[21] I.. Baldassari, A. Siahkoohi, J. Garnier, Solna, and M. V. de Hoop. Taming score- based diffusion priors for infnite-dimensional nonlinear inverse problems. arXiv preprint arXiv:2405.15676 2024.
[22] B. Hosseini, A. W. Hsu, and A. Taghvaei. Conditional optimal transport on function spaces. arXiv preprint arXiv:2311.05677 2023.
[23] P. Hagemann, S. Mildenberger, L. Ruthotto, G. Steidl, and N. T. Yang. Multilevel diffu- sion: Infinite dimensional score-based diffusion models for image generation. arXiv preprint arXiv:2303.04772. 2023.
[24] B. T. Feng. J. Smith, M. Rubinstein, H. Chang, K. L. Bouman, and W. T. Freeman. Score-based diffusion models as principled priors for inverse imaging. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 10520-10531, 2023.
[25] Y. Sun, Z. Wu, Y. Chen, B. T. Feng, and K. L. Bouman. Provable probabilistic imaging using score-based generative priors. IEEE Transactions on Computational Imaging, 2024.
[26] X. Xu and Y. Chi. Provably robust score-based diffusion posterior sampling for alug-and-pe -la- image reconstruction. arXiv preprint 1703, 2024.
[27] A. M. Stuart, J. Voss, and P. Wilberg. Conditional path sampling of sdes and the langevin meme method. 2004.
[28] M. Hairer, A. M. Stuart, J. Voss, and P. Wiberg. Analysis of spdes arising in path sampling. part i: The gaussian case. 2005.
[29] M. Hairer, A. M. Stuart, and J. Voss. Analysis of spdes arising in path sampling part ii: The nonlinear case. 2007.
[30] A. Beskos, M. Girolami, S. Lan, P. E. Farrell, and A. M. Stuart. Geometric meme for infinite- dimensional inverse problems. Journal of Computational Physics, 335:327-351, 2017.
[31] J. Wallin and S. Vadlamani. Infinite dimensional adaptive meme for gaussian processes. arXiv preprint arXiv: 1804.04854 2018
[32] A. Durmus and É. Moulines. High-dimensional Bayesian inference via the unadjusted Langevin algorithm. Bernoulli, 25(4A):2854-2882 2019.
[33] A. Durmus and f. Moulines. Nonasymptotic convergence analysis for the unadjusted Langevin algorithm. The Annals of Applied Probability, 27(3):1511-158) 2017.
[34] A. Dalalyan. Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society Series B: Statistical Methodology, 79(3): 651-676, 2017.
[35] M. Hairer, A. M. Stuart, and S. J. Vollmer. Spectral gaps for Metropolis-Hastings algorithm in infinite dimensions. The Annals of Applied Probubility, 24(6):245--2490 2014.
[36] T. Cui, K. J. Law, and Y. M. Marzouk. comeessooninndpeedden likelihood-informel meme. Journal of Computational Physics, 304:109-13:: 2016.
[37] Cui, G. Detommaso, and R. Scheichl. Multilevel dimension-inddeeendent likelihood-infomme meme for large-scale inverse problems. Inverse Problems, 40(3):035005, 2024.
12

---

[38] A. Beskos, A. Jasra, K. Law, Y. Marzouk, and Y. Zhou. Multilevel sequential monte carlo with dimension dimension-indeeenee independent seelihood-informmn proposals. SIAM/ASA Journal on Uncertainty Quantification, 6(2):762-786, 201 8.
[39] M. Morzfeld, X. T. Tong, and Y. M. Marzouk. Localization for meme: sampling high- dimensional posterior distributions with local structure. Journal of Computational Physics, 380: 1 1-28, 2019.
[40] A. Beskos, G. Roberts, A. Stuart, and Voss. Meme methods for diffusion bridges. Stochastics and Dynamics, 3(03))319-350, 2008.
[41] H. Ma, L. Zhang. X. Zhu, and J. Feng. Accelerating score-based generative models with preconditioned diffusion sampling. In European conference on computer vision, pages 1-16. Springer, 2022.
[42] H. Ma, X. Zhu, J. Feng. and I.. Zhang. Preconditioned score-based generative models. Interna- tional Journal of Computer Vision, pages 1-27, 2025.
[43] G. Franzese, G. Corallo, S. Rossi, M. Heinonen, M. Filippone, and P. Michiardi. Continuous- time functional diffusion processes. Advances in Neural Information Processing Systems, 36, 2024.
[44] G. Franzese and P. Michiardi. Generative diffusion models in infinite dimensions: a survey. Philosophical Transactions A, 383/2299):20240322, 2025.
[45] H. Lim, Kovachki, R. Baptista, C. Beckham, K. Azizzadenesheli, J. Kossaifi, V. Voleti,
J. Song, K. Kreis. J. Kautz, et al. Score-based diffusion models in function space. arXiv preprint arXiv:2302.07444 072000000 2023.
[46] G. Kerrigan, Ley, and Smyth. Diffusion generative models in infinite dimensions. arXiv preprint 12200021...00886, 2022.
[47] S. Lim, E. B. YOON, T. Byun, T. Kang. S. Kim, K. Lee, and S. Choi. Score-based genera- tive modeling through stochastic evolution equations hilbert spaces. Advances in Neural Information Processing Systems, 36, 2024.
[48] S. Bond-Taylor and C. G. Willcocks. oo-diff: Infinite resolution diffusion with subsampled mollified states. arXiv preprint arXiv:2303. 2023.
[49] J. Pidstrigach, Y. Marzouk, S. Reich, and S. Wang. ddinite-dimensione diffusion models. Journal of Machine Learning Research, 25(414):1-52, 2024.
[50] L. Wang. Sirhunen-Loeve expansions and their applications. London School of Economics and Political Science (United Kingdom), 2008
13

---

A Proofs of Section 3
A.1 Proof of Proposition 3.1
By Assumption for the observed modes j & N, the preconditioned Langevin dynamics to the SDE
reduces
d.x,"
Diag (8y(TiH)) + Diag (eG(T)) + ONO -N - xi" ISiSN
UisisN
(16)
ONo Diag
(c)(T)) di 22CxdW
ISISN
with On Diag ISJSN
(1,). For each unobserved mode j > N, we have
dx! ()) = - [s)((;s))++
x di e)(()dt + V2NddW.
(17)
Both [66 and 17) are -hibeinn-Uheebeck (OU) processes. particular, X $ XN in distribu-
tion, where the distribution of X is the stationary distribution of (I6):
XN 00
N (m (T), 8 (T))
with
m (r)
-Diag (s)(T:14) + Diag (E)(T)) CNo AANAN 15JEN
ONo AANy Diag (6)G))
LISJSN
15i5N
-[ Diag (sy(T:)))+ +C) 'Digg (T()))) ISJSN
-2 AA CN C Diag (cG)) ISiSN
+o
ISJEN
and &N (7) is such that solves the Lyapunov equation
Diag (s)(T:1)) + Diag (E](T)) + CCNC---AAAA (T)(T) USiiN
I3JSN
+ (r) Diag (sy (Ti Diag (e](7)) + CNO -ATAN I5jEN
-NN-ANNNN =2ON-
ISiSN
Then
Diag (eg(T)) +0 - 1-1 ISiSN
(~
O
Diag (sy(7;]4) + CN ISjSN
since
Diag (8)(T:14) + Diag (c9(T)) + -2- AANANN AANAN6000 (7)(T) UsisN ON (8 (7)) -1 (()((T) CN,
ISiEN
and
8 (T) CN Diag (sy(T; 14) + +CN Diag (cf(T)) + -0 -0-AANAN ON ISiEN
ISiEN
= (T)))) CCN
For each j N. 17 one-dimensional OU process with rate sy (T( + (T), mean shift
(T) + (-] and noise 22y,. Hence x,0) tgoo x! in distribution, where the
distribution of x the stationary distribution of (17
X& N (- sy(T;H) + (-,,(r;)))+
(T)
A tf() s5(T:H) + (5](T)
These results are valid (T(T;p) + (T) is positive number for all
14

---

A.2
Proof of Theorem 3.1
For each mode j. define
By = My [e_ + +1-
The proof can be divided into two cases: one for the observed modes Σ N, and one for the unobserved modes N. Since the KL divergence for the unobserved modes can be obtained by laking AN = O in the expression for the observed modes, we focus only on the latter.
The marginal distributions of the j-th mode for the approximate and true posterior, for j S N, are given respectively by
1 [
+ + 17' 13'6667] [o (0-9() - 17'e)(7)].
*(/
N
and
1 +o 1 (AAAN)) o (AT [Hg
to 3 [)
(1) =N
The KL divergence between these two Gaussian distributions admits the explicit formula
DKL
#
1 +
1
2 1)515((1))
log
Hg
14g
2 (ANAN))
+o 2AA (ANAA))s + +25707(7)
2
[dj
j4j
+
((1
1
((AA + +2)(67(7)
(A-uu))) 1766((71
jb
- {
+o (
We study its limiting behavior as r - From the first term, we derive
- to -
1
+o - -A + +251e3(7)
log
14y
log
11+0-22sh6 "u;(AA
e (1 -e -- + 2 (A)AN))s + P.j (272)(77)
1
(1- + (E-T 1)pi
+ (1 11---1pp +D e9(o)
log
1+
+ 5 -2 uy (A
log
1
[-r(p,- - -1) + 0(721)
- - (A]
(-r(p)
1)
1 o (A
(18)
15

---

We now consider the other terms of the KL divergence. First, we derive
1
to (ANAAN) 13))6(6)
+
x [ (A)T)) - 17 17)c((7)] - Mi
+o 7
(ANu))
M4
1
(1- + (e- - 1)p
1+ 1 1 (A
1+ (A)
e e (1-E-))p
[o ((Ato) - xr1c)(+)] - σ (AATu))
1 24 AN) 11+ 1 1 (A AN)ia (10
14
?ANo)) -1 27's 3' - -
14 1 ( AN)sS [1+ 1 1 o Ppy(A (F(D- - 1) - ()'=(())+
x (AT 27 15(2((7)] - o (1hee))))
{ (x)(66(66 1 o Ous AN)ss (T(D) - 1) - pj (T()
2T)
14
1 (n
(ATT)); 1+o (A (T(D) 1) -
1イ'c((7)) -)p
+ +0(7)
Taking the square yields
(1, +o + + -2'/5((7)
x [o (AAu)) + (-)
1
(A -AA(())))s)) {
-
[/g
43 (1 0- 1, AN)ss)
17 1536/(7)2
(7)
(A) - - (ATTu)) (T(D) - 1) 1) 22'))(()) + +0(77).
1 1 AN)is
(19)
Next, we consider
1 M3
-2
+ o ((AAAN) + +15
-2
+
(-T(p) - -1) + 21 (7()) (7)7))
(1+o 1+-00pp -2 (A N (AANN )3
1 o (AJ AN)is
13
2
(T(D)
109-1) 1) - 11)1111666611) + -))))))))))+++(177 +069)
(1 2 AN))m))
1 (A
(20)
16

---

Putting
18
19
and 20
together, we obtain
DRL.
(
2
25 (T(
-17 (f(())))))))))))))))))))))))))) - - -1 (()) + +((7).
1 1 (A
B Proofs of Section
B.1 Proof of Proposition 4.1
If assume that A AN is diagonal in (uz), for the observed modes j N the preconditioned
Langevin dynamics becomes
dx( (M)(T; + (T( + Ajo (AA x di -
- -c)(T)] di
+ (2
(21)
while for each unobserved mode j > N, one obtains
dx!" = [T) + [T) x dt - () + /22,dw!")
(22)
Let (t) E[X 1. For the observed modes gives the linear ODEs
N, taking the expectation in the SDE above
dm
(T)( + 3 m ()) + [Λ,o (AND)) - e7(
dt while for each unobserved mode j > N one obtains
dm dt [s)(T: + Eg(] (t) =)))
Given mn) = -m ()) hoth have unique solution. For j $ N,
2 (ATr))
(t) m (0) (0)
- (stt)) (ATAAwss
sy (T::))+ (A
A (4hr)) (T)
()(7))) (T) 2
which for oo decays exponentially fast to the mean
(A (T(()
+T(+:)))+5(T + 2(A
with rate
x0)) = sy(TiM) (T) AANAAN)ss
([e My +1 J - + -2 (AJANN) + +
For the unobserved modes j > N,
(T)
()
m (t) = 6) (0) + Sy(T; + =g((T)
S)(T; + =T(
which converge to the mean
(T(
sy(T:M) + (T)
with rate
166) sy(T:M) ET(
=>ylle- mM (1 e - + +
17

---

B.2 Proof of Theorem 4.1
We consider only the case of the observed modes j v N. since the case for the unobserved modes j > N follows directly by setting Av = 0.
By Proposition 4.1 ensuring uniform convergence rate for 3 using an approximate score function as described in Assumption
T
amounts to solving the equation
d + Ajo (AN (T](T)= = 1,
(23)
where ji My le + e >)pp.
Assume the expansions
2, = + + +
=
Then compute
B3 M1 pg(p; +
1, + 0(T²)
using that Pj = X Substituting into 233
we obtain that ,0 and 2/41 must satisfy
M5 (A, + (0) [ |3 2 | 1)) (
7) + + 0(72) =1
Rearranging the terms, we get 2, Ui + 2/ // - = which gives
- |17 to "(ATAN)))]
and
15 (-1
[4j (0) 1| + 2 to (A +03 = 0,
yielding
= (1ا'ري)) (0)
- (1) 55) -5)
C Non-Gaussian Sampling: Technical Details
C.1 Proof of Proposition 5.1
By ((X) = = (X(0), (x), the prior y has Radon-Nikodym derivative with respect the Gaussian
N(O,, Cμ) given by
dp (X) II exp
[IxxxIIxxxxxtt (-0)(x())).
exp
dN C.)
Since C and Cp are both diagonalized by the same basis (vz), the prior factorizes as a product of independent one-dimensional marginals in the coordinates X():
dμ
()
(X)
II
dN m2) (x6)).
dV Cp
where
du(()) (x(1) x exp (-0)(x(1))
We can then work mode by mode. For each j. define the one-dimensional OU process
x ~ N((0,)))
x = = + v1 ve---
with gb)) ~ NO, A₂) independent of X!) Notice that
(1)
e-/2 Uj ~ ~N (1,, ( 2 Hj e + (1 (1-ee - )
F()
0
(24)
18

---

The OU transition kernel is
p p(X(t)
xx =0) N/ee
7/2 2/000 x0. (1-e ()))))))
where (nu, o (x)) is the density at of the normal distribution with mean μ and variance o². We push forward the prior e e (0, M5) through the OU kernel. Mode by mode, its density is
(x+)
(xp(--
2o)dro
f exp(=0,((o) po.r p (0o, 2d)o0
where (To, denotes the joint density of (X)))
()) _.()). Let jiy = + (1 -TT)
Dividing by the marginal Gaussian density of X we get
(x+)
po. (co, Ør) dxo
exp(-o, (zo))
((0,
pr (2r)
eep( 0, (2o)) xz )doo f
E/exp((- 2, (X, x xr-
(x+) Since 560 (3TiT: = r3O log ((.AA))
+ \,, (4, (x+), obtain
50 (Z,,T; Xj0,
X = xrl
e + (1 - xt0..
X,
C.2 Proof of Proposition 5.2
For each mode j. define
(x((), r) = log E exp((o)(Xx)
x( x
Under Assumption the first N coordinates of the preconditioned Langevin dynamics corre- sponding the observed modes j N satisfy
dx," = Diag (8)(T:/4) + ONO AANAN + Diag ISiSN
(c((7) ISiSN
x.N dt
CNo CNO-BATJU -N Diag (B,8)(1!1))
(1x()))))
+
Diag
(25)
ISJEN
I3/NN
(e)(T) dt + 22C ISiSN
Diag
The SDE for the unobserved modes j N can be obtained by taking AN above:
ax(") = 88 (T; + og x(
(26)
+ -
e dt + /2
Both 25 and 26
preconditioned overdamped Langevin SDEs. In particular, checks that can be written as
25
dxW. (CN /2C
where the potential UN
UN = = XN
1
T
C Diag (s((T:)))+ - + ON'
Diag (15(T) xN ISiEN
2
ISJEN
N
2 AAA
CN'
Diag (c(T)) X
Σ
0 - - (x((()))
ISiEN
1=1
19

---

Its stationary distribution is Ag. which is absolutely continuous with respect to the Lebesgue measure over RN:
d (X*)
dxN & exp(-Un(Xx)))
We split Un into quadratic and non-quadratic terms. Hence
d y (x*)
dx exp (- B (X*, -)) N (r), ( (T)((*)
(T),
where ( N (Xx) is the density at X of the multivariate Gaussian with mean vn N (T)
and covariance 8. (r), with
$ 7) Σ
(ef(T)) (G()))]
(7) ON' Diag (T)(T: to AANAN + ON ISJSN
Diag
ISJEN
( ISJSN
ne (T) = (T) o 2 ANy Ci Diag By the same argument, for each j N, the dinediimal potential of (26) is
x(O)²
2 +7) + +xx(x(()) + +2 (+)6)())
Its stationary distribution therefore
dX oc exp (-B1)(X())T)) (o (( (1))))
where (T) 800 (T (X))) is the density at x(i of the multivariate Gaussian with mean (T) and covariance 500 (T), with
+ 17 (x00, -) = (x((), +)
(T) = ():')((T:1)) + +
m = (T)) -
C.3 Error Analysis in the Non-Gaussian Setting
For the sake of completeness, we present a result analogous to Theorem 3.1 for the non-Gaussian case. the interested reader will notice, the calculations are significantly more involved, but remain relatively straightforward.
Theorem C.1. We assume cT(() = O()), c)(r) = 0(1), and An = Monisjs((AAjp The Kullback-Leible divergence between # and (1) is given by
(06 || m B2(T) + ET(T,,
DRL.
where By(T) is bias term given by
p. (c) + log / - 1A)2 (= (O)
By(T) = 227 e so [x]
X7
- log / (-
1 N N (1))
and ET(T) is an error term
Eş(r) =
++) ((-3))))
0(73/2)
20

---

where
(+)= = Bgg) xx2
[ij (5'05'e9(7) (1 ps) + 2 B BaD0 (As (d)(()>-0 - -dj(x))
1
(x(x)(1 - 2p,)o] - (()((T), (15(())))).111)) U1) / (-0s(o)) 2. 1 unl NC (-D= -)))))(()))))))0))
1 (2)00(3)2 0g(+)) - (1((){1 - 22)))2
2 p (T())
x
2
(z + Pj 1 (T))
1
+
(1 pj) dz.
Hg
2
and
(T) = BBlg) [x² [z(1)]
Pj (T) (T))(T), As3.B55)1 / - ( NNPP (2)')())))))())
x - b0 (z
(r))
Míi
dz,
2
with
Z(E)(T).A / ---5(())=a 2or (Ag) mol? [(
Proof. In the following Eg 50) [bb] and E 5 [+xx) stand for J ((()dd+())) the j-th mode marginal of the approximate posterior distribution #.
(x). Recall that for j s N
di dx())(x(1)
1 exp
- (5)
An,x(( -13) 1²)
20
(-
dN
(-11)
+
27 xT'e((T),
- + 1=y'e3(0) (x(1)),
1
while the true posterior is
dr (x(1)
1
1 -,,0) =11²) dN dv(o, (x0)(X(i))-
20 [ - v3
exp
For the unobserved modes j > N. we set A₁ = -. For each j, we have
# m
DKL
(-tt)
(3)))()))) 1 + +, A5'e9(7) 11) log N0.
E R59
log.V/
[Ag Pty + 27 15
27 1
first term
log 7,50
E (1) (1- 8,)
25
seccod term
third term
21

---

First term
We derive
log.N
1
+ Xy cg(7)
2j 2T(s)(t),
ب + +
1
(x)
[jj
1
1
1og()
log
+ +
2
|
1
1
+ + 13776)
x+ [h
x
+
2 My
and
1
1
1x
10,) = - log(2T)
log p.
2
2 pd
Hence
log.V (- - + +
1 17'10(6)) ") 100,)(0)1))
E Go
2515)(T),
_ +
Lig
log
+Dj +Dzte((t)
1
e_ + (1 - e- Ppj
2
e-
+ (1
1pj
2
1
x²
+
+ DD +557)
Dj (575)(7)
e- + +11--1)p
H4j
We have
= log
2
1
6-
+ (1 - )ーン
1
+ +D (3(6))
= 2
1 ((1 ((1- p))) + (1 -))
and
E (9
M4 e- + (1- (1-e3 ----e--p) + DJ (8())(())
1
1
pj (/1e((6) ?
x²
1
x e (1 - T + +Pj 1 (5((T)
Us
x22
= =[(1 pT} + pj [
(7) 2g0 (a] + 1j'
[4g
x [1 - (1 - (1-p))r - p. [g](]] + +0(?))
Second term
We have
E exp (X)) |x,= =]= // exp((-b)(3)
(_ (2-m,, mn -)mm)(()) (2))
exp
dz,
V2TU,
20r
where
my ETi + - +1---T)); == 1+
1 [Pp)
Pj
r
Pj
1
2
x O(T3),
2
8
and
en
11- T 72
+ 0(7512))
/===| |i e- + (1 e-
4 32
22

---

By the change of variable w = 2-m-(4) ver
we obtain
R exp (do(Xo))
Xx x0
f exp(pxx (x)) V2TUT
1
(2 (-
m) (x))2
exp
dz
20g
/ exp(-00 (Vo, w + mr (z)))
1
exp(- (-w2/2)duu
22T
exp( exp((0,(x)) |+ (> [x)(2)) - o5 (x)] - (z) 2p, )p)) _ + 0(79/3)
where we used the Taylor expansion for exp( 060 (Vrrw w + my (x)) as vi= + 0 and
/ 2nT
/
- expl-w²/ /2) 1.
w²
exp( exp(-u212) =0,
\2TT
Hence
E (0 (z()))
E B log 1 + {>y -((z)~30)(00 [5]()]] (10))(1-2pp))))))) -/000/ _ 0(7312)
2
Third term For the unobserved modes j > N, we analyze
/
e (= NO0
(27)
J o5(0) 6'00-1"5000000000
+ -))))()))~)) +x1"
We use that
(=)==
2pp)) +0
which implies
(1---0000--1111---00000--1331333 )(7913)
Now we consider the density of N (- "1
(++ 14 (+11"111111000000000"01
2
1
exp
2|0 2
(28)
2T 1 + X7 [T)]-1
We have
1
[ - 2
アリ - (7( 0
1 (1- (1-)))
1
(29)
2n 1ij + N, (T)
/2mpy
We now look the exponent of 28 Its numerator reduces to
(z p (T)) -( +,)
- (p)) + (5(()) + +((?),
while the reciprocal of its denominator (28 reduces to
1
1 p,)T p (T))
2p
Then the exponent of (28) can be expanded as
1
22
(TT
2p
(2 Pj (T))
(z + PJ (T))
+
((1-p))- p;)T + + + +0(7))
Pj
2/1
23

---

and the exponential term in 28
becomes
(2 + Pj =)(())
exp
2pg
(30)
2 pj (T)
(2 + Pj (T(()))
((L-p,) (1 (T)) + +
x 1
2/h
Putting together
29 and
30
get that the Gaussian density
28 is expanded
W(-pj
Ej
(T))
+ Pj (T)
(z ()(T))
x
++
((1 p,)T - D5 (7)) 2
Hj
We now expand for small
( dN(- + -,,""")))))))))))00000000) 1a5" +)-1
Let
2T(e((T)) = [0 (5((T), (z))2
We derive
2,(e((T))-1 1 - (2)((((T))
e N ([((-pj(_((() p()()))))()))))))))))))))))))))) (x(00(332 -((00=)--(())))/(0000-----------
of(:)((-2p;))+
x
(T(
+ (2+))'))(7)) 1 (1- + +15 15'63(7)
(7) +
dz
H2
Hj
+ (773)))
Then 27 can be expanded as
log (0) - log 2, (E)())
- -2,(,((t))-1 / (0,0+) (1)p) (2())))))(()))
- +1(())( 22:))) -15 (T))(())))())++ +
N7
+
(2 pj ()())
1
M4
2
x ((1 + + 1 + 1(())( dz + 0
Now let
z, (T) ff - (An N(-p; ()));')(())))))))))))
For the observed modes j < N, we get w;) A,j, log ()(e Agj,H)))
log Z, Agj,
(5je)))), / -0,(z) - 2 N NC (T)))) ((p-- (2(00()))-0((0)
dy(()(1-2pp)))) -x,' - (T))
(2 (7)
(a (2+1ī
1
Pj
(1-p;)) +2, (((() dz + 017312).
24

---

Remark C.1. If o3 is smooth and (T( = (1), then E,(7) + 0 as r + 0.
D Illustrations: Additional Details
Here we provide additional details the theoretical setup underlying the illustrations. All illustrations were generated on Google Colab (13 GB RAM), and all code executions took less than one minute
D.1 Recovering the KL coefficients of the Brownian sheet
The Brownian sheet B(11, 22) is a Gaussian process with zero mean and covariance Cov( (z), #2), B(y,, v2)) min(x1, vi) min(12.32).
Its Karhunen-Loève expansion [50] is
22)
Σ (21111)))))11 (x1, E [0, 1J3, J.k
where Mi NN are independent Gaussian random variables, and
6 = 2in
π
x
sin
π
k
22
2
2
2
Pi.A
k
In Section 6 we truncate the KL expansion after N modes
B (21,
Σ
J.k=1
and consider the inverse problem of recovering the first N2 coefficients from noisy observations corresponding to the first M² < N modes
Vj.k ÚiA + Ej,,
j. < M,
where the prior ūj.. ~ (0, (j.&)) and the noise Ej, NO0 o²). This setup satisfies the assumptions of our theory, since the prior diagonal in the KI. basis (Øj,k) and the forward map is simply the projection onto these modes, so that
AJ..,'* = ō
and zero otherwise. As result, the posterior for each coefficient remains Gaussian
(i.N) 75.k Uj.k ~ u3y = (m,,, oy,)),
with, for j,k S M,
Uj,k = (1 + t -==)+
25
Hjjk Pj + σ² Uj.k:
Hj,k + +
mj.k
and for j > M or ks > M (unobserved modes) the posterior simply coincides with the prior, Uj = 1j,8, m =
Experimental details
In Figure 2 within the theoretical setup described above, we set the noise
level 1 chose N 200, and varied the number of observed modes M² 752 200 to illustrate the dnsecetization-invariin the small errors reported in the fourth column of Figure The preconditioned Langevin dynamics, using the preconditioner Cм Diagi ()g,)). Ay,k = [ + - -1, was run for 10 iterations with a fixed sizp of 10-1 We assumed access to the exact score function, i.e.,
of the preconditioned Langevin sampler. This is confirmed by
0 in 6
'Code reproduce results can be found at https///ghhuu.oooooaoaee///////t
25

---

D.2 Inverse source problem for the heat equation
Let S2 = [0, 1]2 C R.?. Consider f2 [0.
solving the heat equation
Ozu(2, t) = Au(z,t)
(x, t) E f x (0,7), E Ω
u(r, 0) =((),, u(r,t) =0,
E 02 (0.
Set u(11, x2it) = jik== ug (t) V/i. (2), where (1),,s S5.R) are the Dirichlet eigenpairs of
-A on 1].
( = Cis u U (x), E [], 101.100011 0.
We have
= 2 sin(j sin sin(k)r2),,
SI. = x
The coefficients evolve as
(t) Srikh 9j.k.
J5.k = (g. .z
In Section 6 consider the the so-called backward heat equation- the ill-posed inverse problem of recovering the initial condition from noisy measurements of u(- 77) inside
Mj.k e S3.4 9j.k + Ejut:
j, k < M,
by adopting Bayesian approach [ We assume Gaussian prior 9j.k e((0e= and independent Gaussian noise Ej,k N0, o²). The forward map is diagonal in (V),,,) with
A = e - S Sn J k S M.
As result, the posterior for each coefficient remains Gaussian
9j.k Mi. (mmgar U,s)),
with, for j, S M.
&Cou o²
Pi.k (3+27)
o2 e S yj.k-
1/,b e- eB( 02
mj.k
For j M or k > M (unobserved modes), the posterior simply coincides with the prior.
In Figure 3 within the theoretical setup described above, we fixed the noise level at 5 10-3 chose M 15 (i.e. 225 observed modes). and T 0.1. We then ran the preconditioned Langevin exact score function perturbed by a relative error E (O, 0.1²), scaled by
Experimental details
with the optimal preconditioner C from Theorem 4. using the
10 the top row of Figure and by 10-1 in its bottom row, and with zero bias (i.e. E) = 0) to simulate a learned score.
This sampler was run for 5- 103 iterations with fixed step-size of 10-2_ For comparison, we also executed the vanilla Langevin sampler for 1.5- 10 iterations with a fixed step-size of 10-6 To further illustrate the quality of our preconditioned posterior samples, Figure [5] below shows uncertainty quantification for Figure 3 For the first modes, we plot the conditional posterior mean (red), the 95% credible interval (orange shading), and the ground truth (dotted black line).
645
48.0
4314
Figure 5: Uncertainty quantification for preconditioned posterior sampling. Left: 10 10
Right:
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
This work focuses on Langevin-type diffusion algorithm driven by score- based generative priors in infinite dimensions, proving convergence error estimates and existence and form of optimal preconditioner.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can expected to generalize to other settings.
It fine to include aspirational goals as motivation as long as it clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See the last section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model well-specification asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
27

---

Answer:
[Yes]
Justification:
All the details of the proof are provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: See Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
28

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Code is uploaded in single zip file along with additional supplementary materials (e.g. Appendix).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
cttiss.ccppblicc/guu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See main text, Appendix, and code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Does the report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The posterior mean and standard deviation intervals presented throughout the main paper and Appendix reflect the inferred uncertainty, while the posterior samples illustrate the variability and structure of the distribution.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
29

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are of range (e.g. negative error rates).
If error bars are reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Our experiments require less than a minute per run on laptop. Details are reported in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
We conform the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special considera- tion due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our paper mainly focuses theoretical analysis of a popular sampler in infinite dimensions.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
30

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimesss considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This work poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
We use open-source packages.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URI..
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e-g., website), the copyright and terms of service of that source should be provided.
31

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pawpetts.pppwwwiiitaa seporewithcode... thcode. con/datasetsll has curated
licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged reach out the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification:
No new asset is released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the detaset/coodemoodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
32

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The paper mainly a theoretical analysis and does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer our LLM policy hbtpes//hhautpsccco///oeeeeeeeee what should or should not be described.
for
33


Reviews:
[{"review_id": "jYCLkwj9Kq", "reviewer": "Reviewer_jf1T", "summary": "This paper considers an infinite-dimensional version of the Langevin diffusion sampler to solve Bayesian inverse problems of form $y = AX_0 +n$, for $A$ a linear operator, $X_0\\sim \\mu$ an unknown Hilbert space valued random variable, and $y, n$ finite-dimensional, with $n$ given as Gaussian observation noise. Under this setup, the aim is to find the posterior of $X_0$ given the observations. In the infinite-dimensional case the diffusion process is driven by a $C$-Wiener process. This $C$ acts as a preconditioner and the aim of the paper is to understand how this preconditioner influences the error of the learned score. To start the analysis, it is assumed that the prior of $X_0$ is Gaussian. Under this assumption the paper provides the stationary distribution of the preconditioned Langevin diffusion in terms of the error of the score approximation as well as the KL-divergence between the true and approximate posteriors. It also considers how to choose an optimal preconditioner $C$. Finally, the paper generalises these results to the case where the prior is absolutely continuous with respect to a Gaussian reference measure.", "questions": "See the weaknesses for questions.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The submission is well-written and structured, making it easy to follow. Moreover, the mathematics is well presented with all assumptions clearly stated.\n\n2. The analysis provides insights into the optimal preconditioner $C$ in the case of Langevin samplers which I think is a valuable result. It is also shown how the preconditioner influences the convergence. \n\n3. I enjoyed the remarks and comments surrounding the statements of the main theorems and propositions. I found them useful for interpreting the results.\n\n\nWeaknesses:\n\n1. Pidstrigach et al. [48] also provide a similar result about the optimal $C$ based on the Wasserstein-2 distance between the true data distribution $\\mu$ and the learned sample distribution (see Theorem 14 and Section 6). Although the problem setting is a bit different, in that they are studying time-reversals rather than Langevin-based samplers for inverse problems, the results seem very related. Could you provide some more details on the similarities and differences to this result?\n\n2. It is assumed that the data distribution is absolutely continuous with respect to a Gaussian measure. Could you shed some light on how restrictive this assumption is in practice? For example, by providing some examples of applications where this would hold.\n\n3. In the appendix, Theorem 3.1. is also generalised to the non-Gaussian case, however I don’t think this result is mentioned in the main paper. I think it would be nice to include this somewhere; if not the whole statement then perhaps a sentence linking to the result in the appendix.\n\n4. The experiments provided are quite toy, consisting of experiments on Brownian sheets and the heat equation. However, I only see this as a minor weakness, since the main objective of the paper is theoretical and in my view the experiments serve more as an illustration.\n\n5. There’s a small mistake in Line 318, which currently reads “…about nonlinear inverse problem?”. This is minor, and does not affect my judgment of the paper.\n\n\nIn summary, I think this is a nice paper with interesting results and I recommend acceptance. However, I’m currently a little unsure on how different the results are to that in Theorem 14 of Pidstrigach et al. If the authors could make this clearer I would be willing to improve my score.", "was_revised": true, "final_justification": "The rebuttal addressed my main concern between the similarities between a prior result. The paper is well-written and studies an interesting problem.", "rebuttal": "We thank the Reviewer for their positive feedback and thoughtful questions. We are happy to clarify their doubts and hope this could lead to an increase in their score.\n\n1. **Similarities and differences with Theorem 14 of [48].** We thank the reviewer for drawing our attention to this specific result in the foundational work by Pidstrigach et al. We are happy to elaborate on the differences and similarities between the two results, and we plan to include this discussion in the camera-ready version of the paper. \nAs the reviewer correctly notes, the settings are different: Pidstrigach et al. study the problem of sampling an unknown data distribution using a score learned from samples of that distribution, whereas we use a score-based generative model trained to learn the prior in order to sample the posterior. \nSetting these differences aside, and assuming in Pidstrigach et al. that \n$$\n\\mu_{\\text{data}}(dx) \\propto \\exp(-\\frac{1}{2\\sigma^2}\\|Ax- y\\|^2)  \\mathcal{N}(0,C_\\mu)(dx),\n$$ \nwith $C_\\mu= \\text{Diag }(\\mu\\_j)$, and, for simplicity, $A = \\text{Diag}({A}\\_{jj})$, one can show that the squared Wasserstein distance between $\\mu_{\\text{data}}$ and $\\mathcal{N}(0,C)$, with $C=\\text{Diag}(\\lambda_j)$, satisfies \n$$\nW^2_2(\\mu_{\\text{data}}, \\mathcal{N}(0,C))=\\sum_{j=1}^\\infty \\left( \\lambda_j^{1/2} - [\\sigma^{-2}A^2_{jj} + \\mu_{j}^{-1}]^{-1/2} \\right)^2 + \\text{const.}$$ \nIt follows that the optimal $C$ minimizing the upper bound in Eq. (14) of Pidstrigach et al. paper is given by\n$\\lambda_j = [\\sigma^{-2}A^2_{jj} + \\mu_{j}^{-1}]^{-1}$, which coincides with the covariance of the target data distribution---as expected. In particular, under the assumption of a perfect (i.e. error-free) score, our framework leads to the same optimal covariance. This result can be extended to non-Gaussian priors that are absolutely continuous with respect to a Gaussian reference measure ${\\cal N}(0,C_\\mu)$, \nby following the arguments developed after Proposition 5.2 in lines 275-280.\nHowever, we stress that this result does not follow in our case from minimizing an upper bound on the convergence error, as in Theorem 14 of Pidstrigach et al. \nIn this sense, our analysis of preconditioning runs in parallel, with differences arising from the distinct objectives of each work.  For example, our preconditioner is derived explicitly; it follows directly from the mean reversion rate of the Langevin dynamics, meaning that the optimal covariance $C$ we find is not merely minimizing an upper bound---it is, under our assumptions, the best achievable in practice, for the purpose we have in mind (to ensure uniform rate of convergence across all modes). Moreover, our optimal $C$ explicitly incorporates the effect of score estimation errors, whereas theirs does not. We believe that these differences do not reflect a limitation---or superiority---of one work over the other, but rather the distinct aims of the two works: Pidstrigach et al. is bounding the convergence error of the reverse SDE dynamics, while our goal is to ensure uniform convergence across all modes in Langevin sampling.  \nIn this light, the closeness of the two results is particularly interesting. This comparison adds valuable context to our work and helps connect different lines of research on infinite-dimensional score-based generative  modeling and we think reinforces the relevance of both results. We are grateful to the reviewer for pointing this out and will make sure to include the discussion as a remark in the final version of the paper.\n\n2. **Assuming a prior absolutely continuous with respect to a Gaussian measure.** This is a standard assumption in infinite-dimensional PDE-based Bayesian inverse problem theory (see the foundational works of Stuart [3], Knapik et al. [4], and collaborators [a--e]). One typically begins with a Gaussian prior on the unknown field (e.g. Mat\\'ern or squared-exponential covariance), and, via the Bayes' theorem, obtains that the posterior measure is absolutely continuous with respect to the Gaussian prior---a crucial property throughout both theory and applications. Gaussian priors are a popular choice because they can be conveniently defined via covariance operators given by integer or fractional inverse powers of differential operators, are relatively straightforward to sample from, and lead to well--understood H\\\"older and Sobolev regularity of their realizations, which are typically encoded through assumptions on the decay rate of the eigenvalues of the prior covariance operator.  \n Assuming absolute continuity with respect to a Gaussian reference measure preserves these key properties---such as the connection between covariance structure and regularity---while allowing mild non-Gaussian modification (e.g. spatially varying penalties, bounded exponential tilts that induce skewness or sparsity), which have been shown useful in applications like biology [f] and perimeter learning [g]. In addition to their modelling flexibility, such priors offer a structural advantage: they can be regarded as conjugate priors for the inverse problems. That is, the posterior distributions associated to these priors have the same form---they are absolutely continuous with respect to the same reference Gaussian measure. This invariance makes it possible to study how a likelihood function updates a prior distribution, and also to implement sequential strategies in which the posterior from one step naturally serves as the prior for the next.\n\n3. **Include generalization of Theorem 3.1 in the main paper**: We thank the reviewer for the suggestion. We fully agree and will include a more direct reference to Theorem C.1 making use of the additional page allowed in the camera-ready version.\n\n4. **Small mistake in line 318**: Thank you for spotting this typo; we will fix it.\n\n\n**References**\n\n[a] Hairer, Stuart, Voss (2009), Signal Processing Problems on Function Space: Bayesian Formulation, Stochastic PDEs and Effective MCMC Methods\n\n[b] Dashti, Stuart (2011), Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem\n\n[c] Beskos, Roberts, Stuart, Voss, (2008) MCMC methods for diffusion bridges\n\n[d] Cotter, Roberts, Stuart, White (2013) MCMC methods for functions: modifying old algorithms to make them faster\n\n[e] Cotter, Dashti, Stuart (2010) Approximation of Bayesian inverse problems for PDEs\n\n[f] Croix, Durrande, Alvarez (2018) Bayesian inversion of a diffusion evolution equation with application to Biology\n\n[g] Dunlop, Elliott, Hoang, Stuart (2017) Reconciling Bayesian and total variation regularization methods for binary inversion"}, {"review_id": "7SmyY6V1Sq", "reviewer": "Reviewer_pjRJ", "summary": "This paper studies diffusion modeling in the setting of infinite-dimensional linear inverse problems. While the importance of pre-conditioning has been noted for other solutions of inf-dim inverse problems, it has been absent from SGMs, leading to pathological results when the discretization is refined.\n\nThis paper notes the importance of pre-conditioning by starting from a Gaussian setting and establish the existence of an optimal pre-conditioner. The paper then extends this result in the non-Gaussian setting and present similar results. The paper then validates the result from 2 linear inverse problem and illustrates that pre-conditioning is important in robustly recovering the true source of the linear inverse problems.", "questions": "- The paper studies a non-Gaussian setting, but the 2 illustrations seem to both stem from Gaussians. Is this an accurate characterization? Is it possible to illustrate a non-Gaussian example?\n- Is it possible to obtain the optimal pre-conditioner outside of illustrative examples? If not, does this paper point to a possible solution to the problem of choosing a sensible preconditioner in a practical setting?", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "I am quite unfamiliar to the field of Bayesian inverse problems, so I find it quite difficult to assess the novelty of this work with little knowledge of related work. Therefore, while I recommend acceptance for this paper, my confidence level is low. \n## Strengths\nWithin the scope of the linear inverse problem, this paper does a good job of explaining and demonstrating the role of the preconditioner. \n\n- The paper is well-written with an easy-to-follow Gaussian example that illustrates to readers who might not be familiar with the linear inverse problem (such as myself) to understand. \n- The paper's illustrations seem to support its robustness towards discretization, as well as the failure of vanilla Langevin. \n- The idea of an optimal preconditioner seems to be of wider interest \n\n## Weaknesses\n- The numerical illustrations are quite difficult to follow. For example, Figure 4 comes with very little explanation and I find it hard to understand the purpose of the figure. \n- The illustrations are largely synthetic example that does not include learned score functions, but instead rely on the ground truth score functions with perturbations. \n- The assumptions of joint diagonalizability and a diagonal $A^\\top A$ seem quite restrictive.", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the Reviewer for their positive feedback.\n\nWe are happy to clarify our manuscript in response to the Reviewer's thoughtful remarks and questions. We hope this could lead to an increase in their rating (or confidence score).\n\n1. **Explaining Figure 4.** Figure 4 shows the empirical autocorrelation function (ACF) of the first ten modes, comparing preconditioned  versus vanilla Langevin sampling. The $x$-axis is the lag, and the $y$-axis is the autocorrelation, which measures how strongly each mode at step $k$ remains correlated with its initial value. Under preconditioning with the optimal $C$, all the modes ACFs show uniform and rapid decay, meaning that they converge uniformly and quickly to the posterior, as the theoretical analysis of Section 4 anticipated. In contrast, vanilla Langevin displays slow, non-uniform decay. We agree this figure would benefit from a clearer explanation, and we will use the extra page in the camera-ready version to add a more detailed description. We thank the reviewer for the suggestion.\n\n2. **Synthetic examples.** The synthetic examples are proposed to illustrate the theoretical results, which are the main contributions of the paper. This led us to choose examples that satisfy the key assumptions---namely, the conditions on the score error from Assumption 1, Theorem 3.1 and Theorem 4.1. For this reason, rather than using a learned score model, we simulate an imperfect score by perturbing the analytical score of the prior.  This is a common approach in the literature; see, for example, the extended numerical experiments of [24], which we cite in the Introduction of our paper.\n\n3. **Assumptions on co-diagonalization of $C$, $C_\\mu$ and $A^\\top A$.** The co-diagonalizability assumption for $C$ and $C_\\mu$ is standard in the literature on infinite-dimensional diffusion models [20, 21, 48]. As for the assumption that $A^\\top A$ can be diagonalized, it is common in many linear Bayesian inverse problem settings and their theoretical analysis; see, for example, the foundational works of Stuart [3] and Knapik et al. [4]. Moreover, in many classical linear inverse problem where $A$ is compact---such as the heat equation, tomography, or inverse scattering for Schr\\\"odinger-type operators under the Born approximation---one can always find a basis in which $A_N^\\top A_N$ is diagonal. However, we emphasize that the main conclusions of our analysis remain valid even without the diagonalization assumption. For example, the asymptotic expansion in Eq. (10) of Theorem 3.1 can be extended to non-diagonal  $A_N^\\top A_N$, after some leg-work, by replacing scalar expansions with the corresponding matrix-series expansions; the part regarding the higher-order modes remains the same. Likewise, in Section 4 one can verify that, under a perfect score function, the optimal preconditioner still takes the form $C=[C_\\mu^{-1} + \\sigma^{-2} A^\\top A]^{-1}$. We will add a remark including these additional considerations and thank the reviewer for pointing out the need and relevance of these comments.\n\n4. **Non-Gaussian setting and illustrations.** The paper presents analysis for both Gaussian and non-Gaussian settings. But since the key insights are closely aligned in both cases  (see, for example, lines 194-199; 230-236; 272-281), we preferred focusing on the Gaussian case in the illustrations. \nThis allows for explicit expressions for the mean reversion rate, avoiding the added complexity of working with upper and lower bounds of $\\partial^2_j \\phi_j$, and enabling a cleaner illustration of the main results of the paper. That said, our framework does allow for a non-Gaussian examples; we will explore this if such an example adds further insight into the core theoretical contributions of the paper.\n\n5. **Choosing a preconditioner in practical settings.** A simple and practical choice is to select a preconditioner that is as close as possible to the prior covariance; as shown in the analysis of Theorem 4.1, for the higher-order modes, the leading order-term of the preconditioner is simply the inverse of the prior covariance. This choice is especially reasonable when the prior decays quickly, in which case $\\mu_j^{-1}\\gg \\sigma^{-2} (A^T_N A_N)_{jj}$for the lower-order modes. Ideally, any available information about the posterior covariance and the score approximation error can be used to refine this first approximation. We can add a remark on this point,  although we emphasize that a detailed investigation of the computational cost of approximating the optimal preconditioner lies beyond the scope of this work."}, {"review_id": "gt7OVZ9Oqu", "reviewer": "Reviewer_PRj9", "summary": "The authors analyze Langevin dynamics driven by learned scores in the infinite-dimensional (function) space setting. In particular, the authors studied how the approximation error of the learned score impacts the stationary distribution.", "questions": "- What system knowledge is required to compute the preconditioner and what are the computational costs? For example, it seems a complete diagonalization of the forward operator is needed, which can be costly. \n\n- On line 198 the authors discuss the bias introduced by the learned score. How large is this in the numerical examples? For which types of inverse problems is this a major issue and does it give insights when learned priors are useful and when not?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n+ Using score-based models for sampling in inverse problems (rather for generating new data) is an important research direction. The authors provide convincing examples why inverse problems coming from PDE settings should be studied in the infinite-dimensional setting, which is largely ignored in the more common \"generative modeling\" setting.\n\n+ The analysis leads to actionable insights, namely a preconditioner. Numerical results (e.g., Figure 3) demonstrate that using the preconditioner stabilizes the inversion. \n\n+ The authors also treat the case of non-Gaussian priors, as long as they are abs continuous to a Gaussian reference (standard assumption).\n\nWeaknesses:\n- The analysis is present only for the case that A^T A is diagonalizable. The authors comment on this limitation and state that more technical argument can handle the more general case but do not provide details. \n\n- The computational costs of obtaining the preconditioner are not discussed, especially what information about the forward model and the learned score are needed.", "was_revised": true, "final_justification": "The authors provided a few more clarifications. I considered this paper as strong before the rebuttal and the answers confirmed me in this rating.", "rebuttal": "We would like to thank the Reviewer for their positive feedback.\n\nWe are happy to clarify our manuscript in response to the Reviewer's remarks and questions.\n\n1. **Assuming $A^\\top A$ is diagonalizable.**  This assumption is common in many linear inverse problem settings and their theoretical analysis [3, 4]. We adopted it because we believe that focusing on the diagonal case provides the clearest exposition, keeping our analysis as clean, interpretable, and insightful as possible. Moreover, for any linear inverse problem where $A$ is compact, one can always find a basis in which $A_N^\\top A_N$ becomes diagonal. However, what we wrote in the Discussion and Future Work section remains true---the main takeaway of our analysis holds even without the diagonalization assumption. The asymptotic expansion in Eq. (10) of Theorem 3.1 can be extended to non-diagonal  $A_N^\\top A_N$, after some leg-work and under appropriate assumptions on the structure of the score approximation error, by replacing scalar expansions with the corresponding matrix-series expansions; the part concerning the higher-order modes remains unchanged. Similarly, in Section 4 one can verify that, under a perfect score function, the optimal preconditioner still takes the form $C=[C_\\mu^{-1} + \\sigma^{-2} A^\\top A]^{-1}$. We can add a remark including these additional considerations.\n\n2. **Computing the preconditioner.** Indeed, exactly computing the optimal preconditioner would require the diagonalization of $A_N^\\top A_N$, which can be computationally expensive. However, a simple and practical alternative would be to select a preconditioner that is as close as possible to the prior covariance; as shown in the analysis of Theorem 4.1, for the higher-order modes, the leading order-term of the preconditioner coincides with the inverse of the prior covariance. This choice is particularly justified when the prior decays quickly, in which case $\\mu_j^{-1}\\gg \\sigma^{-2} (A^T_N A_N)_{jj}$ for the lower-order modes. Any additional information about the posterior covariance or the\nscore approximation error can then be used to refine this initial approximation. We can add a remark\non this point, although we emphasize that a detailed investigation of the computational cost of\nconstructing the optimal preconditioner lies beyond the scope of this paper and is left for future work.\n\n3. **Bias and implications**. In the numerical examples, we set $\\epsilon^b\\_j =0$. Since formula (10) is an equality, a large value of  $\\epsilon^b\\_j$ would necessarily lead to a large sampling error, preventing an accurate recovery of the posterior mode. In fact, learning an incorrect score can significantly impact the posterior sampling---a challenge that already exists in finite dimensions but can become catastrophic when attempting to control the global error in the infinite-dimensional setting. In this sense, our result highlights the interplay between\nthe data spectral properties and the design of the preconditioned Langevin sampler; understanding how these spectral properties, together with the structure of the inverse problem, influence the learnability of the score  is an important question for future research, especially given that there is currently\nlittle literature on how to control the error arising from training infinite-dimensional score-based\ngenerative models to approximate the true score function (see [20, 42, 43, 45, 48]). Our contribution here is a first step toward a theoretical understanding of this issue,\n   in a setting that allows for rigorous  analysis of when a learned prior can provide useful insight."}, {"review_id": "9K2UN2zRtj", "reviewer": "Reviewer_QcCN", "summary": "The work analyzes Score-based Models for Bayesian inference of linear inverse problems, extending to the case that posterior is supported on an infinite-dimensional space. It is shown that a trace-class preconditioning operator is required for the Langevin diffusion. Sufficient conditions for global convergence in KL divergence are derived and the existence and the form of an optimal preconditioner which depends on the score approximation error and the forward operator are also proved. The analysis covers both Gaussian and non-Gaussian priors.", "questions": "1. In Fig 1, the first row of simulation is described as having growing variance. However, it seems to me that the variance stay the same when the mode index is above 60 or so. Can the authors provide further explanation regarding this?\n\n2. Can the author justify the assumption that $A_N$ is non-singular?\n\n3. How plausible are the conditions, $p_j^{-1} \\epsilon_j^a(\\tau) = O(\\tau)$ and $\\lambda_j^{-1} \\epsilon_j^b(\\tau) = O(1)$ in Theorem 3.1? Can the authors provide some insights on these conditions?\n\n4. The assumption that $A_N^\\top A_N$ is diagonal in Proposition 4.1 and Theorem 4.1 seems to be rather restrictive.\n\n5. The mean reversion rate $\\kappa$ seems to be undefined in Proposition 4.1.\n\n6. Assumption 1 is on the score approximation error, can the authors provide some comments on how this error can be guaranteed to be small as later results depend on this error?\n\n7. In line 280, $|\\epsilon^a_j| \\ll \\lambda_j$ requires approximate error to be small ($\\epsilon^a_j$ need to be small) if the scale of the Brownian motion ($\\lambda_j$) is small. It will be better if the authors can present some examples for validating such sufficient conditions.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Lifting the Bayesian inference algorithms to infinite-dimensional space is well-motivated. The work provides the convergence analysis for both Gaussian and general non-Gaussian priors. It has good amount of originality. \n\nHowever, the work has some restrictive assumptions and the authors do not justify these well and thus the significance is not obvious to me. For example, how can the non-singularity of $A_N$ and $A_N^\\top A_N$ be guaranteed in practical problems? Besides, the sufficient conditions of the optimal preconditioned is not well discussed to provide enough insights. \n\nOverall, it will be better if the authors can provide some examples which satisfies the derived sufficient conditions and thus make the significance of the work more clear and further improve the quality of the work. If the sufficient conditions are easy to validate, then the work will be more of importance.", "was_revised": true, "final_justification": "After the rebuttal with the authors and the discussions from the other reviewers, I decided to raise my score from 4 to 5.\n\nMy original borderline acceptance rating is because I am not familiar with originality and significance of such work. The detailed discussion has helped me understand the paper better and thus improve my confidence on the paper.", "rebuttal": "We would like to thank the Reviewer for their constructive feedback.\n\nWe are happy to clarify our manuscript in response to the Reviewer's questions. We hope that this could lead to an improvement in their assessment of the paper.\n\n1. **Figure 1.** We thank the reviewer for spotting this typo. What we intended to say is that the posterior covariance is _not_ trace class; that is, the eigenvalues of its second moment do not decay at infinity---not that the modes exhibit growing variance. As a result, posterior samples do not belong to $H$, which is theoretically inconsistent with the formulation of the problem. Indeed, a Gaussian measure is supported in $H$ only if its second moment is finite on $H$, which is not the case in Figure 1, since the posterior variance is not decaying at fine scales. We will fix this typo in the camera-ready version of the paper.\n\n2. **Is $A_N$ really non-singular?** We thank the reviewer for pointing out this assumption, which is a leftover from our first  analysis attempts, where we wanted to keep the setting as  simple as possible. In fact, as the reviewer may have noticed, the final analysis presented in the present manuscript does not rely on the non-singularity of $A_N$; nowhere in the main results or proof do we require that $(A_N^\\top A_N)\\_{jj}\\neq0$. The only part that may require a minor adjustment is lines 234-235, but this does not pose any issue: if $(A_N^\\top A_N)\\_{jj}=0$, then the convergence rate for that mode is simply $\\lambda\\_j/\\mu\\_j$. We will remove this assumption in the camera-ready version of the paper.\n\n3. **Assuming $A^\\top A$ is diagonalizable.** This is a common assumption in many linear Bayesian inverse problem settings and their theoretical analysis; see, for example, the foundational works of Stuart [3] and Knapik et al. [4]. Moreover, in many classical linear inverse problems where $A$ is compact---such as the heat equation, tomography, or inverse scattering for Schr\\\"odinger-type operators under the Born approximation---one can always find a basis in which $A_N^\\top A_N$ is diagonal. That said, the diagonalization assumption is not essential to our analysis; it is adopted primarily to keep the presentation as clean, interpretable, and accessible as possible. For example, the asymptotic expansion in Eq. (10) of Theorem 3.1 can be extended to non-diagonal  $A_N^\\top A_N$, after some leg-work, by replacing scalar expansions with the corresponding matrix-series expansions; the part regarding the higher-order modes remains the same. Likewise, in Section 4 one can verify that, under a perfect score function, the optimal preconditioner still takes the form $C=[C_\\mu^{-1} + \\sigma^{-2} A^\\top A]^{-1}$. We will add a remark including these additional considerations.\n\n4. **Missing definition of the mean reversion rate.** Thank you for noticing this; we will include the definition in the camera-ready version.\n\n5. **Conditions on score approximation error (answer to questions 3,6,7).** The affine-error decomposition in Assumption 1 is natural and follows from the co-diagonalization assumption for $C$ and $C_\\mu$, which is standard in the literature on infinite-dimensional diffusion models [20,48].  In lines 194--199, we provide a qualitative analysis of the impact of $\\epsilon^a_j$ and $\\epsilon^b_j$ in Proposition 3.1. From the remarks, it is clear that assuming $p_j \\epsilon_j^a$ to be small and $\\lambda_j^{-1} \\epsilon_j^b$ to be bounded is essential for achieving an accurate sampling of the posterior. Theorem 3.1 then makes these observations more quantitative, with assumptions that directly reflect the insights of Proposition 3.1. Since formula (10) is an equality, any significant deviation of $\\epsilon^b_j$ and $\\epsilon^a_j$ from the stated assumptions would necessarily result in a large sampling error at leading order, preventing an accurate recovery of the posterior mode. In this sense, Theorem 3.1 highlights the interplay between the data spectral properties and the design of the preconditioned Langevin sampler. It provides a sharp characterization of the link between the score approximation  error and the posterior sampling  error, showing that if the score approximation error is too large, it can  significantly impact the overall  posterior sampling error, potentially leading to catastrophic consequences for the error bound. To our knowledge, we are the first ones to provide sufficient conditions that ensure the boundedness of this error in the setting of infinite dimensional Bayesian linear inverse problems. Whether these conditions on the score approximation error are easy to satisfy in practice is an important question for future research, especially given that there is currently no literature on how to control the error arising from training infinite-dimensional score-based generative models to approximate the true score function, see [20, 42, 43, 45, 48]. With this paper, we hope to have laid down not only the theoretical framework but also the practical motivation for why such an analysis is essential to fully understand the infinite-dimensional setting.  We thank the reviewer for bringing up this point; in the camera-ready version, we will include these considerations in the Discussion and Future Work section, clarify the connection between the discussion in lines 194--199 and the assumptions in Theorem 3.1, and elaborate on the implications of Remark 3.1."}]

Author rebuttal:

Meta-review: This manuscript provides a theoretical study of an infinite-dimensional version of the Langevin diffusion sampler. All reviewers agree that the derived results present a valuable and interesting contribution to the field (clearly formulated and well-presented). During the rebuttal phase, the authors could largely clarify all concerns raised during the initial review (also leading to score increases), including how the current results are novel compared to prior work. From my perspective, this is already a solid paper and if all requested changes are included in the final version, this presents a solid contribution to NeurIPS. Hence, I am recommending acceptance.
Decision: Accept (spotlight)