Stability and Generalizability in SDE Diffusion Models with Measure-Preserving Dynamics
Authors: Weitong Zhang
Chengqi Zang
Liu Li
Sarah Cechnicka
Cheng Ouyang
Bernhard Kainz
Venue: NeurIPS 2024 poster (2024)
Keywords: Diffusion modeling
Random dynamical systems
Magnetic resonance imaging
Primary area: diffusion_based_models
TL;DR: Measure-preserving Dynamics in SDE Diffusion

Abstract: Inverse problems describe the process of estimating the causal factors from a set of measurements or data. 
Mapping of often incomplete or degraded data to parameters is ill-posed, thus data-driven iterative solutions are required, for example when reconstructing clean images from poor signals. 
Diffusion models have shown promise as potent generative tools for solving inverse problems due to their superior reconstruction quality and their compatibility with iterative solvers. However, most existing approaches are limited to linear inverse problems represented as Stochastic Differential Equations (SDEs). This simplification falls short of addressing the challenging nature of real-world problems, leading to amplified cumulative errors and biases. 
We provide an explanation for this gap through the lens of measure-preserving dynamics of Random Dynamical Systems (RDS) with which we analyse Temporal Distribution Discrepancy and thus introduce a theoretical framework based on RDS for SDE diffusion models. We uncover several strategies that inherently enhance the stability and generalizability of diffusion models for inverse problems and introduce a novel score-based diffusion framework, the Dynamics-aware SDE Diffusion Generative Model (D^3GM). The Measure-preserving property can return the degraded measurement to the original state despite complex degradation with the RDS concept of stability.
Our extensive experimental results corroborate the effectiveness of D^3GM across multiple benchmarks including a prominent application for inverse problems, magnetic resonance imaging.

Stability and Generalizability in SDE Diffusion Models with Measure-Preserving Dynamics
Weitong Zhang
Chengqi Zang²
Liu Li
Sarah Cechnicka'
Cheng Ouyangl.3
Bernhard Kainz 1.4
Imperial College London, UK, Uniivrsityy of Tokyo, JP, University of Oxford, UK "Friedrich-Alexander University Erlangen-Nümberg, GER
weitong.zhang200 zhang20@impprial..emm uc.
Abstract
Inverse problems describe the process of estimating the causal factors from a set of measurements data. Mapping of often incomplete degraded data parameters ill-posed, thus data-driven iterative solutions are required, for ex- ample when reconstructing clean images from poor signals. Diffusion models have shown promise as potent generative tools for solving inverse problems due to their superior reconstruction quality and their compatibility with iterative solvers. However, most existing approaches are limited to linear inverse problems repre- sented as Stochastic Differential Equations (SDEs). This simplification falls short of addressing the challenging nature of real-world problems, leading to amplified cumulative errors and biases. We provide an explanation for this gap through the lens of measure-preserring dynamics of Random Dynamical Systems (RDS) with which we analyse Temporal Distribution Discrepancy and thus introduce theoretical framework based on RDS for SDE diffusion models. We uncover several strategies that inherently enhance the stability and generalizability dif- fusion models for inverse problems and introduce novel score-based diffusion framework, the Dynamics-aware SDE Diffusion Generative Model (D'GM). The Measure- prreervvgg property return the degraded measurement to the original state despite complex degradation with the RDS concept of stability. Our extensive experimental results corroborate the effectiveness of DMGM across multiple bench- marks including a prominent application for inverse problems, magnetic resonance imaging.
Introduction
Diffusion probabilistic models 5 54 52
have demonstrated impressive performance across various image generation tasks, primarily by modeling diffusion process and then learning associated reverse process. Among the many commonly used approaches [63], diffusion models that incorporate the concept of score functions (28 55 capture the intrinsic random fluctuations of the forward diffusion process, positioning them as good choice for in-depth analysis. Score-based generative models (SGMs) entail gradually diffusing images towards a noise distribution, and then generating samples by chaining the score functions at decreasing noise levels with score-based sampling approaches. One such example of SGM with a score-based sampling technique, known score matching [54] has gained popularity for density estimation [[6]. It employs methods such as
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Langevin dynamics 21 |1 43 30] and SDEs
29 37
31
to simulate the underlying probability
distribution of training samples.
However, vanilla unconditional SGMs can be extended to inverse problems by leveraging an implicit prior distribution, based on the available counterpart measurement, subjected corruption and/or noise. To this end transitionary SGMs enable an iterative recovery of the data from this noisy counterpart, instead of relying on Gaussian white noise starting point [66 37 39
20 34
Intuitively, leveraging priors and a generative capacity into transitionary SGMs offers the possibility of exploring high quality reconstruction and restoration and gaining better performance. However, current transitionary SGMs that incorporate priors have largely overlooked the unreliable quality of the prior and its measurement. Empirically, transitionary SGMs cannot always be trusted in terms of stability and efficiency, especially in a regime of non-uniformly distributed noise or corrupted signal quality [ [66] 377. Hence, the exploitation of transitional learning within SGMs does not come without costs as their advantages vanish in limited data quality settings. Theoretical understanding is notably lacking in this field with the following fundamental open problem: Can we realize reliable transitionary diffusion processes in practice for inverse problems with a theoretical guarantee?
While recent works have started to lay down theoretical foundation for these models, detailed understanding is still lacking. Current best practice advocates for smaller initialisation values (e.g.. noise schedule [66] [20]. instead of large values [15] 61 to ensure that the forward dynamies brings the diffusion sufficiently close to a known prior and simple noise distribution. However, a proper choice of the values conditioned on the prior within a theoretical framework should be preferred for a better approximation of the score-matching objective and higher computational efficiency. To fully facilitate the power of reversion and generation of transitionary SGMs and to mitigate the influence of low-quality measurements for solving inverse problems, this paper provides a measure- preserving dynamics of random dynamical system (RDS) perspective as promising way to obtaín reliable reversion and generation. Notably, our 'measure' is not only the observations (e.g., degraded images), but also represents the invariant probability measure (distribution) of the RDS. This allows to consider the concept of an RDS stability and to frame challenging degradation learning within a measure-pee preeeurving dynamical system. Thus, we can start from a transitionary SGM interpretation of diffusion models and connect RDS to the SDE in transitionary SGMs. The pitfalls (e.g., Instability) are discussed in Sect. 3 and further implications can be found in the Appendix. Transitionary SGMs have not been fully explored before, and we provide a theoretical interpretation of a stationary process as a possible solution.
Our D'GM framework is abstracted from transitionary SGMs, The key to our framework is a station- ary process following
dynamics ensure the stability and generalizability of the diffusion, as well as reducing the influence of accumulated error, distribution bias and degradation discrepancy. Our contributions can be summarised as follows:
1. Temporal Distribution Discrepancy: We conduct a rigorous theoretical examination of the instability issue of transitionary SGMs, measured as Temporal Distribution Discrepancy (i.e., lower bound of modeling error). This analysis sheds light on critical aspects related to stability and generalizability" effectively addressing unexplored fundamental gap in the understanding of solving challenging inverse problems with SDEs.
We propose a solution, DCGM, and an explanation from measure-preserving dynamics of Random Dynamical Systems (RDS). 'Measure" includes both measurements (degraded image) and invariant measures (distribution) of RDS, which allows complex degradation learning and enhances restoration and reconstruction accuracy.
2. GM Framework:
3. Thorough Evaluation:
Our contributions are substantiated by extensive validation. We demon- strate the practical benefits of our DO GM framework across various benchmarks, including challenging tasks such the reconstruction of Magnetic Resonance Imaging (MRI) data.
We address the instability of diffusion models for inverse problems under domain shift and concept drift (unknown and heterogeneous degradation). This leads to what we believe is a completely novel view on the theoretical foundation of how the degradation process is modelled. The result is an approach that is more in line with the original intention of the theory of diffusion. We
'Generaiizabiiity refers to the extent to which out-of distribuuti and domain shift impacts the fidelity of the restoration process. Stability in SDE diffusion models is demonstrated by its resilience to degradation beyond its domain and its consistent ability to restore high-quality images.

---

chose inverse problems as a relevant application area to demonstrate our ideas but also included variety of challenging problem settings explore the generalizability of D&GM. To the best of our knowledge, no other method can handle a diverse range of challenging tasks like real-world dehazing, compressed MRI reconstruction, blind MRI super-resolution, etc. with unified underlying theoretical framework. In Tab. 1 we illustrate the key differences of D&GM compared with SGMs and transitionary SGMs.
2 Preliminaries
SGMs. We will follow the typical construction of the diffusion process x(t). t [0,7] with <<t) E Rd. Concretely, we want x æ00) ~ po()), where po Pdataa and a(T) ~ Pr, where pr is a tractable distribution that can be sampled. In this work, consider the score-based diffusion form of the SDE [55]. Consider the following Itô diffusion process defined by an SDE:
(I) where f: Fd + Rd is the drift coefficient of (t), IR + R is the diffusion coefficient coupled with the standard d-dimensional Wiener process E IRd. By carefully choosing f. 9. one can achieve a spherical Gaussian distribution as + T. For the forward SDE in Eq. there exists corresponding reverse-time SDE 3 55 dx
da fæ, t) + g()
( t) 9()2 V log g())WW
score function
where is the infinitesimal negative time step, and w is the Brownian motion running backwards. The score function Vx log pe (ar) is in general intractable and thus SDE-based diffusion models approximate by training time-dependent neural network under a score function [57]28] (28.
Transitionary SGMs. [1 37 66 34 20 I2 leverage transitionary iterative denoising paradigm
for the inverse problems. In inverse problems, such as super-resolution, we have an (nonlinear, partial, and noisy) observation of the underlying high-quality signal æ. The mapping æ + is many-to-one, posing an iol-posed problem. this case, strong prior on x: is needed for finding a realistic solution. Formally, the general form of the forward (measurement) model is:
(3) where A()): Rd RT oftentimes n < d is the forward measurement operator and n is the measurement noise, assuming ~ N (0, I While sharing similar aim of bridging y and æ in transitionarr SGMs, different mathematical frameworks have been used: [17] employs Inversion by
y A (x) + n2, y. R" x E Rd
Direct Iteration;
37 34 20 model it Mean-reverting SDE.
Transitionary. SGM has become an increasingly important line of SDE research due to the applicability on images with theoretical guarantees. However, they often perfrom poorly in real-world scenarios. To provide theoretical investigation of this gap, we interpret Transitionary SGM as Ornstein- Uhlenbeck (OU) process. This perspective allows us understand the random fluctuations in image degradation as stochastic processes, providing a foundation to integrate random dynamical systems (RDS) with the diffusion process natural extension of the SDE framework involving the OU process.
The Mrassve-preservingg property is introduced from the perspective of RDS: still return to the original state despite severe degradation. Our approach constructs a bridge from measure- prreseriii dynamical system to transitionary SGM through measure- preesure-preserving dynamics, and highlights the Temporal Distribution Discrepancy in Sect. 33 Subsequently, we address this issue of instability: by incorporating a measure- greasure-preserving strategy into the solution of inverse problems, which is detailed in Sect. 4 This covers counterpart modeling, bridging transition from uncertain diffusion modeling to deterministic solutions, yielding significant improvements in both performance and efficiency as demonstrated in Sect. 3 More details can be found in Sect. and Appendix.
The distribution can
3 Instability Analysis: Transitionary SGMs with Corrupted SDE Diffusion
Ornstein- -ulteben- (OU) process. An OU process is common case in transitionary SGMs, where x, is defined using an SDE: dr, = dl + o,dW, W, is standard Brownian motion. A drift term ai can be introduced:
MRI signals defined C and Cd We demonstrate in Sect. that our approach is applicable MRI.
3

---

Table 1: Differences between tfat--ofthheaa art SDE diffusion based approaches.
Model
Theorem Foundazion
Properties
TDD (Prop_2)
Atrrctor
Operatoe Dic. sided one-sided
Ineree problem solver
SGM b3
A/0.
SDE dictusion
Unsolvable
No subset
pnd Ka.
Mean-reveriigg SDF
Incuuiiity
Limited
LinNNas-Lin
Pal (aa)
Schrudinger Bridge
Limited
No attracts
Non-Lin
DMGM (ours)
P4
Measure Preserving RDS
Stubilit
Robust
Mie roff)
two sided
Lin/ Now: Lin/Blind
dæ, 0s zt)dt o,dW
(4)
where μ denotes state mean, reflecting the expected state of the measurement (e.g., corrupted image [37] noisy speech (59) over time. fz. are time-dependent parameters. The drift term corrects deviations from the constant μa, effectively pulling the process towards μ 20) with Stability (in Appx. G opposed pure noise in Eq.
Measure-preserving Dynamics in SDE Diffusion. The solution of the above SDE can be represented by continuous-time random dynamical system yo defined on a complete separable metric space (X, (See precise definition of RDS in Appx. C. More generally, we can extend the RDS two-sided solution operator with flow map. The base flow driven by Brownian motion can be written W (t, (. = Wtt + s, s,w) Ws,
Proposition 1 After extending the solution of the OU process to RDS, the measure-pres measure-preserving RDS should meet the property stt, s;w)z 4 (t s, 0; (.w).. However, OU processes with time- varying coefficients usually do not satisfy this property. In this situation, the system breaks the forward-reverse processes, making it difficult maintain stability.
Intuition 1. A two-sided measure pressurvprreerving random dynamical (MP-RDS) system for- mulation enables us to use the Poincare recurrence theorem [45]. (see precise statement in Appx[), intuitively, with two-sided MP-RDS St, the Poincare recurrence theorem ensures that the system yo; starts from terminal condition 2y, run backward in time, will hit a region (xo e, Fo + <) for small in finite time, where 2g is the high-quality image.
Example Following Intuition and Prop. 1, suppose that the OU's 0 follows a cosine schedule, such that 0, cos(t) for 0 < < T, then for some 0 S s t S T, (tt s; w)x + ip (t - s, 0; (,w)). x because the change of 0, w.r.t. time is not uniform. The OU-process instability exists due Temporal Distribution Discrepancy (Prop. 2).
At a high level, Proposition can be extended show that there exists a compact attracting set at any -00 < t < oo, and this convention has allowed us to characterize the attractor K (w) = N [u, o² /20). The closed-form distribution for y can be complex and may not be tractable depending on the particular scenarios of the actual image degradation process The modification of a and 0 is used to regularize the perturbation and attempt to close the distribution. However, these injections might bypass the stationary process. More details can be found in Appx. D
Instaollity-Temprra Distribution Discrepancy. Given the process OU (Ix, pz; 0) with Eq. 4 where 2T # Tooo for finite T, indicates that the perturbed state cannot move towards the degraded L.Q image and fails matching the thcoretical distribution. This inherent discrepancy further causes bias in the estimation of μr, which gradually accumulates into error in the reverse process.
Temporal Distribution Discrepancy is illustrated by Proposition 2 (proof can be found in Appx.
Proposition 2 Given Eq. 3 and Eq. 4 and assume that the score function is bounded by C in L norm, then the discrepancy between the reference and the retrieved data is, with probability (1 8) least:
1100- OU 11; T, ((x0 - 11)2 -07/20r) -28r + o /20
(5)
maa (Co + 2 log 2 2log 6)

---

Intuition 2. Intuitively, Proposition 2 provides a theoretical measurement on how the differ- ence between finite iteration distribution and the asymptotic distribution of the OU-process in 1.2 could further enlarge the discrepancy between the retrieved image and the actual HQ image. Discrepancies are typically explained by complex degradation monotonic modeling.
For a noisy inverse problem, the retrieved data with any finite I depends on ot, Hi,, T, K (Lipschitz constant). This proposition also correlates [39,, where the lower bound of the distance between the high quality image and retrieved image in L norm in our model is further enlarged by this discrepancy, which correlates to the term ((ro - -1)² (o /20))) e 20r /20r
Another way further minimize this bound is through the term e 202 with [0, T normalized to [o, 1]. What we refer to as 0-schedule corresponds to the exact functional form of Or, several schedules be set here, e.g., constant, linear, cosine, and log. At high level, the discrepancy between the reference and the retrieved data stems from the divergence between the forwarded final state and the low quality image. Eq. 5 can be factored into three constituent parts: the data residual, the stationary disturbance, and the random noise. While conditional diffusion generation entails a trade-off between variability and faithfulness [66]. the persistent discrepancy within the residual has significant impact on the generalizability of solving the transitionary tasks. This also establishes connection with SDEdit 39 and CCDF [12]. When fitting inverse problems involving paired data into diffusion models, while accounting for deviations and degradation, inserting them into Eq. 4 directly may not be the most effective strategy.
During sampling and inference with the degraded input y. the discrepancy identified in Prop. 2 intensifies. The complex degradations in y exacerbate the divergence from the expected μ distribution, significantly impacting the accuracy of the restored data žо. More details are in Appx. F
4 Towards Stability: Measure-preserving Dynamics in SDE Diffusion
In Sect. we extrapolate and theorize the Temporal Distribution Discrepancy on and x in the diffusion model for the inverse problem. Our key idea is combine the stationary process to alleviate the Temporal Distribution Discrepancy problem following the measure-preserviin dynamics from RDS. Recall that our 'measure' not only the measurements (i.e., degraded image), but also represents the invariant measure (distribution) of the RDS.
We begin by describing the forward and reverse processes of the D*GM, which serves stable bridge between the quality data and the counterpart measurement. We adapt score-based training methods estimate this SDE. Following this, we deseribe the essential constructions for preserving the stationary process in the diffusion model and solving for Temporal Distribution Discrepancy on an orthogonal basis compared to current transitionary diffusion models.
Measure-preserving Dynamics with the Stationary Process. Following Prop. I in a SDE Diffusion from 0 to T, the corresponding 'attractors' (states) can be viewed as N (++ (zo p)e- 8. oo - -22) 200-0)0/22) 3/20.). We can guide SDE Diffusion towards a stable and robust solution based on the properties of preserig in RDS. It can be extended to impose that for every t, 28, process. This convention allows us to reduce the regularization on two variables o O, to just one variable to satisfy the property of the peaasure-preeerving dynamics in the asymptotic sense, i.e., lim;-ro (ttt, w; lim o (t s, O; & (,w) This convention allows to characterize the attractor of the system as K K(w) = [N,,, X²).
x x where À is the variance of the designated stationary measure forward
The definition and constraint of the attractor are significant; without imposing this constraint, the seaasure-preserving property cannot be maintained, and the system would degrade into a Coefficient Decoupled SDE (Coe. Dec. SDE), we also analyse this in Fig, 2 and Tab. I in Appx. H
5

---

MRI Reconstruction
MRI Super-resolution
Real Dense Haze Removal
Rain Removal
DMCM L.
Misrare senerndng
Sarbonrry Powesss
M Forword Process
RAA Bubbe Beikeel
High Quality Image
Image
Quality Image
Output
Figure I: Dynamics-aware SDE Diffusion Generative Model (D*GM). When extending transitionary SDEs to random dynamical systems (RDS), their measure-presert pressrre- property should be kept to maintain stability. This corresponds to driving the SDE towards the drift term μs (LQQ). There is Temporal Distribution Discrepancy which results from the gap between the forward estimation and the low quality image in the SDE, With the distribution aligned between FT and μ., the SDE can be made more robust to inverse problems. Reconstruction results for low quality (LQ) images after application of our D*GM method, on different tasks, compared to the ground truth (GT) on two domains The frequency domain: MRI Reconstruction (undersampling factor 8x, 16x, frequency masks are colored red); MRI Super-resolution (up-scaling factor of X4, cross- cross-domant domain evaluation). The image domain: Real Dense Haze Removal; Rain Removal (light, heavy).
Example 2. When - oc, the attractor becomes excessively large, reducing the significance of μ.. SDEs exhibiting this behavior defined Coefficient Decoupled SDEs. In practice, l demonstrates non- non-infinite properties as an input image, while the corresponding sigma and theta are indeed unconstrained. In such decoupled forms, the coefficients of the attractor size increases &, diminishing the significance of
Based on Prop. 2 since the temporal distribution discrepancy always exists as long the running time is finite, and we want the final state ær to be as close as possible to the distribution of I Therefore, we introduce such that given T. the distribution of vr follows A N(1(1 - (1(11_er)) Toe Tre"r,r2>' )"r)), and Too follows .V 72>?), with 7 > 1, we increase the possibility of sample 7T from FT to become closer the distribution of Toc. and thus serves as a plausible initial state for the reverse process. We can control how much to close the distributions, either by increasing the stiffness at the cost of potentially destabilizing the reverse process, or by decreasing r to further smooth the density functions of both distributions at the cost of more reverse iterations.
By connecting the inverse problem with the analysis above, we clarify the discrepancy in the stationary modeling process from meeasrre-presevvee dynamics and thereby improve the generalization of diffusion processes and the accuracy of the reverse process. This is particularly important for accommodating the diversity of degradation states and to ensure accurate sampling.
Forward Process. We describe the forward process as: dr; 0₂ (u xz, )tt To,dWe parameterized by to calibrate the SDE modeling, is the state mean. The parameters 0, and o,, both being time- dependent and strictly positive, correspond to the rate of mean reversion and the stochastic volatility, respectively. The selection of 6, and a, offers flexibility in Tab. 2 c.p., Sect.
Considering the trade-off between complexity and effectiveness, Cos has been chosen for both 0, and ot. This aims capturing complex temporal dynamics computationally tractable manner, thereby optimizing the balance between the performance and calculation convenience.

---

Table 2: (4(2,, t). ve(3e, 8) solutions with various 8e. oG.
()(
(a f)
eat xo + (1 (--e" el lox(1+e"t)
Lin
x²x² - e 0x²)
y
No + (1 - ct los(1-of())
^²x² - 2 lox(
Log
(0 ( sinfe)
+ 1 -((1 sing
1 20 (im sim(es)
Cos
^²x²
Quad
e xxxx xu + (1
0 Jlyy
727(1
220 f1
In the forward process, the mean He approaches the low-quality image with (xx)) = p. while the variance tends toward the stationary variance var (z;) rig² Essentially, the forward SDE transitions the high-quality image to a low-quality counterpart infused with Gaussian noise. The discretized SDE for the forward process is 2t, It.-, + 02,_, (u zt,_,) + -o AW, We employ a transition strategy utilizing a varied stationary variance. Additionally, we execute an unconditional update, which operates without the need for matching in the reverse process. These not only allow image corruption but also provides effective adaptability for improvements.
Reverse Process. The reverse process aims at reconstructing the original image by gradually denois- ing a low quality image. utilizes the score the marginal distribution, denoted as V, lo (x), and is governed by:
dr, [0 22, - (x)) dt + ro,dW.
The reverse-time DMG process of Eq. can be found in Appx. D This closely mirrors the forward process and incorporates additional drift term proportional to the score of the marginal distribution. The ground truth score for this process, necessary for training our generative model, is:
Vx log (x mo) It (x
Vt.
where (2) represents the random attractor of the process time t, and is the variance. Our training objective is defined as the minimization of the expected discrepancy between the predicted and true scores the data distribution:
l" = arg min (x,x))))) we ((
- Vx, log (x, Xo. y)1k
(8)
where w = -1/r2 is a time-dependent weighting function, and Sg denotes the score network parameterized by 9 which approximates the score of the marginal distribution. The optimization is conducted over the network parameters 0, under the expectation with respect to the time variable t, the initial image Xu, noisy image xt, and data y.
5 Experiments
Experimental Settings: We evaluate DºGM on various challenging restoration and reconstruction problems. We initially analyze our method by examining its performance with closely related diffusion formulation variants. Subsequently, we benchmark DOGM against the state-of-the-ar techniques in these domains. For comprehensive evaluation across all experiments, we report the PSNR (25 and SSIM (88) for pixel- and structural-Ieve alignment, LPIPS [72] for measuring perceptual variance. An in-depth description of our implementation is provided in Appx. G
5.1 Stability: Illustrations of the Measure-preserving Dynamics within Diffusion Models
SGM, and Transitionary SGMs vs. D
We perform qualitative and quantitative analyses using variants of closely related formulations for Prop. and 2 and evaluate across (A) SGMs and (B) transitionary SGMs. (A) uses a common score-based SDE, (B) uses Coefficient Decoupled SDE (e.g., variance exploding SDE with the drift term (1) according to Prop. 1 and OU SDE, alongside our D*GM.
7

---

Table Quantitative results for Rain100H and Rain (best in bold and second best underlined)
Table Quantitative results for O- HAZE and Dense-
Rain100H
Raint001.
O-HAZE
Donse-Hau
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
LPIPS
Methods
PSNRT
SSIM
PSNR
SSIM
JORDER [1
26.25
0.835
0.197
36.61
0.974
0.028
DCP PK]
16.78
0.653
12.72
0,442
IRCNN 37
29.12
0.882
0.153
35.17
0.955
0.068
DehuzeNet
0.770
1384
0,430
0.020
GFN
18.16
0.671
MPRNeI 68
30.41
0.891
0.158
36.40
0.965
0.077
GDN 16
18,92
0.672
14.96
0.536
MAXIM 50
30.81
0.903
0.133
38.06
0.977
0.048
Fet-Net
MSHDN 18
22.12
24.36
0.749
0.770
15.30
15.13
0.551
0.549
VP8 (CD)
0.885
0.051
0.968
0.023
IR. SDE (OU)
31.65
0.904
0.047
38.30
0.981
0.014
SGID PTF 1
20.96
0.741
12.49
0.517
DPGM
Restormer
07
23.58
0.768
15.78
0.548
Dehaner
25.11
0.777
16.62
0.560
MI-TF
0.566
D*GM
26.23
0.786
15.83
0.551
19
(aun))
(a) O-HAZE and Dense-Haze.
(b) Rain 100 Rain
Figure 3: Qualitative results for (a) deraining and (b) dehazing.
Following Tab. 1 VPB 73
be regarded
0.00
E: 0.25
E 0.50
a Coefficient Decoupled SDE, and IR-SDE [37] as OU SDE. Our results in Fig. illustrate that D*GM converges stably towards the ex- pected distribution, unlike other methods which exhibit instability or deviation. This highlights the reliance of other techniques, e.g., score- based SDEs, on retrospective measurement con- sistency corrections.
4
Simulated Deraining: We evaluated DCGM to- gether with the state-of-the-art deraining strate- gies: (1) OU SDE method IR-SDE (37). Co-
efficient Decoupled (CD) VPB
733 and other
phose
CNNs 64
56 We use two of the most renowned synthetic raining datasets: Rain100H [65] and Rain100L [65]. Rain 100H
68
Figure Sampling trajectories of SGM, transi- tionary SGMs: Coef. Dec., OU SDE, and D*GM.
contains 1800 pairs images with and without rain, along with 100 test pairs. As for Rain 100L.,
consists 200 pairs for training and 100 pairs for testing. We present results based on the PSNR, SSIM, and LPIPS metrics.
Quantitative results from the two raining datasets are presented in Tab. 3 Based on both distortion and perceptual metrics, DoGM is capable of generating the most realistic and high fidelity results as shown in Fig. 3b
5.2 Generalizability: DM for real-world data
Case Study 1: Dehazing. We utilize the real-world datasets O-HAZE 2 Dense-Hazz []] which contain 45 and 55 paired images, respectively. We use the last 5 images of each dataset as the testing set and rest as the training set following the common split other methods. Results are shown in Tab. and Fig. 3al Our work improves results on O-HAZE both quantitatively and qualitatively. Smaller improvements are observed on Dense-Haze. This can be attributed to the severe signal corruption of the Dense-Hap Haze data. A combination with tailored task-specifie, Transformer-based

---

methods [47 [22 might lead to further performance gains for such data. Such extensions are beyond the focus of this paper. Qualitatively D'GM achieves excellent visual results (Fig. and Fig. Bay.
Case Study 2: MRI Reconstruction. MRI data is represented in the complex-valued frequency domain, which is distinctly different from the natural image domain. We utilized the fastMRI dataset [69]. containing single-channel, complex-valued MRI samples. Implementation details can be found in Appx. G based state-of-the-art reconstruction methods. Although our method does not have a task-specific design, we still get comparable performance (more details and results are provided in Appx. H). Fig. Iillstrrates our reconstruction results from masked k-space data for 8x and 16x acceleration, i.e., under-sampling sampling for faster data acquisition.
For a robust comparison, we benchmarked against a diverse set of deep learning-
Table 5: Quantitative results for fastMRI dataset with acceleration rates x8 and x 16.
Table 6: Quantitative results for IXI MRI SR on unseen datasets.
Undersampling factor 8x
Undersumpling factor 16x
x
Guys
Method
SSIM
LPIPS]
PSNR
SSIM
LPIPS
Methods
PSNR
SSIM
PSNR
SSIM
ZeruFilling
DSCS [1
SwinMR
DAGAN 03
25.
26.95
22.74
35.99
0.709
0.730
0.719
036678
0.291
0.262
0.504
0.254
24.85
23.87
20.04
23.35
0.667
0.673
0,673
0.624
0.42
0.530
0.317
0.327
SFM [9]
EDSR
PDM 70
23.03
23.28
22.89
0.709
0.711
25.18
27.93
25.10
0.727
0.851
0.731
DillusRRecon
0.738
0.68%
ACT 48
22.80
0.707
26.38
0.826
CDMMMR
27.26
01.74
0.236
25.77
0.707
CST [4
23.20
0.214
28.55
0.837
D'GM
27.92
0.240
0.175
25.26
0.201
0.153
D*GM
25.60
0.863
Case Study 3: MRI Super-resolution (SR). 1x dataset is the largest benchmark considered in our MRI SR evaluation. Clinical MRI T2-weighted (T2w) scans are collected from three hospitals with different imaging protocols: HH, Guys, and IOP. For investigating our cross-domain generalization and robustness, challenging task for both MRI SR and natural image restoration, we trained on HH data with k-space truncation, and tested on Guys and IOP with kernel degradation with an up-scaling factor of X4. More details can be found in Appx.
The methods are tested under unseen data conditions, including different acquisition parameters, MRI scanners (different vendors and -trenggtt)gths) and unseen degradations. With D&GM we are able to demonstrate varying degrees of improvement, as well as generalizability to the discrepancy within the training domain and across the test domain as shown Tab. 6 Qualitative results are shown in Fig. I1 and further results across domains in Appx. H
Discussion and limitations
Other works, like VPB (CD) 733. 12SB K4 are based on diffusion bridges assuming that clean and degraded images are already close. Thus, the tractability of the reverse process heavily relies on the validity of the assumed Dirac delta distribution. IR-SDE (37) employs the mean-revertin SDE theorem based on running the reverse SDE with instability. Since unstable errors accumulate in each step, this model will eventually become unable learn the transformation, e.g. degradation. DPS O and CDDB [10] assume that the degradation process is known, or linear operations are directly used to simulate the degradation process, which limits the generalizabilityy of the method. In contrast, D'GM is built on the theorem of Measure-preserving RDS, which bridges clean and degraded image distributions while taking both degradation and measurements into account. Moreover, D&GM can be extended to a two-sided solution operator (tractability) with flow map according to Prop. [
L Limitations. Even though results are better than others when the degradation process is very severly corrupted (e.g., real dehazing), the overall quality of the restored image is still limited, which consistent with Prop. 2. This might be alleviated via guiding the sampling process with priors and enhanced μ., such as posterior sampling or degradation maps on the data manifold, but such approaches are still limited as shown in Tab. 7 and Appx
Computational Complexity vs. Performance. Tab. highlights that prior work is often tailor-made for a specific subset of tasks and thus also generalisation-liite practice. D*GM'' focus on a generic robust solutions from an RDS perspective can mitigate this, while maintaining en-par performance with task-specific approaches in Tab. g
for challenging environments in
httt://bbainnddeeeloomme..t.too dataset

---

Table 7: Comparison to tailor made approaches.
Table 8: D'GM vs. recent deraining works.
Real Dehazing
Diftu, for Inverse Problems.
Transitionary SGMs.
Deraining
Model Complexity
[00
PSB
D.GM
PSNR
SSIM
Param.
FL.P
PSNR1 SSIM
21.35
21.31
24.52
DRSformer
32.17
0.93
242.
0.488
0.991
0.583
0.691
0.786
D7GM
32.21
0.935
36.5M
104.76
7
Conclusion
The proposed D*GM framework enhances the stability and generalizability of SDE-based diffusion methods for challenging inverse problems. Our approach, grounded in peeasrre-preservin dynamies of random dynamical systems, ensures broad applicability and relevance. We demonstrate D effectiveness across various benchmarks, including challenging tasks like MRI reconstruction.
This work was supported by the JADS programme and UK Research and Innovation [UKRI Centre for Doctoral Training in AI for Healthcare grant number EP/S023283/1J HPC resources were provided by the Erlangen National High Performance Computing Center (NHR @ FAU) of the Friedrich-Alexander Aleeeelleandd---nvesstttttt Erlangen-Nümbbrgg (FAU) under the NHR project b14 433dc and b180de. NHR funding is provided by federal and Bavarian state authorities. NHR @ FAU hardware is partially funded by the German Research Foundation (DFG) 440719688 Support was also received by the ERC projects MIA-NORMAL 101083647 as well as DFG 513220538, 512819079.
Acknowledgements:
References
[1] C. O. Ancuti, Ancuti, M. Sbert, and Timofte. Dense-hazes A benchmark image dehazing with dense-haze haze-free images. In ICIP, pages 1014-1018. IEFE, 2019.
[2] C. O. Ancuti, C. Ancuti, R. Timofte, and C. De Vleeschouwer. O-haze: dehazing benchmark with real hazy and haze-free outdoor images. CVPR workshops, pages 754 762, 2018
[3] B. D. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications. 12(3):313-32,, 1982.
[4] H. Bai, J. Pan, X. Xiang, and J. Tang. Self-guided image dehazing using progressive feature fusion. TIP, 31:1217-1229, 2022.
[5] S. Bell-KIis Kligler,,,,r, A. Shocher, and M. Irani. Blind super-resolution kernel estimation using an internal-gan gan. NewrIPS, 32, 2019.
[6] B. Cai, Xu, Jia, Qing. and D. Tao. Dehazenet: An end-to-end system single image haze removal. IEEE TIP, (5(11):5187-519) 2016.
X. Chen, H. Li, M. Li, and J. Pan. Learning sparse transformer network for effective image deraining. In CVPR, 2023.
[8] X. Chu, L. Chen, and W. Yu. Nafssr: Stereo image super-resolution using nafnet. In CVPR, pages 1239-1248, 2022.
[9] H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, and J. C. Ye. Diffusion posterior sampling for general noisy inverse problems. ICLR, 2023.
[10] Н. Chung. J. Kim, and J. C. Ye. Direct diffusion bridge using data consistency for inverse problems. NeurIPS, 2023.
[11] Chung, D. Ryu, M. McCann, M. Klasky, and C. Solving 3d inverse problems using pre-trained 2d diffusion models. In CVPR, pages 22542-22551, 2023.
[12] Chung. Sim, and
Come-closer-difiuee faster: Accelerating conditional diffusion models for 2022.
inverse problems through stochastic contraction. In CVPR, pages
[13] H. Crauel, A. Debussche, and F. Flandoli. Random attractors. Journal of Dynamics and Differential Equations, 9:307-341, 1997.
[14] M. Z. Darestani, J. Liu, and R. Heckel. Test-time training close the natural distribution shift performance gap in deep learning based compressed sensing. /CMI, pages 4754-4776. PMLR, 2022.
[15] V. De Bortoli, J. Thornton, J. Heng, and A. Doucet. Diffusion schrodinger bridge with applications to score- score-based generative modeling. NewrIPS, 34:17695- 2021.
10

---

[16] K. Dehnad. Density estimation for statistics and data analysis. 1987.
[17] M. Delbracio and P. Milanfar. Inversion by direct iteration: An alternative to denoising diffusion for image restoration. TMLR, 2023.
[18] H. Dong, Pan, Xiang, Z. Hu, X. Zhang. F. Wang, and M.-H. Yang. Multi-scale boosted dehazing network with dense feature fusion. In CVPR, pages 2157-20 2167, 2020.
[19] M. EI Helou, R. Zhou, and S. Süsstrunk. Stochastic frequency masking to improve super-resolution and denoising networks. In ECCV, pages 749-766. Springer, 2020.
[20] G. Franzese, S. Rossi. L. Yang. A. Finamore, D. Rossi. M. Filippone, and P. Michiardi. How much is enough? study on diffusion times score-based generative models. Entropy, 25(4))633, 2023.
[21] U. Grenander and M. I. Miller. Representations of knowledge in complex systems. Journal of the Royal Statistical Society: Series B (Methodological), 56(4))549-581, 1994.
[22] C.-L. Guo, Q. Yan, S. Anwar, R. Cong. W. Ren, and C. Li. Image dehazing transformer with transmission- aware 3d position embedding. In CVPR, pages 5812 5820, 2022.
[23] K. He, J. Sun, and X. Tang. Single image haze removal using dark channel prior. IEEE TPAMI, 33(1)233312 2353, 2010.
[24] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33:6840 6851, 2020.
[25] A. Hore and D. Ziou. Image quality metrics: Psnr vs. ssim. In ICPR, pages 2366-2369. IEEE, 2010.
[26] J. Huang, A. I. Aviles-Rivero, C.-B. Schönlieb, and G. Yang. Cdiffiuu: Can we replace the gaussian noise with k-space undersampling for fast mri? In MICCAI, pages 3-12. Springer, 2023.
[27] J. Huang, Y. Fang, Y. Wu, H. Wu, Z. Gao, Y. J. Del Ser, J. Xia, and G. Yang. Swin transformer for fast mri. Neurocomputing. 493:281-308 2022.
[28] A. Hyvärinen and P. Dayan. Estimation of normaazza statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
[29] A. Dolicoeur-Martineau generating data with score-based models. arXiv preprint urXiv:210S.1400S 2021.
R. Piché-Taillefer T. Kachman, and I. Mitliagkas. Gotta go fast when
[30] A. Jalicoeur-Martnccau, R. Piché-Taillefer, and improved sampling for image generation. arXiv preprint arXiv:2009005755 2020.
Combes, and I. Mitliagkas. Adversarial score matching
[31] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion- based generative models. arviv preprint arXiv:2206. 00364, 2022.
[32] B. Laurent and P. Massart. Adaptive estimation of quadratic functional by model selection. Annals of statistics, pages 1302-1338, 2000.
[33] B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee. Enhanced deep residual networks for single image super-resolution. In CVPR workshops, pages 136-144, 2017.
[34] G.-H. Liu, A. Vahdat, D.-A. Huang, E. A. Theadorou, W. Nie, and A. Anandkumar. 12sh: Image-to-image schrbackslash odinger bridge. ICML, 2023.
[35] H.-T. D. Liu, F. Williams, A. Jacobson, S. Fidler, and O. Litany. Learning smooth neural functions via lipschitz regularization. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1-13, 2022.
[36] X. Liu, Y. Ma. Z. Shi, J. Chen. Griddehazenet: Attention-based multi- network for image dehazing. In ICCV, pages 7314-7323, 2019.
[37] Z. Luo, K. Gustafsson, Z. Zhao, Sjölund, and T. B. Schön. Image restoration with mean-reverting stochastic differential equations. JCMI., 2023.
[38] K. Mei, A. Jiang, Li, J. Ye, and M. Wang. An effective single-image super-resolution model using networks. In ICONIP, pages 542 553. Springer, 2018.
squeeze-and-exeitee
[39] C. Meng, Y. He, Y. Song, J. Song, J. Wu, J.-Y. Zhu, and S. Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. ICLR, 2022.
[40] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. ICLR, 2018

---

[41] J. C. Nguyen, A. A. De Smet, B. K. Graf, and H. G. Rosas. Mr imaging- based diagnosis and classification of meniscal tears. Radiographics, (A(4):91-9999, 2014.
[42] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In ICML, pages 8162-8171. PMLR, 2021.
[43] G. Parisi. Correlation functions and computer simulations. Nuclear Physics B, 180(3):378- 384, 1981.
[44] C. Peng. P. Guo, S. K. Zhou, V. M. Patel, and R. Chellappa. Towards performant and reliable undersampled mr reconstruction via diffusion model sampling. In MICCAI, pages 923-633. Springer, 2022.
[45] H. Poincaré. Sur le problème des trois corps et les équations de la dynamique. Acta mathematica, 13()):A3-2270 1890.
[46] X. Qin, Z. Wang, Y. Bai, X. Xic, and H. Jia, Ffa-net: Feature fusion attention network for single image dehazing. In AAAI, 2020.
[47] Y. Qiu, K. Zhang, C. Wang, W. Luo, H. Li, and Jin. Mb-taylorformer: Multi-branch efficient transformer expanded by taylor formula for image dehazing. In CVPR, pages 12802-12813, 2023.
[48] S. Rad, Yu, Bozorgtabar, and J.-P. Thiran. Test-time adaptation for super-resolution: You only need to overfit on a few more images. In ICCV, pages 1845-1854, 2021.
[49] D. Ren, W. Zuo, Q. Hu, Zhu, and D. Meng. Progressive image deraining networks: A better and simpler baseline. In CVPR, pages 3937-3946, 2019.
[50] Ren, L. Ma. J. Zhang, Pan, X. Cao. W. Liu, and M.-H. Yang. Gated fusion network for single image 2018.
dehazing. In CVPR, pages
[51] Schlemper, Caballero, J. V. Hajnal, A. Price, and Rueckert. deep cascade of convolutional neural networks for mr image reconstruction. In IPMI, pages 647-658. Springer, 2017.
[52] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamies. In ICML, pages 2265. PMLR, 2015.
[53] J. Song, C. Meng, and Ermon. Denoising diffusion implicit models. In ICL.R.
[54] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. NeurlPS, 32, 2019.
[55] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2020.
[56] Z. Tu, H. Talebi, H. Zhang, F. Yang. P. Milanfar, A. Bovik, and Y. Li. Maxim: Multi-axis mlp for image processing. In CVPR, pages 5769-5777777 5780, 2022.
[57] P. Vincent. A connection between score matching and denoising autoencoders. Neural computation, 13(7);1661-1674, 2011.
[58] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEFF TIP, 13(4):600612, 2004.
[59] S. Welker, J. Richter, and T. Gerkmann. Speech enhancement with score-based generative models in the complex stft domain. arXiv preprint arXiv:2203. 10022003.77000 /7004, 2022.
[60] H. Wu, Y. Qu, S. Lin, J. Zhou, R. Qiao, Z. Zhang, Y. Xic, and L. Ma. Contrastive lcarning for compact single image dehazing. In CVPR, pages 10551-10560, 2021.
[61] 7. Xiao, K. Kreis, and A. Vahdat. Tackling the generative learning trilemma with denoising diffusion gans. In ICLR, 2021.
[62] G. Yang. S. Yu, H. Dong, G. Slabaugh, P. L. Dragotti, X. Ye, F. Liu, S. Arridge, J. Keegan, Y. Guo, et al. Dagan: Deep de-aliasing generative adversarial networks for fast compressed sensing mri reconstruction. TMI, (71:111 2017.
[63] L. Yang. Z. Zhang, Y. Song, S. Hong. R. Xu, Y. Zhao. Y. Shao, W. Zhang, B. Cui, and M.-H. Yang. Diffusion models: A comprchensive survey of methods and applications. ACM Computing Surveys, 2022.
[64] W. Yang, R. T. Tan, J. Feng, Z. Guo, S. Yan, and J. Liu. Joint rain detection and removal from a single image with contextualized deep networks. IEEE TPAMI, 42(6): 2019.
12

---

[65] W. Yang. R. T. Tan, J. Feng. J. Liu, Z. Guo, and S. Yan. Deep joint rain detection and removal from single image. In CVPR. pages 1357-1366, 2017.
[66] Z. Yue, J. Wang, and C. C. Loy. Resshift: Efficient diffusion model for image super-resolution by residual shifting, NeurIPS, 2023.
[67] S. W. Zamir, A. Arora, S. Khan, M. Hayat, S. Khan, and M.-H. Yang. Restormer: Efficient transformer for high-resolution image restoration. CVPR, pages 5728-5739, 2022.
[68] S. W. Zamir, A. Arora, S. Khan, M. Hayat, S. Khan, M.-H. Yang, and I., Shao. Multi-stage progressive image restoration. In CVPR, pages 14821-14831, 2021.
[69] J. Zbontar, F. Knoll, A. Sriram, T. Murrell, Z. Huang, M. J. Muckley, A. Defazio, R. Stern, P. John- son, M. Bruno, al. fastmri: An open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.0880 2018.
[70] K. Zhang, J. Liang. L. Van Gool, and R. Timofte. Designing practical degradation model for deep blind image super-resolution. CVPR, pages 4791-4800, 2021.
[71] Q. Zhang and Y. Chen. Fast sampling of diffusion models with exponential integrator. ICI.R, 2022.
[72] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unrcasonable effectiveness of deep features perceptual metric. In CVPR, pages 586-595, 2018
[73] L. Zhou, A. Lou, S. Khanna, and S. Ermon. Denoising diffusion bridge models. arXiv preprint arXiv:2309. 16948, 2023.
13

---

Appendix
A
Broader Impacts
Our proposed method offers significant advancements in the restoration of degraded images with minimal risk of hallucinations due to stability guarantees and with applications including MRI reconstruction and super-resolution. In the clinical domain, the adoption of our method for MRI reconstruction must adhere to stringent regulatory and approval processes. The results generated by model should serve auxiliary tool to assist healthcare professionals in their diagnostic and treatment decisions, rather than as a standalone diagnostic tool.
B Intuition of Measure-Preserving Dynamics in SDE:
Consider the analogy stretched rubber band, which naturally seeks to return to its original but does with lot of oscillations when released. This clastic behavior parallels the dynamics of the OU process, where deviations from a mean state are counteracted by a restorative force, guiding the system back towards equilibrium (i.e., final state), with random perturbations.
Our process models the noise as stochastic component that fluctuates around stationary process and improve the OU process with RDS. Measure-preservieee preserving dynamics ensure that while the image undergoes transformations during the denoising process, the overall statistical properties remain consistent (i.e., invariant image features), which cannot be satisfied by vanilla OU processes or previous approaches (Tab. 1).
C Mathematical Preliminaries
Consider a probability space (2, P) accompanied by a standard Brownian motion W,. A stochastic process over the interval 0 s & < I' can be formulated by the following stochastic differential equation (SDE):
dæ. b(t, dtt + at,, xz ddW
Definition Filtration collection of sigma- fields,
: S1555)
termed a filtration if:
1. Fi C. F is a sub-o for every t [0,
2. 1f0 S < S T, then Ft, C Ftx
Here, F. represents the information set at time t.
Definition Strong Solution process x, which F-progressivell measurable, considered strong solution to the SDE given by Eq. 7if:
( < o
almost surely. This captured by:
2t = Do xo + ( b(s, J. (ss,x,)dW., Vt e [0
(10)
Definition 3 Lipschitz Continuity For an N-dimensional stochastic process over t e [0,
adapted to the filtration F, functio wxxx) exhibits Lipschitz continuity in æ if:
1. w(xe) is F-measurable with the requisite dimensions.
14

---

2. There exists non-negative constant K such that, for every 24 and Ve
(w( w(x))l S -
In numerous diffusion models, Lipschitz continuity is inherent, ensuring the existence and uniqueness of the solution the stochastic process. However, for clarity in network design, the emphasis Lipschitz continuity ensures the foundation of neural networks remains consistent.
Definition 4 Random Dynamical System A random dynamical system(RDS) consists of a base flow, the "noise", and cocycle dynamical system on the "physical" phase space, we first discuss one fundamental element of our RDS, the base flow.
Let (.P) be a probability space, the noise space. Define the base flow 0 R x Ω S2 as follows: for each "time" E let 0, Q + Q2 be a measure-pree peresurv-r measurable function: P(F) = TP (F)) for all E. E F and TR
Suppose also that
1. 9. = idg= 52 - Ω2 the identity function on Ω for all 5, t R, 0, 0, =0
That is, 8,,3 E R. forms group of meeasure-preservinn transformation of the noise (2, Now we are ready to define the random dynamical system(RDS).
let (X, d) be complete separable metric space, the phase space. Let i : Rxxxxxx be
(B(RR) & B(X), (())) measurable function such that 1. for all E Ω2 the identity function on X; 2. for (almost) all 92 (t, (p(t,,,)) is continuous; 3. satisfies the (crude) cocycle property: for almost all w E S2
+p (t, (w(W)) (((,ww) (tt s,w)
the case of random dynamical systems driven by a Wiener process R X - X, the base flow 0. 5 > Ω would be given by
9. (L)).(w)) W W(L+ -W(.ss)
Theorem Existence and Uniqueness If the initial condition zo E T.2 is random variable that's independent of W and both μ(00, mo) and o00, vo) E H2, then, provided there exists a constant K that satisfies:
lb(t, - b(t, + (o(t, (o((,)) (t, b(t,))--(t))))
(11)
- <K yl. Vt [0, E R
(an attribute also recognized as Lipschitz continuity), unique strong solution to Eq. 1 exists in IH?2
for every T >> Additionally:
E
sup UST
xel
S C(1 1(10 1)
(12)
holds true, where the constant C depends on both T and K.
D Preliminaries and Proof for Proposition 1
D.1 Proof for reverse-time GC process:
There is one-to-one and onto correspondence between the stochastic differential equation and the
Kolmogorov equation for p (z,, x., , s. which describes the evolution of the underlying probability distribution. Consequently, there should be one-to-one and onto correspondence between
a reverse-time equation for 7, and a Kolmogorov equation for p(r,, lax, 8), 8 >
dx; 0, xt)dt rodW,
We have the corresponding Kolmogorov backward equation given by
Öp (z., x,, =0c(kr ( x1) Op (X., s De.t)
Øt
Oi.
(13)
p2p (I,, Ft, 1)
2 o?
Ox
15

---

The unconditioned Kolmogorov forward equation is given by
Op (I,,8 8 2z,
d (0r(u Iz) (
I2, [2)
01
Oxe
(14)
1 03 (To0] (x Ie. xx,t))
2
Ox7
See 3 for more details on Kolmogorov equations.
Bayes rule gives
p (Te, t, Fs, 8) p (Ir, Fe, 1) (x+, t)
We plug this result into
13 which gives us the Kolmogorov equation
i ot
0
p (It, (1i, Ts.
[ (It, (s
dr
(15)
1 02 [p (J2,, s,, 7
+
2
Ou?
and the expression for f is given by
f (xr,t) = 0e( 10--1) rz))
[p (2,, 7)
p p t) ar,
(16)
=0(( - x4)) - 7² o² log dx,
[p(x,: t)l
Therefore, have that the reverse process corresponds the Kolmogorov equation I6 given by dx: 0c(μ x.) - -2 - log V.p. (xt) + 7 ?o?dW.
Definition Different from deterministic dynamical systems, random dynamical systems usually consider a pullback attractor rather than forward attractor due to the non-autonomousness introduced by the random noise. The pullback attractor (or random global attractor) A(w) for the RDS yp we defined in is a P-almost surely unique random set such that:
A(w)) is a random compact set: A(w) C X almost surely compact and + d(x, A(w))
(F, (BX)) ((X))))mmsurabll function for every X
A(w) is invariant: for all ((t,w)(A(w))
A (D,w) almost surely;
3. Alw) is attractive:
for any deterministic bounded
set
B C
X.
lim;-+o (1p
(B), A(w)) = almost surely.
(XX) denotes the Borel a algebrra generated by the space X where the RDS is defined.
Definition 6 Poincare Recurrence Theorem
Let
EX,
be a finite measure space and let
be a geassrr-prresevvnng transformation. then we have that for any E E E, the set of those points x of F. for which there exists N M such that (x) & E. for all n. N has zero measure. In other words, almost every point of E returns to E. In fact, almost every point returns infinitely often; i.e. ps (fx E there exists N such that (x) & E for all > N\) = O.
D.2 Analysis of Proposition
Proposition After extending the solution of the OU process to RDS, the measure-preserring flow map of the solution should meet the property gtt, s; w)x yo (t 8, O; 0,w) w..... x. However, OU processes with time-narying coefficients are usually not satisfied for this property(can be referred to as time-homogeneity) and thus the stability of the system breaks.
The forward process in SDE notation
dx, = 0c(u xz dtt + o,dWe
16

---

and the solution to the above SDE is given by
u + of oze oze Bei dW,
It = + (2o - uie Bon
where 0z:t - Sn Ozdz.
This solution above can be represented by a continuous random dynamical system (RDS) y defined on complete separable metric space (X, d), where the noise is chosen from a probability space (2, More details can be found in [13].
More generally, can extend the RDS to two-sided, infinite time, define flow map (solution operator) yo :
x TRd + Rd by (t,, 8, (w,rn) == x (t, skw,ro) with w E f2 -00 < & & t co.
The base flow driven by Brownian motion can be explicitly written W (t, (w() [t++s,w)
W(s,w) w).
Now suppose that:
1. The flow map ú,, 6 E IR is a measure-preserving transformations of (2,F, P), with the property that for all L and x X.
=
P-a.s.
(17)
2. (i) ptt,, (p w;w) = (tt, w) for all 8 < & and x E X; (ii) p(t, st w) is continuous in X, for all & t. (iii) for all < and r E X, the mapping
(tt,
measurable from F) to ((X): and (iv) for all f. E X, and P-a.e. w, the mapping s + (t,, s;w) is right continuous at any point. Where B(X) denotes the r-algebra generated by X.
Under assumptions (i), (ii), (iii), (iv) and suppose that for P-a.e. w there exists a compact attracting set K(u (w) time 0, Lee, such that for all bounded sets C X,
d(p((0, s; B, K(w))
as
We can see that the attractor of this system is defined in the pullback sense, such that time is rewind backward before iterating forward. Moreover, the reverse process with any starting time t to is defined as the RDS going backward in time
(ss, (,w, mx)
start from the time t realization and run backwards to s.
the above proposition can be extended to show that there exists a compact attracting set any -co < t < co. and this convention has allowed us to characterize the attractor K K(w) NNu,, X²). when the time becomes finite, for example from 0 to T, the random attractors can be abstractly e -u:e))
viewed as N [ (I.. -
Ba:s X(1
Moreover, an important assumption in the I7 is usually not satisfied for OU processes with time- of X2, where is constant, and varying coefficients, therefore, we impose that for every I, 28, will be the asymptotic variance of the forward process. This convention has allowed us reduce the regularization on two variables ot, Og to just one variable to satisfy [7] and this convention has allowed to characterize the attractor ((w) NN, X2), when the time becomes finite, for example from 0 to T, the random attractors can be abstractly viewed as the Gaussian measure
N p + (x..
μ)ee Bors X(1 e efu:.)
E Proof for Proposition
Proposition Given Fq. and Eq. and assume that the score function is bounded by C' in T,2 norm, then the discrepancy between the reference and the retrieved data is, with probability at least (1 8):
17

---

|00
2 ((20 112 /20) ) 20r + o o /0
(18)
oman (Comma maz + d -2 log 6 2 log 6)
where zo to are the quality reference and sampling data. For a noisy inverse problem scenario, the retrieved data with any finite T always indicates difference depends on dz, M4, X, T, K, where K is the Lipschitz constant for the reverse process.
we have that the absolute value between the theoretical expectation and actual expectation after T period given by
| ((tr))l (1
wa)e 0T
Similarly, the difference between thcoretical variance and T-period variance also has strictly positive difference. where 0: - _ 0,ds.
Therefore, with finite 7", the final state of the forward process can only reach a êr rather than the theoretical stationary distribution, which we denote by Toos we denote the retrieved image after T-periods from theoretical stationary distribution by âo. zo by the ground truth HQ image, 27 the true distribution after iteration,
|| - foun(Eq, to. || I (fo) I co l1
(19)
2 ||| N 12 40 - xo 12
Inside the norm, the first term bounded below, since both dt and t both follow normal distribution and are independent of each other, the difference between those two random variables, we denote by 27, that follows a normal distribution
N ( (Iu 1)e 6. - -1,, 6 c c-20c) x2)
(20)
=N (1r (1 (w)e m)e- Be m))- - e- e-20l))
Therefore, we could rewrite the equality as:
l fou(=q, μ1; to)ll
2
do?
1T
V.
log Pr (x) dt + dw,
dl
1
(21)
do?
(xo 11²ee"
20,
+ 22 2 -20) - Co
dw
2
a²
Since we require == we can find Omar such that < Omaz for all t. The last term only concerns the random noise, according to [32], we have that the last term equivalent to the squared L2 norm of random variable from Wiener process time = 0, with marginal distribution being e e N (0, I). The squared L2 norm of e divided by o². is a x² dd-striibuuiion with d-degrees of freedom, we have the following sided tail bound, according to B2
29,
Pr (1e11//0² (to) > d 2 log 2og 6) I exp(log 8)
Therefore, with probability 1 5, the HQ image and retrieved image has a lower bound of, is the number dimension for 2o).
((ro 11)2 11)=-2)) -20, + 2X²
2 Omax (Co (Coma + d 2 log - 2 log d)
SDE modeling that has suffered complex degradation is considered 'corrupted':
18

---

Example 1. Based on the Prop. The conditional SDE diffusion composed of three general stages, forward, backward, and sampling. Both time-reversal processes have been declared unstable under the degradation process 1a according the Prop. 2.
The following advantages would be offered by our measure-preserving dynamical system when formulated as SDEs:
Intuition 1. A two-sided measure- preserving random MMPamical(MP-RDS) system formulation enables us to use the Poincare recurrence theorem, intuitively, with a two-sided MP-RDS Pr. the Poincare recurrence theorem ensures that the system ff, starts from terminal condition æ7, run backwards in time, will hit a region (zo - e, zi + E) for small in finite time, where zpp is the high-quality image.
measure-preserring
F Temporal Distribution Discrepancy during Sampling
Theorem 2 Suppose that both the drift (x) and diffussion 0g term of stochastic process It is Lipschitz continuous with some constant K, moreover, x E L (F. IR) is a solution to the SDE
dx = b(t, x) + x),
and initial condition bo. ou then we have that
E DT zol?
S CI..
(22)
such that
16 = E [1,"
+ S / lool
|bol dl
and C depends only on T, K, which is the rumning time and Lipschitz constant
Firstly, we have the following relationship:
OT < [oo + ('' [b(t,z)] dt + sup osesT
(23)
Fr xol S (" [b(t,z)) dt
I. o(s,,)
sup USIST
Squaring both sides, taking expectations, and applying the Serkholder-Daiise DavisyGundy inequality, we get:
E |PT
xol?
J [b(1, [0(1,20)1 del
< CR
sup OSIST
(J." [lbol + lxell dt d
f-" T
(24)
< CE
+
(0((, di
( [bo dt + 1'' [ooo + | dt
& CE
Remark: should be noted that the constant C, which depends on and K, varies from line to line.
Next, we show that for any
0, there exists a constant C. > 0 such that:
19

---

SE | ? + Cclg.
sup E 051ST
(25)
Applying Itö's formula, get:
Jxel (x,b(t, xx) + (ott dt + 2x, xz) dB..
(26)
Considering the martingale property of the third term, integrating, and taking expectation on both sides, we have:
//" 20, (ss) ds
2
x +
[ 12 + //¹ [C] + 2z.001 [bol + Clool
ds
(27)
/
FT /
T
S C
E
|r.. ds + 2B
lbol ds + CT2.
Using Gronwall's inequality, we can prove 25 with the result above. Substituting (5 into 24 completes the proof, showing that the distance between 2T and zo in the T,2-sense is bounded by C.,
With theorem 1 in hand, since the OU process is Lipschitz continuous, then the reverse process for the OU process is also Lipschitz continuous.
Now, suppose that the Lipschitz constant for the reverse process is given by K. then we have that in L norm, the distance between any final state øT E N N(,, A) and the initial sta image) is bounded by constant that only depends time T and Lipschitz constant K, and the initial condition for the drift and diffusion term, where we denote as C(K, pt, A), which will be written as C(K, T) for short.
Now, since the time is finite, where theoretically only when T - oo would &r converge to the theoretical stationary distribution, thus, if we denote the sample from 7' time steps by ir, and øT by the sample from the theoretical stationary distribution, then we have E[iT ēT) ] > e(T, K, μr, X), note that the K here is the Lipschitz constant for the forward process, and this distance strictly decrease in T.
Suppose that in inference, when the ground truth æg is unknown, the distance between ground truth zo and sample from the theoretical distribution is bounded from below by
1 Joo || > C(K, +
(28)
Then, we can see that the gap between 2n and रर increases such bound, which is
11:00 - ăT N 120 - Ioo (ØT xoo)
> C(K, T + CT. H,
where 2' is the time step for the forward process, and e is some strictly positive constant. Now suppose that 24 is the solution to the reverse process, which runs for T periods in total, then we have that for E [o, T), denote Fo, fo as the original HQ image and the final state of the reverse process, respectively.
x Quality OU (I Quality pi; to 8) 112
|20 doll |20 ेT (30 2T)
(29)
|||.0 - ir || |1
=x1)lll >
C(K,T)) C(K,T)),
Therefore, the bias created in inference depends on the IQQ image μ, stationary variance, Lipschitz constant, and the time steps T.
20

---

G Stable in Probability
Definition Stable in Probability
Given probability space (Ω, F, P) and standard Brownian motion W., general form of SDE
for a stochastic process Tt, 0 S - is given by
dx = b(t, dt + (t,
(30)
Such that the Lipschitz condition is satisfied, for both b(t, o x)) A solution x(1, w) 0 is said to be stable in probability for 2 0 if for any s 2 0 and & > 0
lim P
sup La x (1) 1>8
8 O.
(31)
2o #0-+
says that the sample path of the process issuing from a point x time will always remain within any prescribed neighbourhood of the origin with probability tending to one as - 0. In practice, this property ensures that the perturbation from the initial state caused by stable process bounded for all i with probability one.
OU addmess unique unconditional stationary solution provided by theorem 19. however, in this example, without specifying the value that determines the stationary variance, i.e., G, If for large a and sample à from the stationary distribution of the OU process, we have that
For example,
P( / 20
(32)
This means that sample from the stationary distribution the forward process could deviate largely from jr, thus making the result no different from the traditional VE(variance exploding) diffusion models that are defined
dr, oidW;
because the variance could be set arbitrarily large no restriction is specified. Therefore, how should such problem be approached most easily? The Lyapunov theorem for stability has provided easy way, without explicitly solving the SDEs, to ensure stability just from the coefficients. Then provide the main theorem that ensures the stability of SDE, first, give the definition of positive definite in the Lyapunov sense
Definition Let denote the family of all continuous nondecreasing functions μ. R4 R4succ that μ(0) = 0 and ju()) > O if > 0. For h > 0, let S,, = {x E R" | < h]. A continuous function [ar, 1) defined on Sr. x (to, oo) is said be positive-definite (in the sense of Lyapunov) if
() = 0 and, for some μ E K,
V(x,t) 2 m((=)
for all tx,t)= E Sh [to,
Then we will use the convention of the Lyapunov quadratic function, such that
Definition 9 Lyapunov quadratic function V given
V ( - x Qz
where Q is symmetric positive-definite matrix.
Theorem The function LV
LV (x.)
+ +b(,,z,) Qu+
(33)
a 2t, Qo (t, xt),
negative-defiitee in some neighhourhood of x. 0 for 2 lo, with respect to system Then the trivial solution of equation is stochastically asymptotically stable.
21

---

Since this theorem is important and the proof will be intuitive in explaining why such condition could ensure stability, the proof will be put here.
Proof:
First, we compute dV(x). which the instantaneous growth of the Lyapunov quadratic function, gives
dV (xx) =V (or + dz,) V (xr))
[aa + dx() Q (ox + dr²) - x Qb (1, xx) dt + xt Qo (1, dB.t (1, diQu, (1, dB+ o (1, Qo (t, dt
xx Qu
Then, take expectation, we can get
E (dV (mi) = x²'Qu (1, di x1, Qu,di+ a (1. Qa (1, LV (xx) dl
Then, if we assume that (x)
-V (x)) kV (xr
such that k is a constant, then
E (V (xr) -VE (2+)},
(34)
E(V (xt)) I exp(-kt).
As can be seen from the proof, the operator LV as the function of the SDE x4 is the expectation of the dV (x:), and the negative semi-definiteness can be regarded as requiring (2:) be contraction. This can be understood from 34 such that
dt B(V(a)))E(((x.)) < k
for & > 0
H Implementation details
Model Implementation: Our exploration into mitigating Temporal Distribution Discrepancy in diffusion models employs two neural network architectures, each catering to different dataset com- plexities.
We utilize the adopted UNet, a staple in DDPM 24 and DDIM frameworks, chosen for its widespread use and strong benchmarking capabilities. By solving the discrepancy through this established structure, we achieve faae--of-the-ar results on synthetic data, showcasing the potential of improving transanary SDE diffusion models in terms of Temporal Distribution Discrepancy and stationary process. Additionally, the use of this prevalent architecture allows for comprehensive analysis and discussion.
Real-world data with its inherent complexity, such as combined degradations, large resolutions, and extensive interdependencies, requires an architecture beyond the conventional UNet. Our improved model incorporates Squceze-and-Exee -xn-ttxitttiion (38) and NAF [ explicitly designed to capture intricate feature interrelations. While these models do not seek to innovate the architectural paradigm, it provides a solid baseline that provides better feature extraction ability than UNet performance in demanding scenarios.
where the improved model incorporates Squeeze-and Excitation 38 to replace the Attention module within NAF g EDSR (33) is employed as the base model
Additional details:
The U-Net we adopted is similar to DDPM described in [37 Bquuezeeand-Exxiiation
TTA-based comparison methods
in MRI super-resolution. In different downstream tasks, we follow the common setting of the
latest compared methods: Deraining (37], Real Dehazing [47], MRI Reconstruction (26), and MRI
super-resolution [[4 and (33] Below more specific details about MRI reconstruction and MRI
22

---

Table 9: Datasets parameters and split setting. IOP details of the sean parameters are not available.
Ditineess
Dait setting
Collection
Seyuence purnmelers
Subjests
Slies
Hospitals
Scanne
Reperition time
Fcho length
Marrix
Recelver call
Source
HH (IXI Brain)
184
60
Hammersmith
Philips 3T
5725
16
192 x 187
Single
Target domain
Gays (IXI Brain]
Guy's Hospital Institute of Psychlany
Philips 1.5T
8178
16 Unknown
Unnormalized
Single Single
IOP (IXI Brain)
30
60
GF 1.5T
Method
PSNR
SSIM
LPIPS
WD UNet
27.31
0.8322
0.227
SN UNet
28.63
0.8687
0.144
UNet
31.03
0,9001
0.058
Table 10: Evaluation of the Lipschitz continuity.
superresolution. For most of the experiments, the training patch-size is set to 128x128 with a batch size of 16. We utilize the Adam optimizer with B3 = 0.9, B2 0.999, learning rate of 10-1 with a decay strategy. Our models are trained on three RTX 6000 GPUs for about four days, each with 40GB of memory. Random seed is 42. All mathematical variants follow the cosine schedule per [42]. The variance À is set to 10 for the OU and stationary processes. the coef. decoupled SDE, a is kept decoupled and does not vary with 0. We observed that the adaptability of T to tasks is contingent upon the task's corruption for model stability and generalizability.
T is hyperparameter as one of the ways to introduce Measure-pressee preserving Dynamics to shape more stable SDE diffusion. It is informed by the deviation between the degraded image and the expected high-quality image. For tasks with moderate intra-domain deviations, T is set to 2, while for tasks with substantial cross-domain and degradation discrepancies, a larger is used. This metric is based on RDS settings of different degradations, rather than learned. A MLP can be used to learn as adaptive metric, although it not comparable with obvious improvements.
SGM, and Transitionary SGMs vs. D We perform qualitative 2 and quantitative |1 analyses using variants of closely related formulations for Prop. I andd 2 evaluate across (A) SGMs and (B) transitionary SGMs. (A) uses common score-based SDE, (B) uses Coefficient Decoupled SDE (e.g., variance exploding SDE with the drift term p)) according to Prop. and OU SDE, alongside our D'GM.
We applied 584 proton-density weighted knee MRI scans without fat suppression. These were subsequently partitioned into a training set (420 scans), a validation set (64 scans), and testing set (100 scans). For each scan, extracted 20 coronal 2D single- channel complex-valued slices, predominantly from the central region with the uniform size of 320 x 320. Our test differs from the 200 reported in (26), possibly as result of the change in the official versions of FastMRI. We subjected all experiments to Cartesian under-sampling masks with undersampling factor 8x and 16x. In the undersampled MRI scans, the acquisition process entails sampling a fractional subset of the Fourier-space (k (k-space), typically governed by mask along the dimension of undersampling rate. Undersampling inherently induces aliasing artifacts in the resultant images. Considering the domain deviation, we did not supplement the extra test set into the final 100 samples. Otherwise, we stayed consistent with the details of the paper. For a robust comparison, we benchmarked against a diverse set of deep learning-bast based state-of of-the-art reconstruction methods, including CNN-based approaches such as D5C5 [51 and DAGAN 62. the Transformer-6 based SwinMR 27 diffusion model-insping DiffuseRecon 44), and the CDifMMR 266.. Quantitative
MRI Reconstruction:
Table 11: Exemplary deraining results of (A) SGM, (B) transitionary SGMs: Coefficent Decoupled SDE, OU SDE, and D*GM.
Method
PSNR
SSIM
LPIPS.
SGM
27.27
0.840
0.144
Coef. Dec. SDE
26.18
0.826
0.205
OU SDE
30.58
0,900
0.051
D*GM (ours)
32.41
0.912
0.040
23

---

results in Tab. exxhibit a differing trend from the image domain. The task task-specific diffusion models achieve better results and are more capable of capturing the complex degradation that occurs in the frequency domain.
Generally, MRI uses mask in the phase-encoding direction (the shortest anatomical direction -1 to model the complex degradation caused by undersampling. In the knee data, based upon the uncertainty of clinical diagnosis (longitudinal artifacts can significantly confuse the diagnosis of meniscal injury), we also masked the frequency direction, which will cause resolution reduction, deterioration of image features, and longitudinal artifacts, more qualitative results can be found in the next section.
MRI Super-resolution: [x[contains contains clinical T1- and T2- T2-weighted scans from three hospitals with different imaging protocols: HH, Guys, and IOP. We selected 184 HH T2 subjects as the source- domain data [train/val/test ratio: 7:1:2], and 30 subjects each from Guys and IOP as two target-domain datasets without degradation, acquisition parameters, datasets, and patient-wise crossovers. The central 60 slices were selected.
We consider two benchmark ideas: (I) Blind SR degradation methods in frequency SFM 19 and spatial PDM [70] domain. (2) Source-free SR adaptation ACT [4] using external priors and test- time adaptation proposed initially for MRI CST [14] with data consistency in the source-domain. We concerned that existing work robustness and generalization focuses medical image segmentation and natural images, and rarely on super-resolution and reconstruction of medical images. Employing the available evaluation criteria in this inevitable problem in practice makes it difficult to reflect the performance of DMGM. Therefore, domain-aware datasets isolation standard for public datasets is designed for the SRR adaptation cross- moss-domain data rather than same-domain in source-free manner. The publicly available data were split into several subsets based on hospitals, scanner, acquisition parameters, modality, and anatomy illustrated in Table Explicit reference standards implicitly correspond to various degradation patterns, thus enabling the isolation of natural degradation patterns in source training domain and target-testing domain. In addition this, different artificial degradation patterns for the subsets in the training and testing domains are employed: K-space truncation downsampling was applied to obtain LR data in the source domain and a kernel degradation [5] was applied in target domain.
We reproduce two types of benchmark ideas on the top of EDSR 33 backbone to achieve the multi- purpose goal: (1) Repurposed blind SR (BSR) for cross-domain data: We utilized BSR degradation methods in frequency SFM (19] and spatial PDM (70) domain. (2) Test-time adaptation (TTA): We compare to a source-free TTA method ACT [4] using external priors and a second TTA method proposed initially for MR reconstruction CST 14 with cycle consistency at source-domain. The settings and adaptation strategies of the comparison methods were used directly.
Lipschitz Continuity for Stationary Process:
We hypothesize that ensuring Lipschitz continuity of the neural network pivotal for the convergence and stability of the diffusion process, particularly in the context of achieving a stationary process. Theoretically, Lipschitz continuity offers two key benefits: (1) it mitigates the impact of small perturbations in input data or model parameters, thus safeguarding against excessive output variability which could cause numerical instabilities or result in an ill-posed problem, and (2) it guarantees the existence of unique solution to the diffusion process, underpinning the reliability and convergence of the numerical methods employed to solve these equations.
To instantiate these theoretical benefits within our architecture, we integrated spectral normalization (SN) [40] and weight decay (WD) [35] into a U-Net structured score network. In the case of SN, is achieved by rescaling each layer's weight matrix W by its spectral norm, F (W7)), to obtain a normalized weight ()(()
w
maz (WJJ- By doing so, we intend to control the overall Lipschitz constant of the network for robust score matching within our diffusion model framework. A quantitive result can be seen in Tab. 10
24

---

A
t-1 +t
X
y
Xt
20
y
Stability
t
A
Basin of attraction
Figure 4: Reverse Initialization with Basin of attraction.
I
Additional Insights
Basin of Attraction in Reverse Sampling: Within diffusion framework, establish forward process with variety to accommodate wide range of potential corruptions, ensuring the desired final distribution close to the expected distribution. We consider that diffusion models inherently encompass the forward operator A within their structure. The transition from high to low- cow-quality images is implicitly encoded in the diffusion pathway, hence the additional forward operator guidance might not contribute supplementary information, which we initially hypothesized would enhance the reverse process.
Thus, the keypoint transfers from the A to the initial y' (Detailed analysis based on Prop. 2 is provided in Appx. FF. Also found this problem fall into the Basin of Attraction (BA) in dynamical system as shown in Fig. 4 BA can be interpreted as the quality of the attractor of the degraded image in the reverse process here. A BA-guided initialization might be a more effective approach for posterior sampling in transitional SDE diffusion models. By initializing the reverse process closer to the expected solution, we may bypass the initial hurdle of distribution mismatch.
More Qualitative Results
25

---

R
LQ
DC (ours)
GT
Figure 5: Deraining results with light rain images of our method.
26

---

LQ
DOGM (ours)
GT
Figure 6: Deraining results with heavy rain images of our method.
27

---

LQ
D3GM (ours)
GT
Figure 7: Dehazing results with real hazy images of our method.
28

---

Frequency encoding direction
Phase encoding direction
8X
16X
8X
16X
LQ
DMGM (ours)
GT
D2 (ours)
LQ
Figure 8: MRI reconstruction results with undersampling rate x8 and x x16, on Frequency-encoding and Phase-encoding directions.
29

---

HH 4X
1Q
DOGM (ours)
GT
Figure 9: MRI super-resolution results with in-domain images of our method.
30

---

10
VI IX 12
RII 10
IOP 4X
II SAT
TM
Guys 4X
1QQ
D2GM (ours)
Figure 10: MRI super-resolution results with cross-domain (different imaging devices and degradation methods) images of our method.
31

---

Phase encoding direction
D'GM (OUSS))
GT
Frequency encoding direction
00000 (nrs)
1.
1.
Phase encoding direction
D 2-- (ours)
CT
Frequency encoding direction
D- -.G (ours
LQ
X8
91
Figure 11: MRI reconstruction results with undersampling rate x8 and x16 of our method, on Phase- encoding and Frequency-encoding directions.
32

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the
paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. The abstract clearly outlines the claims of existing approaches, introduces the novel score-based diffusion framework (D*GM), and highlights the effectiveness of the proposed method through extensive experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
It is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses the limitations of the method under the discussion, and provide more details in the appendix. The main claims made in the paper accurately reflect the paper's 's scope. Thus, we also provide more extra findings beyond the limitation.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
33

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
The paper provides a full set of assumptions and complete proofs for the theoretical results introduced, including the measure-preserve dynamics of Random Dynamical Systems (RDS) and the Temporal Distribution Discrepancy analysis.
Justification:
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper fully discloses all the information needed reproduce the main ex- perimental results, including details about the benchmarks used, such as magnetic resonance imaging, and the effectiveness of the D*GM framework across multiple benchmarks.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
34

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We have provide all the details and instructions along with the paper and the appendix for the reproducible results. The datasets involved in the paper are public.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https://nips.cct
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
/ //nips. t/tips.cc/publl.../ wwtpppblic/guidess.eeeeeespeeeee htwol////g..dessdeeeuuiiiiis wwweeessummsssoooooo..
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: The paper specifies all necessary training and test details, such as data splits, hyperparameters, and optimizer types, ensuring that the results can be fully understood and appreciated. Full details are provided in the supplemental material to complement the core paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
35

---

Justification:
The paper reports evaluation metrics and other appropriate statistical signifi- cance information for the experiments, clearly explaining the factors of variability and the methods used for caleulating these measures. On different downstream experiments, we adapted appropriate evaluation metries, following state-of-ther work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g., negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides detailed information on the compute resources required for the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The research conducted in the paper conforms to the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there special consider- ation due to laws regulations in their jurisdiction).
36

---

10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The paper discusses both potential positive and negative societal impacts of the work, such as the improvement of diffusion models for medical imaging and the risk of misuse in generating deceptive content.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Necessary safeguards are described for the responsible release of models medical and image generators. These safeguards help ensure controlled and ethical use.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
37

---

The paper properly credits the contributors of existing assets used in the research.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. www.coo//daaaaee com/aataeet has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The new assets introduced in the paper, including the D*GM framework, are well-documented, with comprehensive documentation provided alongside the assets. This facilitates their use and further development by the research community.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA. The paper does not involve crowdsourcing or research with human subjects, this section is not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
38

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: NA. The paper does not involve crowdsourcing research with human subjects, so this section is not applicable.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
39


Reviews:
[{"review_id": "vlctdMq2jB", "reviewer": "Reviewer_APgH", "summary": "Given that existing diffusion models are limited to linear inverse problems, this paper proposes to use measure-preserving dynamics of random dynamical systems to formulate a theoretical framework for SDE diffusion models. They uncover several strategies that inherently enhance the stability and generalizability of diffusion models for inverse problems and introduce a score-based diffusion framework, D3GM. The measure-preserving property can return the degraded measurement to the original state despite complex degradation with the RDS concept of stability. Experiments on multiple restoration and reconstruction tasks, such as dehazing, deraining, and MRI reconstruction, demonstrate the stability and generalizability of the proposed D3GM framework.", "questions": "Please refer to the weaknesses.", "limitations": "Yes", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The theoretical results are interesting and open up many potential paths for future investigations.\n\n- Clearly explains the advantages of measure-preserving dynamics in SDE diffusion, and how they motivate algorithmic design.\n\n- For some challenging applications, such as MRI super-resolution, the derived model shows excellent generative capabilities and outperforms some well-known baseline methods.\n\n- The writing is clear and the paper is well structured.", "weaknesses": "- The connection between the measure-preserving property and the proposed D3GM is not well-explained, and how the temporal distribution discrepancy is mitigated within D3GM is not intuitive.\n\n- Why choose the perspective of measure-preserving dynamics of random dynamical systems? What unique advantages does it offer in solving challenging inverse problems? Besides the related instability analysis, are there more intuitive explanations available?\n\n- The baselines used for comparison in the dehazing and deraining tasks are somewhat outdated, which raises questions about the real-world effectiveness of the proposed methods when compared to state-of-the-art approaches.\n\n- Strictly speaking, although the paper provides some theoretical insights, it does not systematically resolve the issues, and the intuitions gained appear somewhat heuristic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer APgH for the detailed review and recognition of our theoretical contributions and innovative use of measure-preserving dynamics.\n\n>Q. Connection between measure-preserving property and D3GM:\n\nThe measure-preserving property ensures that despite complex degradations, the distribution can revert to its original state, maintaining stability and generalizability. This is achieved through the lens of Random Dynamical Systems (RDS), which provide a robust framework for analyzing temporal distribution dynamics and ensuring consistency in the diffusion process. \n\nWe have provided more explanations in Q1, Q2 and Q3 of the general rebuttal and will add more detailed explanations and illustrative examples to clarify this connection in the revised version.\n\n>Q. Choice of measure-preserving dynamics:\n\nUnlike traditional linear approaches, RDS can model non-linear and non-Gaussian degradations, which are common in practical applications. This perspective provides a more comprehensive understanding of the underlying dynamics and improves the stability and accuracy of the reconstruction. We have provided an analogy in appendix B and Q4 of the general rebuttal.\n\n>Q. Outdated baselines in dehazing and deraining tasks:\n\nOur method focuses on robustness and generalization, especially in real-world applications such as MRI reconstruction, dehazing, and deraining. Achieving practical usability in these challenging tasks is a significant accomplishment. Compared to task-specific state-of-the-art methods, our approach excels with significantly reduced FLOPs and complexities (see Table 8, L. 321, P. 9). We have updated our comparisons to include very recent methods in Table 7 (L. 321, P. 9) and provided examples in the rebuttal PDF that show where these advanced methods struggle with real-world degradations.\n\n>Q. Systematic Resolution of Theoretical Insights:\n\nWe appreciate the reviewer's insight that a systematic resolution of our theoretical findings requires further research, aligning with your remark that our work opens up many potential paths for future investigations. In the appendix, we have elaborated on topics such as the basin of attraction and other related issues, sharing all our findings openly. We believe our approach and findings will inspire the community to collectively explore new and exciting directions in diffusion models based on our work. We are committed to further exploring this direction, aiming to bridge generative learning and real-world problem-solving through more robust frameworks. \n\nWe appreciate your constructive feedback and insightful questions. We will incorporate your suggestions to clarify the measure-preserving property and update comparisons. Your input will help us enhance the impact and clarity of our work. Thank you for your valuable input.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "O61iRC2Rb0", "reviewer": "Reviewer_J29t", "summary": "The paper addresses the use of diffusion models in solving inverse problems, which involve estimating causal factors from degraded data. Traditional methods often fall short in real-world scenarios due to accumulated errors and biases. To tackle these issues, the authors propose a new theoretical framework based on measure-preserving dynamics of Random Dynamical Systems (RDS) for Stochastic Differential Equation (SDE) diffusion models. They introduce the Dynamics-aware SDE Diffusion Generative Model (D3GM), which enhances the stability and generalizability of these models. Experimental results, particularly in magnetic resonance imaging (MRI), demonstrate the framework’s effectiveness.", "questions": "I have mentioned several problems in the Weakness section. In addition,\n\n1. For the training of NN, e.g. dehazing, you mentioned there were only 100 pairs images for training and testing. Is the model barely trained with this much data?", "limitations": "I am not aware of any potential negative societal impact of this work.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Solving inverse problems is critical in many real-world applications. Discussing the problem from the measure-preserving dynamical system perspective is interesting.", "weaknesses": "1. It is unclear to me how this work is different from DDBM [1] and Augmented bridge matching [2]. In fact, DDBM's setting is more general by considering diffusion bridges derived from h-transform. That setting covers OU processes (with some reparameterization on t if necessary). \n\n2. The empirical study does not include an evaluation on the sampled image quality. (It is mentioned at L245 that FID will be reported; however, I did not find any in the paper. )\n\n3. The connection between the theoretical work in Sec 3 and the implementation in Sec 4 is unclear. \n\n4. There are multiple choices of drift and diffusion coefficients. However, there are no discussions/ablation studies to show how to choose them. \n\n5. There is no performance comparison with similar implementations like DDBM and I2SB. \n\n[1] Denoising Diffusion Bridge Models, Linqi Zhou, Aaron Lou, Samar Khanna, Stefano Ermon, 2023\n[2] Augmented bridge matching, Valentin De Bortoli, Guan-Horng Liu, Tianrong Chen, Evangelos A Theodorou, Weilie Nie, 2023", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank Reviewer J29t for your valuable feedback and insightful comments. Your suggestions have significantly contributed to highlighting the distinctiveness and clarity of our work.\n\n>Q. Difference from DDBM and Augmented Bridge Matching:\n\nWe thank the reviewer for mentioning these two works. We recognize that DDBM [1] and Augmented Bridge Matching(ABM) [2] (close-source) were not published at the time of our submission. We will include them in the introduction and discussion sections of the final paper. Despite this, our approach remains distinctly different:\n\nBoth DDBM and ABM rely on the theoretical foundation of diffusion bridges and Doob’s h-transform. The h-transform is an almost sure path between two endpoints where DDBM fixes it to be the forward process and learns the reverse. The learned generalized time-reversal ODE is designed for paired image translation, as stated at the end of DDBM’s section 4 [1]. For image restoration tasks with complex degradations unseen during training (i.e., $(\\tilde{x}\\_{hq}, \\tilde{x}\\_{lq}) \\notin q\\_{data}(x,y)$), the DDBM ODE cannot generalize to capture the correct reversal, as shown in rebuttal PDF. DDBM can be challenging to scale due to the reliance on a fixed forward process during training as noted by the authors [7]. \n\nABM faces a similar issue. Despite augmenting the drift with initial sample information to preserve coupling information, it loses the Markovian property and imposes more restrictions in the time-reversal process, making it unsuitable for image restoration tasks with unseen degradations.\n\nIn contrast, D3GM leverages measure-preserving dynamics of Random Dynamical Systems (RDS) to enhance stability and generalizability. Unlike DDBM, which is constrained by its reliance on h-transforms, D3GM’s measure-preserving property allows distributions to revert to their original state despite complex degradations, ensuring robustness and accuracy in non-linear and non-Gaussian scenarios.\n\n>Q. Connection between theoretical work and implementation:\n\nThe measure-preserving property ensures that despite complex degradations, the distribution can revert to its original state, maintaining stability and generalizability. This is achieved through the lens of Random Dynamical Systems (RDS), which provide a robust framework for analyzing temporal distribution dynamics and ensuring consistency in the diffusion process. \nWe have provided more explanations in the Q1 and Q2 of the general rebuttal above and will add more detailed explanations and illustrative examples to clarify this connection in the revised version.\n>Q. Choices of drift and diffusion coefficients:\n\nWe appreciate your comments on the coefficients and schedules. We have provided visual comparisons in the rebuttal PDF. The visual results and discussions in Q5 of the general rebuttal will be incorporated into the final version and appendix.\n\n>Q. Training with limited data:\n\nObtaining paired datasets for real-world corruption scenarios is challenging. To address the limitation of having fewer pairs of images, we employ augmentations on the O-haze (2833×4657 pixels) and Dense Haze (1200×1600 pixels) datasets, adhering to common settings used in previous dehazing works [4, 5, 6]. We enhance the training dataset by randomly cropping patches from these images for each epoch, ensuring that the patches differ each time. This effectively creates a much larger dataset from the small available set, improving the robustness and generalization of our model, as demonstrated in other studies. We will clarify this in the final version and add more details in the appendix.\n\n>Q. Comparison with DDBM and I2SB\n\nRegarding I2SB, we have already compared our approach to I2SB in Table 1 (theoretically, L. 90, P. 3), Figure 2 (quantitatively, L. 248, P. 7), and Table 7 (quantitatively, L. 321, P. 9), discussing various perspectives in our submission. Additional experimental results have been included in the rebuttal PDF to further illustrate our model's performance.\nWe recognize that DDBM was not published at the time of our submission. DDBM shares many similarities with I2SB, and we have provided a specific analysis in the general rebuttal's Q1 and your Q1. Thank you for mentioning these. \n\n>Q. Negative societal impact of this work:\n\nWe would like to clarify that our paper does mention the potential negative societal impacts in appendix A(L. 473, P. 14). We will ensure this point is further clarified in the final version to avoid any misunderstandings.\n\n>Q. Evaluation of FID:\n\nOur paper mistakenly mentioned reporting FID, which was an oversight. We appreciate you pointing this out and will correct it in the final version. Given our contributions, adding FID would not offer additional insights, as PSNR, SSIM, and LPIPS evaluate accuracy and perceptual quality comprehensively. Additionally, we have provided detailed comparisons with state-of-the-art methods, covering practical aspects (datasets, FLOPs, complexity) and theoretical aspects (foundations, mathematical formulations).\n\nThank you once again for your insightful feedback. Your comments have been crucial in refining our work and ensuring its robustness and clarity. We will incorporate the results, discussion and provide further explanations in the final version and appendix to ensure a comprehensive understanding.\n\n[1 ]Denoising Diffusion Bridge Models, L. Zhou, et al., ICLR, 2024\n\n[2] Augmented Bridge Matching, V. De Bortoli, et al., arXiv, 2023\n\n[3] I2SB: Image-to-Image Schrödinger Bridge, G.-H. Liu, et al., ICML, 2023\n\n[4] Mb-Taylorformer: Multi-branch Efficient Transformer Expanded by Taylor Formula for Image Dehazing, Y. Qiu, et al., ICCV, 2023\n\n[5] Image Dehazing Transformer with Transmission-Aware 3D Position Embedding, C.L. Guo, et al., CVPR, 2022\n\n[6] Single Image Dehazing for a Variety of Haze Scenarios Using Back Projected Pyramid Network, A. Singh, et al., ECCV Workshops\n\n[7] https://github.com/alexzhou907/DDBM/issues/3", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eoGKIO5gUb", "reviewer": "Reviewer_z6fz", "summary": "The author(s) of the paper provides a theoretically sound method of a Dynamics-aware SDE Diffusion Generative Model (D^3GM) to enhance the stability and generalizability of inverse problem diffusion models. The authors provide a rigorous mathematical examination of the temporal distribution discrepancy for the instability issue of transitionary score-based generative models. The analysis extends the traditional Ornstein-Uhlenbeck (OU) process to random dynamical systems (RDS), focusing on the stability of SDE. The authors then proposed a novel method (D^3GM) that combines the stationary process to relieve the temporal distribution discrepancy problem following the measure-preserving dynamics from RDS; this method could guide the SDE diffusion to a desired stable solution. The experimental results from the authors also indicate the efficiency of D^3GM under different situations.", "questions": "1. Although this paper has solid theoretical support, the experiment is not as good as the theoretical part. It would be helpful if the authors could add more visual comparisons between these models (I noticed Figure 3, but from the set of results, there seems to be no significant improvement). This would help readers get a clear understanding of the proposed method's exact performance compared with other methods. \n2. at line 217, the COS is chosen to balance 'the trade-off between complexity and effectiveness'. This is unclear; it will be helpful to add more theoretical support or a set of experiments to empirically show COS is better than other options.", "limitations": "Even though the paper fulfills the theoretical gap, the actual improvement of the model performance seems limited.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The authors provide a novel method (D^3GM) integrating measure-preserving dynamics into SDE diffusion models. This is the fundamental contribution of this paper. \n2. Extending the Ornstein-Uhlenbeck process to a random dynamical system is innovative, allowing the community to better understand the instability problems of diffusion models. This paper also provides a detailed mathematical measurement of the discrepancy between the reference and the retrieved data. \n3. The logic of this paper is clear.", "weaknesses": "1. The paper is theoretically sound, but it might be difficult for readers with no math background to follow. Consider adding more explanations for the theoretical part. \n2. See below", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer z6fz for the excellent summary of our paper and for capturing our core contributions so well. We appreciate the positive comments on our clear logic and detailed mathematical analysis.\n\n>Q. Add explanations for non-math background readers:\n\nWe acknowledge that combining theoretical knowledge across multiple domains is challenging. To address this, we have included all relevant mathematical theories and derivations in the appendix. Additionally, we have provided Intuition 1 (Line 142) and Intuition 2 (L. 158), as well as Example 1 (L. 143) and Example 2 (L. 200), at key points to aid understanding. Furthermore, we used a more vivid analogy in Appendix B (L. 480, P. 14) to guide readers through the connection between measure-preserving dynamics and inverse problems.\n\nBesides these enhancements, we will include additional explanations and illustrative examples in the camera-ready version to make the concepts more accessible. Specifically, we will also add more intuitive descriptions and diagrams to bridge the gap between the rigorous mathematical framework and practical understanding.\n\n>Q. Need for more visual comparisons:\n\nWe thank the reviewer for highlighting the importance of Figure 3 and other visual comparisons. In the revised version, we will expand Figure 3 to include more comparative visuals against other baseline methods. \n\nWe will focus more on visual comparisons with state-of-the-art (SOTA) methods. Given time and space constraints, we have included several visual comparisons with SOTA methods in the provided rebuttal PDF.\n\n>Q. Clarification on COS:\n\nWe appreciate the reviewer's comment regarding the use of Cos at Line 217. We have provided visual comparisons in the rebuttal PDF. The visual results and discussions in Q5 of the general rebuttal will be incorporated into the final version and appendix.\n\n>Q. Limitation on the model performance:\n\nWe understand the reviewer's observation regarding the limited apparent improvement in model performance. Our performance primarily lies in the robustness and generalization of resulting models, especially when applied to real-world data. Given the inherently challenging nature of these tasks, achieving practical usability is in itself a significant contribution. \n\nOur method not only demonstrates exceptional stability but also proves its extensive applicability across various practical scenarios, such as MRI reconstruction, dehazing, and deraining. Compared to other task-specific state-of-the-art methods, our approach excels with dramatically reduced FLOPs and complexities in Table 8 (L. 321, P. 9). \n\nWe have provided comparisons with other advanced methods in Table 7 (L. 321, P. 9), and additionally included examples in the rebuttal PDF highlighting where these advanced methods fail to handle real-world degradations.\n\nIt is a pleasure to have our innovative approach and clear logic recognized by you. The theoretical parts will be made more accessible by adding more visual comparisons and explanations. The suggestions you provided have greatly enhanced the completeness of our work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers (APgH, J29t, z6fz) for their insightful comments and recognition of the novelty and strong theoretical foundations, comprehensive experiments and convincing results, and applicability across areas. 

We greatly appreciate the recognition that D3GM opens up many potential paths for future investigations.

>Q1.Uniqueness and differences from Diffusion Bridge models.

Current diffusion theories primarily focus on image editing and translation, e.g.,  Bridging theory and diffusion theory (Table 1, L.90, P. 3), a process that does not include the perturbation of degradation factors and precise reconstruction of the underlying image. Assuming that the forward process can almost surely arrive at the target state $y$(ideal final state of the forward process and the initial state of the reverse process) is the basis for all bridge models, where in the context of our paper, $y$ is the stationary distribution $N(\mu, \tau^2 \sigma^2)$. However, the ‘almost surely’ state $y$ is never reached in finite time $T$, and the discrepancy we defined in Sec3 is the theoretical characterization of this unaddressed issue.

In this paper we provide a completely novel view on the theoretical foundation of how the *degradation* process is modeled. The result is an approach that is more in line with the original intention of the theory of diffusion with Stability and Generalizability. We chose inverse problems as a relevant application area to demonstrate our ideas but also included a variety of challenging problem settings to explore the robustness of $D^3GM$. To the best of our knowledge, no other method can handle a *diverse range* of challenging tasks like dehazing, complex-valued MRI reconstruction, super-resolution, with a *unified underlying theoretical framework*. 

>Q2.Connection between the concepts.

Score-based Generative Models (SGMs)  often perform poorly in real-world scenarios. To provide a theoretical investigation of this gap, we interpret *Transitionary SGMs* as a type of process that includes random fluctuations, i.e., *Ornstein-Uhlenbeck (OU) processes*. This perspective allows us to understand the random fluctuations in image degradation as stochastic processes, providing a foundation to reinterpret SGMs as systems that evolve over time with inherent randomness with the diffusion process as a natural extension of the SDE framework involving the OU process. Such systems are known as random dynamical systems (RDS).

The *Measure-preserving property* is introduced from the perspective of RDS: The distribution can still return to the original state despite complex degradation with *stability*. Stability in RDS ensures that the system does not exhibit erratic or divergent behavior, but instead, remains predictable and consistent in the long run.

>Q3.Motivation for analysis 3 and solution 4.

We address the robustness of diffusion models for inverse problems under domain shift and concept drift (*unknown and heterogeneous degradation*). 

Current SGMs approximate the degradation process as linear and monotonous, which enhances their performance in scenarios traditionally suited to linear transformation. However, this assumption leads to significant shortcomings when faced with complex, real-world data as shown in our experiments. Intuitively, this overly idealized assumption of linear restoration leads to the accumulation of discrepancies (Prop. 2, L.155) during the reverse phase of unstable diffusion sampling, resulting in deviations that prevent a stable backward process.

>Q4. Why we choose the perspective of measure-preserving dynamics of random dynamical systems.

Consider the analogy of a stretched rubber band, which naturally seeks to return to its original but does so with a lot of oscillations when released.  

This elastic behavior parallels the dynamics of the OU process, where deviations from a mean state are counteracted by a restorative force, guiding the system back towards equilibrium (i.e., final state), with random perturbations. 

Our process models the noise as a stochastic component that fluctuates around a stationary process and improves the OU process with RDS. Measure-preserving dynamics ensures that while the image undergoes transformations during the denoising process, the overall statistical properties remain consistent (i.e., invariant image features), which cannot be satisfied by previous approaches (Tab.~1).

>Q5. Choices of Drift and Diffusion Coefficients:

PDF file and below are the different settings and considerations, the final version will include more details and discussion.

*Cos*: The Cos schedule is preferred for its smooth transition and better handling of complex temporal dynamics, maintaining a high signal-to-noise ratio and preventing abrupt loss of original information with a uniform denoising process.

*Log*: The log schedule's initial fast noise reduction increases early epistemic uncertainty, and its need for a large $𝑘$ parameter makes it resemble a constant schedule, losing its late-stage advantages.

*Lin*: The linear schedule fails to account for varying noise complexity, causing faster degeneration of original information and potential underfitting or overfitting during sampling.

*Quad*: The quadratic schedule's aggressive early noise reduction and slow late-stage denoising result in rapid loss of details initially and insufficient denoising later, lacking general improvements.

Diffusion models require high signal-to-noise ratios to achieve fine-grained detail and spatiotemporal coherence, particularly during the reverse process. The COS schedule supports measure-preserving dynamics by providing a consistent and controlled denoising process. It reduces the risk of accumulating errors that can distort the reconstructed image. 

We appreciate the reviewers' constructive feedback and are committed to addressing all the raised concerns in our camera-ready version, ensuring a rigorous and impactful camera-ready version.

Meta-review: The paper studies inverse problems through the lens of diffusion models. The majority of the reviewers agrees that this is a strong submission, in particular, a novel idea, theoretically sound and well presented. Furthermore, the author's rebuttal did a good job in clarifying existing issues, leading to overall score increases. In summary, I am recommending acceptance and encourage the authors to take all comments seriously and adjust the manuscript accordingly for the camera-ready version.
Decision: Accept (poster)