Score-based Self-supervised MRI Denoising
Authors: Jiachen Tu
Yaokun Shi
Fan Lam
Venue: ICLR 2025 Poster (2025)
Keywords: Generalized Denoising Score Matching; Self-supervised Learning; Self-supervised Denoising; Score-based denoising; Medical Image Denoising;
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: Corruption2Self introduces a score-based self-supervised MRI denoising framework that leverages Generalized Denoising Score Matching (GDSM) to achieve high-quality denoising without clean reference images.

Abstract: Magnetic resonance imaging (MRI) is a powerful noninvasive diagnostic imaging tool that provides unparalleled soft tissue contrast and anatomical detail. Noise contamination, especially in accelerated and/or low-field acquisitions, can significantly degrade image quality and diagnostic accuracy. Supervised learning based denoising approaches have achieved impressive performance but require high signal-to-noise ratio (SNR) labels, which are often unavailable. Self-supervised learning holds promise to address the label scarcity issue, but existing self-supervised denoising methods tend to oversmooth fine spatial features and often yield inferior performance than supervised methods. We introduce Corruption2Self (C2S), a novel score-based self-supervised framework for MRI denoising. At the core of C2S is a generalized denoising score matching (GDSM) loss, which extends denoising score matching to work directly with noisy observations by modeling the conditional expectation of higher-SNR images given further corrupted observations. This allows the model to effectively learn denoising across multiple noise levels directly from noisy data. Additionally, we incorporate a reparameterization of noise levels to stabilize training and enhance convergence, and introduce a detail refinement extension to balance noise reduction with the preservation of fine spatial features. Moreover, C2S can be extended to multi-contrast denoising by leveraging complementary information across different MRI contrasts. We demonstrate that our method achieves state-of-the-art performance among self-supervised methods and competitive results compared to supervised counterparts across varying noise conditions and MRI contrasts on the M4Raw and fastMRI dataset. The project website is available at: https://jiachentu.github.io/Corruption2Self-Self-Supervised-Denoising/.

Published as a conference paper ICLR 2025
SCORE-BASED SELF-SUPERVISED MRI DENOISING
Jiachen Tu, Yaokun Shi & Fan Lam
Department Electrical and Computer Engineering University of Illinois at Urbana-Champaign Champaign. IL 61820, USA {itu9, yaokuns2, fanlaml @ feillinois.edl
ABSTRACT
Magnetic resonance imaging (MRI) is powerful noninvasive diagnostic imaging tool that provides unparalleled soft tissue contrast and anatomical detail. Noise contamination, especially in accelerated and/or low-field acquisitions, can significantly degrade image quality and diagnostic accuracy. Supervised learning based denoising approaches have achieved impressive performance but require high signal-to-noise ratio (SNR) labels, which are often unavailable. Self-supervised learning holds promise to address the label scarcity issue, but existing self-supervised denoising methods tend to oversmooth fine spatial features and often yield inferior performance than supervised methods. We introduce Corruption2Self (C2S), novel score-based self-suuerrised framework for MRI denois- ing. At the core of C2S is generalized denoising score matching (GDSM) loss, which extends denoising score matching to work directly with noisy observations by modeling the conditional expectation of higher-SNR images given further corrupted observations. This allows the model to effectively learn denoising across multiple noise levels directly from noisy data. Additionally, we incorporate a reparameterization of noise levels to stabilize training and enhance convergence, and introduce detail refinement extension to balance noise reduction with the preservation of fine spatial features. Moreover, C2S can be extended multi-contrast denoising by leveraging complementary information across different MRI contrasts. We demonstrate that our method achieves state-of of-the-art performance among supervieed methods and competitive results compared super- vised counterparts across varying noise conditions and MRI contrasts on the M4Raw and fastMRI dataset. The project website available at: ht ht githuu. Sel Sar.aaaas-----eeeeaaaaaaaaaaaaaa
1 INTRODUCTION
Magnetic resonance imaging (MRI) is an invaluable noninvasive imaging modality that provides exceptional soft tissue contrast and anatomical detail, playing crucial role clinical diagnosis and research. However, the inherent sensitivity of MRI to noise, particularly in accelerated acquisitions and/or fie-fiedd settings, can impair diagnostic accuracy and subsequent computational analysis. Unlike natural images, MRI data often contains Rician or non-central chi-distributed noise in magnitude images. Denoising has become an important processing step the MRI data analysis pipeline. Higher signal-to-noise ratios (SNR) enable better trade-offs for reduced scanning times increased spatial resolution, both of which can improve patient experience and diagnostic accuracy. Traditional denoising methods, such as Non-Local Mcans (NLM) Froment (2014) 4) and BM3D Mäkinen al. (202)), often rely handcrafted priors such Gaussianity, self-sim self-similarity, or low-rankness. The performance of these methods is limited by the accuracy of their assumptions and often requires prior knowledge specific the acquisition methods used. With the advent of deep learning,

---

Published as a conference paper at ICLR 2025
supervised learning based denoising methods Zhang et al. (2017): Liang al. (2021): Zamir et al. (2022)
have demonstrated impressive performance by learning complex mappings from noisy inputs to clean targets. However, they rely heavily on the availability of high-quality, high-SNR "ground truth" data for training a resource that is not always readily available feasible to acquire in MRI. Acquiring such clean labels necessitates longer sean times, leading to increased costs and patient discomfort. This limitation underscores the need for self-supervised denoising algorithms, which remove or reduce the dependency on annotated datasets by leveraging self gereratedddddd pseudo-labels as supervisory signals. Moreover, supervised approaches often face generalization issues due distribution shifts caused by differences in imaging instruments, protocols, and noise levels Xiang et al. 2023), limiting their utility in real-world clinical settings. Techniques
such Noise2Noise Lehtinen al. (2018), Noise2Void Krull et al. (2019), Noise2Self [Batson & Royer
(2019), and their extensions Xie et al.
2020 Huang et (202)): Pang et al. (202I);Jana et al. (2024); Wang
ct al. (2023) have demonstrated the potential learn effective denoising models without explicit clean targets.
et al. (2020) and Coil2Coil Park et al. (2022). enhance performance. However, these self-supervised methods often have limitations. For example, Noise2Noise requires repeated noisy measurements, which may not be practical. Methods like Noise2 and Noise2Self rely on masking strategies that can limit the receptive field or introduce artifacts, potentially leading to oversmoothing and loss of fine details. Approaches such as Pfaff et al. (2023) and Park et al. (2022) require additional data processing steps, restricting their applicability. In this paper, we introduce Corruption2Self, a score-based self-supervised framework for MRI denoising. Building upon the principles of denoising score matching (DSM) Vincent (2011). we extend DSM to the ambient noise setting, where only noisy observations are available, enabling effective learning in practical MRI settings where high-SNR data are scarce or impractical to obtain. An overview of the C2s workflow is illustrated in Figure [ Additionally, we incorporate reparameterization of noise levels for a consistent coverage of the noise level range during training, leading to enhanced training stability and convergence. In medical imaging, the visual quality of the output and the preservation of diagnostically relevant features are often more critical than achieving high scores on standard metrics. To address this priority, a detail refinement extension introduced balance noise reduction with the preservation of fine spatial feature. Furthermore, we extend C2S to multi-contrast denoising by integrating data from multiple MRI contrasts. This approach leverages complementary information across contrasts, indicating the potential of C2S to better exploit the rich information available in multi-contrast MRI acquisitions.
Recent innovations tailored for MRI, like Patch2Self
further exploit domain- sppecffc characteristics
Forward Corruption
DolX, Asur ho(X,, t) Asep
(X..0)
NuX Amai
(X fau)
DK,,9 9)
D Irael
Figure 1: Overview of the Corruption2Self (C2S) workflow for MRI denoising. Starting from noisy MRI image Xiya the forward corruption process adds additional Gaussian noise to create progressively noisier versions X,, During training, the model learns reverse this process by estimating the clean image Xo from these corrupted observations, despite having access only to noisy data. The denoising function he approximates the conditional expectation E[X, Xz), effectively learning to denoise without clean targets. reparameterized function Do (X,, f), which shares parameters with hg, is used to compute the loss.
We conduct extensive experiments on publicly available datasets, including a low-field MRI dataset, to evaluate the performance of C2S. Our results demonstrate that C2S achieves ofate-of-thh the- performance
2

---

Published as a conference paper at ICLR 2025
among self-supervised methods and, after extending to multi-contrast on the M4Raw dataset, shows state-of- the-art performance among both self-supervised and supervised methods. Notably, we are among the first to comprehensively analyze and compare self-supervised and supervised learning approaches in MRI denoising. Our findings reveal that C2S not only bridges the performance gap but also offers robust performance under varying noise conditions and MRI contrasts. This indicates the potential of self-supervised learning to achieve competitive performance with supervised approaches when the latter are trained on practically obtainable higher-SNR labels, particularly in scenarios where perfectly clean ground truth is unavailable, offering a practical and robust solution adaptable to broader clinical settings.
2 BACKGROUND
2.1 LEARNING-BASE DENOISING WITHOUT CLEAN TARGETS
A central concept in many of the self-supervised denoising methods is J-invariance Batson & Royer (2019), where the denoising function is designed to be invariant to certain subsets of pixel values. Techniques like
Woise2Void Krull et al. (2019) and Sf & Royer (2019) utilize this property by masking pixels and predicting their values based on their surroundings. approximating supervised learning objectives without the need for clean data. Noise2Same Xie et al. (2020) extends these concepts by implicitly enforcing J- invariance via optimizing self- superviedd upper bound. Approaches such as Neighbor2Neighhon Huang al. Pang et al. (2021) create pairs of images from single noisy observation mimic the effect of supervised training objectives with independent noisy pairs. Noise2Score Kim & Ye (2021) exploits the assumption that noise follows an exponential family distribution and utilizes Tweedie's 's formula to obtain the posterior expectation of clean images using the estimated score function via the AR-DAE et al.. (2020) approach. This approach highlights a connection between Stein's Unbiased Risk Estimator (SURE)-basee based denoising methods [Soltanayev & Chun (2018); Kim et al. (2020) and score matching objectives Hyvärinen & Dayan (2005). Specifically, under additive Gaussian noise, the SURE cost function can be reformulat lated as an implicit score matching objective, differing only by a scaling factor and constant term Kim & Ye (021I) In the context of MRI, recent innovations have capitalized on the intrinsic properties of data within self supf-supe framework Moreno López et al. (2021) enhance denoising performance without clean labels. For standard MRI. Praff et al. (2023) utilizes SURE Kim et (2020) and spatially resolved noise maps to improve denoising performance, while Coil2Coil Park jt al. (2022) leverages coil sensitivity exploit multi-coil information effectively. In the realm of diffusion MRI (dMRI), specialized approaches have emerged to address the unique challenges of 4D data. Patch2Self Fadnavis al. 2020) employs a J-invariant approach that preserves critical anatomical details by selectively excluding target volume data from its training inputs, though it requires minimum of ten additional diffusion volumes. DDM2 Xiang et al. (2033) introduces a three-stage framework incorporating statistical image denoising into diffusion models. More recently, Wu et al. Wu et al. (202)) proposed Di-Fusion, a single-stage self-supervised approach that performs dMRI denoising by integrating statistical techniques with diffusion models through Fusion process that prevents drift in results and "Di-" process that better characterizes real-world noise distributions.
(2021). Noisier2Noise |Moran et al. (2020) and Recorruped2Recoruppp
2.2 DENOISING SCORE MATCHING
Denoising Score Matching (DSM) Vincent (01T) is a framework for learning the score function (i.e., the gradient of the log-density) of the data distribution by training model to reverse the corruption process of adding Gaussian noise. In DSM, given a clean data sample X, E Rd and a noise level oe > 0, a noisy observation X, is generated by adding Gaussian noise:X, Xo + o,Z, Z ~ (0 Ia). The objective is to train a denoising function hg Rdd x RR + Rd, parameterized by 8, that estimates the clean image X, from its noisy counterpart X,. This is achieved by minimizing the expected mean squared error (MSE)
3

---

Published as a conference paper ICLR 2025
loss Xx..x. (X,,).)) - - Vincent (201)) demonstrated that optimizing the denoising function ha
using the MSE loss is equivalent to learning the score function of the noisy data distribution (x(x)) up to a scaling factor. Specifically, using Tweedie's 's formula, the relationship between the denoising function and the score function is given by: Vx, log (xx) मे (b,(x,, 1) Xe) By learning ha. we implicitly learn
Vx, log (xx). which forms the foundation of score-based generative Moodels/Song & Ermon (2019); Ho et al. (020). However, DSM relies on access to clean data during training. limiting its applicability in situations where only noisy observations are available. Ambient Denoising Score Matching (ADSM) Daras
et al. (2024) leverages double application of Tweedie's 's formula to relate noisy and clean distributions, enabling score-based learning from noisy observations. While ADSM was originally designed to mitigate memorization in large diffusion models by treating noisy data as form of regularization Daras et al. (2024) its potential for self-supervised denoising remains underexplored. In our work, we bridge the gap of score-based self-supervised denoising in practical imaging applications.
3 METHODOLOGY
Consider clean image Xo H& and its corresponding noisy observation Ka E RR. We formulate the self-supervised denoising problem as estimating the conditional expectation E[X, X which constitutes the optimal estimator of X. in the minimum mean square error (MMSE) sense. While estimating this typically requires clean or high-SNR reference images in a supervised learning setting, we adopt an ambient score-matching perspective inspired by Daras al. (202)), cireumventing the need for clean labels. The noisy image can be modeled as:
(1) where tam. 0 denotes the noise standard deviation at level Edata- and N ~ ~O, represents the noise component. In MRI, the noise N is typically assumed to be additive, ergodic, stationary, uncorrelated, and white in k-space Liang & Lauterbur (2000). When the signal-to-noise ratio (SNR) exceeds two, the noise in the image domain can be well-approximated as Gaussian distributed (Gddjjartsson & Patz (195)). While this Gaussian assumption underpins our theoretical framework, our empirical results suggest robustness even when this condition is not strictly satisfied.
X Xo Tua N,
The noise level 0 tus scales the noise to match the observed noise level in the MRI data. To facilitate self- supervised learning, we introduce a forward corruption process that systematically adds additional Gaussian noise to X defining a continuum of increasingly noisy versions of the data:
X; X + Voi -?. Z. Z~ N~~ (0,,
8> Edata.
(2)
where oe is a strictly monotonically increasing noise schedule function for E (fama; T), with being the maximum noise level. This process allows to model the distribution of the noisy data different noise levels and forms the foundation for our generalized denoising score matching approach. In scenarios where the noise deviates from Gaussianity (e.g., Rician noise in low-SNR regions), pre-processing techniques such the Variance Stabilizing Transform (VST) Foi (201I) can be applied to better approximate Gaussian statistics. Our goal train a denoising function he d R Rd. parameterized by 0. which maps noisy input X, at noise level t to an estimate of either the clean image Xo or a less noisy version corresponding to a target noise level tuage S taar where X ( Xo + o Oungu No. with No- ~ (N,
3.1 GENERALIZED DENOISING SCORE MATCHING
We introduce the Generalized Denoising Score Matching (GDSM) loss, which enables learning denoising function directly from noisy observations by modeling the conditional expectation of a higher-SNR image given further corrupted version of the noisy data (proof provided in Appendix 4).
4

---

Published as a conference paper ICLR 2025
Theorem 1 (Generalized Denoising Score Matching), Let Xo E IR' be clean data sample drawn from the distribution Po(Xo). Suppose that the noisy observation at given data noise level idaaa is
X = Xo + OusuNN, N ~ N~N(O,
and that for any 1. Equation equation
ddata the observation X, is generated according to the forward process described in
Let bg IRd x (Iaau, T] + Red be denoising finntiin parameterized by 0, and fix a target noise level 0 Darnn satisfying S Obrnn S Taa Define the loss function
J(0) = Ex e, X. [l1r(t, (twae)) he (X,,t) + ó(t, (twrae) X.
Xiam
(3)
where is sampled uniformly from (Idana T' and the coefficients are defined by
o, o Aaa
rtt, (ompe))
and ö(t, (tam)) ==
o
of Vawzer
Then any minimizer 0* of (() satisfies
ha- (X. (urpr x.]
other words, the optimal denoising function recovers the conditional expectation of the image with noise level twne given the more heavily corrupted observation X..
Remark 1. The proposed GDSSM framework generalizes several existing methods: when &uagg t GDSM reduces to the standard denoising score matching (DSM) Vincent (011:: when tunge = 0, it recovers
the ambient denoising score matching (ADSM) formulation Daras al. 2024 Moreover, GDSM subsumes
Noisier2Noise Moran et al. (2020) as a special case. By setting Traze 0 and fixing the noise level to
of = (1+ o²) jo Eamu" the coefficients simplify to yt, 0) = Ita? and 8(t, = 1+ thereby generalizing Noisier2Noise continuous range of noise levels. For further details, see Section B.2
a²
To enhance training stability and improve convergence, we introduce a reparameterization of the noise levels. Let (0, ["] be a new variable defined by
o = of o.... T' 07 o...
(4)
The original can be recovered via the inverse of oe. as:
(V + 0
t t=
(5)
Since o, is strictly increasing, o, is well-defined. In practice, T fiaas we approximate 7" R T allow uniform sampling over T and consistent coverage of the noise level range during training, leading to smoother and faster convergence. To further improve training stability, we combine our reparameterization strategy with Exponential Moving Average (EMA) of model parameters. Details of the training dynamics with and without reparameterization (and the effect of EMA) are provided in Appendix I
(see Figure 12
Under this reparameterization, the loss function in Equation equation 12 becomes:
J'(0) = 7.X, [II' (T, Otagge ho (X,, t) + 8'(TT, Otarge
Xx X tan 1?]]
(6)
where the coefficients are:
2 kans
8'(T, Otupa)
o? T
07 oz
Surger
Corollary 2 (Reparameterized Generalized Denoising Score Matching). With our reparameterizatton strategy, the minimizer 0* of the objective J' J') satisfies (proof provided in Appendix 5p:
hg- (X,, = E. [X t X.] vX, E Rd. t Iytata
5

---

Published as a conference paper ICLR 2025
3.2 CORRUPTION2SELE: SCORE-BASED SELF-SUPERVISED MRI DENOISING FRAMEWORK
Building upon the Reparameterized Generalized Denoising Score Matching introduced earlier, we propose Corruption2Selr (C2S) where the objective to train denoising function he that approximates the conditional expectation E [ Xz), where X is a less noisy version of a with noise level tases I ta
Denote D2 (xx,,, k.TT T) as the weighted combination of the network output with the input through skip connections:
Dx >ouI(T, (rage) ho (X,, () + Askip(T, (T, turge Xx;
(8)
where X; imm + o with Z (O,, and t relates to through of (V + )iam) blending coefficients are defined as:
The
(T, Oaagya
ATsip(T, (hangs
(9)
o7 kmm
o7 + keam
o burget
Barge
When Otapp = 0 (maximum denoising); the goal is to predict E(X, X and the coefficients simplify to: Aour (T, 0) = =0?/ + )oum), Actip (1, 0) ofan/ (o² + ob )- Our loss function is expressed as:
(0) = _ P (x). [((0,1 ww(T) [ID,(X,,T)
X
(10)
where X, is constructed by adding noise to Xtan according to the reparameterized noise level t, and w(T) is weighting function designed to balance the contributions from different noise levels. Following practices from prior works Song al. (2020); Kingma al. (2021) 1) Karras et al. 2002)). w(() can be set to (o? + of with a being a hyperparameter controlling the weighting.
During inference, given noisy observation Xtam the denoised output is obtained by:
X lgg tam Iddaaas
(11)
where the trained model ha- approximates E X providing a clean estimate of the image.
While the C2S training procedure effectively approximates X,X,, X,] when Ourpe
it can lead oversmoothing. To maintain balance between noise reduction and feature preservation, we introduce detail refinement extension where the network trained to predict E E X.] with a non-zero target noise level tuge > O, allowing the network to retain controlled amount of noise for preserving finer image textures (details are provided in Appendix G). As shown in Table [ incorporating the detail refinement extension leads to statistically significant improvement in image quality across contrasts. Our denoising model builds upon the U-Net architecture employed in DDPM Ho et al. (2020), enhanced with time conditioning and the Noise Variance Conditioned Mulli-Head Self-Attention (NVC-MSA) module Hatamizadeh et al.) (2023). which enables the self-attention layers to dynamically adapt to varying noise scales. Empirically, we found that setting the noise schedule function a, equal the noise level yields good performance. More details regarding the architecture and implementation are provided in Appendix
4 RESULTS AND DISCUSSION
We first evaluate C2S on the M4Raw dataset, which includes in-vivo MRI data with real noise. The training and validation data use thrre-repetitioonavvraa images, while the test data comprises higher-SNR labels,
6

---

Published as conference paper ICLR 2025
Algorithm 1 Corruption2Self' Training Procedure
Table 1: Effectiveness of detail refinement module on the M4Raw validation dataset. Improvements are statistically significant (*p < 0.05) using paired t-tests.
Require: Noisy dataset Xtauca he, max noise level T, batch size m, total iterations K, noise schedule function d,
Additional details provided in Appendix G
1: for l = 1 to K do
Sample minibatch Xeaama U(0, T), 2
Contrast
PSNR
p-value
(0, Ta)
TI
34.5640.002 - 34.89 038
0.001
of
3:
Compute Amae (T, 0)
Aakip (T,0)
T2 FLAIR
33.84 10.090 - 34.07 10.121
0.001
of
12.44 - 32.58
0.016
Gdate Tduta
77
SSIM
p-value
4:
Recover = or (1 03 + olann
TI T2 FLAIR
0.885+0.002 - 0.892
0.007*
5: Compute X, X Xtduto + 0.7
0.860+0.003 - 0.8667 40.002
0.003*
Compute loss: Lc2s(8) (0)
Em 1e( -
0.81220.001 - 0.818 0.005
Update using Adam optimizer Kingma (014))
created by averaging six repetitions for TI and T2, and four for FLAIR. This setup allows us to assess how well denoising methods perform when evaluated on cleaner test data. Table Table the performance of C2S against classical methods (NLM, BM3D), supervised learning (SwinIR, Restormer, Noise and self- supervised approaches (Noise2Void, Noise2Self, PUCA, LG-BPN, Noisier2Noise, Recornupted2Recorruppe C2S consistently outperforms other self-supervised methods, achieving the highest PSNR and SSIM across all contrasts. Our base C2S model significantly outperforms existing self-supervised methods across all contrasts, achieving PSNRs of 32.59dB. 32.28dB, and 32.43dB for T1, T2. and FLAIR respectively. With our detail refinement extension, C2S further improves performance to 32.77dB/0.99 32.33dB/0.890 and d1/0B.0.87 for T1, T2, and FLAIR contrasts respectively. Notably, recent self-supervised methods like PUCA and LG-BPN, demonstrate lower performance on MRI data. This performance gap can be attributed the blind-spot architecture design, often leading to information loss and oversmoothed results,
Nolay
Label
Noise2Noise
SwisiR
BMBD
Noise2Bel
LG-BPN
Noisier2Neise
R2R
C2S
C29 pernfiraeeed
Figure 2: Comparison of different denoising methods for T1 contrast from the M4Raw dataset.
An important observation is that supervised methods such as SwinIR Restormer, trained on three- ppetition-averager labels, do not significantly outperform self-supervised methods on higher-SNF SNR test data. Supervised models typically learn E Xtyn.. where taat > farget > 0 when multi-repetition averaged samples are used labels. This makes supervised methods less effective at handling shifts higher-SNR test data. In contrast, C2S approximates E[X, X,). allowing it to achieve competitive performance on test data. Empirical results test labels pbree-repetitionn-veeree matching the SNR of the training data (presented
7

---

Published as conference paper ICLR 2025
Methods
TI PSNR SSIM f
T2 PSNR SSIM f
FLAIR PSNR SSIM 1
Classical Non-Learning-Bel Based Methods
NLM Froment (2014)
31.90/ /.899 0.898 32.07 / 0.903
31.17
32.01 /
BM3D Mäkinn et al. 2020
31.20 31.20/0.877 0.877
32.14 2014/0.87 0.873
Supervised Learning Methods
SwinlR Liang et al. (2021)
32.53 / 0.913
31.90 0.891
32.15 0,885
Restormer Zamir al. (2022
32.35 / 0.912 32.59 / 00911
31.79 0.89
32.31 0.886
Noise2Noise Lehtinen et al. 2018
32.37/0.888
32.70 0.871
Self-Supervised Single-Contre Methods
2 Krull et al. (2019)
31,46 / 0.870
30.93 / 0.857
31.17 0.851
Noise2Self Batson Royer (2019)
31.72 0.887
31.18
31.72 0.870
PUCA Jang al. 20224
30.52 / 0.870
29.11/0.827
29.57 / 0.807
LG-BPN et al. (2023
31.15 0.890
60.66/0.866
30.82 / 0.862
Noisier2NoIse Moran al.
2020
31.60/0887
31.45
31.59 0.861
Recornpted2kecorrppcee Pang et al. 2021
31.67 /
31.33 0.870
31.57 / 0.863
C2S
32.59 / 009155
32.28 0.888
32.43 0.872
C2S (w/ Detail Refinement)
32.77 / 0.919
32.33 0.890
32.51 0.876
Table 2: Quantitative comparison of denoising performance on the M4Raw test dataset. Results show PSNR (dB) and SSIM metrics across T1, T2, and FLAIR contrasts. C2S with detail refinement (bold) achieves the best performance among all self-supervised methods across all contrasts, and even outperforms supervised approaches in some cases. Second-best results among self suff-supe methods are underlined.
in Appendix show that supervised methods like SwinIR and Restormer perform better when the noise characteristics of the training and test data are similar.
Methods
PD. 13/255 PSNR / SSTM t
PD. 25/255 PSNR / SSIM t
PDFS, 13/255 PSNR / SSIM 1
PDFS, 25 PSNR SSIM
Classical Non-Learning-Bane Methods
NLM
30.40/ / 0.772 33.16 0.829
21.63 / 0.327 30.58 / 0.755
28.82 0.726 30.64 / 0,705
21.12/0.35
BM3D
28.49 / 0.592
Supervised Learning Methods
Noise2True (Swin1R)
34.44 / 0.868
32.39 / 0,820
31.35 / 0.774
29.55 / 0.665
2oise2True (U-Net)
34.54 0.80 8.8 34.06 0.854
32.61 0.825
31.39 0,775
29.62 / 0.669
Noise2Noise Lehtinen et al. 2018
30.83 0.769
31.33 0.773
29.12 / 0.654
Self-Supervised Methods Noise2 Void Krull et al. (2019
32,19 0.804
29.79 / 0. 0.706
29.50 0.629
27.99 0.558
Noisc2Self Bafson Koyer 2019
32.47 0.808
30.60 0.757
29.32 0.613
28.31 0.563
PUCA Jang el A024
31,03 0.771
29.88 0.740
28.65 0.594
27.54 0.527
LG-BPN Wang 2023
31.15 0.776
30.32 0.751
29.14 0.603
27.77 0.535
Noisier2Noise Moran ct al.
2020
33.18 0.807
30.35 0.741
30.39 0.683
27.83 / 0.559
Recorrupeed2eccorutee Pang al.
2021
33.29 0.810
30.52 / 0.745
30,95 0.752
28.32 / 0.561
C2S
33.36 0.831
30.62 / 0.753
30.72 0.750
28.58 / 0,592
C2S (w/ Detail Resinement)
33.48 0.832
30.67 0.761
30.91 0,756
28.62 0.601
Table 3: Quantitative evaluation on the fastMRI test dataset with simulated noise at two different levels 13/255 and 25/255) across PD and PDFS contrasts.
8

---

Published as a conference paper ICLR 2025
Nolay
Lebel
Nolss2Noise
Swind
BM3D
Noise2Bulf
LG-BPN
Noisier2Niise
R2R
C2S
C2S
Figure Comparison of denoising methods for the PD contrast (a 13/255) from the fastMRI dataset.
To further evaluate the robustness of C2S under different noise levels, we conducted experiments on the fastMRI dataset Zbontar et al. (2018), simulating Gaussian noise with 13/255 and a 25/255. As shown in [3] the same baseline methods are analyzed and C2S consistently achieves the best or comparable results among self-supervised methods. On PDFS with 13/255, Recornppted2Rcoorpptee achieves a slightly higher PSNR (30.95 dB vs. 30.91 dB); however, C2S records the highest SSIM (0.756), indicating better detail preservation. It is worth noting that although the labels in this simulated dataset do not have added synthetic noise, they still contain inherent noise typical in MRI, albeit with higher SNR. Figure 3 demonstrates that our method balances feature preservation and noise removal, resulting in much cleaner visual representations compared to other methods. For additional results on fastMRI, refer to Appendix E
We assessed the effect of reparameterization on training stability and performance by mapping the noise levels E (0. 'i to new scale for more uniform sampling. As shown in Table 4a reparameterization improves PSNR and SSIM across all contrasts. The training dynamics, illustrated Figure 12 confirm the stabilizing effect of reparameterization. The model with reparameterization (blue) shows smoother and faster convergence than the model without (orange), which fluctuates more and converges slower.
M4Raw PSNR 1 SSIM
fastMRI PSNR
Architecture
T1
T2
FLAIR
SSIM 1
Method
PSNR
SSIM f
PSNR
SSIM f
PSNR
SSIM 1
U-Net
33.11
0.865
32.32
0.807
Without Repuram.
31.14
0.837
30.53
0.807
30.43
0.771
DDPM
34,82
0.886
33.48
0.835
With Reparam.
34.43
0.882
33,82
0.860
32,56
0.814
Ours
34.91
0.890
33,63
0.837
(a) Impact of reparameterization of noise levels on the M4Raw dataset. Results are validation results obtained after training for 200 epochs.
(b) Influence of model architecture on the
M4Raw dataset (T1) and fastMRI dataset (PD).
Table 4: Ablation studies on the impact of reparameterization and model architecture.
We evaluated the impact of different architectural choices on the performance of our denoising model. As shown in Table 4bb incorporating time conditioning significantly improves both PSNR and SSIM across the M4Raw (TI contrast) and fastMRI (PD contrast, noise level 13/255) datasets. The best performance is achieved by further incorporating the NVC-MSA module in our model (Appendix C). which allows the model dynamically adapt to varying noise levels by integrating noise variance into the self-attention mechanism.
Our approach demonstrates strong robustness to noise level estimation errors, making it suitable for prac- tical applications where exact noise levels may be unknown. Through extensive experiments detailed in Appendix H we show that C2S maintains stable performance even with significant estimation errors (+50% of the true noise level). This robustness, combined with the incorporation of standard noise estimation

---

Published as conference paper ICLR 2025
techniques (e.g., from the skimage package Van der Walt et al. (2014)), enables our method effectively function as blind denoising model. In practice, we find such noise estimation tools provide sufficiently accurate estimates for optimal model performance, alleviating the need for precise noise level knowledge. More quantitative results and analysis of the effect of noise estimation error are provided in Appendix H
Extending C2S to Multi-Contrast Settings MRI typically involves acquiring multiple contrasts to provide comprehensive diagnostic information. By leveraging complementary information from different contrasts, denoising performance can be further enhanced. To capitalize on this, we extend the C2S framework to multi-contrast settings by incorporating additional MRI contrasts inputs. Figure demoonttattss the visual comparison of different denoising methods for the T1 contrast on the M4Raw dataset. It is evident that using multi-contrast inputs (TI & T2, TI & FLAIR) allows for better structural preservation and detailed reconstructions compared to single-contrast denoising techniques. Quantitatively, Table shows that multi-contrast C2S consistently outperforms classical BM3D, supervised Noise2Noise, and single-contrast C2S in terms of PSNR and SSIM. More details can be found in Appendix D
Target Contrast PSNR/ SSIM f
Best Classical
Best Supervised Noise2Noise
Best Self-Supervised
Multi-Contrast C2S
BM3D
C2S
T1 T2
FLAIR & T1
T2 & FLAIR
32.07 0.903
0.911
0.919
33.57 0..21
33.89 0,922
T2 FLAIR
31.20/ 0.877
32.37 0.886
32.33/ 0.890
33.01 00.95
N/A
33.36 0.901
0.874
32.70/ 0.871
32.51/0.876
N/A
32.71 0.879
32.62 0.877
Table 5: Multi-contrast denoising results on the M4Raw dataset. For the multi-contrast C2S results, entries such as "TI & T2" indicate that T1 and T2 contrasts were used as inputs for denoising the target contrast.
Hotay
Malti Avg. Labe!
BN33
Swislft
NobscHuuse
C25
T2 PLAIR
Figure 4: Comparison of different denoising methods for TI contrast in the M4Raw dataset. The figure showcases Noisy. BM3D, Noise2Noise, and C2S, along with multi-contrast C2S variants (TI T2, TI FLAIR). Multi-contrast C2S preserves more structural details and produces sharper reconstructions.
5 CONCLUSION
We have introduced Corruption2Self, score-based self- supervised denoising framework tailored for MRI applications. By extending denoising score matching to the ambient noise setting through our Generalized Denoising Score Matching approach, C2S enables effective learning directly from noisy observations without the need for clean labels. Our method incorporates a reparameterization of noise levels stabilize training and enhance convergence, well detail refinement extension to balance noise reduction with the preservation of fine spatial features. By extending C2S to multi-contrast settings, we further leverage complementary information across different MRI contrasts, leading enhanced denoising performance. Notably, C2S exhibits superior robustness across varying noise conditions and MRI contrasts, highlighting its potential for broader applicability in clinical settings.
10

---

Published as a conference paper ICLR 2025
ACKNOWLEDGEMENTS
This research was partially supported by the following fundings: NSF-CBET-1144244 and NIH- R35GM142969. The authors would also like to thank Ruiyang Zhao and Yizun Wang for the helpful discussions.
REFERENCES
Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In Proceedings of the 36th International Conference on Machine Learning, volume 97, pp. 524-533, 2019.
Giannis Daras, Alexandros G Dimakis, and Constantinos Daskalakis. Consistent diffusion meets tweedie: Training exact ambient diffusion models with noisy data. arXiv preprini arXiv:2404.10177 2024.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Un- terthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words:
Transformers for image recognition scale. arXiv preprint arXiv:2010.11923 2020.
Shreyas Fadnavis, Joshua Batson, and Eleftherios Garyfallidis, Patch2self: Denoising diffusion mri with self-supervised learning. Advances in Neural Information Processing Systems, 33:16293-16303, 2020.
Chun-Mei Feng, Huazhu Fu, Shuhao Yuan, and Yong Xu. Multi-contrast mri super-rcsolution via multi- stage integration network. In Medical Image Computing and Computer Assisted Intervention-MICAA 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part VI 24, pp. 140-149. Springer, 2021.
Alessandro Foi. Noise estimation and removal in mr imaging: The variance-stabiliiztton approach. In 201 IEEE International symposium on biomedical imaging: from nano to macro, pp. 1809-1814. IEEE, 2011.
Jacques Froment. Parameter-free fast pixelwise non-local means denoising. Image Processing On Line, 4: 300-326, 2014.
Hákon Gudbjartsson and Samuel Patz. The rician distribution of noisy mri data. Magnetic resonance in medicine, 34(6)9110-914, 1995.
Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, and Arash Vahdat. Diffit: Diffusion vision transform- ers for image generation. arXiv preprint arXiv:2312.021333 2023.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbor2neighbor: Self-supervised denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 14781-14790, 2021.
Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Leurning Reseurch, 6(4), 2005.
Hyemi Jang, Junsung Park, Dahuin Jung, Jaihyun Lew, Ho Bae, and Sungroh Yoon. Puca: patch-unshuflle and channel attention for enhanced self-supervised image denoising. Advances in Neural Information Processing Systems, 36, 2024.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35:26565-2657 2022.
11

---

Published as a conference paper ICLR 2025
Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedic's approach self-supervised image denoising without clean images. Advances in Neural Information Processing Systems, 34:864- 874,-87,4408 2021.
Kwanyoung Kim, Shakarim Soltanayev, and Se Young Chun. Unsupervised training of denoisers for low-dose ct reconstruction without full-dose ground truth. IEEE Journal of Selected Topics in Signal Processing, 14 (6):1112-1125,, 2020.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems, 34:2169-22177, 2021.
Diederik P Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014.
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-learning denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2129-2137, 2019.
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. International Conference on Machine Learning, pp. 2965-2974. PMLR, 2018.
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Lue Van Gool, and Radu Timofte. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference computer vision, pp. 1833-1844 1844, 2021.
Zhi-Pei Liang and Paul C Lauterbur. Principles of magnetic resonance imaging. SPIE Optical Engineering Press Bellingham, 2000.
Jae Hyun Lim, Aaron Courville, Christopher Pal, and Chin-Wei Huang. Ar-dae: towards unbiased neural entropy gradient estimation. In International Conference on Machine Learning, pp. 6061-6071, PMLR, 2020.
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022, 2021.
Mengye Lyu, Lifeng Mei, Shoujin Huang, Sixing Liu, Yi Li, Kexin Yang. Yilong Liu, Yu Dong, Linzheng Dong, and Ed X Wu. M4raw: A multi-contrast, multi-repetition, multi-channel mri k-space dataset for low-field mri research. Scientific Data, 10(1):264, 2023.
Ymir Mäkinen, Lucio Azzari, and Alessandro Foi. Collahorative filtering of correlated noise: Exact transform- domain variance for improved shrinkage and patch matching. IEEE Transactions on Image Processing, 29: 8339-8354, 2020.
Nick Moran, Dan Schmidt, Yu Zhong, and Patrick Coady. Noisier2noise: Learning to denoise from unpaired noisy data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12064-12072, 2020.
Marc Moreno López, Joshua M Frederick, and Jonathan Ventura. Evaluation of mri denoising methods using unsupervised learning. Frontiers in Artificial Intelligence, 4:642731, 2021.
Tongyao Pang. Huan Zheng, Yuhui Quan, and Hui Ji. Recorrupted-o--rcoommppp unsupervised deep learning for image denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24522205,, 2021.
12

---

Published as a conference paper ICLR 2025
Juhyung Park, Dongwon Park, Hyeong-Geol Shin, Eun-Jung Choi, Hongjun An, Minjun Kim, Dongmyung Shin, Se Young Chun, and Jongho Lee. Coil2coil: Self-superviser mr image denoising using phased-array coil images. arXiv preprint arXiv:2208.07555 2022.
Laura Pfaff, Julian Hossbach, Elisabeth Preuhs, Fabian Wagner, Silvia Arroyo Camejo, Stephan Kan- nengiesser, Dominik Nickel, Tobias Wuerfl, and Andreas Maier. Self-supervieed mri denoising: leveraging stein's 's unbiased risk estimator and spatially resolved noise maps. Scientific Reports, 13(1):22633 2023.
Shakarim Soltanayev and Se Young Chun. Training deep learning based denoisers without ground truth data.
Advances neural information processing systems, 31, 2018.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 2020.
Stefan Van der Walt, Johannes L Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D Warner, Neil Yager, Emmanuelle Gouillart, and Tony Yu. scikit-image: image processing in python. PeerJ, 2:e453, 2014.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23 (7):1661-1674, 2011.
Zichun Wang, Ying Fu, Ji Liu, and Yulun Zhang. Lg-bpn: Local and global blind-patch network for self- supervised weal-world denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18156-18165, 2023.
Chenxu Wu, Qingpeng Kong. Zihang Jiang, and S Kevin Zhou. Self-supervised diffusion MRI denoising via iterative and stable refinement. In The Thirteenth International Conference on Learning Representations, 2025. URL httth:tt: / www.:eewwwwwwwwe...eeeeeeeeeeeee wwt.erorwrmmiaddeeapeeaee-ee-
Tiange Xiang, Mahmut Yurt, Ali Syed, Kawin Setsompop, and Akshay Chaudhari. ddm Self-supervised diffusion mri denoising with generative diffusion models. arXiv preprint arXiv:2302.03011. 2023.
Yaochen Xie, Zhengyang Wang, and Shuiwang Ji. Noise2same: Optimizing self supervied bound for image denoising. Advances in neural information processing systems, 33:20320-2085 203300, 2020.
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming- Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5728-5739, 2022.
Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew J Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, et al. fastmri: An open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.00
2018.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 26(7):3142-3155 2017.
13

---

Published as conference paper ICLR 2025
A ADDITIONAL EXPERIMENTAL DETAILS
A.1 DATASETS
We evaluated our Corruption2Self (C2S) method on two distinct datasets:
In-vivo Dataset (M4Raw): The M4Raw dataset Lyu et al. (2023) contains multi-channel k-space images across three contrasts (TI-weighted, T2-weighted, and FLAIR) from 183 participants. The dataset was partitioned into training (128 individuals, 6,912 slices), validation (30 individuals, 1,620 slices), and testing (25 individuals, 1,350 slices) subsets. For training and validation, pseudo-grous truth labels were generated by averaging three repetitions for TI/T2-weighted contrasts and two repetitions for FLAIR. Test data utilized higher-SNR labels derived from averaging six repetitions (T1/T2-weighted) and four repetitions (FLAIR), enabling assessment of model generalization to cleaner data. The dataset (20.7GB) is distributed under CC-BY-4.0 license.
Simulated Dataset (fastMRI): We utilized single-coil knee data from the fastMRI dataset Zbontar et
(2018), selecting patient entries matching those in MeNet[Feng et al. (202T) to ensure contrast correspondence. The dataset was structured into training (180 pairs, 6,648 slices), validation (47 pairs, 1,684 slices), and testing (45 pairs, 1,665 slices) subsets. White Gaussian noise with a E [13,25] was introduced to simulate real-world MRI noise patterns. The dataset (31.5GB) distributed under MIT license.
A.2 TRAINING PARAMETERS
All experiments conducted NVIDIA A6000 GPUs. For the M4Raw dataset, optimization performed using Adam with learning rate x 10-4 and weight decay x 10-4 For the fastMRI dataset, Adam configured with learning rate 1 10-1 and weight decay 5 x 10-2 Critical hyperparameters (learning rate, weight decay, batch size, maximum noise level 7)) were optimized based on validation performance. Early stopping was implemented to prevent overfitting, and final models were selected based on optimal validation metrics before test set evaluation.
A.3 EVALUATION PROTOCOL
To assess the efficacy and robustness of the Corruptio2Seelf (C2S) framework, we employed the following evaluation protocol:
Pseudo-Ground Truth Generation: For the in-vivo dataset (M4Raw), pseudo-ground truth labels were generated by averaging the multi-repetition images.
Image Quality Metrics: The quality of the denoised images was quantified using two metrics:
Peak Signal-to-Noise Ratio (PSNR): This metric measures the ratio between the maximum possible power of a signal and the power of corrupting noise, assessing the quality of the denoised image against the reference label.
Structural Similarity Index Measure (SSIM): This metric evaluates the structural similarity between the denoised image and the reference, focusing on luminance, contrast, and structure.
B THEORETICAL RESULTS
Lemma 3. Given the objective sujjectivefunction J(0):
J(0)= = Exx
F2
Fx ... (X,,)) - 11
14

---

Published as conference paper ICLR 2025
where X. (X of (t)I), the function ga (X,, 8) that minimizes J0) is the conditional expectation:
g (X,, t) E ( X₂}
Proof. Our goal is to find go (Xt, 1) that minimizes the objective function J (0). Since the outer expectations over X tn and do not affect the minimization with respect 0, we focus minimizing the inner expectation:
E Ex,|X XX am
|| (x,, t) X Xram
For fixed X, and the optimal function ge- (X,,t) minimizes the expected squared error:
min E Ex |lgo (X,,t)
Ko
According to estimation theory, the function that minimizes this expected squared error is the conditional expectation of Xia given X,:
Bo-
(x,()
EE
(X X,]
Therefore, the function ge- (X,,t) E X Aiee
X, minimizes the objective function J(O).
Theorem (Generalized Denoising Score Matching: restated Theorem D. Let the following assumptions hold:
1. Data Distribution: The clean data vector Xo E TRd is distributed according to po (xo)-
2. Noise Level Functions: The noise level or schedule functions o are strictly positive, scalar-valued funcction of time I, and are monotonically increasing with respect to
3. Noisy Observations at Data Noise Level tdata: The observed data X is given by Xeass
Xo + Orsna Lt where Zi ~ (O,
4. Target Noise Level tnarget S tdota: The target noisy data Xtway defined as X Xo +
Ouang Ztango where Zieups (O,Ia).
5. Higher Noise Levels 2 Idata: For any 2 Idaa the noisy data X, is given by X, Xo + ozZr where Z, (O, Ta).
6. Function Class Expressiveness: The neural network class (ba} } is sufficiently expressive, satisfying the universal upproximation property.
Define the objective function:
(0) Ex.,m xx [ll(t, (tanp) ho (Xe,t) + 8(t, twmgn) X, X 17]
(12)
where is uniformly sampled from (tdmas T) and
o, o
0²
Iurent
(1, (wwge))
8(1, atwnet)
o? of
o² Fanget
Given X and t 2 Edaia X, is sampled as:
X. Xtao
+ No -z oz Zz where Z ~O0,O
(13)
Then, the minimizer 0" of (0) satisfies:
hg- (X,,t) E t X,), VX, E IRd, t 2 tabia
(14)
15

---

Published as conference paper ICLR 2025
Proof. Our goal is to find 0* that minimizes (0). Note that X, can be expressed in terms of both i and X bagge
X, Xtan + Voz oram Zi,
Z ~ La),
X, Luge + Vo7 ouma Zz
Z2 ~ (N),,
Using properties of Gaussian distributions, the score function Vx, log Pr (Xt)
can be written two ways:
X1) X
E
[
Xx
Vx, po
o7
Reass
o
Lunge
Equating these expressions and rearranging terms:
0, tas
0
E Leau
Xx]
E8 (tange
X1]
X
0
o
o Fruzet
Recognizing the coefficients as (t, targe) and 6(t, Orrgga respectively:
E I
Xx] 7(1, trnan E X [ X₂] + 8(1, orna Xt.
Define the auxiliary function:
go (x,() 7(1, ouuaa (Xo:))))) + 5(1, trgan Xt-
Substituting into the loss function J(B), we have:
((0) = E [|Boo (Xe,t) Xeam 12]
This is mean squared error (MSE) objective between ga (X,, () and X.... The objective function becomes:
((0) = Ex 8x E, Xeam
(X,,)X X tsa
By the property of MSE minimization (Lemmal 3.. the function go (Xe, t) that minimizes ((0) satisfies:
go- (X,,t) f) [Xemm X,).
Substituting the earlier expression for E Xim {x(]:
X.] + ō(t, Ounge Xx, (t, tunge Thg (X,,)) t) + ö(t, tunga Xx, = (tt, Oungn E X Aurge
Subtracting ö(t, trana Xx, from both sides:
(tt, o borgs hgg (X,,t) (tt, (tagg EE X Lunn X.].
Since 7t, ouuaa) Ttt, ttuaa):
0 (due to or being strictly increasing and f tdata), can divide both sides by
hg- (X,,t) E X Augge
X,
This completes the proof.
Corollary 5 (Reparameterized Generalized Denoising Score Matching; restated Corollary 2}. Let the
assumptions of the Generalized Denoising Score Matching theorem 4 hold. Additionally, define:
16

---

Published as conference paper ICLR 2025
1. Reparameterization: For & dano define T such that
o² o, Osmm where (0, T and T' is determined by o o7 or recover using the inverse function of 0t, denoted as oi o,:
(15)
Note that given and d
t= (v 07 + Paan
(16)
This inverse fnversefunction exists and is well well-defined due the strictly monotonically increasing property of o
Define the new objective function:
J'( = -x Ex. X [lh' (T, (anger (x,,t, + 8',,, cnange) X Xtam [9]
(17)
where is uniformly sampled from [0, Th and the coefficients are defined as:
o²
olam Fumpe
(T, t
(T, Ttwne)
o7 ossm Towne
o oism O
Given X and t, X, is sampled as:
X X + o
where Z' ZO,,LO.
(18)
Then, the minimizer 0* of (0) satisfies:
he- (xt,X))) Lurger X,), VX, E Rd data-
(19)
Proof. Substituting of into y(t, ohrra and 8(t, trrrye)
from Theorem
we obtain:
o²
2 o o? + oz fugee
(T, otuga) = T(,, otana)
(T Otugel) 8(1, oraggs)
o7 + oi or
Define go (X,,t) = (T, Ouaaa h (), ('(,, Itana )x, The objective function becomes:
J'(0) Ex.s. E, E,Ex,xX....
(Xe, 1) - X
By Lemma 3 the function minimizing J'(() is:
ga- (X,, [Xemm Xx).
From proof of Theorem
we have:
E [ X,] 7/ (5, Turge E [Xtuge X.] + 8'(T, O )X,
Therefore,
ga-(X (x,, t) = (T, tren hg- (x,, t) + 8'(T, Otonp Xx, = (T, Thange E [Xtrpe X,] + 8'(T,
Comparing both expressions, we conclude:
7'(T, terpu hag (X,, t) = 7 (T, Oranpu
E X kanps X(]
Since (T, Iumm) divide both sides by 7T(T, timmp):
hg. (X,,t) E X
X,
This completes the proof.
17

---

Published as conference paper ICLR 2025
B.1 EXTENSION VARIANCE PRESERVING CASE
Our GDSM framework can be naturally extended to the variance preserving (VP) Song al. (2020); Ho et al. case. For example, when Ø turee O, which corresponds to the VP formulation in ADSM Daras et al. In this case, the data model follows:
2020
2024
X Nan
oig Xo + Team Z,
<<< Ctam
<
(20)
Let X. _2 represent clean data, and for any Jiam <0 < 1, the forward corrupted X, is given by:
X, = / o%Xo o, Z, ~ ~(
(21)
Define the objective function:
of
v1 ot X,
(v) Exisn 1,X, t,X,
o ho(X,,t)
-ia of oiam
X Lan
(22)
Then, the minimizer 0* of Jvp(O) satisfies:
hg- (X,X 1) -| X.),
VX, Rd 8 12 data
(23)
B.2 CONNECTION TO NOISIER2NOISE
We demonstrate that Noisier2Noise Moran et al. (2020) emerges as special case of our Generalized Denoising Score Matching (GDSM) framework under specific conditions. Let us establish the correspondence between notations: in Noisier2Noise, X represents the clean image, Y X + N represents the noisy observation with noise N, and Z Y + M represents the doubly-noisy image with additional synthetic noise M. These correspond formulation where X is Xo, Y is X and Z is X,.
From proof of Theorem when setting Oregg = 0, we obtain:
0
o
tan
? of
IE
X
Xo
X,)
X,
(24)
o
In the improved variant of Noise, parameter controls the magnitude of synthetic noise M relative to the original noise N. We can establish that this parameter corresponds to our noise schedule through:
a² of
(25)
Under this relationship, Equation equation
24 becomes equivalent to the Noisier2Noise formulation:
a² I I+a a
E[XiZ|
(26)
I a
This equivalence leads to the characteristic Noisier2Noise correction formula:
(1 + a²
[[X
(27)
a²
18

---

Published as a conference paper ICLR 2025
In the standard case where a 1, this reduces [[ 2 [4] Z, which corresponds to our framework with of = =0 fam
Our GDSM framework offers several advances over Noisier2Noise. First, it provides continuous noise schedule through allowing the model to learn from spectrum of noise levels rather than fixed ratio determined by a.. Second, it introduces explicit time conditioning the network architecture, enabling better adaptation to different noise magnitudes. Third, and perhaps most importantly, it eliminates the need to tune the a parameter, which according to Moran et al. (2020) "difficult or impossible derive the absence of clean validation data." Instead, our approach automatically learns to handle different noise levels through the continuous schedule and time conditioning. Furthermore, GDSM extends beyond the clean image prediction task by supporting arbitrary target noise levels through Uuger providing unified framework for various denoising objectives.
C MODEL ARCHITECTURE
this appendix, we provide a comprehensive description of the architectures employed for both single- contrast and multi-contrast MRI denoising. Our designs build upon the U-Net structure utilized in Denoising Diffusion Probabilistic Models (DDPM) (o et al. (2020), incorporating advanced conditioning and attention mechanisms to enhance performance. For detailed implementation of the Noise Variance Conditioned Multi-Head Self-Attention (NVC-MSA) module, please refer to Appendix C.2
C.I SINGLE-CONTRAST MODEL ARCHITECTURE
Our single-contrast denoising model employs U-Net backhone augmented with time conditioning and the NVC-MSA module Hatamizadeh et al. (2023).
Time Conditioning: The model adapts its processing based on the noise level by integrating time em- beddings into the convolutional layers. This is achieved through adaptive normalization (e.g.. instance normalization followed by an affine transformation conditioned on the time embedding), as introduced DDPM.
NYC-MSA Module:
To enable the network to adjust to varying noise levels, we incorporate the NVC-MSA module into the self-attention mechanisms of the U-Net. The module conditions the attention the current noise variance, allowing the network to effectively capture long-range dependencies and adapt to different noise scales. Mathematically, the queries, keys, and values computed
Q (x(X) + (t((),
(28)
K W Wx(X) bx((),
(29)
V (x(X) + (1),
(30)
where (t), bx (t), and by (t) are learned affine transformations of the time embedding. This NVC-MSA mechanism allows the attention modules to aware of the noise level and adjust their focus accordingly, effectively capturing long-range dependencies. The implementation details and pseudo-code for the NVC-
MSA module are provided Appendix C.2
C.2 NOISE VARIANCE CONDITIONED MULTI-HEAD SELF-ATTENTION (NVC-MSA) PSEUDO CODE
Below is the code snippet of the NVC-MSA module, which is integral to both single-contrast and multi- contrast model architectures. This module conditions the self-attention mechanism on the noise variance level, enabling the model to adapt its attention based on the current noise level.
19

---

Published as a conference paper ICLR 2025
The code snippet encapsulates the core functionalityy of the NVC-MSA module. The module first normalizes the input tensor and generates queries, keys, and values for spatial tokens. It then reshapes and projects the noise variance embeddings using Ixl convolutions. These coise-conditioned components are added the queries, keys, and values before applying the attention mechanism. Finally, the output is rearranged and projected to produce the final feature map.
20

---

Published as conference paper ICLR 2025
def (selff, x, noise_emb)
# Get shape of input tensor b, e, h w x. shape
n
# Normalize the input tensor self.norm(x)
# Generate queries, keys, and values for spatial tokens gkv - self.te_qkvit (x) chhnk dim=1)
9, k, map (lambda t: rearrange (t, "b (h d) -> b (x h d', h-self. heads), qkv)
# Reshape and project noise variance embeddings using 1x1 convolutions noise_end emb
(b, -1, 1, 1) (ncise_emb) noise_k nelf.noise_key_cony (ncise_emb) noise_v = self.noise_value_ccom (noise_emb)
noise_q colf.noise_querr
# Rearrange the projected noise variance embeddi nga
noise_q rearrange (noise_q, se_q, "b
v
(x y) h h-sel f. heads) (eedd))
noise rearrange (noise k, "b (h x v V. 8
(x y) h d', b=self heads!
noise_v v = rearrange (noise_ (b (h d)
x d
V. 6
(x y) h d', h=self. heads)
I Add noise variance-dependent components to queries, keys, and values
q q noise_q
k k noise_k
v v ncise_v
Apply attention mechanism out self. k, v)
Rearrange and project the output
out - rearrange (out, fb (x h -> b (h d) y', x-h, y-w)
return self aalftt_oour (out)
Listing I: Code snippet for NVC-MSA implementation
MULTTICCONTRSS C2S
D.1 MULTI-CONTRAST MODEL ARCHITECTURE
The multi-contrast denoising model extends the single-contrast architecture handle multiple input contrasts, thereby enhancing denoising performance by leveraging complementary information.
Multicontrast Fusion: The model accepts multiple contrast inputs concatenating the primary and complementary contrast images (e.g., T2). An initial convolution layer extracts feature embeddings from the fused contrasts, which then processed through the U-Net architecture.
NVC-MSA Module: Similar to the single-contrast model, the multi-contrast model integrates the NVC-MSA module into its self self-attention mechanisms. By conditioning the attention the noise variance level the
21

---

Published as conference paper ICLR 2025
model can adaptively adjust its focus based on the current noise level, as detailed in the pseudo-code provided in Appendix C.2
Output Head:
Following the U-Net processing, the output head generates single-channel image representing the denoised primary contrast image.
The architecture can dynamically adjust accommodate any number of input contrasts by modifying the input layer accordingly. Although our current implementation is based on U-Net, the NVC-MSA mechanism is compatible with other architectures, such as Vision Transformers Dosovitskiy et al. (2020) Liu et al. (202T), where it can replace standard multi-head self-attention modules to enhance model complexity and performance. This extension remains an avenue for future research.
Flexibility and Extensions:
D.2 MULTI-CONTRAST C2S ALGORITIM
The multi- contiitt C2S framework extends the Angle-contrass algorithm to leverage complementary informa- tion from auxiliary contrast images. Given collection of noisy multi-contrast training images
((Kiso, C.)
where X E R is the noisy target contrast image and C, E Rd represents e auxiliary noisy contrast images, goal is to estimate the clean target contrast image X₂ using both X and C. In the multi- contrast setting, we focus on the case where farget = (i.e., Tamps = 0), aiming to directly estimate the MMSE estimator Xo Xim.. CJ. While the single-contrast C2S incorporates a detail refinement extension with non-zero target noise level, we leave the exploration of such extensions and more advanced contrast fusion architectures for multi-contrast C2S as future work. For the implementation of the denoising function Do(X, C), the conditioning auxiliary contrasts C is achieved through CNN encoder architecture that extracts features from each auxiliary contrast image. These extracted features are then concatenated with the features from the target contrast image in the feature space, allowing the model effectively integrate complementary information from all available contrasts. The concatenated features are subsequently processed through the U-Net backbone with NVC-MSA modules, as in the single-contrast case. Following the reparameterization strategy introduced in Section 3.1, we define reparameterized function Do(X,, C)
as:
D& (X,, C)= >Tuu(TT Ougea) he (X,,t C) (T, Otage Xx,
(31)
where X, = Xtym + o. with Z ~ (00 Iz), and the coefficients remain consistent with the single-contrast
case:
0²
2 oanne
Aoa((T Otuga)
Askip (T, Ituan)
(32)
o²
o
? ora Piune
The multi-contrast C2S loss function is then formulated as:
LMC (2)())
Ex ien Peaa (x) (((T) Do(XX, (C) X
(33)
2
C-p(c)
(0.a)
The complete training procedure for multi-contrast C2S is outlined in Algorithm
22

---

Published as conference paper ICLR 2025
Algorithm 2 Multi-Contrast Corruption2Sel/Trainne Procedure
Require: Noisy multi-contrast dataset {(Xi Fan C') hg, max noise level T, batch size m, total iterations K, noise schedule function d+
1: for k 1 to K do
Sample minibatch (XXt C')] ~ ~2(00 (O)
o? dapp x? =2t0tan
Compute )mer(T, am
Recover = oo (V07 + +
5: Compute X, + Xian +0,
6: Compute loss: L = 2m Di w(T)
|| AOa (T, 0) be (X, C²) + (T) X Update I using Adam optimizer minimize L
During inference, given a noisy target contrast observation X and auxiliary contrasts C, the denoised output is obtained by:
X h1g- (X data
C)
(34)
where the trained model hg- approximates E[X, X tym CJ. providing clean estimate of the target contrast image that benefits from the complementary information in the auxiliary contrasts.
E ADDITIONAL RESULTS ON FASTMRI
In this section, we present additional results the fastMRI dataset, evaluating the performance of various denoising methods across different noise levels and contrasts.
Table 6 summarizes the performance comparison between the baseline method (Without Reparameterization) and our proposed Corruption2Self (C2S) approach, under four configurations: PD with o 13 255, PDFS with 13/255, PD with 25/255, and PDFS with 25/255.
Method
13/255 PSNR 1 SSIM
13/255 PSNR 1/ SSIM
25 25/255
PDFS, 25, /25 PSNR 1 SSIM 1
PSNR /SIM
Without Reparam.
32.65 / 0.821
30.08 / 0.676
30.16/
0.550 28.62 0.601
With Repuram.
33.48 / 0.832
30.67 / 0.761
30.91 0.756
Table 6: Impact of reparameterization of noise levels on the fastMRI dataset. The "Without Reparam." contains estimated baseline results, while "With Reparam." represents the proposed method with reparameterization.
ADDITIONAL RESULTS ON M4RAW
F. VISUAL COMPARISON OF DENOISING METHODS
this section, we provide a visual comparison of several denoising methods applied to T1, T2, and FLAIR contrast images in the M4Raw dataset. The denoising methods evaluated include Noise2Noise, BM3D, SwinIR, R2R, 2elf, and C2S, with Multi-repetition Averaged Label serving as the ground truth reference for comparison.
23

---

Published as a conference paper ICLR 2025
horey
PAMER
Figure 5: Comparison of different denoising methods for PD contrast (noise level 13/255) in fastMRI.
Figure 6: Comparison of different denoising methods for PDFS contrast (noise level 25/255) in fastMRI.
24

---

Published as conference paper ICLR 2025
holey
SAIRR
Figure 7: Comparison of different denoising methods for Tl contrast in M4Raw. The top row shows the original noisy image and results from Noise2Noise, BM3D, and SwinIR. The bottom row includes the multi-repetition averaged label, R2R, Noise2Self, and C2S methods. A zoomed-in section of each image is presented below each corresponding brain image for detailed comparison.
Heday
Mutti-ep. Aug. Label
BM33
Rolss2Self
RZR
Swiriitt
Nelsee2oolse
12:2710817
SLA7TERTS
12.3710113
Figure 8: Top: Comparison of different denoising methods for T1 contrast in M4Raw. Bottom: Comparison of different denoising methods for T2 contrast in M4Raw.
25

---

Published as conference paper ICLR 2025
holey
SAIRR
Figure 9: Comparison of different denoising methods for FLAIR contrast in M4Raw.
Figure 10: Comparison of different denoising methods for FLAIR contrast in M4Raw.
F.2 EVALUATION ON MATCHING-SNR TEST LABELS
When evaluated on test data matching the SNR of the training data, supervised methods like SwinIR and Restormer achieve the best performance, with PSNR improvements over self-supervised methods. This indicates that supervised approaches excel when the training and testing conditions are similar. However, the performance gap narrows when considering C2S, which attains PSNR and SSIM values comparable to those of the supervised methods.
These observations highlight a limitation of supervised methods: they may not generalize well scenarios where the test data has different noise characteristics higher SNR than the training data. In contrast, C2S demonstrates robust performance across different SNR levels, achieving competitive results on both Matching-SNR and higher-SNR test labels. This suggests that self-supervised approach is resilient to variations noise levels and better generalizes to cleaner images without relying on clean ground truth during training.
26

---

Published as conference paper ICLR 2025
Methods
TI PSNR SSIM f
T2
FLAIR PSNR SSIM 1
PSNR SSIM f
Classical Non-Learning-Bell Based Methods
NLM Froment (2014)
34.65 0.897 35.27 / 0.900
33.72/0.873
32.83 0.830 33.22 / 0.841
BM3D Makinnn et al. 2020
34.01 / 00875
Supervised Learning Methods
SwinlR Liang et al. (2021)
36,09 / 0.926 35.96 0.926 34.82 / 0.892
34.57 0.902
34.34 0.909
Restormer Zamir al. (2022
34.13 0.898
34.21 / 0.908
Noise2Noise Lehtinen et al. 2018
33.92 / 0.861
33.73 0.879
Self-Supervised Single-Contres Methods
2 Krull et al. (2019)
32.83 / 0.870
31.73 0.857
30.90 0.821
Noise2Self Batson & Royer (2019)
34.17 0.883
32.64 0.847
31,96 / 0.823
Recorrupted22Kecrmmupp Pang al. 2021
33,60 0.801
32.93 0.820
32.42 0.794
C2S (Ours)
36.11 0.925
34.87 0.904
34.15 0.898
Table 7: Quantitative results on the M4Raw dataset evaluated on thrre-repetition-wveree (matching training and validation SNR). Mean PSNR and SSIM metrics are reported. The best results among self-supervised methods are in bold.
labels
G DETAIL REFINEMENT ALGORITHM
In this appendix, we present the Detail Refinement algorithm an extension to the primary Corruption2Self (C2S) training procedure. While the primary stage focuses on learning the conditional expectation E[X, X,] for maximum denoising, this aggressive approach may lead to the loss of fine details and important features. The Detail Refinement stage addresses this issue by training the network to predict E E[X, i X,] with a non- zero target noise level Iugge > 0, allowing the preservation of intricate structures and textures. Empirically, we discovered that uniformly sampling furget from the interval [0, [ams] during training already yields good results and leave more advanced tuning process to future work. This sampling strategy provides good balance between noise reduction and detail preservation, allowing the network to learn a range of refinement levels adaptively.
G.1 Loss FUNCTION FOR DETAIL REFINEMENT
The loss function for the Detail Refinement stage similar the primary C2S stage but introduces target noise level 0 busges The denoised output D& (X,,7) trene) is defined as:
Dg(X: T, Trage) >oun tmgge) (X,, 1) + (T, Otarge) X,,
where ho (x,,,))) t) is the denoising network parameterized by 0, with input X, and noise level t, and Aoaa (T, tuga and )aki((T, otuge) are blending factors between the network output and the noisy input.
The loss function is given by:
w(T) Da (x, otuna)) X 112]
Lrefine (0) = Xia. Perse (x)
(35)
[0
targe (0,
Z (0,L
where:
27

---

Published as conference paper ICLR 2025
Xtam Plas (x) is the noisy observed data.
20 T is the reparameterized noise level uniformly sampled from [0, where T is the maximum noise level.
Ourge ~ U0, taaml is the target noise level, sampled uniformly from the interval (0, Otaml-
X
X Karn + o.Z, where Z ~ NO Ia) is Gaussian noise.
Aour(T (T, Irana)
and Askip (T, traan)) are defined as:
o
Xour tange)
Askip Otage)
(36)
0²
tan
Suree
0
2 Oarge
The goal of this refinement stage is retain controlled amount of noise, preventing the loss of fine details and features while still performing denoising.
G.2 ALGORITHM IMPLEMENTATION
The Detail Refinement training procedure minimizes the loss function Crefine (0) over the network parameters
0. The steps are follows:
Algorithm 3 Detail Refinement Training Procedure
Require: Noisy dataset {x: )im denoising network hg, max noise level T, batch size m, total iterations Kyeline
noise schedule function
1: for lc = 1 to Keeinn do
Sample minibatch (X 1m
u(0, NO
Sample a turps Ud(0, dtam
Compute Aoua (T, Ooapp
o²
07
kan aan
-a Longa
Compute Xakp Otampa)
Pune
+o Faaes
Recover = o, (V07 + 2
7: Compute X; + X. sau + o,
8: Compute loss: L = Zm Di= w(T) Do X., tugge) X ym
Update 0 using Adam optimizer Kingma 2014) to minimize L
H ROBUSTNESS TO NOISE LEVEL ESTIMATION ERROR
In practical applications, precise noise level estimation be challenging, making robustness to estimation errors a crucial property for denoising models. This section provides a comprehensive analysis of our model's performance under various degrees of noise level misestimation, demonstrating its capability function effectively as a blind denoising model.
H.1 EXPERIMENTAL SETUP AND RESULTS
We evaluate our pretrained model's ' robustness by systematically varying the input noise level estimation fama from -50% (underestimation) +50% (overestimation) of the true noise level. Table 8 presents the quantitative results across T1, T2, and FLAIR contrasts on the M4Raw test set, while Figure visualizes these trends.
28

---

Published as conference paper ICLR 2025
Noise Level Estimation
M4Raw Dataset (PSNR SSIM 1)
T1
T2
FLAIR
-50%
32.6004 0.9154
32.3210 / 0.8890
32.9496 0.8757
40%
32.5958/ 009958/0.115
32.3125 / 0.8888
32.9498 / 008759
-30%
32.5910/0.915
32.3044 0.8886
32.9478 00.8761
-0%%
1.586610.9115
32.2964 0.8884
32.9436 / 0.8762
-10%
92.5810/099149
32.2882 / 0.8881
32.9373 / 0.8763
0%
32.5758 $2.5758/0.9148 0.91448
32.2812 / 0.8879
32.9297 / 0.8774
+ +10%
32.5704 17700/00.911 0.91447
32.2700 / 0.8876
32,9192 0.8763
+20%
32.5649 0.9144
32,2593 0.8873
32.9076 0.8763
+30%
32.5592 / 0,9145
32.2468
008870
32.8947 / 0.8766
+40%
32.5535 0.9143
32.2322 / 0.8866
32.8805 0.8760
+50%
32.5475 / 0.9142
32.2151 0.8862
32.8655 0.8759
Table Performance analysis under varying noise level estimations on the M4Raw dataset. The model demonstrates remarkable stability across all contrasts, with particularly strong performance under slight underestimation. The and indicate underestimation and overestimation of the noise level, respectively.
P5NR vs. Nolse Level Estimation Emrr
SSIM Noise Level Estimation Error
0.910
(68)
0,900
T1 SSM
0.895
T2 55MM
FLAIR 55M
TI PSUR
0.890
FSNR
FLAIR PSNR
0.885
32.3
32.2
0.875
-40
-20
20
40
-40
-20
20
Neise Level Estimation Error (%)
Noise Level Estimation Error (%)
Figure 11: Visualization of model performance under varying noise level estimations. The plots demonstrate consistent stability across all contrasts, with minimal performance degradation even under significant estima- tion emror (+50%).
H.2 ANALYSIS AND DISCUSSION
Our experiments reveal several key findings: Overall Stability: The model maintains remarkably stable performance across all tested estimation errors, with maximum PSNR variations of only 0.053 dB, 0.106 dB, and 0.084 dB for TI, T2, and FLAIR contrasts, respectively. Asymmetric Response: Interestingly, the model shows slightly better performance under noise level underestimation compared overestimation. For instance, with TI contrast, a 50% underestimation achieves a PSNR of 32.6004 dB, outperforming both the noise level (32.5758 dB) and 50% overestimation (32.5475 dB). While all contrasts exhibit robust performance, the degree of stability varies. Tl shows the most stable response, while demonstrates slightly higher sensitivity estimation errors.
29

---

Published as conference paper ICLR 2025
M4Raw T1
fastMRI PD25
T
PSNR
SSIM t
PSNR
SSIM
3
33.30
0.869
30.58
0.740
5
33.91
0.879
31.01
0.765
34.91
0.890
30.41
0.751
15
34.35
0.882
30.42
0.743
20
34.43
0.882
30.52
0.755
Table 9: Influence of maximum corruption level T on the M4Raw validation dataset.
H.3 PRACTICAL IMPLEMENTATION
our implementation, we utilize the sk skimage package Van der Walt al. (2014) for noise level estimation, which proves sufficient for optimal performance. The model's demonstrated robustness suggests that even relatively simple estimation techniques can provide adequate noise level approximations for effective denois- ing. Our findings reveal significant practical advantages: the model readily adapts to real-world scenarios where exact noise levels are unknown, while standard noise estimation tools consistently deliver near-optimal performance. Furthermore, the observed slight preference for underestimation indicates that conservative noise level estimates may be advantageous in practice.
While future work could explore more sophisticated noise estimation techniques, particularly for extreme cases, our current results demonstrate that the model's inherent robustness already makes it highly practical for real-world applications. This robustness, combined with the effectiveness of standard noise estimation tools. enables our approach to function reliably as blind denoising model, requiring minimal assumptions about the underlying noise characteristics.
I ADDITIONAL ABLATION STUDIES
1.1 EFFECTIVENESS OF MAXIMUM CORRUPTION LEVEL T
We also analyzed the impact of the maximum corruption level T. All models were trained for 300 epochs with the same hyperparameters. As shown in Table performance generally improves with higher T, peaking 10 for the M4Raw dataset (TI contrast). For the fastMRI dataset (PD contrast, 25/255 noise level), the best performance occurs at T'' = 5. These results indicate that while increasing T' generally benefits performance, excessively high corruption levels may not lead to further improvements within the given training budget and could require longer training times to converge.
I.2 EFFECTIVENESS OF REPARAMETERIZATION AND EMA ON TRAINING DYNAMICS
In our framework, we leverage reparamcterization strategy to address the challenges associated with noise level sampling. The theoretical foundation of GDSM (see Theorem 1) requires sampling noise levels > daaaa However, directly sampling t ~ (f T can be sensitive to errors in noise level estimation. To mitigate this, we instead sample an auxiliary variable T (0, T) and define the corresponding variance via
o
We now describe two key perspectives that elucidate why this reparameterization improves the training process:
30

---

Published as conference paper ICLR 2025
Stable Noise Level Sampling. By recovering the original noise level through the transformation
1= o.
(V pham
the additive term of ensures that the recovered remains valid even when due is underestimated. This robustness contrasts with the direct sampling approach, where inaccuracies in estimating ddtta would directly affect the noise level range.
Consistent Coverage Across Data Samples. Sampling т from fixed interval (0, guarantees that the entire noise level range is consistently covered during training, irrespective individual sample noise characteristics. Although approximate T z= T for practical implementation, the ixed-range sampling of T avoids the variability inherent in the direct sampling of t (which would otherwise depend on each sample's s noise level). This consistency is particularly beneficial for datasets with heterogeneous noise levels, leading to smoother convergence and more stable training dynamics.
In addition to reparameterization, we apply Exponential Moving Average (EMA) with decay rate of 0.999 during training. The combination of these techniques not only enhances stability but also accelerates convergence. Figure 12 presents comparative analysis of the PSNR and SSIM metrics the M4Raw FLAIR dataset over the first 125 training epochs (T = 5 and Jurget = 0). The results demonstrate that the joint application of reparameterization and EMA stabilizes the training dynamics.
22
0.00
With Reperandderrtate
2.50
19ch
Esect
Figure 12: Effectiveness of Reparameterizatioo in Noise Level Adjustment on the M4Raw FLAIR validation Dataset. Comparison of PSNR and SSIM metrics on the validation set for different model configurations over the first 125 training epochs (T' 5, Ourget 0). The combination of reparameterization and EMA (0.999) yields more stable training dynamics and improved convergence.
J GUIDELINES FOR SELECTING AND USING MAXIMUM CORRUPTION LEVEL
J.1 THEORETICAL BOUNDS FOR MAXIMUM CORRUPTION LEVEL
Following insights from score-based generative models Song et al. 22020), the maximum corruption level be theoretically chosen as large the maximum Euclidean distance between all pairs of training data points to ensure sufficient coverage for accurate score estimation. Our analysis of MRI datasets reveals the following maximum pairwise distances:
31

---

Published as conference paper ICLR 2025
M4Raw Dataset:
TI contrast: 56.72
T2 contrast: 47.99
FLAIR contrast: 65.20
FastMRI Dataset:
PD (a 13/255): 114.23
PDFS (a = 13/255): 102.59
PD (a 25/255): 115.53
PDFS (a 25/255): 102.94
J.2 PRACTICAL CONSIDERATION AND TRAINING DYNAMICS
While theoretical bounds provide upper limits, our empirical studies show that significantly smaller values of T can achieve optimal performance while maintaining computational efficiency. Table [0 demonstrates the relationship between T and training convergence:
Max Corruption Level (T)
Epochs to Converge
20
204
15
183
10
125
5
79
42
Table 10: Relationship between maximum corruption level and training convergence M4Raw dataset (TI).
J.3 ALTERNATIVE: VARIANCE PRESERVING (VP) FORMULATION
An alternative to the variance exploding (VE) formulation is the variance preserving (VP) formulation (Appendix B. Here, az is sampled uniformly from (otau: 1), eliminating the need to estimate T. This approach avoids explicit selection of T, the maximum corruption level is bounded by 1.
The corruption process in this formulation given by:
of with additional noise levels sampled such that oum < 0, This principled approach maintains theoretical guarantees and achieves comparable performance to VE in our experiments.
Xo o Z, 0 t 1,
(37)
J.4 PRACTICAL GUIDELINES
Based on our analysis, we recommend starting with T for datasets exhibiting moderate noise levels, as this provides effective baseline for most applications. For datasets with higher noise levels or more complex noise patterns, gradually increasing up to 20 may yield improved results, though practitioners should note that training time increases approximately linearly with T. Throughout the training process, careful monitoring validation metrics is essential to determine optimal stopping points and assess the effectiveness of the chosen corruption level. In cases where dataset characteristies make the selection of T particularly challenging, the VP formulation offers a principled alternative that maintains theoretical guarantees while eliminating the need for explicit 7 selection.
32


Reviews:
[{"review_id": "r5XXx4w42w", "reviewer": "Reviewer_2ysz", "summary": "This paper presents Corruption2Self (C2S), a score-based self-supervised denoising framework specifically designed for MRI data. C2S uses a reparameterized noise schedule and applies Generalized Ambient Denoising Score Matching (GADSM) to extend traditional score matching approaches to scenarios where only noisy data are available. Key contributions include the introduction of a reparameterized noise level function to stabilize training, as well as a multi-contrast extension to leverage complementary information across MRI contrasts. Experimental evaluations on M4Raw and fastMRI datasets show competitive results for C2S, often surpassing both classical and self-supervised methods in denoising performance metrics.", "questions": "1. Given that this method is closely aligned with score-based DDPM, what aspects of C2S differentiate it from classical diffusion-based approaches, aside from the reparameterized noise level function?\n1. How does the noise level estimation error affect denoising quality in practical scenarios? Is there a threshold or method for mitigating significant deviations in noise estimation?\n1. How would the training duration change if a higher maximum corruption level $T$ were used? Does this require additional training epochs to reach convergence, as hinted at in the ablation study?\n1. Could the authors clarify why adding T1 contrast data in the multi-contrast experiments led to reduced performance? This is counterintuitive, as T1 contrast generally provides valuable anatomical information.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* Reparameterization of Noise Levels: The proposed reparameterization of noise levels is a noteworthy contribution, offering enhanced training stability and convergence. This allows the model to sample uniformly across the noise range, leading to smoother training curves and better generalization.\n* Comparison with Self-Supervised and Supervised Methods: The paper includes extensive quantitative comparisons with self-supervised and supervised denoising models, establishing C2S as a strong self-supervised alternative in terms of PSNR and SSIM.\n* Multi-Contrast Extension: Incorporating multi-contrast data is a beneficial approach that leverages complementary MRI contrasts, enhancing structural preservation and improving the quality of denoised images.", "weaknesses": "* Limited Novelty Beyond Classical Denoising Diffusion Probabilistic Models (DDPM): The use of a score-based approach is similar to DDPM without substantial differentiation. Although the reparameterization is innovative, the rest of the framework closely resembles classical score-based diffusion models, raising concerns about the originality of the overall approach.\n* Noise Level Estimation Error Not Clearly Specified: While Figure 5 attempts to show robustness to noise level estimation error, the specific impacts and handling of these errors in practical settings remain unclear. Further clarity on this aspect would strengthen the model’s practical applicability.\n(Fluctuations in Training Stability: Although the reparameterization claims to stabilize training, Figure 2 indicates some fluctuations, suggesting that stabilization might not be consistent across all noise levels. This could impact reproducibility and model robustness, especially under varying noise conditions.\n* Parameter Specification in Equations: The notation for key parameters, such as $\\lambda_{out}$ ​and $\\lambda_{skp}$​ , lacks clear definitions in the methods section, which could hinder understanding and replication of the proposed framework.\n* Effects of Corruption Level (T) in Training: The paper does not provide guidance on the relationship between higher corruption levels (T) and the required number of training iterations to achieve optimal performance. This omission could limit the applicability of the method in datasets with different noise characteristics or levels of corruption.\n* Counterintuitive Results in Multi-Contrast Experiments: In the multi-contrast experiments, incorporating T1 contrast data seems to worsen the results. This is surprising since T1 typically offers structure-rich information that could enhance denoising. An analysis of this phenomenon would provide deeper insights into the limitations of C2S in multi-contrast applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5SR6W6SkYV", "reviewer": "Reviewer_9qMe", "summary": "This paper proposed Corruption2Self, a self-supervised MRI denoising method based on ambient diffusion (training diffusion models using corrupted data). The authors proposed an algorithm named Reparametrized Generalized Ambient Denoising Score Matching and showed its superior performance compared to a number of baselines, supervised and self-supervised methods, on M4Raw (containing pairs of real noise and ground truth) and FastMRI datasets (synthetic noisy images).", "questions": "- Authors may want to point out clearly how MRI denoising is different from natural image denoising. What are the challenges? Why does it worths special attention? \n- In the multi-contrast experiment, how extra contrasts were used as inputs? They were directly modeled by diffusion models (i.e. a channel of X_0 ... X_T) or as a conditional channel (i.e. diffusion models learn p(target contrast|other contrasts))?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The proposed algorithm GADSM has sound math groundings. \n- Authors did extensive experiments to compare the proposed algorithm to a number of baselines and showed its superior performance.\n- Authors discussed the application of the algorithm to multi-contrast MRI, which is an overlooked field in MRI denoising.", "weaknesses": "- This paper's originality seems to be limited. The proposed method Reparametrized GADSM is a straightforward extension to ADSM [1]. In addition, authors failed to point out the challenges when applying self-supervised denoising methods in natural images to MRI images. It seems to be that except for the multi-contrast part, the others are natural extensions of techniques that have already been tested on natural images.\n- The paper's problem setup is very similar to Noiser2noise [2], both handling Gaussian noise with known sigma. I think it is an important baseline to be tested. \n- The self-supervised method used as baselines in this paper are too outdated. There are a number of newer methods under the category of blind-spot network (J-invariance) such as LG-BPN [3] and PUCA [4], which has more powerful architectures to increase the accuracy in predicting the value in blind spots, therefore better PSNR/SSIM numbers. Even though these methods were proposed to handle noise with spatial correlation (i.e. not pixelwise independent), the architectures can be easily optimized for independent noise (by making the blind spot be just 1 pixel). \n- In Fig. 4, C2S seems to have more blurry results than R2R. Some details seem to be harder to see.\n- The dataset shown in Fig. 4 seems to be unfair for supervised method, since the label is very noisy. Authors may want to re-consider the statement that \"the potential of self-supervised learning to match or even surpass supervised methods in MRI denoising\" (Introduction).\n- The multi-contrast experiments are not fair to other baselines such as Noise2noise and R2R, since the other contrasts can be easily included as an extra channel in the model input to boost performance. \n- Is hallucination problem of generative diffusion models a concern here? How can it be addressed?\n\n[1] Daras G, Dimakis AG, Daskalakis C. Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data. arXiv preprint arXiv:2404.10177. 2024 Mar 20.\n[2] Moran N, Schmidt D, Zhong Y, Coady P. Noisier2noise: Learning to denoise from unpaired noisy data. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020 (pp. 12064-12072).\n[3] Wang Z, Fu Y, Liu J, Zhang Y. Lg-bpn: Local and global blind-patch network for self-supervised real-world denoising. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023 (pp. 18156-18165).\n[4] Jang H, Park J, Jung D, Lew J, Bae H, Yoon S. PUCA: patch-unshuffle and channel attention for enhanced self-supervised image denoising. Advances in Neural Information Processing Systems. 2024 Feb 13;36.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "B7eemFMxH2", "reviewer": "Reviewer_jWG5", "summary": "This paper proposes Corruption2Self, a score-based self-supervised framework for MRI denoising. \n\nMotivation:\n\nThe goal is to learn denoising directly from noisy data (without relying on high-quality labels). Their framework aims to fix the label scarcity and over-smoothing of finer details issues in existing supervised and self-supervised methods respectively. \n\nContributions:\n\nThe framework comprises a generalized ambient denoising score matching (GADSM) loss followed by reparametrization to improve convergence and detail refinement extension to preserve finer spatial features. Further, the authors extend the framework to incorporate additional MRI contrasts to improve performance. The authors finally claim that their framework achieves state-of-the-art performance among self-supervised methods and comparable performance among supervised methods.", "questions": "1. Robustness of C2S (M4Raw):\nIn the context of matching test-train SNR on M4Raw dataset, the authors claim that \n    1. (Line 327-328) supervised methods such as SwinIR and Restormer perform better when the noise characteristics of train and test data are similar\n    2. (Line 328-330) C2S achieves better generalization.\nAccording to \n    1. Table 3 (Results where test data SNR > train data SNR): C2S perform similar to SwinIR for eg.\n    2. Table 7 (Results where test data SNR ~ train data SNR): C2S perform similar to SwinIR (although both have higher metrics here).\n\n    It seems that both methods perform similar to each other in both the conditions?\n    Are the metrics higher in latter because the labels in Table 7 are noisier than in Table 3? I'm not sure if we can conclude that C2S is more generalized and supervised is not from the above data?\n\n2. I'd be interested to know the intuition behind why reparametrization works. Mathematically, it seems very similar except that we sample ${\\tau}$ ~ $U(0, T]$ rather than ${\\tau}$ ~ $U(0, T']$ due to $T >> T'$ . Does this approximation help in convergence?\n\n3. Does reparametrization and detail refinement extension help in the case of FastMRI dataset also? Curious as effectiveness for only M4Raw dataset have been reported.\n\n4. How would one estimate `T` (max corruption level) before training the model?\n\n5. Given that complementary pair of contrasts improve denoising, curious to know if all the 3 contrasts can be used (eg. T1, T2 and FLAIR) for instance to further improve denoising?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper comprehensively analyzes and applies existing self-supervised and supervised denoising approaches in the context of MRI\n\n2. The paper is well written, original and provides great detail into the workflow of the Corruption2Self (C2F) framework.\n\n3. Incorporation to reparametrization (Table 4) and extensions to multi-contrast settings to improve denoising.\n\n4. Showcases robustness of methodology on varied noise level estimations compared to true noise (Table 9)", "weaknesses": "### A. Detail refinement extension claim\nAccording to the metrics in Table 1, it is unclear if the detail refinement extension is being effective. The improvements in PSNR / SSIM does not seem notable. It would be helpful to include an error bar (for the table), statistical significance test (to show notability) and visuals to show effectiveness.\n\n### B. Applicability and impact\nThe paper can cover how their workflow can be used in practice while denoising. The following aspects can add more value-add in terms of real-world impact to the paper.\n1. Estimation of $σ_{tdata}$ in real-time during inference.\n2. Given that traditional methods such as BM3D work decently well (maybe 1-3 points less performant than C2S), how difficult is it to go about training / deploying C2S in the MRI denoising workflow than using non-learning based methods?\n3. Would this methodology potentially result in MRI image acquisition?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)