DDM$^2$: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Models
Authors: Tiange Xiang
Mahmut Yurt
Ali B Syed
Kawin Setsompop
Akshay Chaudhari
Venue: ICLR 2023 poster (2023)
Keywords: Unsupervised MRI Denoising
Diffusion Models

Abstract: Magnetic resonance imaging (MRI) is a common and life-saving medical imaging technique. However, acquiring high signal-to-noise ratio MRI scans requires long scan times, resulting in increased costs and patient discomfort, and decreased throughput. Thus, there is great interest in denoising MRI scans, especially for the subtype of diffusion MRI scans that are severely SNR-limited. While most prior MRI denoising methods are supervised in nature, acquiring supervised training datasets for the multitude of anatomies, MRI scanners, and scan parameters proves impractical. Here, we propose Denoising Diffusion Models for Denoising Diffusion MRI (DDM^2), a self-supervised denoising method for MRI denoising using diffusion denoising generative models. Our three-stage framework integrates statistic-based denoising theory into diffusion models and performs denoising through conditional generation. During inference, we represent input noisy measurements as a sample from an intermediate posterior distribution within the diffusion Markov chain. We conduct experiments on 4 real-world in-vivo diffusion MRI datasets and show that our DDM^2 demonstrates superior denoising performances ascertained with clinically-relevant visual qualitative and quantitative metrics.

Published as a conference paper at ICLR 2023
DDM SELF-SUPERVISED DIFFUSION MRI DENOIS- ING WITH GENERATIVE DIFFUSION MODELS
Tiange Xiang, Mahmut Yurt, Ali B Syed, Kawin Setsompop & Akshay Chaudhari Stanford University fxtiange, myurt, alibsyed, kawins, akshayac/estanffrddeed
ABSTRACT
Magnetic resonance imaging (MRI) is common and life-saving medical imaging technique. However, acquiring high signal-to- agnai-to-noise ratio MRI scans requires long scan times, resulting in increased costs and patient discomfort, and decreased throughput. Thus, there is great interest in denoising MRI scans, especially for the subtype of diffusion MRI scans that are severely SNR-limited. While most prior MRI denoising methods are supervised in nature, acquiring supervised training datasets for the multitude of anatomies, MRI scanners, and scan parameters proves impractical. Here, we propose Denoising Diffusion Models for Denoising Diffusion MRI (DDM²), self-supervised denoising method for MRI denoising using diffu- sion denoising generative models. Our three-stage framework integrates statistic- based denoising theory into diffusion models and performs denoising through conditional generation. During inference, we represent input noisy measurements as a sample from an intermediate posterior distribution within the diffusion Markov chain. We conduct experiments on 4 real-world in-vivo diffusion MRI datasets and show that our DDM demonstrates superior denoising performances ascertained with dinically-relevan visual qualitative and quantitative metrics. Our source codes are available at: https: /tt::://gttttbubu com/ St FMMMMMMI/DDD
INTRODUCTION
Magnetic resonance imaging (MRI) is a non-invasive clinical imaging modality that can provide life-saving diagnostic information. Diffusion MRI is a subtype of MRI commonly used in oncologic and neurologic disorders (Bihan 2003 Bihan et al. 2006). which can quantitatively assess micro- structural anatomical details. However, diffusion MRI scans suffer from severe signal to noise ratio (SNR) deficits, hindering diagnostic and quantitative accuracy. Image SNR can be improved either by lowering image resolution, which further reduces diagnostic utility, by increasing the total scan time, which already can require 10+ minutes of time in the MRI scanner. Thus, there is large interest in decreasing diffusion MRI scan times to improve patient throughout in hospitals and the patient experience. While image SNR is governed by the underlying MRI physics, applying post- processing denoising techniques fast and low-SNR MRI acquisitions can improve overall image SNR. Developing such methods to improve SNR of diffusion MRI scans is an unsolved problem that may improve the efficacy of the millions of such scans performed in routine clinical practice annually.
Supervised machine learning techniques have previously been proposed for MRI denoising, however, such methods are limited by their clinical feasibility. is clinically impractical acquire paired high- and low-SNR diffusion MRI scans across various anatomies (e.g., brain, abdomen, etc), diffusion weighting factors (and resultant SNR variations), MRI field strength and vendors, and clinical use- cases. Such large distributional shifts across heterogeneous use-cases leads to fundamental drop in model performance (Darestani et al. 202T). The diversity of data and the need for effective denoising methods motivates the use of unsupervised denoising techniques for diffusion MRI.
To address these challenges, the major contributions of our work are three-fold: (i) We propose DDM for unsupervised denoising of diffusion MRI scans using diffusion denoising models. Our three- stage sulf-supe approach couples statistical self-denoising techniques into the diffusion models (shown in Figure [ (ii) 2DM2 allows representing noisy inputs samples from intermediate state in the diffusion Markov chain to generate fine- -rainer denoised images without requiring ground truth references; (iii) We evaluate our method four real-world diffusion MRI datasets

---

Published as a conference paper at ICLR 2023
that encompass the latest and longer established MRI acquisition methods. DDM demonstrates state-of-the--al denoising performance that outperforms second best runner-up by an average of 3.2 SNR and 3.1 contrast-to-noise ratio (CNR) points, across this large range of MRI distributions.
2
RELATED WORK AND BACKGROUND
2.1 DIFFUSION MRI
Diffusion MRI sensitizes the MR signal to the movement of protons within an object being imaged using tailored imaging gradients (small steerable magnetic fields). Given that both magnitude (overall duration that gradients are turned on, termed as b-value) and directions (linear combination of the three independent gradient directions) can be steered, the diffusion weighting is considered vector. A typical diffusion MRI scan for resolving tissue microstructure is 4D nature, with dimensions of spatial coordinates and 1 dimension of diffusion vectors. Depending use-case, the number of diffusion vectors can range from to upwards of 100. Prior approaches for improving diffusion MRI sequences exist; however, these require highly-task-specifie models (Kaye) et al. 2020 Gibbons et al. 2018). These methods scan the same volume multiple times, average these multiple low-S SNR images to generate high- Ligh-SNR image, and subsequently, train supervised model to transform a single low-SNR image to the averaged high-SNR image. Such methods cannot generalizable diverse anatomies, MRI scanners, and image parameters of diffusion MRI scans.
the clinical
2.2 STATISTIC-BASED IMAGE DENOISING
Given a noisy measurement x, an inverse problem is to recover the clean image y defined by:
Aιy t
where A is linear coefficient and denotes the additive noise. Without losing generality, one usually represent Agz sample from the Gaussian distribution, (N, AÿI). When paired ground truth images y are available, a neural network can then trained to approximate y through direct supervisions. However, supervised training proves infeasible when y are missing in the datasets. Attempts have been made to relax the reliance on supervised signal from clean images, by using noisy images alone. Based the assumption that the additive noise is pixel-wise independent, Noise2Noise (Lehtinen et al. 2018) claimed that to denoise an noisy measurement x towards another measurement x' is statistically equivalent to the supervised training on y up to a constant:
L(((('), x) = [x(('') - x = [x0('') - y21 + const.
(2)
Based the above idea, Noise2Self (Batson & Royer 2019) further designed the J IIvariainee theory that achieves self-supervised denoising by using the input x itself. With the statistical independence characteristic of noise, Noise2 (Krull et al. (2019) and Laine al. (Laine et al.) 2019) extended the self-supervised strategy to a so-called "blind-spot' technique that masked out image patches and designed particular neural networks predict the unmasked pixels. Neighbor2Neighhor (Huang let al., 2021 creates noisy image pairs by dividing sub-samples and uses them to supervise each other. Without losing pixel information, Recorrupted2Recoruu
(Pang al. 202T) was proposed keep all pixels intact and achieve self-supervised denoising by deriving two additional corrupted signals. To better utilize 4D MRI characteristics, Patch2Self (Fadnavis et al. 2020) was proposed to utilize
multiple diffusion vector volumes as input (x'} and learn volume- wise denoising.
Building on the statistical theory studied above, this work we achieve unsupervised MRI denoising through a generative approach. Compared to Patch2Self that requires large number of volumes (e.g. > 60) as a guarantee to denoise single volume, DDM can efficiently denoise MRI scans acquired with very few diffusion directions and limited number of volumes (e.g. 5). This is clinically relevant since common clinical diffusion MRI scans use fewer than 10 directions. Moreover, unlike Patch2Self, our trained denoiser is universally applicable for the entire 4D sequence and no repeated training is needed for denoising volumes in different directions.
2.3 DIFFUSION GENERATIVE MODEL
A diffusion model (Sohl-Dickstein et al. 2015) denotes parameterized Markov chain with T discretized states S Sj... . that can be trained to generate samples to fit given data distribution. Transition of this chain is bi-directional and is controlled by pre-defined noise schedule 31.-..T-
2

---

Published as a conference paper at ICLR 2023
Reverse
Unconditional sampling (DOPM)
Conditional sampling (DDM))
S
S
SO
S
S
S
p(s (z)
((
So
model 5
Denolsing
X
Volume-wii wise processing [Patch2Self)
Slice-wise processing (DDM.)
Input
Inpur ti-
outpput
Inpur tle
Indus
input
output
Input
Figure 1: Top Left: 2DM generates clean MRI through reverse diffusion. Top Right: Uncondi- tional sampling process (Ho et al. 2020) (starts from a noisy prior 2) v.s. our conditional sampling process (starts from noisy measurement x). Bottom: Volume-wise processing (Fadnavis al.) 2020) (requires a large number of input volumes) v.s. our slice-wise processing (requires very few input volumes). Gray blocks indicate accessible 3D volumes in a MRI sequence, orange blocks indicate the input priors {x'} and blue blocks indicate the one-pass output.
The forward process or the diffussin process ((S{{S,_1) gradually adds Gaussian noise to the posterior samples at every state until the signal is completely corrupted. The reverse of the above diffusion process or the reverse process tries to recover the prior signal with parameterized neural network F:
T ps (S (S. II
p(rr t=1
(S(S = ((S +),
(3)
Starting from Sr = (0, I), the reverse transition continues till the first state So: valid sample from the given data distribution. Following (Ho et al. 2020). we set of = 3r and hold 31,... as hyperparameters. In what follows, refer this kind of set up as the baseline diffusion model.
Recent works have extended diffusion models to improve sampling quality or efficiency. Song et al. (Song & Ermon 2019) investigated the density of data distribution and incorporated Langevin dynamics and score matching methods (Hyvärinen, (2005) into diffusion model. In follow-up work, Song et (Song et al. 2021b) formulated score based generative models as solutions to Stochastic Differential Equation (SDE). Recently, Noise2Score (Kim & Ye 2021) also trained score function and in a manner similar ours to perform score-bas based diffusion model for blind image denoising. Rather than transiting through the diffusion Markov chain, the authors validated its ability denoiser by following the Tweedie's 's formula (Efron 201T). the other hand, Denoising Diffusion Implicit Model (DDIM) (Song al. 2021a) presented non non-Markoving acceleration technique for the reverse process that enables the sampling of a data in only few states. Similarly, Watson et al. [Watson] al. 2022) proposed generalized family of probabilistic samplers called Generalized Gaussian Diffusion Processes (GGDP) to reduce the number of states without compromising generation quality. By using knowledge distillation, Salimans ef proved that faster reverse process can be learnt from a slower one in the student-teacher manner iteratively (Salimans & Ho 2022). Instead of starting from Sr, our method initiates the reverse process from an intermediate state that also accelerates generation. DDM is compatible with the above advances that can be utilized for further accelerations.
Prior works also adopted the diffusion model to recover data across modalities. WaveGrad Chen et al,, 2021), DiffWave (Kong et al., 202T). and BBDM (Lam et al.) (2022) are for speech/audio generation. Luo er al. (Luo & Hu/ 2021). Zhou et al. (Zhou et al., (021). and GeoDiff (Xu et 2022) are for graph-structure data generation. Chung et al. (Chung et al. 2021) and Song et al. (Song et al. 2022) are for medical image inverse problem solving. Chung et al. (Chung et al. 2022) concurrently explored sampling noisy inputs posteriors in the diffusion Markov chain for image denoising and super resolution using coarse noise approximations via image eigenvalues. In contrast, integrate statistical theory and self-supervised noise estimation into diffusion models to account for noise distribution shifts. We note that there was no open source code available for (Chung et al. 2022) for using it as comparison method.
3

---

Published as a conference paper at ICLR 2023
3 METHODS
3.1 PROBLEM DEFINITION AND OVERVIEW
In this section, we show that clean image approximations y can be generated through a diffusson model with a known noise model g. To condition diffuusin sampling on the noisy input x, our idea is to represent as sample from a posterior at some data-driven intermediate state St in the Markov chain and initiate the sampling process directly from x through p(S,|x). A comparison between the unconditional sampling process (Ho et 2020) that starts from p(Srlz), N/ I) and our conditional sampling process is outlined in Figure Right.
There are now three questions need answers: (i) How to find a noise model 9 in an unsupervised manner? (ii) How to match x to an intermediate state in the Markov chain? and (iii) How to train the diffusion model as a whole when both noise model and the best matched states are available? To mitigate parameter tuning, DDM is designed in three sequential stages solutions w.r. the above questions: (Stage I) Noise model learning; (Stage II) Markov chain state matching; and (Stage III) Diffusion model training. The overall framework is outlined in Figure 2 All experiments below are performed on 2D slices, however 3D volume results are shown in the appendix.
3.2 STAGE I: NOISE MODEL LEARNING
It is a common assumption that noisy signals can be obtained by corrupting the clean signals with noise sampled from an explicit noise model (Prakash et al., 2021). Without prior knowledge on the corruption process, it is difficult to find an effective mapping between x and y. Following the J-Invariance theory, one can approximate solution by learning a denoising neural network from the noisy images themselves. To be more specific, with some prior signals (x') as input, a denoising function & can be trained to generate a clean approximation y of &f {x'} such that y x y = (((x')).
For 4D MRI sequence X E Rxxxxxxi with / 3D volumes (acquired at diverse gradient directions) of size w x h d, Patch2Self learns & by aggregating / /-1 volume patches as (x') } and regressing one clean voxel (3D pixel) y E RIxIx1 for the remaining volume (Fadnavis et al. 2020). This training process repeats for all of the 3D volumes in X to obtain / volume-specific denoisers. There are three major drawbacks to such denoising strategy: (i) the denoising function & needs to be trained for each volume individually, which inevitably demands long and repeated training process for MRI sequences with several volumes. (ii) the denoising quality heavily relies on the size of {x') number of accessible volumes in the 4D sequence (see studies in section 4.3): (iii) when inspecting in 2D slices, the denoised results yield significant spatial inconsistencies due to its volume wise regression.
Inspired by the performance of Patch2Self, but considering its challenges, we learn a slice-to-slice mapping function for the entire sequence instead of a patch-to-voxe mapping. This enables & to regress an entire 2D slice at time by taking also slices to form {x'}. Such a simple modification overcomes the above three drawbacks: (i) only single & needs to be trained for the entire sequence; (īi) denoising quality remains stable when forming {x'} with very few volumes (see studies in section ).:): (iii) better spatial consistency can be observed in the denoised slices. A comparison between our modification and original Patch2Self is briefly shown in Figure I Bottom. We retain this wlice-wise processing scheme throughout all three stages in our framework.
3.3 STAGE II: MARKOV CHAIN STATE MATCHING
When optimal mapping function is learned by +. the noise model G can be obtained by fitting the approximated residual noise E to a Gaussian ((a21) with zero mean and a variable standard deviation a. Without any restrictions, not necessarily has mean value of zero and a direct fitting will lead to distribution mean shift. In this way. we propose to explicitly complement ě's mean value
u4 |all Σ to be zero and calibrate the denoising result y accordingly that balances Equation |
Azp.
=
A
The complemented can be then used to fit the 9 and estimate a. Recall that in the diffusion model, noise schedule $1,....7 . is pre-defined that represents the noise level at every state in the Markoy
4

---

Published as a conference paper at ICLR 2023
X
Eq.
1I
State Matshing Eq. 5
Gaussian
Dencising function
Denoising fundtio
prlors
y
Imput prices xx'
g NO o²I)
|I|
Eq. and shufflere!
Eq. 2
y
Eq.
Spead
Denosinng function
Denoising funcion FF
inpul prices X')
y
train denoising function + in self-supervised manner to estimate an initial noise distribution. In Stagg II, the estimated noise by & is used to fit Gaussian noise model G. We match standard deviation of this noise model sampling noise schedule 3 and represent x as a sample at St. With both G and t, in St Stage III, we train another denoising function F in the diffusion model for the reverse process p= x). All three stages take as inputs and returns 2D slices as discussed in section 3.2
Figure 2: Our three stage DDM In Stage
the diffusion
chain. We determine a matching state of by comparing & with all possible posteriors p(S,)-o and vB, Specifically, a state is matched when time stamp t is found that minimizes the distance:
arg min II olp,,
where || |P denotes the p-norm distance. Since is a discrete integer within a finite interval: {1. T), cast the optimization problem into surrogate searching problem.
match at state S, indicates that, with this particular noise schedule 3, there exists at least one possible sample from the posterior state St in the baseline unconditional generation process that sufficiently close to the given input x. In this way, a denoised image can be sampled at state So through the reverse process iteratively pSSo(S.).
3.4 STAGE III: DIFFUSION MODEL TRAINING
In diffuuson models, during the reverse process, additional denoising function F is trained to predict é or y at each transition step. As inputs to F, clean images y are first corrupted through a prior sampling (y)) Aιy λ2z, where (N I) (Equation 1. Following DDPM and its hyper- parameters, rewrite A Va, and A2 vI di in terms of 35 āe = -I'' Olgy Or, In unsupervised denoising with unavailable, denoised images y are approximated by 4.
8r.
Unlike the supervised training schema that can train on clean ground truth y directly, is sub- optimal to supervise F with the Stage I estimated y in the unsupervised denoising scenario: F would degenerate to become an inverse function of that traps to interpolate only within the solution space of &b To enable F extrapolation to wider and more accurate solution space and to potentially improve denoising, we introduce two simple modifications as below.
Noise shuffle. First, we modify the sampling protocol for the diffusion process from ((A.l)) va.y vT- 0z to (S,,)y)
Vary + ffuffle(E), where shuffle(-) denotes the spatial shuffling operation and is the residual noise predicted by 4. Under the assumption of noise independence, such noise shuffle operation forces F to learn from the implicit noise distribution modelled by T rather than the explicit noise model
reduce the distribution disparity in between.
J-Inarriance optimization. Second, we modify the loss function to train F. Borrowing the objective let F optimize towards the original noisy input x instead of The objective
used in Stage therefore becomes arg miny [[,(S, ā,) x||2. Such a modification not only minimizes the distance the statistical 'ground truth' but also the approximation error brought by T.
5

---

Published as a conference paper at ICLR 2023
Noisy Image
DIP
Noise2Noise
Patch2Self
Noise2Score
Ours
Inference Time: 2.8
gSlider
Stanford HARDI
Inference Time: 1.3
Figure 3: Qualitative results on the most challenging datasets. Inference time per slice is reported alongside. DDM not only suppresses noise the most but also restores anatomical structures the best.
4 EXPERIMENTS
4.1 DATASETS AND EVALUATIONS
Datasets. Experiments were first conducted on an in-house dataset brain diffusion MRI (gSlider) that has 0.5mm isotropic resolution, which acquired under institutional review board approval using a recent advanced diffusion MRI sampling technique. The gSlider data had dimensions of 128 x 128 x 160 50 (X x Y x Zx diffusion directions) with 1-value=1000) To evaluate the generalizability of DDM2, additional experiments were done on other publicly-available brain diffusion MRI datasets acquired with different protocols with less advanced MRI encoding for image SNR and resolution: (i) Sherbrooke 3-Shell dataset (dimensions = 128 x 128 x 64 x 193) with denoising occurring on the b-value=1000 images (amongst the available b-value=2000 and 3000 images) (Garyfallidis et al., 2014); (ii) Stanford HARDI (dimensions = 106 x 81 x 76 x 150) with b-value=2000 (Rokem) 2016): (ii) Parkinson'ss 's Progression Markers Initiative (PPMI) dataset (dimensions = 116 x 116 72 x 64) with b-value=2000 Marrek et al.) 2011). Experiments to add noise to raw k-space data for creating realistic noisy reconstructed images were performed on the SKM-TEA knee MRI dataset (Desai et al. 2021a) (results in Appendix section G
Evaluations. We considered state-of-the-an methods for comparisons with DDM2: (i) Deep Image Prior (DIP), classic self-supervised denoising method (Ulyanov al. 2018); (ii) Noise2Noise (N2N), a statistic-based denoising method (Lehtinen et al. 2018); (ūi) Patch2Self (P2S), state-of- the-art MRI denoising method; (iv) Noise2Score (N2S), recent busee denoising method. We note that assessing perceptual MRI quality a challenging research problem (Mittal et al. (011). as there is lack of consensus image quality metrics, especially in an unsupervised reference-free settings (Chaudhari et af.) (019) (Woodard & Carley-Spencer (006). Metrics such as PSNR and SSIM require reference ground truth image, yet they still have poor concordance with perceptual image quality and consequently, evaluating on downstream clinical metrics of interest provides a better measure of real-world utility (Mason et al., 2020) Adamson al. 202T). Thus, we utilized Signal- -ggnal-to-Noiee Ratio (SNR) and Contrast-to-Nonn Noise Ratio (CNR) to quantify denoising performance and report mean and standard deviation scores for the full 4D volumes. As both metrics require regions of interest segmentation to delineate foreground and background signals, we follow the approach outlined in (Fadnavis et al.) 2020). We segment the Corpus Callosum using (Garyfallidis) (FA) maps.
al. 2014) to generate quantitative and clinically-relevant Fractional
Implementation details. A typical noise schedule (Ho et al. 2020 Saharia et al. 202I) follows a 'warm-up' scheduling strategy. However, for most of MRI acquisitions, noise level is mild and the matched Markov chain state S. will be close to Sy. This leads to very few transitions in the reverse process and eventually poor generation quality. We therefore modify 3 to follow a reverse 'warm-up' strategy such that /3 remains at 5e -5 for the first 300 iterations and then linearly increases to 1e between (300, 1000] iterations. See supplementary materials for more details.

---

Published as a conference paper at ICLR 2023
Noisy Input
Patch2Self
Noise2Score
Ours
50
-25
Reusive ONR
Maric
DIP
N2N
P2S
N25
Ours
Figure 4: Left: Quantitative fractional anisotropy map comparisons (major differences highlighted with arrows). Right: DDM demonstrates statistically significant SNR/CNR improvements (two- sided t-test, le p-values) over all competing methods.
4.2 MAIN RESULTS
Qualitative results. In Figure 3 we visualize denoising results on axial 2D slices for each com- parison method for the gSlider and HARDI datasets (additional dataset images in supplementary). Compared to DIP, all other methods demonstrate better MRI denoising abilities. Although the simplest Noise2Noise model and Noise2Score models achieve promising denoising, both suffer from excessive blurring of pertinent details in the underlying anatomical structures. Utilizing the infor- mation provided in all different volumes, Patch2Self therefore able restore most of the details. Unfortunately, this method is not robust to heavy noise and cannot produce clean estimations for the noisier Sherbrooke and gSlider datasets. Noise2Score, intermediate denoising quality between Noise2Noise and Patch2Self but with excessive blurring. By integrating statistical denoising into diffusion models, D not only suppresses noise the most but also restores anatomical details. Two neuroradiologists reviewed the DDM images and commented that there were no new hallucinated lesions or regions with excessive blurring.
score-based denoising method, yields
Quantitative results. We calculated SNR/CNR on the entire Stanford HARDI dataset. The segmen- tation quality of the corpus callosum regions impacts metric calculation and an explicit tuning of Rols would lead to unfair comparisons. To avoid this, we used scripts provided by (Garyfallidis et al.) 2014) for Rol segmentation and metric calculation. In Figure Right, we show box plots of relative SNR/CNR by subtracting the scores of the input noisy images from the denoised ones. Compared to DIP, N2N, and P2S, our method shows significant statistical improvements on both metrics (two-sided t-test, p-value < 1 1-4). Compared the most advanced score- score-based denoising method N2S, method demonstrates an average of 0.95/0.93 improvements on SNR/CNR respectively.
The SNR/CNR measures depict how 2DM improves best-case results, depicting its excellent denoising capabilities, while maintaining worst-case results compared to the comparison methods. We note that N2S did have improved worst-case results than DDM2 but this likely due to excessive image blurring induced by the method (Figure J. Note that without ground truth references, SNR used approximate indications of image quality.
As a diffusion model based approach, iterative inferences are necessary for DDM This by its nature becomes a disadvantage from the perspective of inference time, especially when comparing to single forward pass-based CNN/MLP based methods. However, due to the novel design of Markov chain state matching. 2DM2 yields 5-10 times inference speed up than DDPMs. Moreover, since many diffusion MRI applications are performed in post-processing. improved SNR at the expense of few minutes of inference can still benefit downstream clinical tasks.
Fractional anisotropy comparisons. Besides quantitative and qualitative evaluations with 2 we examine how the denoising quality translates to downstream clinical tasks such as creation of Fractional Anisotropy (FA) maps using the various image denoising methods. To calculate FA, we used the volumes acquired by the first diffusion directions along with the 10 b-vauee= volumes. Representative results displayed in Figure Left indicate that the proposed DDM yields enhanced and less noisy capture of diffusion directions of fiber tracts.
4.3 EXTENSIVE ABLATION STUDIES
Modifications at Stage I. In section 3.2 several upgrades were made upon Patch2Self for relaxing the reliance large number of volumes (expensive obtain during MRT acquisition) and for better
7

---

Published as a conference paper at ICLR 2023
Relative SNR
Relative CNR
Patch25elf Stage!
Patch2Self n=G3
Ratch25elf n-63
Ours
Ours
10
10
n=2
(size [x] input volummes
Nolsy Input
Final
Ours Stage!
w/o state matching w/o noise shuffle
w/o I J-Invariance
Ablative results on number accessible volumes n used inputs. Bottom: Qualitative results of the ablation studies. Results obtained by different ablation settings are labeled by different identifiers. Major differences are zoomed and are highlighted by red arrows.
Figure Top:
spatial consistency in the denoised results. In Figure 5 Top, we compare our Stage against Patch2Self by inputting limited number of volumes n (size of {x'}). Patch2Self can achieve decent denoising performance when using large number of volumes (e.g. n>>0).. When small, Patch2Self fails to denoise the data properly. However, our simple modifications leverage spatial priors and achieve impressive denoising result even for n = With more accessible volumes, our Stage I maintains its denoising performance at high level. Considering both denoising quality and clinical applicability, we adopted n = 2 as our final setting.
Necessity of the multi-stage design. With the learned noise model from Stage I, method learns an additional diffusion model to restore more subtle details and reduce the approximated statistical error. To compare whether alone, we compare our final results to the denoised results obtained at Stage (Figure 5 A, more comparisons be found in supplementary materials). Due to the upgrades made in section 3.2 Stage I already achieves satisfactory denoising results. However, similar to other statistic-based methods, Stage cannot restore high-frequency details and always converges smooth solutions. This emphasized the novelty of using a diffusion model to generate self-consistent details to further restore the underlying anatomy.
three-stage method performs better than the Stage
Necessity of Stage II. In section 3.3 state S, the Markov chain. This done by minimizing the distance between the fitted standard deviation and the pre-defined noise schedule S. Instead of finding the most matched state, we evaluate whether is possible to naively represent x as the middle state S,= regardless of B. Given that the reverse process starts from z ~ (0, I), representing x as S may overestimate the noise level. Consequently, excessive signals will be generated and be imposed into the denoising results as shown in Figure B. These hallucinations are not valid restorations of the underlying anatomical structure though they may lead to high SNR/CNR scores.
we propose to represent x as sample from an intermediate
Modifications at Stage III. As mentioned section 3.4 two modifications are designed to assist diffusion model training. We visualize the denoising results by disabling the noise shuffle operation (Figure 5 C) and the J-Invariance optimization (Figure D). By learning from an explicit noise model and optimizing towards the Stage I results, observe significant structural degradation in the denoised images. Moreover, both the results differ considerably from the Stage I results (Figure A), which further validates the need of our modifications to be used in Stage IIT.
8

---

Published as a conference paper at ICLR 2023
A
B
A Noisy Input
Results of 3D volume-wise DDM
Results of 2D slice-wise DDM?
Residual plots between A and
Residual plots between A and
Figure 6: Comparisons between 31D volume-wise DDM and 2D slice-wise DDM Results shown on the transverse, Coronal, and Sagittal planes. Major differences are highlighted by arrows.
3D volume-wise DDM Other than 2D slice-wise processing schema of DDM 3D volume-wise schema could be another natural design. However, we show its inferior denoising performances via experiments as demonstrated in Figure 6 The residuals between the original and denoised images maintain more structure in the 3D methods than 2D methods, which depicts subpar denoising performance. To match computational burden of 2D slice-wise experiments, the largest possible patch size of 16 x 16 x 16. In supplementary materials, we discussed the impact of patch size and presented additional results. There are three drawbacks can be observed: (i) Volume- wise DDM requires larger networks with 3x model parameters; (ii) Contextual dependencies in the longitudinal dimension (across slices) might be weaker for datasets acquired with thick slices due to varying noise characteristics within and across slices. This limits 3D networks, whereas 2D networks can capture the information within the isotropic slice: (iii) Most advances in diffusion models focus on 2D sequential acquisitions, which can be easily adopted to improve DDM
experimented with
DISCUSSIONS
MRI noise considerations. Noise in MRI typically follows Rician distribution for background regions and transitions to Gaussian in foreground regions where the SNR>3 (Andersen 1996 Robson al. 2008). Moreover, linear image reconstructions result in complex Gaussian noise remaining Gaussian in the image domain also (Robson et al. 20088.. Modern image acceleration techniques simply perturb the magnitude of noise spatially (based on coil and subject patterns) but not modify the Gaussian distributions (Pruessmann et al. 1999). Consequently, using DDM2 for denoising relevant foreground image regions is consistent with the underlying image noise statistics. For future work in non-linear image reconstructions (such as learning- -basd reconstructions (Chaudhari et al.) 2020)), our framework still allows data-driven fitting of different noise profiles with a corresponding noise model and score function suggested by Noise2Score.
Limitations. As with most DDPM applications, DDM primarily limited by slow inference durations. However, one switch to faster networks explore acceleration techniques in (Song et al. 202 Moreover, DDM2 may hallucinate actual lesions due to its generation nature. While this not in our results. it can be mitigated by enforcing data consistency during inference. Systematic radiologist-driven studies may also assess anatomic integrity and expert quality metrics.
6 CONCLUSION
In this paper, we present 2 a self-supervised denoising method for MRI. Our three-stage method achieves MRI denoising via a generative approach that not only reduces noisy signals but also restores anatomical details. In Stage I, a denoising function is trained to capture an initial noise distribution. In Stage II, we fit a noise model to match the noisy input with an intermediate state in the diffusion Markov chain. In Stage III, we train a diffusion model to generate clean image approximations in an unsupervised manner. In experiments, we demonstrate that DDM2 outperforms existing state-of-the ahr- denoisers for diffusion MRI methods both qualitatively and quantitatively.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGEMENT
We thank Congyu Liao for his feedback and provision of the gSlider data. This work was supported by NIH grants RO1 AR077604, R01 EB002524, and RO1 AR079431. Additional research support was provided by the Stanford Radiological Sciences Laboratory Neuroimaging Research grant.
ETHICS STATEMENT
All datasets used in this study were acquired under institutional review board (IRB) approval for human subjects research. No additional metadata is provided to enable re identification of the individuals involved. The proposed denoising technique has the ability of reducing MRI scan time or enabling noisier data acquisitions on older MRI scanners. This presents large opportunity to make life-saving MRI technology available and more accessible patient populations that would not ordinarily be exposed to it due to initial infrastructure and maintenance costs of expensive MRI scanners. Further clinical validation of the proposed technique will be necessary prior to routine usage in these scenarios, however, in order to ensure lack of visual hallucinations and comparable diagnostic outcomes compared to the current standards care.
REPRODUCIBILITY STATEMENT
Our proposed method is fully reproducible with respect to the main results reported in section 4.2 For methodology & implementation details, readers are referred to our source codes, which are available at: httpps://qithie http://....ccottttttooottt//////
REFERENCES
Philip M Adamson, Beliz Gunel, Jeffrey Dominic, Arjun D Desai, Daniel Spielman, Shreyas Vasanawala, John M. Pauly, and Akshay Chaudhari. SSFD: Self-supervised feature distance as an MR image reconstruction quality metric. In NewrlPS 202/ Workshop on Deep Learning and Inverse Problems, 2021. httttttst//ppeeeeeeesss
net
Tz 6M_
Anders H Andersen. On the rician distribution of noisy mri data. Magnetic resonance in medicine, 36(2):331-32) 1996.
Joshua Batson and Loic Royer. Noise2self: Blind denoising by self-supervision. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97
of Proceedings of Machine Learning Research, pp. 524-533. PMLR, 2019. URL http:/7
proceedings ngs . .ml..rr...... wwwpssvvvv wtt///aatooniio 9a html
Denis Le Bihan. Looking into the functional architecture of the brain with diffusion MRL. Nature Reviews Neuroscience, 4(6))469 480, June 2003. doi: 10. 10.1038/nmm1199 URL https::/dol
orc/10.1038/nmm1111
Denis Le Bihan, Cyril Poupon, Alexis Amadon, and Franck Lethimonnier. Artifacts and pitfalls in diffusion MRI. Journal of Magnetic Resonance Imaging, 24(3):478-485 488, 2006. doi: 10. 1002/mmii 20683.
/oo. org. 1 1002/ 002/jmrr... 20683
Akshay S. Chaudhari, Kathryn J. Stevens, Jeff P. Wood, Amit K. Chakraborty, Eric K. Gibbons, Zhongnan Fang, Arjun D. Desai, Jin Hyung Lee, Garry E. Gold, and Brian A. Hargreaves. Utility of deep learning super-resolution in the context of osteoarthritis MRI biomarkers. Journal of Magnetic Resonance Imaging, 51(3):768-777 779, July 2019. doi: 10.1002/jmri.26872. URL
https: www. /ttps://doi.org/10 1002/ 1ew....2.6222
Akshay S. Chaudhari, Christopher M. Sandino, Elizabeth K. Cole, David B. Larson, Garry E. Gold, Shreyas S. Vasanawala, Matthew P. Lungren, Brian A. Hargreaves, and Curtis P. Langlotz. Prospective deployment of deep learning in mri: A framework for important considerations, challenges, and recommendations for best practices. Journal of Magnetic Resonance Imaging, 54 (2):357-37)) August 2020. doi: 10. 10.1002/jmri.2733.1 URL https: doi. org/10..100... imri 27331
10

---

Published as a conference paper at ICLR 2023
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Waveg- rad: Estimating gradients for waveform generation. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria. May 3-7, 2021. OpenReview.net, 2021. URL httts:://ppenrett ew
ww..et.......e.......e net/ forum? hatppoumii-NsssMiia. Nc.. a080
Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye. Come-closer-diffuse- tional diffusion models for inverse problems through stochastic contraction. CoRR, abs/21 2021.
Accelerating condi-
http:://axxvvvvc
abs 2112. 05146
Hyungjin Chung, Eun Sun Lee, and Jong Chul Ye. MR image denoising and super-resolution using regularized reverse diffusion. CoRR, abs/2203, 12621, 2022. doi: 10.48550/rrXiv2223...55 URL /ttps:://oi..rr//
www.448555riii.. 2203. 12621
Mohammad Zalbagi Darestani, Akshay $ Chaudhari, and Reinhard Heckel. Measuring robustness in deep learning based compressive sensing. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learming Research, pp. 2433-2444. PMLR, 18-24 Jul 2021. URL httpth::/ppoceeeiiiiiet mlr.
html
Arjun Desai, Andrew Schmidt, Elka Rubin, Christopher Sandino, Marianne Black, Valentina Mazzoli, Kathryn Stevens, Robert Boutin, Christopher Ré, Garry Gold, Brian Hargreaves, and Akshay Chaudhari. Skm-tea: A dataset for accelerated mri reconstruction with dense image labels for quantitative clinical evaluation. In J. Vanschoren and S. Yeung (eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, volume 1, 2021a. URL
neurips cttpp/pppp.ccpppppe//22111 ottp://aeec...r.......oooooooaa.
f F111:/06666652221009900003333333 oouneeeeeeeeeppppeaaroooorroeeee
Arjun D Desai, Batu M Ozturkler, Christopher M Sandino, Robert Boutin, Mare Willis, Shreyas Vasanawala, Brian A Hargreaves, Christopher M Ré, John M Pauly, and Akshay S Chaudhari. Noise2recon: Enabling joint mri reconstruction and denoising with semi-supervised and self- supervised learning, 2021b. URL httph::t////rttttt org. wwwgg/ab///1101 00075
Arjun D Desai, Andrew M Schmidt, Elka B Rubin, Christopher Michael Sandino, Marianne Susan Black, Valentina Mazzoli, Kathryn Stevens, Robert Boutin, Christopher Re, Garry E Gold, et al. Skm-tea: A dataset for accelerated mri reconstruction with dense image labels for quantitative clinical evaluation. In Thhirty-fftth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021c.
Arjun D Desai, Beliz Gunel, Batu M Ozturkler, Harris Beg, Shreyas Vasanawala, Brian A Harg- reaves, Christopher Ré, John M Pauly, and Akshay S Chaudhari. Vortex: Physics-driven data augmentations using consistency training for robust accelerated mri reconstruction, 2022. URL
https://arxiv wttp:o/g//aaoooooooooooooossssss 2111 02549
Bradley Efron. Tweedic's s formula and selection bias. Journal of the American Statistical Association, 106(496):1602-1614 2011. doi: 10. 10.1198/jasa2011 1.mml 1181.
Shreyas Fadnavis, Joshua Batson, and Eleftherios Garyfallidis. Patch2self: Denoising diffusion MRI with self-supervised learning. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tiee Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6- 2020, virtual, 2020. URL htll:tttpp::/prrceeediii
neuri wttpssccpppppppee 2t0.:/24440..7220000002222222--3
Eleftherios Garyfallidis, Matthew Brett, Bagrat Amirbekian, Ariel Rokem, Stéfan van der Walt, Maxime Descoteaux, and Ian Nimmo-Smith. Dipy, library for the analysis of diffusion MRI doi: 10.3389//inff.2144.0000 URL https
data. Frontiers Neuroinformatics, 8:8, 2014.
//doi 3389/fninf 201 00008
Eric K. Gibbons, Kyler K. Hodgson, Akshay S. Chaudhari, Lorie G. Richards, Jennifer J. Majersik, Ganesh Adluru, and Edward V.R. DiBella. Simultaneous NODDI and GFA parameter map generation from subsampled q-space imaging using deep learning. Magnetic Resonance in Medicine, (1(4:23399 2411, November 2018. doi: 10.11002mmm.27555 URL https://doi oorg/10 100.mmmmmmmmmmmm
11

---

Published as a conference paper at ICLR 2023
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan- Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Anmual Con- ference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL http:::/tt//tt....tt http::a::///cceeeeddeeee neurips. wtw.//pppr////////// Abst
Tao Huang, Songjiang Li, Xu Jia, Huchuan Lu, and Jianzhuang Liu. Neighbornneighbor: Self- supervised denoising from single noisy images. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 14781-14790. Computer Vision Foundation IEEE, 2021. URL / 0221//21100000022.2222 pdf
Aapo Hyvärinen. Estimation of non-normalized statiistt statistical models by score matching. J. Mach. Learn. Res., 6:695-709, 2005. htt.ttt:: oog.l...rrroooooooo httpp//.rg/appppp////////aaaaaaa
html
Zahra Kadkhodaie and Fero P Simoncelli. Solving linear inverse problems using the prior implicit in denoiser. arXiv preprint arXiv:2007.13644 2020.
Elena A. Kaye, Emily A. Ahern,, Cihan Duzgol, Ida Häggström, Erich Kobler, Yousef Mazaheri, Maggie M. Fung, Zhigang Zhang, Ricardo Otazo, Hebert A. Vargas, and Oguz Akin. Acceler- fnffrusioo-weiighed
ating prostate diffusion- weighted MRI using a guided denoising convolutional neural network: Retrospective feasibility study. Radiology: Artificial Intelligence, 265)e200007, August 2020. doi: 10.1 10.1148/raai2220020000
K Kwanyoung Kim and Jong Chul Ye. Noise2score: Tweedie's approach to self-superriseed image denoising without clean images. CoRR, abs/2106.07009, 2021. URL http:/ttt/taxii...rrrto
abs/2106.. 07009
Zhifeng Kong. Wei Ping. Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://
openreview wwt. wttt//ooumiid=aa..
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void - learning denoising from single noisy images. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA. USA, June 16-20, 2019, pp. 2129-2137. Computer Vision Foundation IEEE, 2019, 000223. http::tt::/rrrm httppsss/aaxxi..ccopp//////11111 10980. pdf
doi:
Samuli Laine, Tero Karras, Jaakko Lehtinen, and Timo Aila. High-quality self- supervipe deep image denoising. In Advances in Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver. BC, Canada, pp. 6968 0968-6978, 2019. URL http:/ttpp./pprcceeennggsgg// 2019/hash/ 7D07:hee/27Bcccaaact333000711113
htm_
Max W. Y. Lam, Jun Wang, Dan Su, and Dong Yu. BDDM: Bilateral denoising diffusion models for fast and high-quality speech synthesis. In International Conference Learning Representations, 2022. htthtttss///ppenie
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miik Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. In Jennifer G. Dy and Andreas Krause (eds.). Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmüssun, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 2971-2980, PMLR, 2018. //proceedings ing. ml wwwpppsssss///.
nen 8a html
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual. June 19-25, 2021, pp. 2837-2845, Computer Vision Foundation / IEEE, 2021. URL httpttt////// http://wwiiii.g....aa//// 21C3. 01
Kenneth Marek, Danna Jennings, Shirley Lasch, Andrew Siderowf, Caroline Tanner, Tanya Simuni, Chris Coffey, Karl Kieburtz, Emily Flagg, Sohini Chowdhury, Werner Poewe, Brit Mollenhauer,
12

---

Published as a conference paper at ICLR 2023
Paracelsus-Elena Klinik, Todd Sherer, Mark Frasier, Claire Meunier, Alice Rudolph, Cindy Casaceli, John Seibyl, Susan Mendick, Norbert Schuff, Ying Zhang, Arthur Toga, Karen Craw- ford, Alison Ansbach, Pasquale De Blasio, Michele Piovella, John Trojanowski, Les Shaw, Andrew Singleton, Keith Hawkins, Jamie Eberling, Deborah Brooks, David Russell, Laura Leary, Stewart Factor, Barbara Sommerfeld, Penelope Hogarth, Emily Pighetti, Karen Williams, David Standaert, Stephanie Guthrie, Robert Hauser, Holly Delgado. Joseph Jankovic, Chris- tine Hunter, Matthew Stern, Baochan Tran, Jim Leverenz, Marne Baca, Sam Frank, Cathi- Ann Thomas, Irene Richard, Cheryl Deeley, Linda Rees, Fabienne Sprenger, Elisabeth Lang, Holly Shill, Sanja Obradov, Hubert Fernandez, Adrienna Winters, Daniela Berg, Katharina Gauss, Douglas Galasko, Deborah Fontaine, Zoltan Mari, Melissa Gerstenhaber, David Brooks, Sophie Malloy, Paolo Barone, Katia Longo, Tom Comery, Bernard Ravina, Igor Grachev, Kim Gallagher, Michelle Collins, Katherine I.. Widnell, Suzanne Ostrowizki, Paulo Fontoura, Tony Ho. Johan Luthman, Marcel van der Brug, Alastair D. Reith, and Peggy Taylor The parkinson progression marker initiative (ppmi). Progress in Neurobiology, 95(4):629-633 http://oo.........o..oooooooo.aa //www. , sci sciencedi recttieetteoo com/ http://c........ee......eeee///t
2011. ISSN 0301-0082. doi:
URL https Bio-
logical Markers for Neurodegenerative Diseases.
Allister Mason, James Rioux, Sharon E. Clarke, Andreu Costa, Matthias Schmidt, Valerie Keough, Thien Huynh, and Steven Beyea. Comparison of objective image quality metrics expert radiologists' scoring of diagnostic quality of MR images. IEEE Transactions on Medical Imaging, (XX4):1044-1722 April 2020. doi:
1109/ 2019 9. 2930338
Christopher A Metzler, Ali Mousavi, Reinhard Heckel, and Richard G Baraniuk. Unsupervised learning with stein's 's unbiased risk estimator. arXiv preprint arxiv: 10531, 2018
Anish Mittal, Anush K. Moorthy, and Alan C. Bovik. Blind/referencellss image spatial quality
evaluator. In Michael B. Matthews (ed.), Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers, ACSCC 2011, Pacific Grove, CA, USA, November 6-9, 2011 1, pp. 723-727. IEEE, 2011. doi: 10. 10.1109/ACSSC.22111110000 1109/ ACSCCCCCC...CO 2011. 61 6109099
URL attps:///oli org/10
Tongyao Pang, Huan Zheng, Yuhui Quan, and Hui Ji. Recorupted-oo-rcoomuppie Unsupervised deep learning for image denoising. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pp. 2043-2052. Computer Vision Foundation / IEEE, 2021.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lever, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32: 8026-8037, 2019.
Mangal Prakash, Alexander Krull, and Florian Jug. Fully unsupervised diversity denoising with convolutional variational autoencoders. In 9th International Conference on Learning Repre- sentations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL
htttps://peeneeviee net/
Klaas Pruessmann, Markus Weiger, Markus B Scheidegger, and Peter Boesiger. Sense: sensitivity encoding for fast mri. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 42(5):952-962. 1999.
Philip M Robson, Aaron K Grant, Ananth J Madhuranthakam, Riccardo Lattanzi, Daniel K Sodickson, and Charles A McKenzie. Comprehensive quantification of signal-to-noise ratio and g-factor for image-based and k-space-based parallel imaging reconstructions. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 60(4):895-907 2008
Ariel Rokem. Stanford HARDI surfaces. 10 2016. doi: 0000044/m...gshharr4433333333333
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pp. 234-241. Springer, 2015.
13

---

Published as a conference paper at ICLR 2023
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. CoRR, abs/2104.07636, 2021. URL
httts::/aaxxis org/
07636
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffiusion models. In International Conference on Learning Representations, 2022. URL
Jaseha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Francis R. Bach and David M. Blei (eds.), Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pp. JoLR.org 2015. httpatppp/ppocceednngeeo mlr mtt.pppess//33///llldiiiiiaea html
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th Interna- tional Conference on Learning Representations. ICLR 2021. Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d' Alché-Buu, Emily B. Fox, and Roman Gameett (eds.), Advances in Neural Information Processing Systems 32: Annual Confer- ence on Neural Information Processing Systems 2019. NeurlPS 2019, December 8-14, 2019, Van- couver, BC, Canada, pp. 11895-11907 2019. URI htt.:p::s//pp.reeeeegeeeeee neure neuri wesuurrpp. cc/ paper/ 201 22209//hashhe httphhh301111ff23 2eleeee22277770533777 d5a 16d133371166ccdd3 2al55aa77aaadc33377777733333-333
html
Yang Song. Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In 9th Interna- tional Conference on Learning Representations, ICLR 2021, Virtaal Event, Austria, May 3-7, 2021. OpenReview.net, 2021b. WRRphttsss////eeee. rew www//oouuue 1d=:= IISEPTTIG2200 III
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations, 2022. URL / wttp:/weeeeooteeo.oooeeeeeeoooo wwweet//ooummmee
http::/wwwww..ttttooorooaaaaaaaa JOuGI
Dmitry Ulyanov, Andrea Vedaldi, and Victor S. Lempitsky. Deep image prior. In 2018 IEEE Conference on Computer Vision and Patlern Recognition, CVPR 2018, Sali Lake City, UT, USA, June 18-22, 2018. pp. 9446-9454. Computer Vision Foundation / IEEE Computer Society, 2018.
doi: 10.1109/CVPR.20 2098000944. URL http:://penaacess..... wtwpeeeaaaaccss..sceeeeeeeeeeeee thecvf com/ content cvpr 201 httpi//htmmmmllamaaaee
2018 B-18-_________ html
Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. Learning fast samplers for dif- fusion models by differentiating through sample quality. In International Conference Learning Representations, 2022. URL https: / http::///oppeeeeeeeeeeee wttp:pewwweeeeeeeeeeeeeeeeeeeeee fesee//foummmiddee VFB VFBjuE BHEp
Jeffrey P. Woodard and Monica P. Carley-Spencer.. No-reference image quality metrics for structural MRI. Neuroinformaatics, 4(3):243 2006. doi: 10.1385/NI:433:243. URL httpt:h/t///dt /oo
10g/10.133//11::::222
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. International Conference on Learning Representations, 2022. URL
hwwp///eeeeeeeeeeeeee eew net/ fanee/ffrrie forum wttpmmmmm.ddicccx/eeeeeeo
Linqi Zhou, Yilun Du, and Jiajun Wu. 3d shape generation and completion through poxnt-voxel diffusion. In 2021 IEEE/CVF International Conference on Computer Vision. ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pp. 5806-5815. IEEE, 2021. doi: 10. 109/CCCV4892..0211 00577. URL http:///0a///////////oo///////// 00577
14

---

Published as a conference paper at ICLR 2023
A TABLE OF SYMBOLS
For notation simplicity, we adopted alphahetic symbols in this paper to represent essential components in framework. For better symbol-name correspondences, here justify implications of all symbols used in the paper in Table to help readers comprehend.
Symbols
Implication
target noisy input noisy input prior
x'
y
clean ground truth
residual noise
scale coefficients balancing y and Gaussian noise as in Equation Stage I estimated clean image of x Stage I estimated residual noise arithmetic mean of denoising function in Stage I denoising function in Stage III state in the Markov chain
y
Me
x(-)
F())
noise model standard deviation of noise distribution noise schedule in diffusion models
3
Table 1: Table of symbols used in this paper.
B EXPERIMENTAL DETAILS
Implementation details. We implement both denoising functions 4 and F via U-Net Ronneberger al., 2015) with modifications suggested (Song al. 20216 Saharia al.
Stage Chen et al. 2021, we train F. condition on oe, 1 (1, The Adam optimizer was used optimize both
2021
we set the size of (x') n 2. In Stage III, inspired by (Song & Ermon 2019
networks with fixed learning rate of 1e-4 and a batch size of 32. We trained & for 1e steps and
F for le steps from scratch. All experiments were performed on RTX GeForce 2080-Ti GPUs in
PyTorch (Paszke et al., 2019).
Evaluation details. Our 2 compared against state-of-the-ar unsupervised denoising methods. For fair comparisons, we aligned the total number of parameters in DDM and all comparison methods: For U-Net based methods, we adopted the exact architecture as in DDM2: For MLP based methods, aligned the number of parameters.
(i) Deep Image Prior (Ulyanov al.) 2018) trains a neural network on random inputs to overfit a specific noisy image. The network is claimed to restore low-level consistencies exist in the target image an intermediate state of the training process. In our experiments, since there is no reliable quantitative metrics like PSNR, we pause network training based on SNR and the visual quality of the reconstructions; (ii) Noise2Noise (Lehtinen et al. 2018) constructs training pairs of two independent noisy measurements of the same target and trains network to transform one measurement to the other. 4D MRI by their nature own independent noisy samples acquired at different gradient directions. In our experiments, we train Noise2Noise model by using the same slices at different volumes: (iii) Patch2Self (Fadnavis et al., 2020) generalizes Noise2Noise and Noise2Self for patch-wise MRI denoising. In experiments, we follow their official implementation and adopted MLP as the regressor and patch radiance for their best performance; (iv) Noise2Score (Kim & Yel 2021) trains neural network estimate a noise-dependent score function to find the clean posterior distribution of noise inputs. In our experiments, estimate the score function under the same assumption of Gaussian noise.
C INTUITIVE DEMONSTRATION OF MARKOV CHAIN STATE MATCHING
In section 3.3, claim that matched state at denotes the minimum p-norm distance achieved between VB, and a (the standard deviation of the Gaussian fitted on the input image x). To verify the match intuitively, we show visual comparisons between x and the noise corrupted stage
15

---

Published as a conference paper at ICLR 2023
Noisy input
Stage matched
State matched
Noisy input
Stage matched
State #500 matched
Figure 7: Visual demonstration of different matches in the Markov chain. Our St.age matched state generates samples that look sufficiently close to the noisy input, while arbitrarily chosen states (e.g. state #500) cannot match the noisy input well.
10
warm-uu
reverse warm-up
0.8
Noisy input
0.4
0.2
Our denoised
200
500
(State
200
#state
600
800
1000
Figure 8: Comparison of the baseline warm-up up based schedule and our reverse warm-up based schedule. Our modified schedule allows noisy inputs represented later state than the baseline, which leads to more transition steps and better denoising quality.
result (So - Vory +6. (0,,,3 prior sample at the state St. A successful match leads to close image distributions (x and c in similar appearances), while x and may look considerably different in unsuccessful matches.
As shown in Figure 7 compared to naive representation at the middle Markov chain (state #500, as studied section 4.3), our stage matches the input in the Markov chain better. The successful match indicates there exists at least one possible sample drawn from the posterior distribution at the matched state that is sufficiently close to the noisy input. Therefore our claim in section 3.1 can be well proved. Without proper state matching strategy, the final generation results will be in poor quality (section 4.3 B).
D NOISE SCHEDULE WITH REVERSE WARM-UP
Noise levels in most of MRI datasets small. Matching them in Stage II with commonly used 'warm-up' based noise schedule 3 leads to few transitions in the sampling process and eventually poor generation quality. Considering this, adopted 'reverse warm-up' setting for
We compare x VIIL 3.) of the warm-up schedule (31 1e- Br 5c ratio= 0.7)
and our reverse warm-up schedule (31 5--5, 3r 1 ratio= 0.7) in Figure g Recall that x = Aςy + Azz (Eq. 1). During the sampling process, our reverse warm-up based strategy raises the value of À faster than the baseline strategy, hence reveals the underlying anatomical structures in MRIs much faster. This allows us to represent noisy inputs at a later state (e.g. state #200) in the Markov chain to enable more reverse process transitions. The baseline warm-up based schedule, the other hand, represents noisy inputs at a much earlier state (e.g. state #5). In this way, the final denoising result will be generated in very few transitions and eventually in inferior quality.
16

---

Published as a conference paper at ICLR 2023
Unconditionally sampled gSlider data
Unconditionally sampled Sherbrooke data
Figure 9: Unconditional sampling of DDM on our gSlider and the Sherbrooke datasets. All of generations are realistic due the restored anatomical details.
E UNCONDITIONAL SAMPLING OF DDM
Differing from all competing methods, besides functioning as denoiser, our diffusion model based method is capable of generating new (nonexistent) images that fit given dataset. We here conducted experiments to generate realistic MRI scans through unconditional sampling from our trained diffusion model. The generation results for our gSlider dataset and the Sherbrooke dataset are shown in Figure
These forged images successfully restored essential anatomical details in the MRI scans, which can be used to enlarge the dataset further to assist down-streaming tasks.
F OUTLIER DETECTION
As a diffusion model based method, the denoising performance of DDM2 is relatively input-dependent. In order to detect on which cases 2DM performs the best and which the worst, here present simple technique that can be adopted during DDM2 inference to detect poorly denoised outliers.
Specifically, we compute RMSE scores between the initial noisy input (S₂) and every image (S,-o) generated in the reverse diffusion process. For good denoising, RMSE values will increase sig- nificantly over iterations (represents denoised results dissimilar to inputs). For poorly denoised outliers, RMSE values will change slowly (represents denoising results similar to the original noisy measurement).
We provide demo results in Figure 10 Clearly, the poorly denoised slice (bottom slice) cannot achieve as high RMSE value as the well denoised one (top slice). This technique easy in implementation and can be adopted to detect poorly denoised outliers by 2DM effectively.
G EVALUATION ON SIMULATED DATA
Apart from the experiments conducted real-world datasets, we investigate whether DDM can still recover clean anatomical patterns by simulating noisy k-space data. We leverage the SKM-TEA dataset [Desai al. (2021c) for implementing these simulations since provides raw, complex, multi-echo, and multi-coil k-space. To simulate the impact of adding additional complex noise to the k-space data, we leverage k-space noise addition strategies that have been validated in prior work (Desai et al.) 2021bb 2022). Briefly, we sample random complex Gaussian noise with varying standard deviations (to simulate different SNR levels) and apply these noise maps to each coil k-space perform image reconstruction to transform the k-space data into magnitude
data. Subsequently, images with varying levels of noise. We simulated noisy images at 4 different SNR levels in total. We also experiment how the number of prior volumes (i.e. n) impact the performance of Patch2Self and DDM2 under these simulation settings. In this case, we conduct experiments with (n=1) and (n=10) prior volume. While changes are made the hyper-passmeters of DDM2, we follow
17

---

Published as a conference paper at ICLR 2023
Noisy Image
DDM
RMSE plots
Well denoised
5
Sp
Poorly denoised
5
So
Figure 10: RMSE plot comparisons between well denoised and poorly denoised slices.
the suggestions in Patch2Self and increase its patch radius for both experiments. Since there is an available ground-truth compare to, we compute quantitative (PSNR and SSIM metries) results reported for the simulated noisy images and the resulting 2DM and Patch2Self denoised volumes.
In Figure we show the denoising results on the simulated noisy images with only n= prior volume available. When input images are in high SNR, DDM demonstrates on- on-par results to Patch2Self. On low SNR images, DDM surpasses Patch2Self significantly terms of both visual quality and quantitative metrics. The same phenomenon can be also observed in Figure [2] where results are obtained with n= 10 prior volumes. Noteworthy, when the input images are highly corrupeed (the left most column), 2DM still recovers valid anatomical pattern due its generative nature. Patch2Self, on the other hand, fails to denoise on low SNR inputs.
H ADDITIONAL COMPARISONS
Compare with different Patch2Self settings. the main paper, we re-implemented Patch2Self with MLP denoisors the same network scale as DDM for fair comparison. Here compare against official Patch2Self implementations under different settings. Specifically, we compare with three different denoisors (MLP, OLS, Ridge) and explore larger patch radius. Note that our CPUs crashed when patch radius is larger than 1, yielding its unaffordable computational costs.
Denoising results shown in Figure 13
No significant differences be observed from the results for Patch2Self with different denoisors. Although setting a larger patch radius improves the results slightly, computational costs increase cubicly and common CPUs with 32 GB memory cannot afford.
Compare with classic Gaussian noise denoisors. Under our Gaussian noise assumption, one may wonder if classic denoising methods can be directly applied on dMRI data and our three-stage DDM is not necessary. Here, we compare with two commonly used classic methods to prove the essence of 2 Unbiased Risk Estimator (SURE) Metzler al. (2018) and the Implicit Prior (IP) Kadkhodaie Simoncelli (2020). Moreover, we also combine these two methods and use one to refine the results from another to if performance boosts be observed.
Denoising results are shown in Figure 14 As common denoisors for Gaussian noise, both SURE and IP fail to denoise dMRI signals and the combination of these doesn't increase denoising quality
18

---

Published as a conference paper at ICLR 2023
Increasing SNR
Low SNR
High SNR
Ground Truth
Noisy Image
SSiMf:
SSIM: 77 P3NE,3L.
SSM 0:96
PANR:
SSIM-- 0138 PONR: 21.0
SSIM: 0.68
SSIM 0.86
SSIM 0:92
PENA 26.3
PSNE-34.1
PSUR 37.2
Ours
Ours Residual
SS/M 0.52 1ONB-166
ESIM. 0.45
EBIM: C:83
SEIM C36
P2S
P25 Residual
Figure 11: Results simulated data with n= 1 shows that in low-noise (high SNR) settings, DDM2 and Patch2Self exhibit comparable performance. However, as noise levels increase (lower SNR), DDM maintains denoising performance, while Patch2Self is unable to perform denoising. All images best viewed zoomed in.
expected. We note that Noise2Score Kim & Ye (2021) is recent method that extends SURE for better score estimation. After applying IP as a post-processor for Noise2Score, the final denoising results still significantly inferior than ours. Therefore, claim that, with the Gaussian noise assumption, 2DM still learns meaningful noise distributions from the dataset itself, which can not be achieved by any of the comparison methods.
Compare under mixed b-value images. In the main paper, we mainly focused on homogeneous b-values since these represent the most clinical and research MRI protocols (where one b- value shell sampled across multiple directions). However, we are also interested denoising methods can function images of mixed b-values. We adopt the Sherbrooke 3-Shell dataset for this experiment. In details, we train on mixture of b-value=1000 and b-value=2000 images and try to denoise the b b-value=2000 ones.
Denoising results in Figure 15 depict that the resultant images, although denoised, have a high amount of blurring. This is most likely due the fact that different b-values have different gradient strengths, and as a result, different distortion artifacts induced by the echo planar readout. This distortion across b-values manifests blurring in the denoised image.
ADDITIONAL RESULTS
DTI Diffusivity results. In addition the factional anisotropy maps presented in section 4.2 and Figure 4 here we perform additional DTI measurements of Axial Diffusivity (AD), Mean Diffusivity (MD), and Radial Diffusivity (RD) to further validate DDM According to the comparison
19

---

Published as a conference paper at ICLR 2023
Increasing SNR
Low SNR
High SNR
Ground Truth
Noisy Image
PSNR:: 12.1
SSN 28
PENRUSLO
PSNR, 42.7
S3IM:
331M:0.67
5SIM:
0,34
25.2
30.5
Ours
Ours Residual
SSIM- 0.56
SSIM 0.86
SSIM 8.96
PSNR: 28.9
PSHRUB2
46.7
P2S
P2S P25 Residual
Figure 12: Results on simulated data with n= 10 supporting volumes. These images indicate that the performance of DDM is comparable independent of how many supporting volumes are available, unlike Patch2Self which requires increasing number of volumes that may not always be clinically available, to achieve reasonable performance. All images best viewed in zoomed in.
Noisy Image
P25 (MLP)
P2S (OLS)
P2S (Ridge)
P2S (r=1)
Ours
Additional comparison with different Patch2Self settings depict similar performance across the hyperparameter space for Patch2Self. All Patch2Self models still suffer from how SNR, compared to DDM
Figure 13:
results shown Figure 16 both AD of Patch2Self and Noise2Score show very smooth signals outside the ventricles. However, 2DM2 shows lot more of the natural structure.
Qualitative results. Additional qualitative comparisons on the four datasets are presented in Figure 17 Obviously, DDM not only reduces noise the most but also restores anatomical details the best.
we present slice-wise improvements the patch-wise processing (Fadnavis al. 2020). To validate our design choice, section 5 we presented an alternative design of DDM that functions on 3D volume patches rather than 2D slices. An intuitive comparison between the two designs outlined in Figure 18.
3D volume-wise 2DM In section 3.2
The best results for 3D volume-wiss 2DM were achieved patch size 16 16 16 16-also the largest volume patches we can afford our GPU memory. Empirically, we found that when functioning on small-size patches, diffusion models fail denoise but instead overfit and return the exact inputs.
20

---

Published as a conference paper at ICLR 2023
Noisy Image
SURE
SURE IP
N25 + IP
Ours
Figure 14: Additional comparison with classic Gaussian noise denoisors.
P2S
P2S (2000&1000)
Ours (2000)
Ours (2000&1000)
Noisy Image
(2000)
Figure 15: Additional results when training on mixed b-value images exhibit adequate denoising but excessive blurring, likely due different distortion profiles between the b=1000 and b=2000 scans that lead to anatomical inconsistencies.
Comparisons at the transwerse, coronal, and sagittal planes between the 2D slice-wise 2DM and 3D volume-wise DDM2 are shown in Figure 6 and Figure 19
Stage I Comparisons. In Figure 20 we present additional qualitative comparisons between the Stage I results and our final results. Clearly, Stage of DDM2 alone cannot restore indicative geometries and textures of the brain. With our Stage II and Stage III, the diffusion model is able to generate such missing contents for more promising denoising results.
Noise residual comparisons. In Figure 21 present noise residual comparisons between Patch2Self and DDM2. Our method suppresses more noise and does not show any anatomical structures in the corresponding residual plots.
21

---

Published as a conference paper at ICLR 2023
Noisy Image
Patch2Self
Noise2Score
Ours
Axial Diffusivity
Mean Diffusivity
Radial Diffusivity
Figure 16: Comparison results of Axial Diffusivity, Mean Diffusivity, and Radial Diffuusivity.
22

---

Published as a conference paper at ICLR 2023
Noisy Image
DIP
Noise2Noise
Patch2Self
Noise2Score
Ours
gSlider
Sherbrooke
Stanford HARDI
PPMI
Figure 17: Additional qualitative results. 2DM2 demonstrates superior denoising quality in both image restorations and faithful reproduction of the original image contrasts. Arrows depict regions of hyperintense signal that may be caused by additional anatomical structure being represented by the other diffusion directions. Best viewed in zoomed in.
2D slice-wise DDM?
3D valume-wise DDM
outtout
Figure 18: Intuitive comparisons between Left: 2D slice-wise DDM? (adopted) and Right: 3D
volume-wis wise DDM?.
23

---

Published as a conference paper at ICLR 2023
Figure 19: Additional comparisons at the transverse, Coronal, and Sagittal planes. A: Noisy input. B:
results of volume-wise DDM C: results of slice-wise DDM
gSlider
Sherbrooke
Noisy Image
Ours Stage!
Ours Final
Figure 20: Additional comparisons between DDM Stage I results and DDM final results on our gSlider dataset and the Sherbrooke dataset. Arrows indicate possible artifacts caused during denoising, potentially due to partial volume effects affecting the edges of the grey matter or possible hallucinations induced by the denoising blurring. Major differences are highlighted.
Patch2Self
DDM2 residuals
Noisy Image
Patch2Self
residuals
DDM
gSlider
Sherbrooke
Figure 21: Residual noise comparisons between Patch2Self and 2
24


Reviews:
[{"review_id": "cFlDvxcK6t", "reviewer": "Reviewer_M8jP", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new method for the denoising of diffusion weighted images with diffusion denoising models. It brings about a series of modifications on existing literature that actually determines higher performance compared to state-of-the-art.\n\nThe article presents extensive validation which is highly appreciated, and shows results from comparing state-of-the-art techniques. My main concern with this is the lack of synthetic data validation, which is something that (unfortunately) many articles on the subject lack. I think that when talking about denoising it would be somehow mandatory to display synthetic results, even more when denoising is achieved through machine learning, where the risk of introducing \"artefactual allucinations\" in the images is very high.\n\n", "summary_of_the_review": "The article tackles a challenging problem. It comes up with new ideas/modifications with respect to the state of the art. It achieves remarkable results although the full extent of these is difficult to evaluate. This is not just a problem of this article but of all the comparing state-of-the-art works, since they do not provide results on synthetic data. This is a problem with machine learning/generative techniques for denoising of medical data, because allucinations can cause dramatic effects. I would not use this denoising method on data from patients. Nevertheless, the methodology described in the article is certainly of interest for the community and can inspire other researchers. As such, the article has a great value and represent a step forward in the good direction. Despite synthetic validation is missing, favorable comparison with many competing methods has been clearly shown.\n\nTo summarize, the article discloses some key ideas to solve the denoising problem using diffusion methods. These ideas are valuable and of interest, and lead to improved performance. The problem statement is clearly formulated, and the literature well acknowledged. I recommend  the acceptance of the article as is. If a second revision stage is foreseen, I would recommend the authors to take into account my comments.", "strength_and_weaknesses": "The article is interesting, inspiring and introduces a series of improvements that are of interest for the community, as also is the shown performance. I like the formulation of the solution to the problem in three stages and the advantage of each of them is clearly presented with quantitative information.\n\nThe main weakness remains, in my opinion, the lack of synthetic validation. Despite many \"real data\" examples it is not possible to assess the amount of allucinations generated by the methodology and the amount of intensity bias introduced with each pixel.\n\nIn my experience with Patch2Self on synthetically generated data I could prove that the estimation of quantitative measurements (e.g. the estimation of diffusivities) is subject to a bias, which is not present, for instance, with methdos based on MPPCA. In this sense, without synthetic results it is not possible to assess this kind of performance.\n\nI note, especially in figure 11 and 14 , that indeed the DDM2 method may introduce relevant allucinations. For instance, in fig. 11 please look at the first raw, and observe the cerebrospinal fluid areas in the middle of the brain, which have very different shapes compared to the noisy image. This is an example of \"invented\" tissue contrast. Another example is found by looking at the first column image of each row of figure 14 (left hand side). In the bottom left part of the brain (just beneath the red square) there is an \"island\" of tissue contrast that is better preserved in the stage 1 compared to the final result.\n\nI think that the authors should point out with arrows to this kind of differences. While reading I had the impression that authors were stressing a bit too much the fact that there were very few allucinations, and in my opinion, that is not the case. In fact, I strengthen the comment about the lack of synthetic experiments!\n\n\nI have other points that I would like the author to address:\n- Does the presented method generalize to the case when the noise variance varies spatially? In my opinion it does not, so please mention that a limitation.\n- Noise in MRI is spatially correlated (use of Partial Fourier, for example): what are the consequences of this for the approach?\n- Noise in MRI is non-Gaussian (e.g. Rician): how does this affect the proposed method which clearly only works for Gaussian data? In section 5 authors mention SENSE, which does not perturb the Gaussian nature of the noise on complex images, but still yields Rician magnitude images!\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "bGN_3-OY8T-", "reviewer": "Reviewer_RsQ4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new method for denoising diffusion MRI data using recent advances in reverse diffusion. The method proposed a three stage approach and builds on top of Patch2Self which is the state-of-the-art method in the field of Diffusion MRI. The method is developed using generative models which indeed show potential for denoising applications.", "summary_of_the_review": "While the method is new, it has quite a few shortcomings in its current form. Answering the above questions and improving the validation analyses should fix the issues.\n", "strength_and_weaknesses": "Strengths\n\nThe proposed method makes use of diffusion models to perform the denoising. While the approach seems useful, there are some questions which need to be answered:\n\nWeakness\n\nThe official implementation of Patch2Self does not have MLP as a regression model. It is stated clearly in the GitHub repository of the Patch2Self implementation that DIPY implementation is the official one. Was this the one that was used? If yes, please can you provide the model parameters set? \nIn the supplement of the Patch2Self paper, the 1st figure shows that MLP performs worse than OLS and Ridge. Why was this chosen as the regression model for comparison?\nThe authors of the paper compare the performance of DDM2 with DIP, Noise2Noise and Noise2Score. DIP and Noise2Noise are not valid for dMRI data on may accounts. Was care taken to ensure that the training was performed on dMRI data? I would assume that these models would not do a good job out-of-the box. \n\nThe authors say that the Patch2Self was compared in all cases with patch radius 0 and MLP model. But as per the Patch2Self paper (Fig. 6C), if the number of volumes is lower, the patch radius needs to be increased. This makes the comparison in Fig. 5 incorrect.  The official implementation of Patch2Self needs at least 10 volumes for optimal performance. \n\nIt is also unclear why 2D slice-to-slice would not have structure in the residuals as opposed to 3D volume-to-volume mapping. This needs to be discussed in further detail. \n\nIn Figure 15, for the Sherbrooke dataset, the residuals are flipped -- please rectify this mistake. \n\nThe authors do not do a good job on validating the performance of the denoiser. While FA is an important metric the way SNR and CNR is computed is invalid. Furthermore the comparison needs to be done on different microstructure measures. \n\nGenerative models typically run the risk of hallucinating structures -- which may not be visible in the residuals. An analysis showing this needs to be added. This can be shown via DEC FA maps and Tractography analyses. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HJM-tGEafm", "reviewer": "Reviewer_Bjyp", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a self-supervised denoising method for diffusion MRI with diffusion model. The model consists of three steps which 1) learn noise pattern, 2) match an input to different diffusion stages and 3) diffusion model that generate denoised image. The model is validated on 4 different dataset to demonstrate generalization capability of the method.  ", "summary_of_the_review": "I am not a super expert in diffusion model so I cannot judge very well on the novelty, but I think the differences of this work from the prior ones are well explained. \nText is written clearly to demonstrate the authors' ideas and they are well validated through the experiments. ", "strength_and_weaknesses": "Strength:\n- Clear text and step-by-step description of the method. \n- Differences from prior works well explained. \n\nQuestion (rather than weakness):\n- It would be nice to have an explanation how prior denoising of MIR are performed with supervision, as typical denoising methods for images are unsupervised. \n- In Stage I, patches are replaced by image slices and slice-to-slice mappings are learned. There must be drawbacks from such a coarse mapping (e.g., losing local structures in the image space) and I am wondering if the authors have any thoughts on this. \n- Is there any backbone model to approximate necessary functions within the framework?\n- What are $\\lambda$ in equation (4)?\n- In the method, it is not clear where the self-supervised training is occuring. \n- Is the noise in typical MRI or diffusion MRI iid or systematic? The noisy images in the experiments are synthesized with different $\\beta$ levels (correct me if I misunderstood) and I am not sure if they truly mimic the noise pattern in reality. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "AkyFZFnwEP", "reviewer": "Reviewer_Wzbi", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides an approach for self-supervised denoising of MRI data using diffusion-based generative models. The approach is applied to denoise diffusion-weighted MRI, hence the name DDM2. A three-stage approach is used to train/denoise: 1. learning a denoising function with existing self-supervised approaches; 2. fitting the noise statistics to a Gaussian; and 3. training a conditional diffusion model using the noisy data and noise statistics. The method is evaluated on four different datasets of diffusion-weighted MRI. Comparisons to other self-supervised methods are shown, as well as ablation experiments.\n", "summary_of_the_review": "The authors develop a three-stage approach to denoising diffusion-weighted MRI using self-supervised denoising diffusion models. The method appears to improve over other self-supervised baselines, though some details remain unclear. The contribution is clear and the work is well-organized and novel.\n", "strength_and_weaknesses": "Strengths:\n- The landscape of self-supervised image denoising is quite crowded and can be difficult to navigate. The authors do a pretty good job contextualizing their work within this space.\n- The approach to training a conditional diffusion model through spatially shuffling the noise residual is interesting.\n- The ablation results lend credibility to the need for the three-stage approach, differentiating it from other self-supervised denoising works\n- Extensive experiments are conducted on different datasets\n- The paper is very detailed and code to reproduce results are provided\n\nWeaknesses:\n- It is unclear how this work fits in with other denoising methods such as [1] and [2]. For example, as the noise is assumed Gaussian, could [1] simply be used to denoise the volumes. Furthermore, could the denoiser in [1] be used according to the strategy in [2] to further refine the solution? To me this seems more straightforward than the proposed three-stage approach.\n- How would the approach compare to running annealed Langevin dynamics using a pre-trained MRI diffusion model (for example, from [3]) with an identity A operator? In other words, is the scan-specific nature of the method necessary?\n- As the approach is not actually specific to diffusion MRI, and the authors report that even n=1 is sufficient for learning the denoiser, could a different MRI dataset (where GT is known) be used to validate the methods (with noise added on top of the GT)? For example, from FastMRI or SKM-TEA?\n- It is unclear from reading the paper if multiple diffusion directions is necessary\n- It seems like all the datasets have multiple scans at the same b-value (for example 10 have b=0 and 60 have b=1000). Are these b-value images collected with different diffusion directions? What would happen if the images with the same b-value and direction are simply averaged together? Could this be used as a GT, either for comparison or for some other supervised approach?\n- It is unclear how the dimensions of the data correspond to the directions/b-values. Please explain the dimension sizes as they relate to the # of directions and b-values.\n- is the method only applicable to denoising with multiple b-values? do the b-values need to be the same for denoising to work?\n- It is said that two neuroradiologists reviewed the images. Can you please comment more on this evaluation? How many images were reviewed per dataset? why wasn't a larger evaluation conducted using the neuroradiologists, for example to rate image quality, SNR, CNR, fine details, etc?\n- Could the authors test (or at least comment) on the impact of Rician vs. Gaussianity, for instance with GT data and additive noise? It was unclear in the text whether the data used are complex-valued or magnitude only, and what the impact is of the Gaussianity assumption.\n\n[1] Christopher A Metzler, Ali Mousavi, Reinhard Heckel, Richard G Baraniuk, Unsupervised learning with Stein's unbiased risk estimator, arXiv preprint arXiv:1805.10531\n[2] Zahra Kadkhodaie, Eero P. Simoncelli, Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser, https://arxiv.org/abs/2007.13640 \n[3] Ajil Jalal, Marius Arvinte, Giannis Daras, Eric Price, Alexandros G Dimakis, Jon Tamir, Robust compressed sensing mri with deep generative priors.  Advances in Neural Information Processing Systems\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster