Robust Compressed Sensing MRI with Deep Generative Priors
Authors: Ajil Jalal
Marius Arvinte
Giannis Daras
Eric Price
Alex Dimakis
Jonathan Tamir
Venue: NeurIPS 2021 Poster (2021)
Keywords: Compressed sensing
generative priors
MRI reconstruction
Langevin dynamics
Inverse problems

Abstract: The CSGM framework (Bora-Jalal-Price-Dimakis'17) has shown that deep
generative priors can be powerful tools for solving inverse problems.
However, to date this framework has been empirically successful only on
certain datasets (for example, human faces and MNIST digits), and it
is known to perform poorly on out-of-distribution samples. In this
paper, we present the first successful application of the CSGM
framework on clinical MRI data. We train a generative prior on brain
scans from the fastMRI dataset, and show that posterior sampling via
Langevin dynamics achieves high quality reconstructions. Furthermore,
our experiments and theory show that posterior sampling is robust to
changes in the ground-truth distribution and measurement process.
Our code and models are available at: 
\url{https://github.com/utcsilab/csgm-mri-langevin}.

Robust Compressed Sensing MRI with Deep Generative Priors
Ajil Jalal" ECE, UT Austin ajiljalal@utexen edu
Marius Arvinte" ECE, UT Austin arvintelutexas edu
Giannis Daras CS, UT Austin gi giamissdaaasuutexas edu
Eric Price CS, UT Austin ecprice@cs ute.as.
Alexandros G. Dimakis
Jonathan Tamir ECE, UT Austin j jtamir@utexas edu
ECE, UT Austin
dimakisčaustinn wtw.us.eeeee.... edu
Abstract
The CSGM framework Saa---alll--r---Diiie erative priors can be powerful tools for solving inverse problems. However, to date this framework has been empirically successful only on certain datasets (for example, human faces and MNIST digits), and it is known to perform poorly on samples. In this paper, we present the first successful application of the CSGM framework on clinical MRI data. We train generative prior on brain scans from the fastMRI dataset, and show that posterior sampling via Langevin dynamics achieves high quality reconstructions. Furthermore, our experiments and theory show that posterior sampling is robust to changes in the ground- -ruut distribution and measurement process. Our code and models are available at: /tgpt:://ithubb www.mmucssaab///ooooco..oaaaaaat hla.ll/g//mmmmmmmeeee mri meri-lannevin.
17) has shown that deep gen-
Introduction
Compressed sensing [23, 15] has enabled reductions to the number of measurements needed for successful reconstruction in variety of imaging inverse problems. In particular, it has led shorter scan times for magnetic resonance imaging (MRI) [62, 90], and most MRI vendors have released products leveraging this framework to accelerate clinical workflows. Despite their successes, sparsity- based methods are limited by the achievable acceleration rates, as the sparsity assumptions are either hand-crafted or are limited to simple learned sparse codes [72, 73].
More recently, deep learning techniques have been used as powerful data-driven reconstruction methods for inverse problems [49, 68]. There two broad families of deep learning inversion techniques [68]: end-to-end supervised and Distributionnleammin approaches. End-to-end supervised techniques
training set of measured images and deploy convolutional neural networks (CNNs) and other architectures to learn the inverse mapping from measurements to image. Network architec- tures that include both CNN blocks and the imaging forward model have grown in popularity, as they combine deep learning with the compressed sensing optimization framework, see e.g. [32, 3, 64). End-to-end methods are trained for specific imaging anatomy and measurement models and show excellent performance in these tasks. However, reconstruction quality is known to suffer when applied of distribution, and recently has been shown severely degrade [4, 19] under certain types of natural measurement and anatomy perturbations.
In this paper study deep learning inversion techniques based on distribution learning. These models are trained without reference to measurements, and so easily adapt to changes in the measurement
Ajil Jalal and Marius Arvinte contributed equally this work.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

process. The most common family of such techniques, known also as Compressed Sensing with Generative Models (CSGM) [13] uses pre-trained generative models as priors. Generative models are extremely powerful at representing image statistics and CSGM has been successfully applied to numerous inverse problems [13, 34] including non-linear phase retrieval [35], and improved
with invertible models [6], sparsity based deviations [21], image adaptivity [42], and posterior sampling [79, 45J. These methods have only recently been applied to MRI and have not yet been shown to be competitive with supervised end-to-end methods. The very recent work [53] trains StyleGAN for magnitude-only DICOM images but requires the presence of side-information and studies Gaussian, real-valued measurements for reconstruction. The deviation from the true MRI measurement model and the use of magnitude images are known to be problematic when evaluating performance [77]. Another work [54] trained an Invertible Neural Network complex- valued single-coil MR images and showed very good performance in comparison to sparsity and GAN priors. Untrained and unamortized generators [37] have also been recently explored [19], showing promising results in some cases. Further, [17] studies the harder problem of learning a generative model for a class of images using only partial observations, first proposed in AmbientGAN [14].
In this paper we train the first score-based generative model [80] for MR images. We show that we can faithfully represent MR images without any assumptions the measurement system. As consequence, we are able to reconstruct retrospectively under-sane pnder-sampled MRI data under a variety of realistic sampling schemes. We show that reconstruction algorithm is competitive with end-to-end supervised training when the test-data are matched to the training data and that is robust to various outt-f-diisribution shifts, while in some cases end-to-end methods significantly degrade.
1.1 Contributions
We successfully train a score-based deep generative model for complex-valued T2- T2-weighted brain MR images without any assumptions on the measurement scheme. When applied to multi-coil MRI reconstruction under the CSGM framework, we achieve competitive performance compared to end-to-end deep learning methods when the test-time data are sampled within distribution.
We give evidence that posterior sampling should give high-quality reconstructions. First, we show that for any measurements (including the Fourier measurements in MRI) that posterior sampling with the correct prior is within constant factors of the optimal recovery method; second, even if the prior is wrong but gives a mass to the true distribution, we show that posterior sampling for Gaussian measurements is nearly optimal with just an additive (log(1/a)) loss.
We empirically show that our approach is robust to test- rest-time distribution shifts including different sampling patterns and imaging anatomy. The former is unsurprising given that our model was trained without knowledge of the measurement scheme. As consequence, our approach provides a degree of flexibility in choosing scan parameters a common situation in routine clinical imaging. Perhaps surprisingly, the latter indicates that specialized training set may offer sufficient regularization for larger class of images. In contrast, we empirically show that end-to-end methods do not always enjoy the same robustness guarantees, in some cases leading severe degradation in reconstruction quality when applied out-of-distribution.
Our method can be used to obtain multiple samples from the posterior by running Langevin dynamics with different random initializations. This allows us to get multiple reconstructions which can be used to obtaín confidence intervals for each reconstructed voxel and visualize our reconstruction uncertainty on voxel-by-voxel resolution. Uncertainty quantification can be incorporated into end-to-end methods, e.g., using variational auto-encoders [24], but this requires changes to the architecture. Our method does not require any modification and multiple reconstruction samplers can be run in parallel.
Our main results are succinetly summarized in Figure 1: we achieve equivalent reconstruction performance using reduced training set when evaluated in in-distribution and are robust when evaluated out-of-distribution.
1.2 Related Work
Generative priors have shown great utility to improving compressed sensing and other inverse problems, starting with [13], who generalized the theoretical framework of compressed sensing and restricted eigenvalue conditions [85, 23, 12, 15, 40, 11, 10, 25] for signals lying on the range of

---

Fully-Sampled MVUF
L1-Wavelet
ConwDeroder
MoDI
E2E-Vamett
Langevin (Durs)
In-dittibation
2S 020 PAM 3h4
20 20 CIR 35 5545
25 03
Suk. 28 1540
05 PAh AN
SON S2MOOW
55H
25 OA PAR THO
20 WO SIR: 15-
SAR 194
22 RID SAR
OW PAN 33-0
Horizonttll
SSM SHE 24:24
SSM 0.0 PAR 17-
SEM 0ND MAR
SSM 000 MA
Figure 1: Comparison of reconstruction methods for in-distribution, sampling and anatomy shift images. methods and hyperparameters were optimized on T2-weighted brain scans and tested at higher accelerations, horizontal masks, and on knee & abdomen scans. Our reconstructions are competitive with statc-of-the-as methods, and introduce fewer artifacts out of distribution. All measurements multicoil k-space from the NYU fastMRI dataset and the supervised baselines trained from scratch on MVUE targets for fair comparison.
vertical sampling mask,
deep generative model [29, 55, 81). Lower bounds in [51, 61, 48] established that the sample complexities [13] are order optimal. The approach in [13] has been generalized to tackle different inverse problems [47, 35, 7, 71, 60, 63, 74, 9], and different reconstruction algorithms [21, 50, 69, 27, 26, 64, 37, 38, 18]. The complexity of optimization algorithms using generative models have been analyzed in [28, 39, 58, 36). Our prior work shows that posterior sampling is restance-optimat for compressed sensing [45]. and satisfies certain fairness guarantees without explicit information about protected sensitive groups [46].
Using compressed sensing for multi-coil MRI reconstruction has led rich body of work in the past two decades [62, 20, 87, 75). See [22] and the recent special issue [44] for an overview of these methods. Classical approaches impose sparsity in a well-chosen basis, such as the wavelet domain [62], or apply shallow learning that leverages low-level redundancy in the images [72, 73, 93]. Recent research has demonstrated the superior performance of deep neural networks for MR image reconstruction [76, 32, 3, 82, 83). A broad class of approaches is represented by end-to- unrolled methods, which use deep networks as learned data priors in the image [3, 32, 82] or k-space domain [84]. Recent work has also investigated the performance of untrained methods [89, 38] for MR reconstruction and has shown competitive results. much less explored line of research is MR image reconstruction with generative priors. The work in [67] proposes CSGM-like algorithm that finetunes an entire pre-trained generator that requires a carefully tuned optimization algorithm during inference.
3

---

45
45
Langevin (Ours)
L1-Wavelet
ConvDrcoder
40
40
McDL
PSNR (aB)
PSNR (dB)
EZE-Varnet
35
35
30
30
25
25
12
2
3
4
6
8
12
R
(a) Brains Vertical Mask
(b) Brains Horizontal mask
45
45
PSNR (dB)
40
PSNR (dB)
40
35
35
30
30
25
25
12
2
4
8
12
R
(c) Abdomens
(d) Proton Density Knees
Figure 2: Average test PSNR in various scenarios, across range of acceleration factors R. Higher R indicates smaller number of acquired measurements. All methods and hyperparameters were optimized brains with an equispaced vertical mask. Our approach mostly shows best performance lowest reconstruction variance both in- and out-of-distribution at test-time. Shaded regions indicate 95% confidence intervals. Note that we trained baselines MVUE images and hence these numerical values should not be compared with those in literature trained on RSS images (see Appendix A.1 for more detailed discussion).
System Model and Algorithm
2.1 Multi-coil Magnetic Resonance Imaging
MRI is medical imaging modality that makes measurements using an array of radio-frequency coils placed around the body. Each coil is spatially sensitive to local region, and measurements acquired directly the spatial frequency, or k-space, domain. To decrease sean time, reduce operating costs, and improve patient comfort, reduced number of k-space measurements clinical use and reconstructed by incorporating explicit or implicit knowledge of the spatial sensitivity maps [78, 70, 30). Formally, the vector of measurements y/i CL acquired by the jth coil can be characterized by the forward model [70]:
acquired in
y/ PFS Wss, i 1, 1,..,, Ne.
where x" E CN is the image containing N pixels, S, is operator representing the point-wise multiplication of the tt coil sensitivity map, F is the spatial Fourier transform operator, P represents the k-space sampling operator, and assume w; NN (0, for simplicity. Importantly, note that the same under-sampling operator is applied to all N₂ coils.
The acceleration factor R denotes the degree of under-sampling in the k-space domain, i.c., R = N Due to the multiple coils, the measurements may not be compressive for small R. However, due to redundancy between the coils, the measurements are compressive for moderate values of R (even if N. L > N) [41]. Also note that we use the true acceleration R, and this does not match values in fastMRI [56]* on certain sampling patterns.
'https: //gp::////ttubb com/ catpauu.ccomtaccoooooooooo httpre/rrreereeeeeeaaaa////////e http///mann/aassaa ters tertttrr/iaeeee /attrr//aaa///aa /ttpa//daaaaaaaapaaaac line 247 has the fastMRI definition of equispaced acceleration factors.

---

Given multi-coil measurements y, sensitivity maps represented by S and the sampling operator P. the goal of MR image reconstruction is estimate the underlying image variable x², Prior work formulates this as regularized optimization problem:
arg lyy Arl + ^Q(x),
where we use the operator A CMxX with M No- L) to subsume the discrete approximation to all linear effects, and Q is a suitably chosen functional prior for the image variable x. For example, to enforce a sparsity prior, one can penalize the & norm in the wavelet representation of æ [62]. More recent approaches involve learned regularization terms parameterized by deep neural networks [76, 32, 3]. These models are typically trained end-to-end using fixed training set and certain assumptions about the sampling operator. In the sequel, we present how score-based generative models can be combined with the posterior sampling [45] mechanism to reformulate (2) and achieve good quality reconstructions without any priori assumptions about the sampling scheme.
When k-space is fully sampled at the Nyquist rate and no regularization is applied, the solution to (2) corresponds to the minimum-varianee unbiased estimator (MVUE) of x², denoted by ÊMVUE [70]. Given fully sampled k-space data, this estimate can act as reference image for evaluating reconstruction error as well as for end-to-end training. Alternatively, a reference image called the roof-sum-of-squares (RSS) estimate can be formed by taking the inverse Fourier transform of each coil and subsequently applying the lg norm for each pixel across the coil dimension, i.c.
No PRSS Σ DE (F y.)l" where F is the Hermitian transpose of F (here the inverse DFT).
Although the RSS estimate biased estimator, it often used it does not make any assumptions about the sensitivity maps, which are not explicitly measured by the MRI system. However, even if solving (2) results in perfect recovery of z², there will be a bias when comparing the result TRss and thus the RSS and MVUE cannot be directly compared numerically.
2.2 Posterior Sampling
The algorithm we consider is posterior sampling [45]. That is, given an observation of the form y Ax + w, where y E CM A E CMN,, ~ (O, d² and xt ~ 14, the posterior sampling recovery algorithm outputs E. according to the posterior distribution μ(-ly).
In order to sample from the posterior,
Langevin Dynamics [8]. Assuming have access to V. log u())y)), we can sample from μ(( y) by running noisy gradient ascent:
E++1 f + 1/p Va, log (( + 2 So, ~~~N00
(3)
Prior work [8] has shown that as + oo and n) + O, Langevin dynamics will correctly sample from μu(zly). In practice, vanilla Langevin Dynamics are slow to converge. Hence, the work in [79] proposes annealed Langevin Dynamics, where the marginal distribution of x at iteration f modelled as br * No. 88) and the generative model is trained to estimate the score function fmn (.) = V., log((u (0, (x.))
Since the distribution of y/ is Gaussian in Eqn (2), we obtain Va, log (()((r)))) ^ (y Any) We
where 7t + 0 decreasing find that it is also helpful to anneal this term, and it to (y(y-Az, 0²
sequence. An application of Bayes' rule gives: Va, log ((zl())) = =f f(x;; 3r) + 0²+7?
(
Putting everything together, our final algorithm is: for xo ~ ~N,(0, I) and for all t 0,... ... ,T TT 1,
F441 t xs me (F(ossBB)+ A4 Art)) -0-A) + 2 So, Se - N(0
(4)
22 0
Note that the parameters T, (T were fixed during training of the generative model, and hence the only hyperparameters during inference are (n a and (xx)--1 Scripts in our codebase describe hyperparameter values used in our experiments.
5

---

3 Theoretical Results
Background and Notation. We first introduce background and notation required for theoretical results. || refers to the E norm. In this section alone, for simplicity of exposition, we will assume that all matrices and vectors are real valued.
For two probability distributions μu, i on some normed space Ω2 and for any 2 1, the Wasserstein- q [91, 5J and Wasserstein-co [16] distances are defined as:
(EE E VEtl(NN)) ((,,))
1/9
(H, = inf
1 [[]]
()) inf sEl((u.))
7- supl| =1 (u,e)en?
where 1I(,,)) denotes the set of joint distributions whose marginals are μ4, v. The above definition says that Wo (j, v) S E, and (u, v) ~ 7, then ||u -ull S E almost surely.
The (c.,))-aprroxingg required to cover 1 mass under a distribution.
covering number [45], is defined as the smallest number of e-radius balls
Definition 3.1 ((s, -approxxiee covering number). Let μ be a distribution on R.. For some parameters > 0, 8 E [0, 1], the (e, -aappoooie covering number of μ1 is defined as
Cove,a(p) ;= min {k: [U B(a;,) +)] 2 1 - 6, E R R
where Blx, E) is the &a bull of radius centered at
Distributional robustness under Gaussian measurements. First, we consider mismatch be- tween the ground-truth distribution, denoted by μ,, and the generator distribution, denoted by v. Prior work [45] has shown that if (i) Wy( v) I for some q > 1 and (ii) we are given M 2 O(log Cov,,6(M) Gaussian measurements, then posterior sampling with respect to v will recover x² ~ i up to an error of e/51/a with probability 1 - 8. Closeness in Wasserstein distance is reasonable assumption in certain examples, such as when μ is the distribution of celebrity faces and y is the distribution of generator trained on FlickrFaces [52]. However, this assumption is unsatisfactory when we consider distributions of abdominal and brain MR scans, for example, since images of these anatomies look entirely different.
We define the following weaker notion of divergence between distributions. Informally, this new definition tells us that v and μ are "close" if they can each be split into components which are close in Wo distance, such that the close components contain a sufficiently large fraction under v and μ. Formally, this is defined as:
Definition 3.2 ((6, aee divergence). For two probability distributions v and μ. and parameters a, E [0, 1]. the (6, a) divergence is defined as
(o, = inf(s 2 0: 3p² ,1",, v/ v" E M(R s.t. μ (1- + +pp v= (1-a)// + av", Woo v/) e.}
Lemma 1. highlights that this is a strict generalization of Wasserstein distances, in the sense that closeness in Wasserstein distance implies closeness in this new divergence.
Since the (8, a> divergence is a generalization of Wasserstein distances, it is not clear that the main Theorem in [45] holds for distributions that are close in this new divergence. The following result shows rather surprising fact: if (8, a (H, S then posterior sampling with M
O (log T-))) + log (u)) measurements will still succeed with probability 2 16)).
Theorem 3.3. Let 8, E [0, 1]. and E >> 0 be parameters. Let μt, be arbitrary distributions over S e. Let x² Is and suppose y = Axx + w, where E RMN and Rd are i.i.d. Gaussian normalized such that Agj NN0 1/M) and w, ~ -N(0, o² /M), with a 2 E. Given and the fixed matrix A, let & be the output of posterior sampling with respect to
R satisfying (ooo))Wo
Then for M 2 0 [log| (T-a)= + min(log Cov,, ((4),, Cov Cov),(vv))) there exists universal con-
stant > 0 such that with probability at least ( over A, W,
Pr |||2" - 2|| 2 cle + x)] < 8 6 R(M). a* (-,18~

---

Fully-Sample
Wavele
SSIM 0.91
PSNR: 34. 94-1
SSIM 0.66
PSNR: 30 94-1
McDL
E2E Vame
Ours
SSIM:
SS/M
PSNR
SSIM PSNR:
Figure 3: Comparative reconstructions 2D abdominal sean with uniform random under-sampling in horizontal direction at I 4. None of the methods were trained to reconstruct abdomen MRI. Our method uses score-based generative model trained on brain images (as explained) and obtains good reconstructions. The red arrows indicate missing details artifacts the kidney structure.
For our running example of y/ being a generator trained on brain scans, and the distribution of abdominal scans, we can set // be the distribution of our generator restricted abdominal scans, and we can let yí be the distribution restricted to "inliers" in μ. This shows that even if our generator places an exponentially small probability mass(i.e., can still recover abdominal scans with polynomial additive increase in the number of measurements (i.e., log(1/(1 - a))).
< 1) on the set of abdominal scans, we
Near-optimality under arbitrary measurement processes. The previous result required Gaus- sian matrices to handle the distribution shift. Our next result shows that for an arbitrary measurement process, and assuming that there is distribution shift between the generator and the ground truth distribution, posterior sampling is almost the best algorithm for this fixed measurement process. This result also shows that posterior sampling is good with respect any metric.
Theorem 3.4. Let .(..,) be arbitrary metric over R.N R. Let x² and let y A(xx)) be measurements generated from x² for some arbitrary forward operator A .R. - RRM, Then if there exists an algorithm that uses y as inputs and outputs x' such that
d(x" x') with probability 1 -5,
then posterior sampling &o ~ p(-ly) will satisfy
d(æ" 2) 25 with probability
> 1 28.
Remark on combining these results. Our theoretical results above show that posterior sampling (1) highly robust distribution shift under Gaussian measurements, and (2) accurate with arbitrary measurements without distribution shift. A natural hope would be to combine these two results and show that is robust to distribution shift under Fourier measurements. Unfortunately, this not true for general distributions: for example, if and are both random distributions over Fourier-sparse signals, then Fourier measurements will usually give zero information about the signal, so cannot convince the sampler to sample near y rather than v.
4 Experimental Results
We perform retrospective under-sampling in all experiments. i.e., given fully-sampled k-space measurements from the NYU fastMRi [56, 94] and Stanford MRI [1] datasets, we apply sampling masks and evaluate the performance of all considered algorithms on the reconstructed data. Depending on scan parameters (e.g., 3D seans for the Stanford knee data Appendix F), we appropriately slice and sample the data in the proper dimension
not commit any inverse crime [31, 77].

---

We first highlight that an advantage of the proposed approach is the invariance to the sampling scheme during training. In contrast, this is a design choice that must be made for supervised end-to-end methods, which here were trained on equispaced, vertical sampling masks, following the fastMRI 2020 challenge guidelines [94, 66]. As our results show, this affords us a significant degree of robustness across wide distribution of sampling masks during inference.
We train a score-based model, NCSNv2 [80], on small subset of scans from the NYU fastMRI brain dataset. Specifically, we train using T2-weighted images at a field strength of 3 Tesla for a total of 14,539 2D training slices. We calculate the MVUE from the fully sampled data and use the ESPIRiT algorithm [87, 43] applied to the fully-sampled central portion of k-space to estimate the sensitivity maps. The backbone network for our model is a RefineNet [59]. Since the generator's output is expected to be complex- -anplex-valued, we treat the real and imaginary parts as separate image channels. Details about the architectures are given in Appendix G.
We use an 4 Wavelet regularized reconstruction algorithm [62] as parallel imaging and compressed sensing baseline. This aims to solve the optimization problem given in (2) with Q(x) || x) æ||1. where W is a 2D Wavelet transform. We use the publicly available implementation from the BART toolbox [88, 86] and optimize the regularization hyper-parameter using the same subset of samples from the brain dataset that was used to train our method. We find that À = 0.01 performs the best on the training data and use this value for all experiments. We consider three different deep learning baselines: MoDL [3]. E2E- VarNet [82], and the ConvDecoder architecture [19].
baselines from scratch on the same training dataset our method, at acceleration factors R = (3. 6} and equispaced under-sampling, with a supervised SSIM loss on the magnitude MVUE image, for 40 and 15 epochs, respectively, using a batch size of 1. For the ConvDecoder baseline, we use the architecture for brain data in [19] that outputs a complex image estimate and optimize the number of fitting iterations on a subset of samples from the training data. We find that 10000 iterations are sufficient to reach a stable average performance at R = 3. Put together, all of our baselines are tailored estimate the complex image 2, thus all comparisons are fair. We evaluate reconstruction performance using the complex MVUE of the fully sampled data as reference image and measure the peak signal-to-nn to-noee ratio (PSNR) and structural similarity index (SSIM) [92] between the absolute values of the reconstruction and ground-trute truth MVUE images.
We train the MoDI. and
4.1 In-Distribution Performance
In this experiment, we test all models using the same forward model that matches the training conditions for the baselines: vertical, equispaced sampling patterns. Examples of various sampling patterns are shown in Appendix C.
Figure (top three rows) shows qualitative results and Figures 2a & 5a respectively show PSNR & SSIM values, for the case where there is no mismatch between the training and inference sampling patterns. As the baselines were trained to maximize SSIM at R 3 & 6, we that they achieve better SSIM scores than us at these accelerations, although there is clear aliasing in the baselines at R 6. We achieve better PSNR values these accelerations, which supports the claim that our method does not overfit to a particular metric (Theorem 3.4). This also highlights the importance of qualitative evaluations in medical image reconstruction and the limitations of existing image quality metrics [65]. From the third row of Figure 1, and Figures 2a & 5a, we notice that our method surpasses baselines at higher accelerations.
We find that f Wavelet suffers both qualitatively and quantitatively at high acceleration factors, while the ConvDecoder is also competitive architecture, but incurs large computational cost. When benchmarked on an NVIDIA RTX 2080Ti GPU, our method takes 16 minutes and 0.95 GB of memory to reconstruct high high-resolution brain scan, whereas the Conv takes longer than 80 minutes and 6.6 GB of memory. While our method is limited by the inference time and is not in the range of end-to-end models (where reconstruction takes at most the order of seconds and 3.5 GB of memory), multiple scans can be reconstructed in parallel due to the reduced memory footprint.
4.2
Performance
Test-time sampling pattern shifts. Here we consider shifts in the forward sampling operator at test-time, while still evaluating on the same anatomy as the training conditions. We measure
8

---

Ground Reconstruction
Deviation
Figure 4: Our method successfully recovers fine details and provide an estimate of the reconstruction error. The left column shows knee from fastMRI dataset, along with an annotated meniscus tear (indicated by red arrow in zoomed inset). Given measurements al an acceleration factor of R = 4, we obtain 48 independent reconstructions via posterior sampling. The second column shows the pixel-wise average of reconstructions, the third column shows the pixel- wise standard deviation, and the fourth column shows the magnitude of the error between the ground truth and the mean reconstruction. Note that our generative prior never seen such pathology, as it was trained on T2-weighted brain scans.
robustness by evaluating the average incurred performance loss when the sampling pattern changes. Recall that our proposed approach does not any explicit information about the sampling pattern P during training, hence we anticipate the highest degree of robustness.
Figure (fourth row) shows qualitative reconstructions when the measurements are obtained from an equispaced, horizontal sampling mask, with an acceleration factor = 3. can be observed that the reconstructions output by E2E-VarNet show aliasing artifacts. Based the statistical results in Figure 2b & 5b, our method retains its performance.
Furthermore, this experiment reveals that MoDL is more robust to this type of mask shift when compared VarNe,,,, even though it uses smaller network. This explained by the fact that E2E- VarNet does not use external sensitivity map estimates, but uses deep neural network for end- to-end map estimation. While this improves performance on in disdiitiibution samples, the performance drop is strong evidence that accurate sensitivity map estimation is vital for robust generalization, and both our proposed approach and MoDL. benefit from the external ESPIRIT algorithm, which compatible with different sampling patterns.
We do note that retrospectively flipping the horizontal and vertical sampling direction is not necessarily representative of prospective sampling in the horizontal direction due to the discrete nature of the phase encoding direction in MRI, and this may contribute the higher scores compared to the vertical mask experiments.
Test-time anatomy shifts. We now consider the more difficult problem of reconstructing different anatomics than the ones seen during. This was previously investigated in [19], which concluded that all methods suffer drastic shift due to the various changes in scan parameters between body parts. In contrast to prior work, our main finding is that the proposed score-based model retains a significant degree of robustness under these shifts, and outputs excellent qualitative reconstructions. In some cases, some end-to-end methods retain robustness as well.
Figures 2c & 5c show PSNR and SSIM scores obtained reconstructed abdominal scans obtained from [1] at different acceleration factors. This represents both an anatomy and sampling pattern shift, and it can be seen that our method, MoDL,, and the Waveeeeeet algorithm retain their competitive advantage, while the ConyDecoder and E2E- VarNet suffer severe performance losses. Figure further shows qualitative comparison of reconstructed abdominal scan at R 4, with highlighted artifacts. Appendix E shows another abdomen scan.
Finally, Figures 2d & 5d show PSNR and SSIM scores obtained on fastMRI knee reconstructions, while Figure (bottom row) shows the accompanying qualitative plots. This anatomy challenging especially because of the poor signal-to to-istt ratio conditions, which can be seen even in the ground- truth image. It can noticed that this is the most severe shift for all methods, but our approach still shows the best performance at R = 2, 4 and a significantly lower variance. Appendix D shows more examples of knee reconstructions with and without fat suppression, and Figure 20 shows metrics on fat suppressed knees.

---

4.3 Uncertainty Estimation
Our method can also provide uncertainty estimates for each reconstructed pixel by running multiple reconstruction samplers. For given observation V, we can obtain independent samples 21, _KK ~ u(-ly), for K sufficiently large. Now, using the conditional mean estimate ā = - / K, we can
compute the pixel-wise standard deviation V VE - =//KK, and this gives an estimate of the error in each pixel. As shown in Fig 4, the pixel-wise standard deviation is a good estimate of the ground truth eπror x ø Additionally, notice that the reconstructions are able recover fine details such as the annotated meniscus tear' in Fig 4 and predict low uncertainty for these features.
Figure 17 in Appendix D shows another example of an annotated meniscus tear. Figures 18 and 19 show comparisons with baselines on the same examples.
4.4 Radiologist Study
We have conducted preliminary blind assessment of overall image quality with two board-certified radiologists and one faculty member who uses neuroimaging for their research. These experts were not involved in our research. We have found that our algorithm was ranked best for knee scans, and tied with the baselines for abdominal and brain scans, supporting our robustness claims in the paper. For more details, please see Appendix H.
5 Limitations
We reported PSNR and SSIM values as they are correlated with radiologist evaluation upto an extent, and our preliminary radiologist study in Section 4.4 suggests the feasibility of clinical adoption. These metrics do not capture the needs real-world radiologists, and a more detailed study is required before the proposed techniques be clinically adopted.
Though promising, our initial results were still limited to fast spin-echo imaging only and all data were retrospectively under-sample sampled. Further study is required to demonstrate prospective performance in a larger body of heterogeneous MRI data. Our method also currently requires a high compute cost at inference time, well the need for pre-trained generative model. Clinical use requires fast reconstruction in addition to fast scanning. Future work should investigate whether score-based based models can be trained without fully-samples training reducing computation time.
well investigate approaches to
Finally, there are potential issues related to discrimination. Specifically, it is possible that the quality of the reconstructed images varies across protected attributes, such as gender or race [57].
Conclusions
This paper reports the first successful application of the CSGM framework for robust multi-coil MR image reconstruction under realistic sampling conditions, and provides theoretical evidence for the robustness of posterior sampling. Our score-based model was trained a small subset of brain MRI scans without any explicit information about the sampling scheme. This shows state-of-the- art performance under severe distributional shifts, making model applicable in wide range of clinical settings.
Our method shows considerable degree of generalization to out-of-distribution samples such as abdomen and knee MRI, even when trained exclusively on brain MRI. Notably, these scans were acquired using different MRI vendors with different pulse sequence parameters and at different institutions. We postulate that adding small set of diverse training samples to our generative model could further improve robustness, and hypothesize that these samples may not necessarily be restricted to MR images.
The results presented in this work represent an important step to applying deep learning models in the clinic, as there is natural variation in sampling, image orientation, receive coils, scanner hardware, and anatomy in clinical practice.
"httts://ddssee fastmri. otrtii//rr///1999
10

---

7 Acknowledgements
Ajil Jalal, Giannis Daras and Alex Dimakis have been supported by NSF Grants CCF 1763702, 1934932, AF 1901281, 20087 10, 2019844, the NSF IFML 2019844 award as well as research gifts by Western Digital, Interdigital, WNCG and MLL, computing resources from TACC and the Archie Straiton Fellowship.
Eric Price has been supported by NSF Award CCF-1751044 (CAREER), NSF Award CCF-2008868, and NSF IFML 2019844.
Marius Arvinte and Jon Tamir have been supported by NSF IFML 2019844 award, ONR grant NO0014-194 1--4-199--25900 NIH Grant U24EB029240, and an AWS Machine Learning Research Award.
We thank the anonymous NeurlPS reviewers for their helpful and considerate feedback. Finally, we would like thank the experts who graciously helped with our image assessment study.
Funding Transparency Statements
8.1 Funding (financial activities supporting the submitted work):
Funding in direct support of this work: NSF Grants CCF 1763702, 1934932, AF 1901281, 2008710 10, 2019844, 1751040, 2008868, the NSF IFML 2019844 award, ONR grant N00014-19-1-2555
NIH Grant U24EB029240, and an AWS Machine Learning Research Award, as well as research gifts by Western Digital, Interdigital, WNCG and MLL, computing resources from TACC and the Archie Straiton Fellowship.
8.2 Competing Interests (financial activities outside the submitted work):
Additional revenues related to this work: Internship at Intel and Google.
References
[1] http://mridata.co org/.
[2] https://pingouin-staaee stats. org/ orgp/generaaedd/iiggninn intraclass_com html.
[3] Hemant K Aggarwal, Merry P Mani, and Mathews Jacob. Modl: Model-based deep learning architecture for inverse problems. IEEE transactions on medical imaging, 38(2):394 405,, 2018.
[4] Vegard Antun, Francesco Renna, Clarice Poon, Ben Adcock, and Anders C Hansen. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proceedings of the National Academy of Sciences, 117(48):3008833)) -3095,, 2020.
[5] Martin Arjovsky, Soumith Chintala, and Léon Bottou.
Wasserstein gan. arXiv preprint
arXiv:1701.077 2017.
[6] Muhammad Asim, Ali Ahmed, and Paul Hand, Invertible generative models for inverse problems: mitigating representation error and dataset bias. arXiv preprint 1rXXiv1905.1167. 2019.
[7] Muhammad Asim, Fahad Shamshad, and Ali Ahmed, Solving bilinear inverse problems using deep generative priors. CoRR, abs/1 04073, 3(4):8, 2018.
[8] Dominique Bakry and Michel Émery. Diffusions hypercontractives. In Seminaire de probabilités XIX 1983/84, pages 177-206. Springer, 1985.
[9] Eren Balevi, Akash Doshi, Ajil Jalal, Alexandros Dimakis, and Jeffrey G Andrews. High dimensional channel estimation using deep generative networks. IEEE Journal on Selected Areas in Communications, 39(1):18-30 2020.
[10] Richard G Baraniuk, Volkan Cevher, Marco F Duarte, and Chinmay Hegde. Model-based compressive sensing. IEEE Transactions on Information Theory, 56(4):1982-2001, 2010.
[11] Richard Baraniuk and Michael Wakin. Random projections of smooth manifolds. Founda- tions of computational mathematics. 9(1:51177, 2009.

---

[12] Peter Bickel, Ya' acov Ritov, and Alexandre B Tsybakov. Simultaneous analysis of lasso and dantzig selector. The Annals of Statisties, 37(4):175--1732, 2009.
[13] Ashish Bora, Ajil Jalal, Eric Price, and Alexandros G Dimakis. Compressed sensing using generative models. In Proceedings of the 34th Interattional Conference on Machine Learning- Volume 70, pages 537-546. JMLR. org, 2017.
[14] Ashish Bora, Eric Price, and Alexandros G Dimakis. Ambientgan: Generative models from lossy measurements. ICLR, 2:5, 2018.
[15]
Emmanuel J Candes. The restricted isometry property and its implications for compressed sensing. Comptes rendus mathematique, 346(9-10):589-59)) 2008.
[16] Thierry Champion, Luigi De Pascale, and Petri Juutinen. The o-Wasserstein distance: solutions and existence of optimal transport maps. SIAM Journal on Mathemutical Analysis, 4001): 2008.
Local
[17] Elizabeth K Cole, Frank Ong. Shreyas S Vasanawala, and John M Pauly. Fast unsupervised mri reconstruction without fully-sampled ground truth data using generative adversarial networks. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 3988-3997, 2021.
[18) Giannis Daras, Joseph Dean, Ajil Jalal, and Alexandros Dimakis. Intermediate layer op- timization for inverse problems using deep generative models. International Conference on Machine Learning. 2021.
[19] Mohammad Zalbagi Darestani, Akshay Chaudhari, and Reinhard Heckel. Measuring robustness in deep learning based compressive sensing. International Conference on Machine Learning.
2021.
[20] Anagha Deshmane, Vikas Gulani, Mark A Griswold, and Nicole Seiberlich. Parallel mr imaging. Journal of Magnetic Resonance Imaging, 36(1):55-72, 2012.
[21] Manik Dhar, Aditya Grover, and Stefano Ermon. Modeling sparse deviations for compressed sensing using generative models. arXiv preprint 1807.01800.114. 2018.
[22] Mariya Doneva. Mathematical models for magnetic resonance imaging reconstruction: An overview of the approaches, problems, and future research areas. IEEE Signal Processing Magazine, 37(1):24-32, 2020.
[23] David L Donoho. Compressed sensing. IEEE Transactions on information theory, 52(4):1289-
1306, 2006.
[24] Vineet Edupuganti, Morteza Mardani, Shreyas Vasanawala, and John Pauly. Uncertainty quantification in deep mri reconstruction. IEEE Transactions Medical Imaging, 40(1):239-
250, 2021.
[25] Yonina C Eldar and Moshe Mishali. Robust recovery of signals from structured union of subspaces. IEEE Transactions on Information Theory,
2009.
[26] Alyson K Fletcher, Parthe Pandit, Sundeep Rangan, Subrata Sarkar, and Philip Schniter. Plug-in estimation in high-dimensional linear inverse problems: A rigorous analysis. In Advances in Neural Information Processing Systems, pages 7440-7449, 2018.
[27] Alyson K Fletcher, Sundeep Rangan, and Philip Schniter. Inference deep networks in high dimensions. In 2018 IEEE International Symposium on Information Theory (ISIT), pages 1 1884-188. IREE, 2018
[28] Fabian Latorre Gómez, Armin Eftekhari, and Volkan Cevher. Fast and provable admm for learning with generative priors. arXiv preprint /007003307 2019.
[29] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu. David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014.
[30] Mark A. Griswold, Peter M. Jakob, Robin M. Heidemann, Mathias Nittk,, Vladimir Jellus, Jianmin Wang, Berthold Kiefer, and Axel Haase. Generalized autocalibrating partially parallel acquisitions (grappa). Magnetic Resonance in Medicine, 47(6):
2002.
[31] Mauhieu Guerquin- Kern, Laurent Lejeune, Klaas Paul Pruessmann, and Michael Unser. Re- alistic analytical phantoms for parallel magnetic resonance imaging. IEFE Transactions on Medical Imaging, (1(3):626-636. 2011.
12

---

[32] Kerstin Hammernik, Teresa Klatzer, Erich Kobler, Michael P Recht, Daniel K Sodickson, Thomas Pock, and Florian Knoll. Learning variational network for reconstruction of acceler- ated mri data. Magnetic resonance in medicine, 79(6):30553300 3071, 2018.
[33] Kerstin Hammenikk, Jo Schlemper, Chen Qin, Jinming Duan, Ronald M Summers, and Daniel Rueckert. Systematic evaluation of iterative deep neural networks for fast parallel mri re- construction with sensitivity-weeighee coil combination. Magnetic Resonance in Medicine, 2021.
[34] Paul Hand and Babhru Joshi. Global guarantees for blind demodulation with generative priors.
In Advances in Neural Information Processing Systems, pages 1 1131--1554, 2019.
[35] Paul Hand, Oscar Leong, and Vlad Voroninski. Phase retrieval under a generative prior. In Advances in Neural Information Processing Systems, pages 9136-9146, 2018.
[36] Paul Hand and Vladislav Voroninski. Global guarantees for enforeing deep generative priors by empirical risk. arXiv preprint arXiv: 07576, 2017.
[37] Reinhard Heckel and Paul Hand. Deep decoder: Concise image representations from untrained non-convolutional networks. arXiv preprini arXiv: 1xXiv:811000382, 2018.
[38] Reinhard Heckel and Mahdi Soltanolkotabi. Compressive sensing with un-trained neural net- works: Gradient descent finds the smoothest approximation. arXiv preprint urXiv:2005.03991, 2020.
[39] Chinmay Hegde. Algorithmic aspects of inverse problems using generative models. In 2018 56th Annual Allerton Conference on Communicatiin, Control, and Computing (Allerton), pages 166-172 172. IEEE, 2018.
[40] Chinmay Hegde, Michael Wakin, and Richard G Baraniuk. Random projections for manifold learning. In Advances in neural information processing systems, pages 641-648. 2008
[41] Feng Huang, Sathya Vijayakumar, Yu Li, Sarah Hertel, and George R Duensing. A soft- ware channel compression technique for faster reconstruction with many channels. Magnetic resonance imaging, 26(1):133-144 2008.
[42] Shady Abu Hussein, Tom Tirer, and Raja Giryes. Image-adaptive gan based reconstruction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 3121-3129, 2020.
[43] Siddharth Iyer, Frank Ong, Kawin Setsompop, Mariya Doneva, and Michael Lustig. Sure- based automatic parameter selection for espirit calibration. Magnetic Resonance in Medicine, 84(6):3423-3437, 2020.
[44] Mathews Jacob, Jong Chul Ye, Leslie Ying, and Mariya Doneva. Computational mri: Com- pressive sensing and beyond [from the guest editors). IEEE Signal Processing Magazine, 37(1):21-23, 2020.
[45] Ajil Jalal, Sushrut Karmalkar, Alexandros G Dimakis, and Eric Price. Instance-optimal com- pressed sensing via posterior sampling. International Conference on Machine Learning, 2021.
[46] Ajil Jalal, Sushrut Karmalkar, Jessica Hoffmann, Alexandros G Dimakis, and Eric Price. Fairness for image generation with uncertain sensitive attributes. International Conference on Machine Learning, 2021.
[47] Ajil Jalal, Liu Liu, Alexandros G Dimakis, and Constantine Caramanis. Robust compressed sensing using generative models. Advances in Neural Information Processing Systems, 33, 2020.
[48] Shirin Jalali and Xin Yuan. Solving linear inverse problems using generative models. In 2019 IEEE International Symposium on Information Theory (ISIT), pages 512-516. IEFE, 2019.
[49] Kyong Hwan Jin, Michael T. McCann, Emmanuel Froustey, and Michael Unser. Deep convolu- tional neural network for inverse problems in imaging. IEEE Transactions on Image Processing, 26(9):450-4422) 2017.
[50] Maya Kabkab, Pouya Samangouei, and Rama Chellappa. Task-aware compressed sensing with generative adversarial networks. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
[51] Akshay Kamath, Sushrut Karmalkar, and Eric Price. Lower bounds for compressed sensing with generative models, arXiv preprint arXiv:1912.0220 2019.
13

---

[52] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition, pages 4401 -4100, 2019.
[53]
Varun A Kelkar and Mark A Anastasio. Prior image- reconstruction using style-based generative models. arXiv preprint arXiv:2 102. 1255,, 2021.
[54] Varun A Kelkar, Sayantan Bhadra, and Mark A Anastasio. Compressible latent-space invert- ible networks for generative model-constrainedl image reconstruction. IEEE Transactions on Computational Imaging, 7:209-223, 2021.
[55] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 14, 2013.
[56] Florian Knoll, Jure Zbontar, Anuroop Sriram, Matthew J Muckley, Mary Bruno, Aaron Defazio, Mare Parente, Krzysztof J Geras, Joe Katsnelson, Hersh Chandarana, et al. fastmri: A publicly available raw k-space and dicom dataset of knee images for accelerated mr image reconstruction using machine learning. Radiology: Artificial Intelligence, 1 2020.
[57] Agostina Larrazabal, Nicolás Nieto, Victoria Peterson, Diego H Milone, and Enzo Ferrante. Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences, 1 12722)::25922-122))
2020.
[58] Qi Lei, Ajil Jalal, Inderjit S Dhillon, and Alexandros Dimakis. Inverting deep generative models, one layer at a time. In Advances in Neural Information Processing Systems, pages 13910-13919, 2019.
[59] Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid. Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1925-1934, 2017.
[60] Zhaoqiang Liu, Selwyn Gomes, Avtansh Tiwari, and Jonathan Scarlett. Sample complexity bounds for 1-bit compressive sensing and binary stable embeddings with generative priors. arXii preprint arXiv:2002.016697, 2020.
[61] Zhaoqiang Liu and Jonathan Scarlett.
lower bounds for compressive sensing with generative models. arXiv preprint arXiv: htt.://984.744,... 2019.
[62] Michael Lustig, David Donoho, and John M Pauly. Sparse mri: The application of compressed sensing for rapid mr imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, (8(6):1182-11 1195, 2007.
[63] Morteza Mardani, Enhao Gong, Joseph Y Cheng, Shreyas Vasanawala, Greg Zaharchuk, Marcus Alley, Neil Thakur, Song Han, William Dally, John M Pauly, et al. Deep generative adversarial networks for compressed sensing automates mri. arXiv preprint /rXiv:1706.0005/ 2017.
[64] Morteza Mardani, Enhao Gong, Joseph Y Cheng, Shreyas S Vasanawala, Greg Zaharchuk, Lei Xing, and John M Pauly. Deep generative adversarial neural networks for compressive sensing mri. IEEE transactions on medical imaging, 38(1): 2018.
[65] Allister Mason, James Rioux, Sharon E Clarke, Andreu Costa, Matthias Schmidt, Valerie Keough, Thien Huynh, and Steven Beyea. Comparison of objective image quality metries to expert radiologists' scoring of diagnostic quality of mr images. IEEE transactions on medical imaging, 39(4): 1072, 2019.
[66] Matthew J. Muckley, Bruno Riemenschneider, Alireza Radmanesh, Sunwoo Kim, Geunu Jeong, Jingyu Ko, Yohan Jun, Hyungseob Shin, Dosik Hwang, Mahmoud Mostapha, Simon Arberet, Dominik Nickel, Zaccharie Ramzi, Philippe Ciuciu, Jean-Luc Starek, Jonas Teuwen, Dimitrios Karkalousos, Chaoping Zhang, Anuroop Sriram, Zhengnan Huang, Nafissa Yakubova, Yvonne W. Lui, and Florian Knoll. Results of the 2020 fastmri challenge for machine learning mr image reconstruction. JEEE Transactions Medicul Imaging, pages 2021.
[67] Dominik Narnhofer, Kerstin Hammernik, Florian Knoll, and Thomas Pock. Inverse gans for accelerated mri reconstruction. In Wavelets and Spursity XVIII, volume 11138, page 11138A. International Society for Optics and Photonics, 2019.
[68] Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. arXiv preprint 27X1v:2005.06001, 2020.
14

---

[69] Parthe Pandit, Mojtaba Sahraee-Araakan, Sundeep Rangan, Philipp Schniter, and Alyson K Fletcher. Inference with deep generative priors in high dimensions. arXiv preprint arXiv: 1911,0340, 2019.
[70] Klaas P Pruessmann, Markus Weiger, Markus B Scheidegger, and Peter Boesiger. Sense: sensitivity encoding for fast mri. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, (2(5):5229662. 1999.
[71] Shuang Qiu, Xiaohan Wei, and Zhuoran Yang. Robust one-bit recovery via relu gener- ative networks: Improved statistical rates and global landscape analysis. arXiv preprint arXiv: 05368, 2019.
[72] Saiprasad Ravishankar and Yoram Bresler. Mr image reconstruction from highly undersampled k-space data by dictionary learning, IEEE Transactions Medical Imaging, 30(5):1028-100 1041, 2011.
[73] Saiprasad Ravishankar and Jeffrey A. Fessler. Data-driven models and approaches for imaging, In Imaging and Applied Opties 2017 (3D, AIO, COSI, IS, MATH, PcAOP), page MW2C.4. Optical Society of America, 2017.
[74] JH Rick Chang, Chun-Liang Li, Barnabas Poczos, BVK Vijaya Kumar, and Aswin C Sankara- narayanan, One network to solve them all-solving linear inverse problems using deep projection models. In Proceedings of the IEEE International Conference on Computer Vision, pages 5888-589 5897, 2017.
[75] Sebastian Rosenzweig, Hans Christian Martin Holme, Robin N Wilke, Dirk Voit, Jens Frahm, and Martin Uecker. Simultaneous multi-slice mri using cartesian and radial flash and regularized nonlinear inversion: Sms-nlinv. Magnetic resonance in medicine, 79(4):2057-2066 2018.
[76] Jo Schlemper, Jose Caballero, Joseph V Hajnal, Anthony N Price, and Daniel Rueckert. A deep cascade of convolutional neural networks for dynamic mr image reconstruction. IEEE transactions Medical Imaging, 37(2):491-503, 2017.
[77] Efrat Shimron, Jonathan Tamir, Ke Wang, and Michael Lustig. Subtle inverse crimes: Na)," ively training machine learning algorithms could lead to overly-optimistic results. arXiv preprint
arXiv:2109.08237, 2021.
[78] Daniel K Sodickson and Warren J Manning. Simultaneous acquisition of spatial harmonics (smash): fast imaging with radiofrequency coil arrays. Magnetic resonance in medicine, 38(4)559 1997.
[79] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pages 11918-11938 2019.
[80] Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. arXiv preprint arXiv:2006.0900 19 2020.
[81] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference Learning Representations, 2021.
[82] Anuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C Lawrence Zitnick, Nafissa Yakubova, Florian Knoll, and Patricia Johnson. End-to-end variational networks for accelerated mri reconstruction. In International Conference on Medical Image Computing and Computer- Assisted Intervention, pages 64-73. Springer, 2020.
[83] Anuroop Sriram, Jure 7bontar, Tullie Murrell, C. Lawrence Zitnick, Aaron Defazio, and Daniel K. Sodickson. Grappanet: Combining parallel imaging with deep learning for multi-coil mri reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020.
[84] Anuroop Sriram, Jure Zbontar, Tullie Murrell, Lawrence Zitnick, Aaron Defazio, and Daniel K Sodickson. Grappanet: Combining parallel imaging with deep learning for multi-coil mri reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14315-14322, 2020.
[85] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267-288, 1996.
15

---

[86] Martin Uecker, Christian Holme, Moritz Blumenthal, Xiaoqing Wang, Zhengguo Tan, Nick Scholand, Siddharth Iyer, Jon Tamir, and Michael Lustig. mrirecon/bart: version 0.7.00, March 2021.
[87] Martin Uecker, Peng Lai, Mark J Murphy, Patrick Virtue, Michael Elad, John M Pauly, Shreyas Vasanawala, and Michael I Lustig. Espirit-an eigenvalue approach to autocalibrating parallel mri: where sense meets grappa. Magnetic resonance in medicine, 71(3):900-1001, 2014.
[88] Martin Uecker, Frank Ong, Jonathan Tamiir, Dara Bahri, Patrick Virtue, Joseph Y Cheng, Tao Zhang, and Michael Lustig. Berkeley advanced reconstruction toolbox. In Proc. Intl. Soc. Mag. Reson. Med, volume 23, 2015.
[89] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9446-9454, 2018.
[90] Shreyas S. Vasanawala, Marcus T. Alley, Brian A. Hargreaves, Richard A. Barth, John M. Pauly, and Michael Lustig. Improved pediatric mr imaging with compressed sensing. Radiology, 256(2):607-616, 2010. PMID: 20529991.
[91] Cédric Villani. Optimal transport: old and new, volume 338. Springer Science Business Media, 2008.
[92] Zhou Wang and Alan C Bovik. Modern image quality assessment. Synthesis Lectures on Image, Video, and Multimedia Processing, 2(1):1-156, 2006.
[93] Bihan Wen, Saiprasad Ravishankar, Luke Pfister, and Yoram Bresler. Transform learning for magnetic resonance image reconstruction: From model- learning to building neural networks. IEEE Signal Processing Magazine, 37(1):41-53, 2020.
[94] Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang. Matthew J. Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, Mare Parente, Krzysztof J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzal, Adriana Romero, Michael Rabbat, Pascal Vincent, Nafissa Yakubova, James Pinkerton, Duo Wang. Erich Owens, C. Lawrencee Zitnick, Michael P. Recht, Daniel K. Sodickson, and Yvonne W. Lui. fastMRI: An open dataset and benchmarks for accelerated MRI. 2018
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes] See Section 3, 4, 4.3
(b) Did you describe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See Section 5
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] Yes we have read the ethics review guidelines.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all thcoretical results? [Yes] In the state- ments of Theorem 3.3, Theorem 3.4.
(b) Did you include complete proofs of all theoretical results? [Yes] In Appendix B
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)? [Yes] Our GitHub link contains all relevant information
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] In Appendix G
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] All quantitative plots have 95% confidence intervals.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] Yes, in section and Appendix G
4. If you using existing assets (c.g., code, data, models) or curating/releasing assets...
16

---

(a) If your work uses existing assets, did you cite the creators? [Yes] Our algorithm uses the NCSNv2 generative model, NYU fastMRI dataset, and Stanford MRI datasets, all of which have been cited.
(b) Did you mention the license of the assets? [Yes] In Appendix G
(c) Did you include any assets either in the supplemental material
URL.? [Yes] To the best of our knowledge, we are the first to train generative models for complex valued MR scans, and include this in the GitHub link in Appendix G
(d) Did you discuss whether and how consent was obtained from people whose data you' re using/curating? [Yes] We used the NYU fastMRI and Stanford MRI datasets, both of which have this covered in their terms agreement.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [Yes] We used the NYU fastMRI and Stanford MRI datasets, both of which have been anonymized.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, applicable? [N/A] We didn't 't do any human subject experiments
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A] We didn't do any human subject experiments
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A] We didn't do any human subject experiments
17


Reviews:
[{"review_id": "tyeRP-wFydI", "reviewer": "Reviewer_nt1J", "summary": "This paper proposed the robust MR image reconstruction framework based on CSGM (Compressed Sensing with Generative Modeling). Compared to the other MR reconstruction algorithms, the proposed method shows consistently superior reconstruction results for both in-distribution (same k-space sampling mask/direction, same anatomy, and same contrast) and out-of-distribution (different k-space sampling direction/mask, different anatomy, and different contrast). Also, it gives the uncertainty estimation of its reconstruction results in pixel-wise manner by running multiple reconstruction samplers.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality  \n- The most of the backbone algorithms are comes from the other works: NCSNv2, RefineNet, etc. However, the originality of this paper sufficiently comes from the how to build the robust reconstruction performance on out-of-distribution samples & its theoretical and mathematical background support. \n\nQuality & Clarity\n- The author tackles the out-of-distribution problem for MR reconstruction. And their CSGM based method makes sense for its robustness and it shows the superior MR reconstruction results for various out-of-distribution cases which are fit for their assumption and mathematical background.\n\nSignificance\n- Many of MR reconstruction algorithms are still focusing on the end-to-end supervised learning (accelerated/sampled data to full-sampled data) and it results to the poor performance on out-of distribution datasets while it results to the excellent performance on in-distribution datasets. Since the proposed method shows impressive robustness of various out-of-distribution datasets (sampling direction/anatomy/contrast), it might give positive impact on real clinical MR acceleration fields.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The followings are the suggestions and my questions  to improve the paper for better understanding of readers:\n\n- Flow of Algorithm might be helpful to the readers: Sample preparation steps such as coil-sensitivity extraction, different sampling masks/different anatomy images for the training/test phase for out-of-distribution, etc.\n\n- Too little explanation of the main score model, NCSNv2 & backbone model, RefineNet.\n\n- Actually, in clinic, they usually fixed the k-space sampling pattern at each acceleration rate (by MRI manufacturer) and they want to maximize the performance of reconstruction for that fixed sampling masks. Thus, the comparison results for the different sampling direction (Phase Encode for anterior-posterior VS. Phase Encode for lateral-later) seems less effective to support the usefulness of the robust algorithm. I think the different contrast images are better examples (T1w, T2w, FLAIR, etc.)\n\n- The author mentions that their results are superior because the the comparative reconstructions show some aliasing artifacts or miss some details. However, some radiologist might score better on  the comparative one to examine the anatomy/lesion even it has some artifacts. The blind examination by the radiologist might be helpful as further work.\n\n-- Figure.4 (a): The 'in-distribution' reconstruction performance of MoDL and E2E-Varnet seems too bad. e.g. less than SSIM 0.8 while the acceleration rate R is only 2? And the PSNR of MoDL and E2E-Varnet results are less than 25dB @only R=2 while the proposed method 40dB (appendix Fig.19(a))? \n\n-- Figure.4 (b) : Why some results of out-of-distribution ((b)Mask Shift Brain: @R=8, R=12 ) show superior than (a) in-distribution method.\n\n-- Figure.4 (d): Is there any discussion why the proposed method shows worse results @ higher acceleration rates, R=8 & R=12?\n\nMinor concerns\n- Please, make sure that figures&tables stand alone, complete and informative in itself.(e.g.Fig.1 what 'mask shift'/'anatomy shift' means)\n\n- Mention about 'inverse crime' (line#214): the authors carefully slice the 3D knee scans in proper dimension to simulate the appropriate real MR acquisition. However, most of the MR image in the dataset(I think all of them), oversampled in k-space Read-Out direction(usually x2) and remove the out of the FOV part in the reconstructed images. Thus, I was wondering if the author did the proper inversion of this process (1.attach the blank bins on the image in Read-Out direction, then  Fourier Transform + retrospective sampling). \n\n- Use capital letter correctly in references(mri, Modl, mr, etc)\n\n- Please, specify the parameters for SSIM metric in appendix.\n\n- Figure.3 the figures have same intensity scale? GT, mean reconstruction VS. std, error? The intensity of error images seems higher than the GT. Please, specify them.", "review_text": "", "extra_scores": {"ethical_concerns": ".", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "11"}, "extra_text": {}}, {"review_id": "qh0xo1T-Ysn", "reviewer": "Reviewer_pPk8", "summary": "This paper propose to use score-based generative models for robust compressed MRI. The paper claims that CSGM framework performs poorly on out-of-distribution samples, and the proposed posterior sampling-based method is robust to changes in the image distribution and measurement process. ", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I list my main concerns below. \n\n- The main contribution of this paper seems to be in using score-based generative models for MRI. The main approach/algorithm is based on prior work that uses a score-based generative model to perform Langevian dynamics for posterior sampling. \n- The paper claims that they are the first to apply CSGM framework on clinical MRI data. This seems to be a strong claim because several other paper have used fastMRI data with generative model-based reconstruction. \n- The paper also discussed the challenges related to shifts in the distribution of images and measurements. \n- The challenge of shift in image distribution at train and test times seems real and important to me, but I do not understand why measurement distribution is of any concern/relevance. It seems obvious to me that the generative model-based methods can work with any measurement pattern. This is different from end-to-end networks that learn to map measurements to the images, and difference between measurement patterns at training and testing is a concern.\n- The problem of mismatch between train and test images of generative models is important. This paper provides empirical results that show the proposed method is robust to such mismatch. At the same time, most of other methods they compare against in Figure 4c also see robust to such image distribution mismatch. I wonder what is the significance of the proposed method in that case? \n- The paper does not compare with the original CSGM framework in which you use a trained generative model and solve an optimization problem similar to the one in Eq. 2. \n- SSIM is a metric often used for natural image quality assessment. It is not necessarily a good metric for MR images. Please report PSNR as well. \n- Theoretical results: I feel there is a mismatch between the theoretical and empirical results in the paper. Gaussian measurements do not exist in MRI, so what is the purpose of discussing those results? Theorem 3.4 talks about general operators, but it is also a generic result that does not take into account any specific component of the proposed generative model-based method. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Brief discussion on limitation in the conclusion section.", "review_text": "", "extra_scores": {"needs_ethics_review": "Yes", "ethics_review_area": ["Responsible Research Practice (e.g., IRB, documentation, research ethics)"], "time_spent_reviewing": "5"}, "extra_text": {"ethical_concerns": "MRI is a popular imaging scheme and we need thorough analysis and test to make claims about generalization/accuracy of the proposed methods. I did not find an adequate discussion on ethical concerns in the paper. The paper actually claim that they are the first to show results on clinical data, but they probably should rephrase it as publicly available real data. "}}, {"review_id": "TtKu4NK8Z5", "reviewer": "Reviewer_zR76", "summary": "The paper applies deep generative priors successfully to accelerated MRI on clinical data. \n\nThe paper trains a score-based generative prior on brain scans from the fastMRI dataset. To recover an image from sub-sampled multi-coil data, the paper imposes the prior by performing posterior sampling via Langevian dynamics. The paper provides numerical results showing that this approach gives very good image quality. Besides the standard setup of in-distribution evaluation, the paper also provides numerical results for different sampling patterns and for a distribution shift from one anatomy to another, and again shows very good results even when applied to one anatomy and trained on another anatomy. \nThe paper also emphasizes that their method can obtain multiple reconstructions with different random initializations which can be used to quantify the uncertainty of the reconstructions. \nFinally, the paper provides theoretical results for posterior sampling. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Comments:\n1. The ranking of algorithms on in-distribution images (Section 4.1, Figure 4(a)) is inconsistent with the results in the literature. In the FastMRI leaderboard, Varnet outperforms ConvDecoder (Deep Decoder) on the Brain dataset, while in Figure 4a, Varnet is by 0.1 in SSIM  worse than ConvDecoder. This is a significant difference. \nPerhaps the reason is that the paper evaluates on MVUE images, but uses the VarNet pre-trained on RSS images (but fine-tuned on MVUE images). \nIn any case, it is important to understand where this difference comes from, because if in fact, the Varnet outperforms ConvDecoder as consitently reported in the literature, then Varnet would outperform Lagevian (i.e., the method in the paper), since ConvDecoder is very close to Lagevian. This in turn would change a major conclusion of the paper saying that 'we achieve competitive performance compared to end-to-end deep learning methods when the test-time data are sampled within the distribution'. \n\n2. Overall the experimental setup is very good and well described. However, why use MVUE images instead of RSS as the ground truth? The paper's numerical results are based on the FastMRI dataset and the FastMRI competition and all associated results are based on using the RSS images as groundtruth. Choosing MVUE as the groundtruth makes a comparison to the literature difficult and also leads to the complication that the VarNet used in the paper under review is trained on the RSS images, but then applied to a MVUE images (albeit it is fine-tuned on MVUE images). \n\n3. In the distribution shift section: It would be interesting to specify whether the results are obtained by applying the CSGM method with the same hyperparameters for changing sampling patterns, or if the paper tunes the hyperparameters of the methods on the new sampling patterns. This is important to know, as [20] has shown that hyperparameter tuning on the target distribution significantly impacts the performance on out-of-distribution data. \n\n4. The paper emphasizes that the Lagevian sampling method can obtain multiple reconstructions with different random initializations which can be used to quantify the uncertainty of the reconstructions. The paper writes in the corresponding Section 4.3 'as shown in Fig. 3, the pixel-wise standard deviation is a good estimate of the ground truth error': I really don't see this conclusion from Figure 3. It would be great to have a quantitative result for this statement.\n\n5. Theorem 3.4 states that if there is an algorithm that reconstructs $x^\\ast$ drawn from some distribution $\\mu$ from $y = A(x^\\ast)$ with $\\epsilon$ accuracy, then posterior sampling succeeds with $2\\epsilon$-accuracy. That's an interesting statement, but it would be good to also discuss the catch: for a given forward operator, it is typically difficult to check whether an $\\epsilon$-accurate algorithm exists, and we don't have access to the distribution of $\\mu$, but only to an estimate through the generative model. Also, it would be great if the paper could comment on whether the Langevin Dynamics converges to a sample from the posterior distribution, in order to clarify whether the recovery results Thm 3.3 and 3.4 apply to the algorithm that is used in the numerical results section.\n\nSignificance and originality:\nThe paper provides evidence that imposing a prior (i.e., recovery via the SCGM framework) can give very competitive results for clinical MRI. This is very interesting, as prior works did not achieve competitive results for real-world medical imaging data by imposing a generative prior. Imposing a prior has a number of advantages, in particular, that the method requires no retraining when changing the forward model. If the SCGM framework can really give results that are competitive to the state-of-the-art end-to-end neural networks, this would be a significant development.\n\nQuality and clarity:\nThe paper is well written overall and describes the setup and experiments well along with the assumption and the setup (apart from a few details pointed out above and in the minor comments below). The paper evaluates the method well, and the experimental setup makes a lot of sense, the only major concern outlined in comment 1 above is the significant discrepancy of the relative ranking of algorithms relative to the literature. \n\nSummary:\nI really enjoyed the paper and would be happy to see a (revised version) at the conference. My reason for not rating it higher, my concern in explained in comment 1. If this can be convincingly resolved, without the need to change the main claim of the paper, the paper would be a clear accept in my opinion, and I'll change my rating accordingly then.\n\n\n\nMinor comments:\n- Fig 1: Would be good to state the acceleration factor in the caption for reference, and whether this is single or multi-coil reconstruction.\n- The paper considers 'test time sampling patterns shifts': I fully agree that a major advantage of the CSGM framework is that it can be easily adapted to changes in the sample process, without the need of re-training. I wouldn't call this a distribution shift as the image distribution doesn't change, but again this is a minor point. \n- A related work is 'Compressible Latent-Space Invertible Networks for Generative Model-Constrained Image Reconstruction' which also showed good results on the FastMRI dataset, albeit not on the original k-space data and not on multi-coil data, and with a different algorithm, but also by imposing a generative prior via the CSGM framework.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "LuyaV6A6-tt", "reviewer": "Reviewer_HfL5", "summary": "This paper considers the problem of compressed sensing (CS) using generative priors, which can also specifically be characterized on a multi-coil MRI setup. The algorithm itself is a more general version of the CS setup which typically considers Gaussian measurements.\n\nThe authors consider a posteriori sampling procedure to invert the measurements $y = Ax^* + w$ by estimating $\\hat{x} \\sim \\mu(\\cdot|y)$ (modeled via deep generative prior) assuming $x^* \\sim \\mu$, given $A$ (MRI setup or general), $y$ and zero-mean $\\sigma^2$ variance Gaussian noise $w$ . Authors use an annealed Langevin Dynamics approach to sample from the posterior, which has been previously used in [73] for sampling from generative priors. Here the score function is modeled via a deep generative neural network.\n\nAuthors also present two theoretical results: one on the closeness of reconstruction $\\hat{x}$ and $x^*$ under distribution shift of the deep generative prior and true distributions (assuming Gaussian A), and another on the quality of posteriori sampling as compared to any other kind of reconstruction procedure under aribitrary (not neccessarily Gaussian) measurements assuming no distributional shift.\n\nAuthors present several experiments on the fastMRI and Stanford MRI datasets and compare to baselines such as L1/wavelet, MoDL, E2EVarNet and ConvDecoder and show  improvement reconstructions on both in-distribution and out-of distribution samplings in terms of SSIM.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper considers a measurement agnostic approach to inverting compressively sensed images under deep generative prior. Specifically authors use the model NCSNv2 from [74] for this purpose, which is a score based generative model. This paper uses an annealed Langevin dynamics algorithm to sample from posterior $\\nu$ and show experimentally that this approach works better in terms of SSIM to prior art on both in-distribution and out-of-distribution reconstructions at various acceleration rates (under sampling rates).  \n\nThrough theorem 3.3, the authors present a theoretical result for $\\hat{x}$ estimated according distribution $\\nu$, assuming a distributional shift from the true distribution $\\mu$ of $x^*$ under gaussian measurements $A$. The main claim of this result is that if $\\mu,\\nu$ are Wasserstein close by factor $\\epsilon$, then the $\\ell_2$ estimation error between $x^*$ and $\\hat{x}$ is lower bounded with high probability.\n\nThrough theorem 3.4, the authors show that this posterior sampling procedure produces a reconstruction within an additional $\\epsilon$-error  ($2\\epsilon$) of any other reconstruction procedure that is bounded by $\\epsilon$ with high probability (for any sampling process A).\n\nThe main contributions of this paper are theoretical results and the application of annealed langevin dynamics to CSGM via posteriori sampling (the algorithm itself is incremental in terms of derivation from prior art). \n\nThe paper is very well written highlighting the key contributions which is backed by their theorem statements and experiments. Experiments on Fast MRI are convincing as well. The paper can benefit from some minor changes as follows.\n\nFigure 1 can benefit from presenting reconstruction SSIM or MSE.\n\nSection 2.1. is the regime of experiments $compressive$? Since CSGM typically refers to $M<N$. Please make the relation between samples M and dimensionality N explicit. \n\nLine 113 is $N_c \\cdot L = M$?\n\nPrior to Line 220, the exact formulation of the deep generative model isn't clear or explicit. Please introduce a paragraph discussing choice of deep generative prior. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Deep neural networks have been successfully used for MRIs and are of importance to society for furthering medical imaging research. I do not think there are negative societal impacts of this work since the authors present a clear ground truth based on MVUE (no regularization, full samples). ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors provide the first demonstration that Compressed Sensing with Generative Priors can be a competitive approach for MRI reconstruction relative to end-to-end, L1, and untrained neural network methods.  The authors additionally demonstrate that the method is more robust than baselines in the context of certain distribution shifts between training and inversion.  The paper also provides novel theoretical guarantees about distributional robustness of posterior sampling approach.  

There was some concern from one of the reviewers about consistency with the literature about the performance of baseline methods, likely due to training being on RSS images with fine-tuning on MVUE images.  The authors should provide commentary about this issue, as discussed with the reviewers during the rebuttal. 

The authors should add additional comments that the paper should not be used for medical purposes without subsequent study by medical professionals, as per the ethics review.

Decision: Accept (Poster)