# Benchmarking MRI Reconstruction Neural Networks on Large Public Datasets
###### Abstract  
Deep learning is starting to offer promising results for reconstruction in Magnetic Resonance Imaging (MRI). A lot of networks are being developed, but the comparisons remain hard because the frameworks used are not the same among studies, the networks are not properly re-trained, and the datasets used are not the same among comparisons. The recent release of a public dataset, fastMRI, consisting of raw k-space data, encouraged us to write a consistent benchmark of several deep neural networks for MR image reconstruction. This paper shows the results obtained for this benchmark, allowing to compare the networks, and links the open source implementation of all these networks in Keras. The main finding of this benchmark is that it is beneficial to perform more iterations between the image and the measurement spaces compared to having a deeper per-space network.  
image reconstruction; neural networks; deep learning; fastMRI; OASIS; MRI 202  
a regularization term reflecting our prior on the data. The need for regularization comes from the fact that the problem is ill-posed since the sampling in the Fourier space, called k-space, is under the Nyquist-Shannon limit. However, these classical reconstruction methods exhibit two shortcomings.  
* They are usually iterative involving the computation of transforms on large data, and therefore, take a lot of time (2 min for a \(512\times 512-500\) um in plane resolution slice [7], on a machine with 8 cores).
* The regularization is usually not perfectly suited to MRI data (it is indeed very difficult to come up with a prior that perfectly reflects MR images).  
This is where learning comes in to play, and in particular, deep learning. The promise is that it will solve both the aforementioned problems.
* Because they are implemented efficiently on GPU and do not use an iterative algorithm, the deep learning algorithms run extremely fast.
* If they have enough capacity, they can learn a better prior of the MR images from the training set.  
One of the first neural networks to gain attention for its use in MRI reconstruction was AUTOMAP [8]. This network did not exploit a problem-specific property except the fact that the outcome was supposed to be an image. Some more recent works [9; 10; 11] have tried to inspire themselves from existing classical methods in order to leverage problem specific properties but also expertise gained in the field. However, they have not been compared against each other on a large dataset containing complex-valued raw data.  
A recently published dataset, fastMRI [12], allows this comparison, although it is still to be done and requires an implementation of the different networks in the same framework to allow for a fairer comparison in terms of, for example, runtime.  
Our contribution is exactly this, that is:  
* Benchmark different neural networks for MRI reconstruction on two datasets: the fastMRI dataset, containing raw complex-valued knee data, and the OASIS dataset [13] containing DICOM real-valued brain data.
* Provide reproducible code and the networks' weights ([[https://github.com/zachariearamzi/fastmri-reproducible-benchmark](https://github.com/zachariearamzi/fastmri-reproducible-benchmark)]([https://github.com/zachariearamzi/fastmri-reproducible-benchmark](https://github.com/zachariearamzi/fastmri-reproducible-benchmark))), using Keras [14] with a TensorFlow backend [15].  
While our work focuses on classical MRI modalities reconstruction, note that other works have applied deep learning to other modalities like MR fingerprinting [16] or diffusion MRI [17]. The networks studied here could be applied but would not benefit from some invariants of the problem, especially in the fourth (contrast-related) dimension introduced.