Z ERO -S HOT S ELF -S UPERVISED L EARNING FOR MRI R ECONSTRUCTION
A BSTRACT
Deep learning (DL) has emerged as a powerful tool for accelerated MRI reconstruction, but often necessitates a database of fully-sampled measurements for training.
Recent self-supervised and unsupervised learning approaches enable training without fully-sampled data.
However, a database of undersampled measurements may not be available in many scenarios, especially for scans involving contrast or translational acquisitions in development.
Moreover, recent studies show that database-trained models may not generalize well when the unseen measurements differ in terms of sampling pattern, acceleration rate, SNR, image contrast, and anatomy.
Such challenges necessitate a new methodology to enable subject-speciﬁc DL MRI reconstruction without external training datasets, since it is clinically imperative to provide high-quality reconstructions that can be used to identify lesions/disease for every individual .
In this work, we propose a zeroshot self-supervised learning approach to perform subject-speciﬁc accelerated DL
MRI reconstruction to tackle these issues.
The proposed approach partitions the available measurements from a single scan into three disjoint sets.
Two of these sets are used to enforce data consistency and deﬁne loss during training for selfsupervision, while the last set serves to self-validate, establishing an early stopping criterion.
In the presence of models pre-trained on a database with different image characteristics, we show that the proposed approach can be combined with transfer learning for faster convergence time and reduced computational complexity.
I NTRODUCTION
Magnetic resonance imaging (MRI) is a non-invasive, radiation-free medical imaging modality that provides excellent soft tissue contrast for diagnostic purposes.
However, lengthy acquisition times in MRI remain a limitation.
Accelerated MRI techniques acquire fewer measurements at a subNyquist rate, and use redundancies in the acquisition system or the images to remove the resulting aliasing artifacts during reconstruction.
In clinical MRI systems, multi-coil receivers are used during data acquisition.
Parallel imaging (PI) is the most clinically used method for accelerated MRI, and exploits the redundancies between these coils for reconstruction (Pruessmann et al., 1999;
Griswold et al., 2002).
Compressed sensing (CS) is another conventional accelerated MRI technique that exploits the compressibility of images in sparsifying transform domains (Lustig et al., 2007), and is commonly used in combination with PI.
However, PI and CS may suffer from noise and residual artifacts at high acceleration rates (Robson et al., 2008; Sandino et al., 2020).
Recently, deep learning (DL) methods have emerged as an alternative for accelerated MRI due to their improved reconstruction quality compared to conventional approaches (Hammernik et al., 2018; Knoll et al., 2020b; Akc¸akaya et al., 2022).
Particularly, physics-guided deep learning reconstruction (PG-DLR) approaches have gained interest due to their robustness and improved performance (Hammernik et al., 2018; Hosseini et al., 2020b).
PG-DLR explicitly incorporates the physics of the data acquisition system into the neural network via a procedure known as algorithm unrolling (Monga et al., 2021).
This is done by unrolling iterative optimization algorithms that alternate between data consistency (DC) and regularization steps for a ﬁxed number of iterations.
Subsequently, PG-DLR methods are trained in a supervised manner using large databases of fully-sampled measurements (Hammernik et al., 2018;
Aggarwal et al., 2019).
More recently, self-supervised learning has shown that reconstruction quality similar to supervised PG-DLR can be achieved while training on a database of only undersampled measurements (Yaman et al., 2020).
While such database learning strategies offer improved reconstruction quality, acquisition of large datasets may often be infeasible.
In some MRI applications involving time-varying physiological processes, dynamic information such as time courses of signal changes, contrast uptake or breathing patterns may differ substantially between subjects, making it difﬁcult to generate high-quality databases of sufﬁcient size for the aforementioned strategies.
Furthermore, database training, in general, brings along concerns about robustness and generalization (Eldar et al., 2017; Knoll et al., 2020c).
In MRI reconstruction, this may exhibit itself when there are mismatches between training and test datasets in terms of image contrast, sampling pattern, SNR, vendor, and anatomy.
While it is imperative to have high-quality reconstructions that can be used to correctly identify lesions/disease for every individual , the fastMRI transfer track challenge shows that pretrained models fail to generalize when applied to patients/scans with different distribution or acquisition parameters, with potential for misdiagnosis (Muckley et al., 2021).
Finally, training datasets may lack examples of rare and/or subtle pathologies, increasing the risk of generalization failure (Knoll et al., 2019; 2020c).
In this work, we tackle these challenges associated with database training, and propose a zero-shot self-supervised learning (ZS-SSL) approach, which performs subject-speciﬁc training of PG-DLR without any external training database.
Succinctly, ZS-SSL partitions the acquired measurements into three types of disjoint sets, which are respectively used only in the PG-DLR neural network, in deﬁning the training loss, and in establishing a stopping strategy to avoid overﬁtting.
Thus, our training is both self-supervised and self-validated.
In cases where a database-pretrained network is available, ZS-SSL leverages transfer learning (TL) for improved reconstruction quality and reduced computational complexity.
Our contributions can be summarized as follows:
• We propose a zero-shot self-supervised method for learning subject-speciﬁc DL MRI reconstruction from a single undersampled dataset without any external training database.
• We provide a well-deﬁned methodology for determining stopping criterion to avoid overﬁtting in contrast to other single-image training approaches (Ulyanov et al., 2018).
• We apply the proposed zero-shot learning approach to knee and brain MRI datasets, and show its efﬁcacy in removing residual aliasing and banding artifacts compared to supervised database learning.
• We show our ZS-SSL can be combined with with TL in cases when a database-pretrained model is available to reduce computational costs.
• We show that our zero-shot learning strategies address robustness and generalizability issues of trained supervised models in terms of changes in sampling pattern, acceleration rate, contrast, SNR, and anatomy at inference time.
B ACKGROUND AND R ELATED W ORK 2. A CCELERATED MRI A CQUISITION M ODEL
In MRI, raw measurement data is acquired in the frequency domain, also known as k-space.
In current clinical MRI systems, multiple receiver coils are used, where each is sensitive to different parts of the volume.
In practice, MRI is accelerated by taking fewer measurements, which are characterized by an undersampling mask that speciﬁes the acquired locations in k-space.
For a multi-coil MRI acquisition, the forward model is given as
[Equation]
where x is the underlying image, y i is the acquired data for the i th coil, P Ω is the masking operator for undersampling pattern Ω , F is the Fourier transform, C i is a diagonal matrix characterizing the i th coil sensitivity, n i is measurement noise for i th coil, and n c is the number of coils (Pruessmann et al., 1999).
This system can be concatenated across the coil dimension for a compact representation
[Equation]
where y Ω is the acquired undersampled measurements across all coils, E Ω is the forward encoding operator that concatenates P Ω F C i across i ∈ { 1 , . . .
, n c } .
The general inverse problem for accelerated MRI is given as
[Equation]
where the (cid:107) y Ω
− E Ω x (cid:107) 22 term enforces consistency with acquired data (DC) and R ( · ) is a regularizer.
PG-DLR WITH A LGORITHM U NROLLING
Several optimization methods are available for solving the inverse problem in (3) (Fessler, 2020).
Variable-splitting via quadratic penalty is one of the approaches that can be employed to cast Eq.
(3) into two sub-problems as
[Equation]
where µ is the penalty parameter, z ( i ) is an intermediate variable and x ( i ) is the desired image at iteration i .
In PG-DLR, an iterative algorithm, as in (4a) and (4b) is unrolled for a ﬁxed number of iterations (Liang et al., 2020).
The regularizer sub-problem in Eq.
(4a) is implicitly solved with neural networks and the DC sub-problem in Eq.
(4b) is solved via linear methods such as gradient descent (Hammernik et al., 2018) or conjugate gradient (CG)
(Aggarwal et al., 2019).
There have been numerous works on PG-DLR for accelerated MRI (Schlemper et al., 2018; Hammernik et al., 2018;
Aggarwal et al., 2019; Liang et al., 2020;
Yaman et al., 2020).
Most of these works vary from each other on the algorithms used for DC and neural networks employed in the regularizer units.
However, all these works require a large database of training samples.
2. S UPERVISED L EARNING FOR PG-DLR
In supervised PG-DLR, training is performed using a database of fully-sampled reference data.
Let y n ref be the fully-sampled k-space for subject n and f ( y n Ω , E n Ω ; θ ) be the output of the unrolled network for under-sampled k-space y n Ω , where the network is parameterized by θ .
End-to-end training minimizes (Knoll et al., 2020b; Yaman et al., 2020)
[Equation]
where N is the number of samples in the training database, E n full is the fully-sampled encoding operator that transform network output to k-space and L ( · , · ) is a loss function.
2. S ELF -S UPERVISED L EARNING FOR PG-DLR
Unlike supervised learning, self-supervised learning enables training without fully-sampled data by only utilizing acquired undersampled measurements (Yaman et al., 2020).
A masking approach is used for self-supervision in this setting, where a subset Λ ⊂ Ω is set aside for checking prediction performance/loss calculation, while the remainder of points Θ = Ω \ Λ are used in the DC units of the PG-DLR network.
End-to-end training is performed using the loss function
[Equation] [Equation]
Z ERO -S HOT S ELF -S UPERVISED L EARNING FOR PG-DLR
As discussed in Section 1, lack of large datasets in numerous MRI applications, as well as robustness and generalizability issues of pretrained models pose a challenge for the clinical translation of DL reconstruction methods.
Hence, subject-speciﬁc reconstruction is desirable in clinical practice, since it is critical to achieve a reconstruction quality that can be used for correctly diagnosing every patient.
While the conventional self-supervised masking strategy, as in (Yaman et al., 2020) can be applied for subject-speciﬁc learning, it leads to overﬁtting unless the training is stopped early (Hosseini et al., 2020a).
This is similar to other single-image learning strategies, such as the deep image prior (DIP) or zero-shot super-resolution (Ulyanov et al., 2018; Shocher et al., 2018).
DIP-type approaches shows that an untrained neural network can successfully perform instance-speciﬁc image restoration tasks such as denoising, super-resolution, inpainting without any training data.
However, such DIPtype techniques requires an early stopping for avoiding over-ﬁtting, which is typically done with a manual heuristic selection (Ulyanov et al., 2018; Hosseini et al., 2020a; Darestani et al., 2021).
While this may work in a research setting having a well-deﬁned automated early stopping criterion is critical to fully harness the potential of subject-speciﬁc DL MRI reconstruction in practice
Early stopping regularization in database-trained setting is conventionally motivated through the bias-variance trade-off, in which a validation set is used as a proxy for the generalization error to identify the stopping criterion.
Using the same bias-variance trade-off motivation, having a validation set can aid in devising a stopping criterion, but this has not been feasible in existing zero-shot learning approaches, which either use all acquired measurements (Ulyanov et al., 2018;
Senouf et al., 2019) or partition them into two sets for training and deﬁning loss (Hosseini et al., 2020a).
Hence, existing zero-shot learning techniques lack a validation set to identify the stopping criterion.
ZS-SSL Formulation and Training:
We propose a new ZS-SSL partitioning framework to enable subject-speciﬁc self-supervised training and validation with a well-deﬁned stopping criterion.
We deﬁne the following partition for the available measurement locations from a single scan , Ω :
[Equation]
where (cid:116) denotes a disjoint union, i.e. Θ , Λ and Γ are pairwise disjoint (Figure 1).
Similar to Section 2.4, Θ is used in the DC units of the unrolled network, and Λ is used to deﬁne a k-space loss for the self-supervision of the network.
The third partition Γ is a set of acquired k-space indices set aside for deﬁning a k-space validation loss.
Thus, ZS-SSL training is both self-supervised and self-validated.
In general, since zero-shot learning approaches perform training using a single dataset, generation of multiple data pairs from this single dataset is necessary to self-supervise the neural network (Quan et al., 2020).
Hence, we generate multiple (Θ , Λ) pairs from the acquired locations Ω of the single scan.
In ZS-SSL, this is achieved by ﬁxing the k-space validation partition Γ ⊂ Ω , and performing the retrospective masking on Ω \ Γ multiple times.
Formally, Ω \ Γ is partitioned K times such that
[Equation] [Equation]
where Λ k , Θ k and Γ are pairwise disjoint,
i.e. Ω = Γ (cid:116)
Θ k (cid:116) Λ k , ∀ k .
ZS-SSL training minimizes
[Equation]
In the proposed ZS-SSL, this is supplemented by a k-space self-validation loss, which tests the generalization performance of the trained network on the k-space validation partition Γ .
For the l th epoch, where the learned network weights are speciﬁed by θ ( l ) , this validation loss is given by:
[Equation]
Note that in (9), the network output is calculated by applying the DC units on Ω \ Γ =
Θ (cid:116) Λ , i.e. all acquired points outside of Γ , to better assess its generalizability performance.
Our key motivation is that while the training loss will decrease over epochs, the k-space validation loss will start increasing once overﬁtting is observed.
Thus, we monitor the loss in (9) during training to deﬁne an early stopping criterion to avoid overﬁtting.
Let L be the epoch in which training needs to be stopped.
Then at inference time, the network output is calculated as f ( y Ω , E Ω ; θ ( L ) ) , i.e. all acquired points are used to calculate the network output.
ZS-SSL with Transfer Learning (TL): While pretrained models are very efﬁcient in reconstructing new unseen measurements from similar MRI scan protocols, their performance degrades significantly when acquisition parameters vary (Muckley et al., 2021).
Moreover, retraining a new model on a large database for each acquisition parameter, sampling/contrast/anatomy/acceleration, may be very computationally expensive (Knoll et al., 2019).
Hence, TL has been used for re-training DL models pre-trained on large databases to reconstruct MRI data with different characteristics (Knoll et al., 2019).
However, such transfer still requires another, often smaller, database for re-training.
In contrast, in the presence of pre-trained models, ZS-SSL can be combined with TL, referred to as ZSSSL-TL, to reconstruct a single slice/instance with different characteristics by using weights of the pre-trained model for initialization.
Thus, ZS-SSL-TL ensures that the pretrained model is adapted for each patient/subject, while facilitating faster convergence time and reduced reconstruction time.
E XPERIMENTS 4. D ATASETS
We performed experiments on publicly available fully-sampled multi-coil knee and brain MRI from fastMRI database (Knoll et al., 2020a).
Knee and brain MRI datasets contained data from 15 andreceiver coils, respectively.
Fully-sampled datasets were retrospectively undersampled by keeping 24 lines of autocalibrated signal (ACS) from center of k-space.
FastMRI database contains different contrast weightings.
For knee MRI, we used coronal proton density (Cor-PD) and coronal proton density with fat suppression (Cor-PDFS), and for brain MRI, axial FLAIR (Ax-FLAIR) and axial
T2 (Ax-T2).
Different types of datasets and undersampling masks used in this study are provided in
Figure 7 in the Appendix.
4. I MPLEMENTATION D ETAILS
All PG-DLR approaches were trained end-to-end using 10 unrolled iterations.
CG method and a
ResNet structure (Timofte et al., 2017) were employed in the DC and regularizer units of the unrolled network, respectively (Yaman et al., 2020).
The ResNet is comprised of a layer of input and output convolution layers, and 15 residual blocks (RB) each containing two convolutional layers, where the ﬁrst layer is followed by ReLU and the second layer is followed by a constant multiplication (Timofte et al., 2017).
All layers had a kernel size of 3 × 3 , 64 channels.
The real and imaginary parts of the complex MR images were concatenated prior to being input to the ResNet as 2-channel images.
The unrolled network, which shares parameters across the unrolled iterations had a total of 592,129 trainable parameters.
Coil sensitivity maps were generated from the central 24 × 24 ACS using ESPIRiT (Uecker et al., 2014).
End-to-end training was performed with a normalized (cid:96) 1 - (cid:96)loss (Adam optimizer, LR = 5 · 10 − 4 , batch size = 1) (Yaman et al., 2020).
Peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) were used for quantitative evaluation.
4. R ECONSTRUCTION M ETHOD C OMPARISONS
In this work, we focus on comparing training strategies for accelerated MRI reconstruction.
Thus, we use the same network architecture from Section 4.2 for all training methods in all experiments.
We note that the proposed ZS-SSL strategy is agnostic to the speciﬁcs of the neural network architecture.
In fact, the number of network parameters is higher than the number of undersampled measurements available on a single slice, i.e. dimension of y Ω .
As such, different neural networks may be used for the regularizer unit in the unrolled network, but this is not the focus of our study.
Supervised PG-DLR: Supervised PG-DLR models for knee and brain MRI were trained onslices from 15 and 30 different subjects, respectively.
For each knee and brain contrast weighting, two networks were trained separately using random and uniform masks (Hammernik et al., 2018) at an acceleration rate (R) of 4 (Knoll et al., 2020c).
Trained networks were used for comparison and
TL purposes.
We note that random undersampling results in incoherent artifacts, whereas uniform undersampling leads to coherent artifacts that are harder to remove (Figure 7 in Appendix) (Knoll et al., 2019).
Hence, we focus on the more difﬁcult problem of uniform undersampling, while presenting random undersampling results in the Appendix.
Self-Supervision via Data Undersampling (SSDU) PG-DLR: SSDU (Yaman et al., 2020)
PGDLR was trained using the same database approach as supervised PG-DLR, with the exception that
SSDU performed training only using the undersampled data (Sec. 2.4).
DIP-Recon: We employ a DIP-type subject-speciﬁc MRI reconstruction that uses all acquired measurements in both DC and deﬁning loss (Senouf et al., 2019; Jafari et al., 2021)
[Equation] [Equation]
We refer to the reconstruction from this training mechanism as DIP-Recon.
DIP-Recon-TL refers to combining (10) with TL.
As mentioned, DIP-Recon does not have a stopping criterion, hence early stopping was heuristically determined (Figure 8 in the Appendix).
Parallel Imaging:
We include CG-SENSE, which is a commonly used subject-speciﬁc conventional
PI method (Pruessmann et al., 1999; 2001), as the clinical baseline quality for comparison purposes.
4. A UTOMATED S TOPPING AND A BLATION S TUDY
The stopping criterion for the proposed ZS-SSL was investigated on slices from the knee dataset.
The k-space self-validation set Γ was selected from the acquired measurements Ω using a uniformly random selection with | Γ | / | Ω | = 0 .
The remaining acquired measurements Ω \ Γ were retrospectively partitioned into disjoint 2-tuples multiple times based on uniformly random selection with the ratio ρ = | Λ k | / | Ω \ Γ | = 0 .
4 ∀ k ∈ { 1 , . .
, K } (Yaman et al., 2020).
Figure 2a shows representative subject-speciﬁc training and validation loss curves at R = 4 for K ∈ { 1 , 10 , 25 , 50 , 100 } .
As expected, training loss decreases with increasing epochs for all K .
The k-space validation loss for K = 1 decreases without showing a clear breaking point for stopping.
For K > 1, the validation loss forms an L-curve, and the breaking point of the L-curve is used as the stopping criterion.
K = 10 is used for the rest of the study, while noting K = 25, 50 and also show similar performance.
Figure 2b shows loss curves on a Cor-PD slice with and without transfer learning.
ZS-SSL-TL, which uses pre-trained supervised PG-DLR parameters as initial starting parameters, converges faster in time compared to ZS-SSL, substantially reducing the total training time.
Average computation times for single-instance reconstruction methods are presented in Table 2 in the Appendix.
Similarly, corresponding reconstruction results for the loss curves in
Figure 2a and b are provided in Figure 9 in the Appendix.
4. R ECONSTRUCTION R ESULTS
In the ﬁrst set of experiments, we compare all methods for the case when the testing and training data belong to the same knee/brain MRI contrast weighting with the same acceleration rate and undersampling mask.
These experiments aim to show the efﬁcacy of the proposed approach in performing subject-speciﬁc MRI reconstruction, while removing residual aliasing artifacts.
We also note that this is the most favorable setup for database-trained supervised PG-DLR.
In the subsequent experiments, we focus on the reported generalization and robustness issues with database-trained PG-DLR methods (Knoll et al., 2019; 2020c; Defazio et al., 2020; Muckley et al., 2021).
We investigate banding artifacts, as well as in-domain and cross-domain transfer cases.
For these experiments, we concentrate on ZS-SSL-TL, since ZS-SSL has no prior domain information, and is inherently not susceptible to such generalizability issues.
Comparison of Reconstruction Methods: In these experiments, supervised and SSDU PG-DLR are trained and tested using uniform undersampling at R = 4, representing a perfect match for training and testing conditions.
Figure 3a and b show reconstruction results for Cor-PD knee and Ax-FLAIR brain MRI datasets in this setting.
CG-SENSE reconstruction suffers from signiﬁcant residual artifacts and noise ampliﬁcation in Cor-PD knee and Ax-FLAIR brain MRIs, respectively.
Similarly, both DIP-Recon and DIP-Recon-TL suffer from residual artifacts and noise ampliﬁcation.
Supervised PG-DLR achieves artifact-free reconstruction.
Both ZS-SSL and ZS-SSL-TL also perform artifact-free reconstruction with similar image quality.
Table 1 shows the average SSIM and PSNR values on 30 test slices.
Similar observations apply when random undersampling is employed (Figure 10 in the Appendix).
For the remaining experiments, we investigate the generalizability of database-pretrained models using supervised PG-DLR as baseline due to its higher performance,
[Table]
while noting SSDU PG-DLR, which is a self-supervised database-trained model, may also be used as a baseline if needed.
Banding Artifacts: Banding artifacts appear in the form of streaking horizontal lines, and occur due to high acceleration rates and anisotropic sampling (Defazio et al., 2020).
These hinder radiological evaluation and are regarded as a barrier for the translation of DL reconstruction methods into clinical practice (Defazio et al., 2020).
This set of experiments explored training and testing on Cor-PDFS data, where database-trained PG-DLR reconstruction has been reported to show such artifacts (Defazio et al., 2020; Muckley et al., 2021).
Figure 4 shows reconstructions for a Cor-PDFS test slice.
While DIP-Recon-TL suffers from clearly visible noise ampliﬁcation, supervised PG-DLR suffers from banding artifacts shown with yellow arrows.
ZS-SSL-TL signiﬁcantly alleviates these banding artifacts in the reconstruction.
While supervised PG-DLR achieves slightly better SSIM and PSNR (Table 3 in the Appendix), we note that the banding artifacts do not necessarily correlate with such metrics, and are usually picked up in expert readings (Defazio et al., 2020; Knoll et al., 2020c).
In-Domain Transfer: In these experiments, we compared the in-domain generalizability of database-trained PG-DLR and subject-speciﬁc PG-DLR.
For in-domain transfer, training and test datasets are of the same type of data, but may differ from each other in terms of acceleration and undersampling pattern (Figure 11 in the Appendix).
In Figure 5a, supervised PG-DLR was trained with random undersampling and tested on uniform undersampling, both at R = 4.
Supervised PG-DLR fails to generalize and suffers from residual aliasing artifacts (red arrows), consistent with previous reports (Knoll et al., 2019; Muckley et al., 2021).
Similarly, DIP-Recon-TL suffers from artifacts and noise ampliﬁcation.
Proposed ZS-SSL-TL achieves an artifact-free and improved reconstruction quality.
In Figure 5b, supervised PG-DLR was trained with uniform undersampling at R =and tested on uniform undersampling at R = 6.
While both supervised PG-DLR and DIP-Recon-TL suffers from aliasing artifacts, ZS-SSL-TL successfully removes these artifacts.
Average PSNR and
SSIM values align with the observations (Table 3 in the Appendix).
Cross-Domain Transfer: In the last set of experiments, we investigated the cross-domain generalizability of database-trained PG-DLR compared to subject-speciﬁc trained PG-DLR.
For crossdomain transfer, training and test datasets are of the different data characteristics and generally differ in terms of contrast, SNR, and anatomy (Figure 11 in the Appendix).
Figure 6 shows results for the case when the testing contrast/SNR and anatomy differs from training contrast/SNR and anatomy, even though the same R = 4 uniform undersampling is used for both training and testing.
In Figure 6a, supervised PG-DLR was trained on Cor-PDFS (low-SNR), but tested on Cor-PD (high-SNR and different contrast).
In Figure 6b, supervised PG-DLR was trained on Ax-FLAIR (brain MRI) and tested on Cor-PD (knee MRI).
In both cases, supervised PG-DLR fails to generalize and has residual artifacts (red arrows).
Similarly, DIP-Recon-TL suffers from artifacts and noise.
ZS-SSLTL achieves an artifact-free improved reconstruction.
For both cross-domain transfer experiments, similar results were observed for brain MRI (Figure 12 in the Appendix).
Average PSNR and SSIM values match these observations (Table 3 in the Appendix)
C ONCLUSIONS
We proposed a zero-shot self-supervised deep learning method, ZS-SSL, for subject-speciﬁc accelerated DL MRI reconstruction from a single undersampled dataset.
The proposed ZS-SSL partitions the acquired measurements from a single scan into three types of disjoint sets, which are used only in the PG-DLR network, in deﬁning the training loss, and in establishing a validation strategy for early stopping to avoid overﬁtting.
In particular, we showed that with our training methodology and automated stopping criterion, subject-speciﬁc zero-shot learning of PG-DLR for MRI can be achieved even when the number of tunable network parameters is higher than the number of available measurements.
Finally, we also combined ZS-SSL with transfer learning, in cases where a pre-trained model may be available, for faster convergence time and reduced reconstruction time.
Our results showed that ZS-SSL methods perform similarly to database-trained supervised PG-DLR when training and testing data are matched, and they signiﬁcantly outperform database-trained methods in terms of artifact reduction and generalizability when the training and testing data differ in terms of image characteristics and acquisition parameters.
In fact, the subject-speciﬁc nature of
ZS-SSL ensures that it is agnostic to such changes in acquisition parameters.
As such, the proposed work is able to provide good reconstruction quality for each subject, and may have signiﬁcant implications in the integration of DL reconstruction to clinical studies.
We note that hyperparameters, such as learning rate may be adjusted based on the domain for further improvements.
It is also noteworthy that the subject-speciﬁc ZS-SSL eliminates the requirement for large training sets.
This may also facilitate the use and appeal of DL reconstruction for recently developed acquisitions, as well as pilot studies that are often performed to determine the acquisition parameters/acceleration rates of large-scale imaging studies, such as the Human Connectome Project (HCP) (Ugurbil et al., 2013).
Finally, while we concentrated on physics-guided models in MRI reconstruction, our ideas and results may inspire further work in related image restoration problems, as well as for generative models or data-driven problems without a data consistency term. A CKNOWLEDGEMENT This work was partially supported by NIH R01HL153146, P41EB027061, U01EB025144; NSF CAREER CCF-1651825.
There is no conﬂict of interest for the authors.