Model-based deep learning for inverse problems in imaging
Creators: name: Pramanik, Aniket
Publisher: name: University of Iowa
Year: 2023
Subjects: public abstract

description: Magnetic Resonance Imaging (MRI) systems provide incredible soft tissue contrast for the organ of interest in a non-invasive fashion. However, it is a notoriously slow imaging modality. A challenge with the acquisition of MRI data is the long scan time, especially for kids and older adults. Long acquisition times are also associated with extensive motion artifacts. A typical approach is to accelerate the data acquisition process by collecting fewer samples and interpolate the unknown ones using an image reconstruction algorithm. The state-of-the-art are model-based deep learning (MoDL) algorithms. Although MoDL approaches are fast and provide impressive performance, those require extensive memory which makes their applicability to higher dimensional problems challenging. MoDL approaches are inspired from compressed sensing based iterative optimization algorithms. However, these methods lack convergence guarantees to the global minima and are also fragile to adversarial attacks. MoDL is fragile to distribution shifts and therefore cannot be deployed in clinic since a wide variety of MRI acquisition settings are encountered. In the context of parallel MRI reconstruction, MoDL suffers from motion artifacts arising from mismatch in calibration and main scans mainly due to the explicit requirement of the MRI scanner’s coil sensitivity information.
description: We propose to solve the above-mentioned limitations. In the work titled “Deep-SLR”, we introduce a deep learning approach for calibration-less Parallel MRI recovery. It is a deep learning extension of the structured low-rank (SLR) algorithms that can perform calibration-less recovery of Parallel MRI. Since SLR algorithms are extremely slow, we propose a model-based deep learning approach for it to make it computationally faster and also to exploit the power of deep learning in obtaining improved performance over conventional SLR methods. This method has been found to be significantly faster and performing better than existing SLR algorithms. We have tested its performance in parallel MRI, single-channel MRI and also in multi-shot echo planar imaging.
description: We address the memory limitations of MoDL by utilizing a deep equilibrium (DEQ) network. The proposed approach requires memory equivalent to a single set of the optimization blocks which makes it efficient for higher dimensional problems and we show this benefit in performing 3D reconstructions of cardiac MRI using the proposed method. The convolutional neural network (CNN) used in this framework is Lipschitz constrained to ensure convergence of the network to a unique fixed point. We show that this method provides improved robustness to adversarial perturbations due to its contractive nature obtained through Lipschitz constraints. Its performance in the context of parallel MRI recovery is at par with conventional MoDL approaches.
description: We develop an acquisition adaptive MoDL framework (Ada-MoDL) for parallel MRI recovery. This framework consists of an additional multi-layer perceptron (MLP) that maps the acquisition information of the data into a set of scalar weights to have selectivity in feature maps generated by the CNN. The architecture is trained with data from multiple acquisition settings and we show that it requires fewer training data per setting compared to a conventional MoDL trained for a specific setting. The MLP parameters are around five percent extra. This approach also performs better than a single MoDL network trained with data from multiple settings. We consider variation in acquisition settings in terms of contrast, field-strength and acceleration. The proposed framework is a single trained model that can replace multiple acquisition setting specific models, thus avoiding training, storing and switching between them. This is beneficial in a clinical setup.
description: Overall, these works have shown promising results and have fairly improved image reconstruction tasks. The methods are general enough to be extended to other inverse imaging problems such as denoising, super-resolution, inpainting etc. These methods have also provided insights into the robustness of the models to adversarial attacks which is a common problem in the world of deep learning. Also, the memory efficient technique using DEQs can provide the pathway to solve higher dimensional inverse problems.