2 2 m a r c h 2 0 1 8 | V O L 5 5 5 | N a T U r E | 4 8 7 LETTEr Image reconstruction is essential for imaging applications across the physical and life sciences, including optical and radar systems, magnetic resonance imaging, X-ray computed tomography, positron emission tomography, ultrasound imaging and radio astronomy 1-3 . During image acquisition, the sensor encodes an intermediate representation of an object in the sensor domain, which is subsequently reconstructed into an image by an inversion of the encoding function. Image reconstruction is challenging because analytic knowledge of the exact inverse transform may not exist a priori, especially in the presence of sensor non-idealities and noise. Thus, the standard reconstruction approach involves approximating the inverse function with multiple ad hoc stages in a signal processing chain 4,5 , the composition of which depends on the details of each acquisition strategy, and often requires expert parameter tuning to optimize reconstruction performance. Here we present a unified framework for image reconstructionautomated transform by manifold approximation (AUTOMAP)which recasts image reconstruction as a data-driven supervised learning task that allows a mapping between the sensor and the image domain to emerge from an appropriate corpus of training data. We implement AUTOMAP with a deep neural network and exhibit its flexibility in learning reconstruction transforms for various magnetic resonance imaging acquisition strategies, using the same network architecture and hyperparameters. We further demonstrate that manifold learning during training results in sparse representations of domain transforms along low-dimensional data manifolds, and observe superior immunity to noise and a reduction in reconstruction artefacts compared with conventional handcrafted reconstruction methods. In addition to improving the reconstruction performance of existing acquisition methodologies, we anticipate that AUTOMAP and other learned reconstruction approaches will accelerate the development of new acquisition strategies across imaging modalities. The paradigm shift from manual to automatic feature extraction in a host of machine learning tasks including speech recognition 6 and image classification 7 has demonstrated the advantage of allowing real-world data to guide efficient representation through a structured training process. This strategy is mirrored in biological organisms for refining visual perception in a process known as perceptual learning 8 . Human visual reconstruction of time-domain neural codes into the percept image is trained through experience during cognitive development into adulthood. This conditioning on prior data has been shown to be critical to robust performance in low signal-to-noise settings 9 , which are fundamentally challenging for artificial imaging systems across disciplines and applications. In contemporary medical imaging, faithful reconstruction of noisy image acquisitions is of particular importance as the clinical push for faster scanning increasingly relies on acquisition strategies that result in a reduction of the signal-to-noise ratio, be they undersampled magnetic resonance imaging (MRI), or low-dose X-ray computed tomography imaging.
In a recent approach similar to the work presented here, REF ) studied image reconstruction for various medical imaging modalities by deep learning the transform between sensor and image domain.
Image reconstruction by domain-transform manifold learning
Nature
Nature
Computer Science Medicine