Limited capture range, and the requirement to provide high quality initialization for optimization-based 2D 3D image registration methods, can significantly degrade the performance of 3D image reconstruction and motion compensation pipelines. Challenging clinical imaging scenarios, which contain significant subject motion such as fet al in-utero imaging, complicate the 3D image and volume reconstruction process. In this paper we present a learning based image registration method capable of predicting 3D rigid transformations of arbitrarily oriented 2D image slices, with respect to a learned canonical atlas co-ordinate system. Only image slice intensity information is used to perform registration and canonical alignment, no spatial transform initialization is required. To find image transformations we utilize a Convolutional Neural Network (CNN) architecture to learn the regression function capable of mapping 2D image slices to a 3D canonical atlas space. We extensively evaluate the effectiveness of our approach quantitatively on simulated Magnetic Resonance Imaging (MRI), fet al brain imagery with synthetic motion and further demonstrate qualitative results on real fet al MRI data where our method is integrated into a full reconstruction and motion compensation pipeline. Our learning based registration achieves an average spatial prediction error of 7 mm on simulated data and produces qualitatively improved reconstructions for heavily moving fetuses with gestational ages of approximately 20 weeks. Our model provides a general and computationally efficient solution to the 2D 3D registration initialization problem and is suitable for real-time scenarios.
The 2D 3D registration methods, by @cite_30 , @cite_25 and @cite_15 , use CNNs to compute the unknown transformation of a given 2D slice with respect to a reference 3D volume while @cite_27 @cite_31 @cite_46 @cite_13 @cite_32 @cite_17 @cite_36 use general registration algorithms to register slices to an initial 3D target. In both cases, a motion free reference initial volume is required for successful 2D 3D registration, which is not guaranteed to be obtainable from a clinical scan due to unpredictable subject motion. @cite_33 proposed to perform slice-to-volume registration by minimizing the energy of weighted Mean Square Difference (wMSD) of all slice intersection intensity profiles. This method does not require an initial target registration volume nor intermediate 3D reconstructions. The authors were able to recover motion up to 15 mm of translation and 30 @math of rotation for individual slices".
@cite_30 @cite_31 @cite_33 @cite_46 @cite_36 @cite_32 @cite_27 @cite_15 @cite_13 @cite_25 @cite_17
2344328023 2166966312 2156875969 2135264048 1994514622 2090612612 2755211235 2950997711
This paper aims to solve a fundamental problem in intensity-based 2D 3D registration, which concerns the limited capture range and need for very good initialization of state-of-the-art image registration methods. We propose a regression approach that learns to predict rotations and translations of arbitrary 2D image slices from 3D volumes, with respect to a learned canonical atlas co-ordinate system. To this end, we utilize Convolutional Neural Networks (CNNs) to learn the highly complex regression function that maps 2D image slices into their correct position and orientation in 3D space. Our approach is attractive in challenging imaging scenarios, where significant subject motion complicates reconstruction performance of 3D volumes from 2D slice data. We extensively evaluate the effectiveness of our approach quantitatively on simulated MRI brain data with extreme random motion. We further demonstrate qualitative results on fet al MRI where our method is integrated into a full reconstruction and motion compensation pipeline. With our CNN regression approach we obtain an average prediction error of 7 mm on simulated data, and convincing reconstruction quality of images of very young fetuses where previous methods fail. We further discuss applications to Computed Tomography (CT) and X-Ray projections. Our approach is a general solution to the 2D 3D initialization problem. It is computationally efficient, with prediction times per slice of a few milliseconds, making it suitable for real-time scenarios. Capturing an enclosing volume of moving subjects and organs using fast individual image slice acquisition has shown promise in dealing with motion artefacts. Motion between slice acquisitions results in spatial inconsistencies that can be resolved by slice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms are, however, typically very slow, specialised to specific applications and rely on approximations, which impedes their potential clinical use. In this paper, we present a fast multi-GPU accelerated framework for slice-to-volume reconstruction. It is based on optimised 2D 3D registration, super-resolution with automatic outlier rejection and an additional (optional) intensity bias correction. We introduce a novel and fully automatic procedure for selecting the image stack with least motion to serve as an initial registration target. We evaluate the proposed method using artificial motion corrupted phantom data as well as clinical data, including tracked freehand ultrasound of the liver and fet al Magnetic Resonance Imaging. We achieve speed-up factors greater than 30 compared to a single CPU system and greater than 10 compared to currently available state-of-the-art multi-core CPU methods. We ensure high reconstruction accuracy by exact computation of the point-spread function for every input data point, which has not previously been possible due to computational limitations. Our framework and its implementation is scalable for available computational infrastructures and tests show a speed-up factor of 1.70 for each additional GPU. This paves the way for the online application of image based reconstruction methods during clinical examinations. The source code for the proposed approach is publicly available. Motion degrades magnetic resonance (MR) images and prevents acquisition of self-consistent and high-quality volume images. A novel methodology, Snapshot magnetic resonance imaging (MRI) with volume reconstruction (SVR) has been developed for imaging moving subjects at high resolution and high signal-to-noise ratio (SNR). The method combines registered 2D slices from sequential dynamic single-shot scans. The SVR approach requires that the anatomy in question is not changing shape or size and is moving at a rate that allows snapshot images to be acquired. After imaging the target volume repeatedly to guarantee sufficient sampling every where, a robust slice-to-volume registration method has been implemented that achieves alignment of each slice within 0.3 mm in the examples tested. Multilevel scattered interpolation has been used to obtain high-fidelity reconstruction with root-mean-square (rms) error that is less than the noise level in the images. The SVR method has been performed successfully for brain studies on subjects that cannot stay still, and in some cases were moving substantially during scanning. For example, awake neonates, deliberately moved adults and, especially, on fetuses, for which no conventional high-resolution 3D method is currently available. Fine structure of the in-utero fet al brain is clearly revealed for the first time and substantial SNR improvement is realized by having many individually acquired slices contribute to each voxel in the reconstructed image. In recent years, postprocessing of fast multislice magnetic resonance imaging (MRI) to correct fet al motion has provided the first true 3-D MR images of the developing human brain in utero. Early approaches have used reconstruction based algorithms, employing a two-step iterative process, where slices from the acquired data are realigned to an approximate 3-D reconstruction of the fet al brain, which is then refined further using the improved slice alignment. This two step slice-to-volume process, although powerful, is computationally expensive in needing a 3-D reconstruction, and is limited in its ability to recover subvoxel alignment. Here, we describe an alternative approach which we term slice intersection motion correction (SIMC), that seeks to directly co-align multiple slice stacks by considering the matching structure along all intersecting slice pairs in all orthogonally planned slices that are acquired in clinical imaging studies. A collective update scheme for all slices is then derived, to simultaneously drive slices into a consistent match along their lines of intersection. We then describe a 3-D reconstruction algorithm that, using the final motion corrected slice locations, suppresses through-plane partial volume effects to provide a single high isotropic resolution 3-D image. The method is tested on simulated data with known motions and is applied to retrospectively reconstruct 3-D images from a range of clinically acquired imaging studies. The quantitative evaluation of the registration accuracy for the simulated data sets demonstrated a significant improvement over previous approaches. An initial application of the technique to studying clinical pathology is included, where the proposed method recovered up to 15 mm of translation and 30? of rotation for individual slices, and produced full 3-D reconstructions containing clinically useful additional information not visible in the original 2-D slices.