Figure 1: X-ray equipment (a) has widespread availability enabling the creation of 2D x-ray imagery (here of a Cheetah) at low cost and with high speed. Regrettably, 2D x-rays do not reveal the full 3D structure (b), here visualized as an iso-surface, available to equipment such as a CT scanner. Our deep learning-based result recovers the full 3D density volume (c) from a single x-ray, allowing for prototypical applications (d) such as cut-away to reveal the cranial interior, re-rendering under different viewpoints, or stereoscopic rendering of legacy x-ray material (here red-cyan anaglyph) or stylized-re rendering in a new modality, here a colorful transfer function. As many different 3D volumes could produce the same 2D x-ray image, inverting this process is challenging. We show that recent deep learning-based convolutional neural networks can solve this task. As the main challenge in learning is the sheer amount of data created when extending the 2D image into a 3D volume, we suggest firstly to learn a coarse, fixed-resolution volume which is then fused in a second step with the input x-ray into a high-resolution volume. To train and validate our approach we introduce a new dataset that comprises of close to half a million computer-simulated 2D x-ray images of 3D volumes scanned from 175 mammalian species. Future applications of our approach include stereoscopic rendering of legacy x-ray images, re-rendering of x-rays including changes of illumination, view pose or geometry. Our evaluation includes comparison to previous tomography work, previous learning methods using our data, a user study and application to a set of real x-rays.
REF uses a deep network to reconstruct computed tomography of different mammalian species from single X-ray images.
Single-image Tomography: 3D Volumes from 2D Cranial X-Rays
Comput. Graph. Forum
Comput. Graph. Forum
Computer Science