Random mesh projectors for inverse problems

We propose a new learning-based approach to solve ill-posed inverse problems in imaging . We address the case where ground truth training samples are rare and the problem is severely ill-posed—both because of the underlying physics and because we can only get few measurements . This setting is common in geophysical imaging and remote sensing . We show that in this case the common approach to directly learn the mapping from the measured data to the reconstruction becomes unstable . Instead , we propose to first learn an ensemble of simpler mappings from the data to projections of the unknown image into random piecewise-constant subspaces . We then combine the projections to form a final reconstruction by solving a deconvolution-like problem . We show experimentally that the proposed method is more robust to measurement noise and corruptions not seen during training than a directly learned inverse . 1 INTRODUCTION . A variety of imaging inverse problems can be discretized to a linear system y = Ax + η where y ∈ RM is the measured data , A ∈ RM×N is the imaging or forward operator , x ∈ X ⊂ RN is the object being probed by applyingA ( often called the model ) , and η is the noise . Depending on the application , the set of plausible reconstructions X could model natural , seismic , or biomedical images . In many cases the resulting inverse problem is ill-posed , either because of the poor conditioning ofA ( a consequence of the underlying physics ) or because M N . A classical approach to solve ill-posed inverse problems is to minimize an objective functional regularized via a certain norm ( e.g . ` 1 , ` 2 , total variation ( TV ) seminorm ) of the model . These methods promote general properties such as sparsity or smoothness of reconstructions , sometimes in combination with learned synthesis or analysis operators , or dictionaries ( Sprechmann et al . ( 2013 ) ) . In this paper , we address situations with very sparse measurement data ( M N ) so that even a coarse reconstruction of the unknown model is hard to get with traditional regularization schemes . Unlike artifact-removal scenarios where applying a regularized pseudoinverse of the imaging operator already brings out considerable structure , we look at applications where standard techniques can not produce a reasonable image ( Figure 1 ) . This highly unresolved regime is common in geophysics and requires alternative , more involved strategies ( Galetti et al . ( 2017 ) ) . An appealing alternative to classical regularizers is to use deep neural networks . For example , generative models ( GANs ) based on neural networks have recently achieved impressive results in regularization of inverse problems ( Bora et al . ( 2018 ) , Lunz et al . ( 2018 ) ) . However , a difficulty in geophysical applications is that there are very few examples of ground truth models available for training ( sometimes none at all ) . Since GANs require many , they can not be applied to such problems . This suggests to look for methods that are not very sensitive to the training dataset . Conversely , it means that the sought reconstructions are less detailed than what is expected in data-rich settings ; for ∗S . Gupta and K. Kothari contributed equally . an example , see the reconstructions of the Tibetan plateau ( Yao et al . ( 2006 ) ) . In this paper , we propose a two-stage method to solve ill-posed inverse problems using random low-dimensional projections and convolutional neural networks . We first decompose the inverse problem into a collection of simpler learning problems of estimating projections into random ( but structured ) low-dimensional subspaces of piecewise-constant images . Each projection is easier to learn in terms of generalization error ( Cooper ( 1995 ) ) thanks to its lower Lipschitz constant . In the second stage , we solve a new linear inverse problem that combines the estimates from the different subspaces . We show that this converts the original problem with possibly non-local ( often tomographic ) measurements into an inverse problem with localized measurements , and that in fact , in expectation over random subspaces the problem becomes a deconvolution . Intuitively , projecting into piecewise-constant subspaces is equivalent to estimating local averages—a simpler problem than estimating individual pixel values . Combining the local estimates lets us recover the underlying structure . We believe that this technique is of independent interest in addressing inverse problems . We test our method on linearized seismic traveltime tomography ( Bording et al . ( 1987 ) ; Hole ( 1992 ) ) with sparse measurements and show that it outperforms learned direct inversion in quality of achieved reconstructions , robustness to measurement errors , and ( in ) sensitivity to the training data . The latter is essential in domains with insufficient ground truth images . 2 RELATED WORK . Although neural networks have long been used to address inverse problems ( Ogawa et al . ( 1998 ) ; Hoole ( 1993 ) ; Schiller and Doerffer ( 2010 ) ) , the past few years have seen the number of related deep learning papers grow exponentially . The majority address biomedical imaging ( Güler and Übeylı ( 2005 ) ; Hudson and Cohen ( 2000 ) ) with several special issues1 and review papers ( Lucas et al . ( 2018 ) ; McCann et al . ( 2017 ) ) dedicated to the topic . All these papers address reconstruction from subsampled or low-quality data , often motivated by reduced scanning time or lower radiation doses . Beyond biomedical imaging , machine learning techniques are emerging in geophysical imaging ( Araya-Polo et al . ( 2017 ) ; Lewis and Vigh ( 2017 ) ; Bianco and Gertoft ( 2017 ) ) , though at a slower pace , perhaps partly due to the lack of standard open datasets . Existing methods can be grouped into non-iterative methods that learn a feed-forward mapping from the measured data y ( or some standard manipulation such as adjoint or a pseudoinverse ) to the model x ( Jin et al . ( 2016 ) ; Pelt and Batenburg ( 2013 ) ; Zhu et al . ( 2018 ) ; Wang ( 2016 ) ; Antholzer et al . ( 2017 ) ; Han et al . ( 2016 ) ; Zhang et al . ( 2016 ) ) ; and iterative energy minimization methods , with 1IEEE Transactions on Medical Imaging , May 2016 ( Greenspan et al . ( 2016 ) ) ; IEEE Signal Processing Magazine , November 2017 , January 2018 ( Porikli et al . ( 2017 ; 2018 ) ) . either the regularizer being a neural network ( Li et al . ( 2018 ) ) , or neural networks replacing various iteration components such as gradients , projectors , or proximal mappings ( Kelly et al . ( 2017 ) ; Adler and Öktem ( 2017b ; a ) ; Chang et al . ( 2017 ) ) . These are further related to the notion of plug-and-play regularization ( Venkatakrishnan et al . ( 2013 ) ) , as well as early uses of neural nets to unroll and adapt standard sparse reconstruction algorithms ( Gregor and LeCun ( 2010 ) ; Xin et al . ( 2016 ) ) . An advantage of the first group of methods is that they are fast ; an advantage of the second group is that they are better at enforcing data consistency . Generative models A rather different take was proposed in the context of compressed sensing where the reconstruction is constrained to lie in the range of a pretrained generative network ( Bora et al . ( 2017 ; 2018 ) ) . Their scheme achieves impressive results on random sensing operators and comes with theoretical guarantees . However , training generative networks requires many examples of ground truth and the method is inherently subject to dataset bias . Here , we focus on a setting where ground-truth samples are very few or impossible to obtain . There are connections between our work and sketching ( Gribonval et al . ( 2017 ) ; Pilanci and Wainwright ( 2016 ) ) where the learning problem is also simplified by random low-dimensional projections of some object—either the data or the unknown reconstruction itself ( Yurtsever et al . ( 2017 ) ) . This also exposes natural connections with learning via random features ( Rahimi and Recht ( 2008 ; 2009 ) ) . 3 REGULARIZATION BY RANDOM MESH PROJECTIONS . The two stages of our method are ( i ) decomposing a “ hard ” learning task of directly learning an unstable operator into an ensemble of “ easy ” tasks of estimating projections of the unknown model into low-dimensional subspaces ; and ( ii ) combining these projection estimates to solve a reformulated inverse problem for x . The two stages are summarized in Figure 2 . While our method is applicable to continuous and non-linear settings , we focus on linear finite-dimensional inverse problems . 3.1 DECOMPOSING THE LEARNING PROBLEM . Statistical learning theory tells us that the number of samples required to learn an M -variate LLipschitz function to a given sup-norm accuracy is O ( LM ) ( Cooper ( 1995 ) ) . While this result is proved for scalar-valued multivariate maps , it is reasonable to expect the same scaling in L to hold for vector-valued maps . This motivates us to study Lipschitz properties of the projected inverse maps . We wish to reconstruct x , an N -pixel image from X ⊂ RN where N is large ( we think of x as an√ N × √ N discrete image ) . We assume that the map from x ∈ X to y ∈ RM is injective so that it is invertible on its range , and that there exists an L-Lipschitz ( generally non-linear ) inverse G , ‖G ( y1 ) −G ( y2 ) ‖ ≤ L ‖y1 − y2‖ . In order for the injectivity assumption to be reasonable , we assume that X is a low-dimensional manifold embedded in RN of dimension at most M , where M is the number of measurements . Since we are in finite dimension , injectivity implies the existence of L ( Stefanov and Uhlmann ( 2009 ) ) . Due to ill-posedness , L is typically large . Consider now the map from the data y to a projection of the model x into some K-dimensional subspace S , where K N . Note that this map exists by construction ( since A is injective on X ) , and that it must be non-linear . To see this , note that the only consistent2 linear map acting on y is an oblique , rather than an orthogonal projection on S ( cf . Section 2.4 in Vetterli et al . ( 2014 ) ) . We explain this in more detail in Appendix A. Denote the projection by P Sx and assume S ⊂ RN is chosen uniformly at random.3 We want to evaluate the expected Lipschitz constant of the map from y to P Sx , noting that it can be written as P S •G : E ‖P S •G ( y1 ) − P S •G ( y2 ) ‖ ≤ √ E ‖P S •G ( y1 ) − P S •G ( y2 ) ‖ 2 ≤ √ K NL ‖y1 − y2‖ where the first inequality is Jensen ’ s inequality , and the second one follows from E ‖P Sx‖2 = Ex > P > SP Sx = x > E ( P > SP S ) x and the observation that EP > SP S = KN IN . In other words , random projections reduce the Lipschitz constant by a factor of √ K/N on average . Since learning requires O ( LK ) samples , this allows us to work with exponentially fewer samples and makes the learning task easier . Conversely , given a fixed training dataset , it gives more accurate estimates .

Summary: Given an inverse problem, we want to infer (x) s.t. Ax = y, but in situations where the number of observations are very sparse, and do not enable direct inversion. The paper tackles scenarios where 'x' is of the form of an image. The proposed approach is a learning based one which trains CNNs to infer x given y (actually an initial least square solution x_init is used instead of y).