Towards Sample-Optimal Compressive Phase Retrieval with Sparse and Generative Priors

1 Introduction . In this paper , we consider the ( real-valued ) phase retrieval problem , which aims to recover a signal x ∈ Rn from noisy magnitude-only measurements : yi = |〈ai , x〉|+ ηi , i = 1 , 2 , . . . , m , ( 1 ) where ai ∈ Rn is the i-th sensing vector , and ηi represents additive noise . This problem arises naturally in areas such as diffraction imaging , X-ray crystallography , microscopy , optics and astronomy [ 8 ] , where it is often difficult or even impossible to observe the linear measurements directly , and one can only record the magnitudes or intensities ( squared magnitudes ) . In many real applications , to reduce the required number of measurements , it is of interest to exploit structure in the signal being estimated . In particular , for applications related to signal processing and imaging , it is well-known that the underlying signal typically admits a sparse or approximately sparse 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . representation in some known basis [ 40 ] . Motivated by this , and considering the popularity of the standard compressive sensing ( CS ) problem [ 16 ] , the sparse phase retrieval problem has attracted significant research interest . Moreover , inspired by the successful applications of deep generative models in many fields [ 15 ] , recently , a new perspective of CS has emerged , for which the sparsity assumption is replaced by a generative model assumption . That is , instead of assuming sparsity , the signal is assumed to be close to the range of a generative model [ 5 ] . In addition to the theoretical developments in [ 5 ] , the authors also provide impressive numerical results showing that for some imaging applications , using generative priors can significantly reduce the required number of measurements ( e.g. , by a factor of 5 to 10 ) for recovering the signal up to a given accuracy . Follow-up works of [ 5 ] include [ 58 , 22 , 45 , 67 , 12 , 29 ] , just to name a few . In this paper , we focus on providing recovery guarantees with near-optimal sample complexity bounds ( e.g. , optimal up to constant factors ) for the compressive phase retrieval problem , considering both sparse and generative priors . 1.1 Related Work . In this subsection , we provide a summary of some relevant works , which can roughly be divided into ( i ) the general phase retrieval problem with no structural assumptions on the signal , ( ii ) sparse phase retrieval , and ( iii ) phase retrieval with generative models . General phase retrieval : A wide range of approaches have been designed to solve the phase retrieval problem . Error-reduction approaches [ 18 , 14 ] work well in practice , but they lack provable guarantees . Convex methods such as PhaseLift [ 9 ] and PhaseCut [ 62 ] lift the phase retrieval problem to a higher dimension , and typically suffer from high computation cost . Convex relaxations that operate in the natural domain of the signal are proposed in [ 19 , 2 ] . However , these convex relaxation based methods are not empirically competitive against widely-used non-convex methods . Following the seminal work of Netrapalli et al . [ 42 ] , several works have studied theoreticallyguaranteed non-convex optimization algorithms for phase retrieval [ 8 , 10 , 63 , 72 ] . These algorithms start from a spectral initialization method , and then use an iterative algorithm ( e.g. , alternating minimization or gradient descent ) to further decrease the approximation error . For the general phase retrieval problem , the optimal sample complexity of O ( n ) can be achieved by certain non-convex methods [ 10 , 63 , 72 ] . Interestingly , optimal ( up to a logarithmic factor ) sample complexity guarantees have also been provided in the case of a random initialization [ 55 , 11 ] . It is worth mentioning that some variants of the classic Wirtinger Flow ( WF ) algorithm [ 8 , 10 ] and trust-region methods [ 55 ] consider squared measurements : yi = |〈ai , x〉|2 + i , i = 1 , 2 , . . . , m , ( 2 ) and minimize the intensity based empirical loss function : fI ( w ) : = 1 2 ∥∥y − |Aw|2∥∥2 2 , ( 3 ) where A ∈ Rm×n is the sensing matrix with its i-th row being aTi , and | · | is applied element-wise . Nonetheless , numerical results suggest that algorithms minimizing the amplitude based empirical risk ( with the measurement model ( 1 ) ) f ( w ) : = ‖y − |Aw|‖22 ( 4 ) are usually more efficient in computation [ 63 , 72 , 54 ] . Based on this , we focus on the measurement model ( 1 ) and the associated loss function ( 4 ) . Sparse phase retrieval : A variety of algorithms have been devised for sparse phase retrieval [ 44 , 53 , 50 ] . In particular , there exist various non-convex optimization based algorithms , including thresholded/projected WF [ 7 , 54 ] , sparse truncated amplitude flow [ 64 ] , compressive phase retrieval with alternating minimization [ 28 ] , and sparse phase retrieval by hard iterative pursuit [ 6 ] . All of these approaches are analyzed under the assumption of using a sensing matrix with i.i.d . Gaussian entries . In addition , similar to non-convex methods for the general phase retrieval problem , all of these approaches start with a spectral initialization step , and then use an iterative algorithm to refine the initial vector . More specifically , in the initialization step , these approaches try to accurately estimate the support of the underlying signal . Unlike the general phase retrieval setting in which optimal sample complexity guarantees are provided for some practical algorithms , for non-convex methods of sparse phase retrieval , the typical sample complexity requirement is O ( s2 log n ) , where s is the sparsity level . This does not match the information theoretic lower bound [ 49 , 57 ] ; the bottleneck comes from the accurate estimation of the support set [ 64 , 54 , 28 ] . Improved guarantees are also available in the noiseless setting under more restrictive assumptions on the magnitudes of the non-zero entries [ 68 ] . In addition , for sparse phase retrieval , recovery guarantees with respect to globally optimal solutions of both the intensity and amplitude based loss functions have been provided in [ 13 , 26 , 24 , 23 , 69 ] , achieving the optimal sample complexity O ( s log n ) . However , designing a computationally efficient algorithm achieving the optimal sample complexity for s-sparse vectors remains an important open problem . Phase retrieval with generative models : Phase retrieval using generative models has been studied in [ 20 , 25 , 27 , 52 , 66 , 1 ] . Extensive numerical experiments for phase retrieval with generative models , as well as both Gaussian and coded diffraction pattern measurements , have been presented in [ 52 ] . Algorithms proposed in both works [ 20 , 52 ] minimize the objective function directly over the latent variable z ∈ Rk using gradient descent , where k is the latent dimension . The corresponding objective function is highly non-convex , and performing gradient descent directly over z limits the explorable solution space , which may lead to getting stuck in local minima . We highlight two particularly relevant works in more detail . To guarantee favorable global geometry for gradient methods , the authors of [ 20 ] assume a ReLU neural network generative model with i.i.d . zero-mean Gaussian weights and no offsets . In addition , the neural network needs to be sufficiently expansive such that ni ≥ Ω ( ni−1 log ni−1 ) , where ni represents the number of neurons in the i-th layer . Under such conditions , a sample complexity O ( kd2 log n ) is obtained , where d is the number of layers . In another related work studying an approximate message passing algorithm , the authors of [ 1 ] maintain i.i.d . Gaussian weights but slightly relax to general activation functions ( not only ReLU ) and ni ≥ Ω ( ni−1 ) . Under these conditions , the high-dimensional regime is studied ( n , m , k →∞ with m/n kept fixed ) , and an asymptotic analysis is given ( not yet established as fully rigorous ) . We also note that [ 1 ] focuses on the case that z ∼ Pz for some separable Pz . Both [ 1 ] and [ 20 ] focus on the noiseless case , though [ 1 ] states that the extension to noisy phase retrieval is possible . Recovery guarantees for non-convex optimization algorithms for phase retrieval with pre-trained ReLU neural network priors and untrained neural network priors have been provided in [ 25 , 27 ] , with assumptions of noiseless measurements and the existence of a good initial vector . Recovery guarantees for phase retrieval with generative models can also be found in [ 66 ] , within a more general setting of using differentiable but unknown link functions . The weaker assumption of an unknown link function comes at a price ; namely , it is an open problem to handle signals with representation error [ 47 ] . Moreover , the assumption of differentiability makes it is not directly applicable to the measurement model ( 1 ) , and the loss function considered in [ 66 ] is different from both ( 3 ) and ( 4 ) , which are widely adopted for phase retrieval . 1.2 Contributions . • We show that for compressive phase retrieval with i.i.d . Gaussian measurements , when the signal is close to the range of an L-Lipschitz continuous generative model with bounded kdimensional inputs , roughly O ( k logL ) samples suffice for ensuring the closeness between the signal and any vector minimizing the amplitude-based empirical loss function ( 4 ) . • To address the sample complexity barrier posed by spectral initialization , we propose a relevant optimization problem , and show that roughly O ( k logL ) samples are sufficient to guarantee that any globally optimal solution of the optimization problem is close to the true signal . This suggests the plausibility of practical spectral initialization algorithms that are able to find such a global optimum , though we do not attempt to provide one that can provably do so . • We adapt our analysis to the case of sparse phase retrieval , and show that roughly O ( s log n ) samples suffice for a similar recovery guarantee , where s is the sparsity level and n is the ambient dimension . This matches the information-theoretic lower bound up to constant factors for broad signal-to-noise ratios , and up to a logarithmic factor in general ; see for example [ 7 , Theorem 3.2 ] and [ 34 , Section 6.1 ] ) . • Motivated by our theoretical findings for sparse phase retrieval , we propose a spectral initialization method based on sparse principal component analysis . We verify on synthetic experiments that our initialization method significantly outperforms several popular spectral initialization methods of sparse phase retrieval ( in the sense of relative error ) . This further corroborates the optimal sample complexity guarantee in our theory .

Summary: This paper considers the problem of noisy compressive phase retrieval (CPR) of $n$ dimensional signal under two different priors: (i) $L$-Lipschitz continuous generative model with latent dimension $k$ and (ii) underlying sparsity $s$. The main claims of this paper are theoretical in nature, where the authors derive the sample requirements for existence of a solution for CPR under both types of priors (assuming there exists an algorithm that can  solve the CPR minimization). The sample requirements obtained are of the order required for a linear compressive sensing problem, and therefore roughly sample optimal.  Additionally the authors use a _truncated_ spectral initialization algorithm for CPR, which they show has solution that lies close enough to the ground truth signal using only O($k\log L$) (and O($s\log n$)) samples.  The authors design a related spectral initialization technique which can be combined with any other off-the-shelf CPR algorithms, while requiring fewer sample requirements then prior state of art.   Spectral initialization has been previously used for the general phase retrieval problem as well as under sparsity and structured sparsity priors, however incurring increased sample requirements. In this paper the authors prove closeness of a truncated spectral initialization to ground truth in sparse phase retrieval which brings down the sample requirement from O($s^2 \log n$) to O($s\log n$).  They also show similar guarantees for the case of deep generative priors.