Title: A Fast and Provable Algorithm for Sparse Phase Retrieval
Venue: NeurIPS 2023
Accepted: no
Keywords: Sparse Phase Retrieval

Nonconvex Optimization

Quadratic Convergence
Abstract: We study the sparse phase retrieval problem, which aims to recover a sparse signal from a limited number of phaseless measurements. Existing algorithms for sparse phase retrieval primarily rely on first-order methods with linear convergence rate. In this paper, we propose an efficient second-order algorithm based on Newton projection, which maintains the same per-iteration computational complexity as popular first-order methods. The proposed algorithm is theoretically guaranteed to converge to the ground truth (up to a global sign) at a quadratic convergence rate after at most $\mathcal{O}\big(\log (\Vert\boldsymbol{x}^{\natural} \Vert /x_{\min}^{\natural})\big)$ iterations, provided a sample complexity of $\mathcal{O}(s^2\log n)$, where $\boldsymbol{x}^{\natural} \in \mathbb{R}^n$ represents an $s$-sparse ground truth signal. Numerical experiments demonstrate that our algorithm not only outperforms state-of-the-art methods in terms of achieving a significantly faster convergence rate, but also excels in attaining a higher success rate for exact signal recovery from noise-free measurements and providing enhanced signal reconstruction in noisy scenarios.

Abstract

We study the sparse phase retrieval problem, which aims to recover a sparse signal from a limited number of phaseless measurements. Existing algorithms for sparse phase retrieval primarily rely on first-order methods with linear convergence rate. In this paper, we propose an efficient second-order algorithm based on Newton projection, which maintains the same per-iteration computational complexity as popular first-order methods. The proposed algorithm is theoretically guaranteed to converge to the ground truth (up to a global sign) at a quadratic convergence rate after at most O ( log( ∥ x ♮ ∥ /x ♮ min ) ) iterations, provided a sample complexity of O ( s 2 log n ) , where x ♮ ∈ R n represents an s -sparse ground truth signal. Numerical experiments demonstrate that our algorithm not only outperforms state-of-the-art methods in terms of achieving a significantly faster convergence rate, but also excels in attaining a higher success rate for exact signal recovery from noise-free measurements and providing enhanced signal reconstruction in noisy scenarios.

1 Introduction

We study the phase retrieval problem, which involves reconstructing an n -dimensional signal x ♮ using its intensity-only measurements:

where each y i represents a measurement, a i denotes a sensing vector, x ♮ is the unknown signal to be recovered, and m is the total number of measurements. The phase retrieval problem arises in various applications, including diffraction imaging [1], X-ray crystallography [2, 3], and optics [4], where detectors can only record the squared modulus of Fresnel or Fraunhofer diffraction patterns of radiation scattered from an object. The loss of phase information complicates the understanding of the scattered object, as much of the image's structural content may be encoded in the phase.

Although the phase retrieval problem is ill-posed and even NP-hard [5], several algorithms have been proven to succeed in recovering target signals under certain assumptions. Algorithms can be broadly categorized into convex and nonconvex approaches. Convex methods, such as PhaseLift [6, 7], PhaseCut [8], and PhaseMax [9, 10], offer optimal sample complexity but are computationally challenging in high-dimensional cases. To improve computational efficiency, nonconvex approaches are explored, including alternating minimization [11], Wirtinger flow [6], truncated amplitude flow [12], Riemannian optimization [13], Gauss-Newton [14, 15], and Kaczmarz [16, 17]. Despite the nonconvex nature of its objective function, the global geometric landscape lacks spurious local minima [18, 19], allowing algorithms with random initialization to work effectively [20, 21].

The nonconvex approaches previously mentioned can guarantee successful recovery of the ground 32 truth (up to a global phase) with a sample complexity m ∼ O ( n log a n ) , where a ≥ 0 . This 33 complexity is nearly optimal, as the phase retrieval problem requires m ≥ 2 n -1 for real signals and 34 m ≥ 4 n -4 for complex signals [22]. However, in practical situations, especially in high-dimensional 35 cases, the number of available measurements is often less than the signal dimension ( i.e. , m<n ), 36 leading to a need for further reduction in sample complexity. 37

38

39

40

41

42

43

44

In this paper, we focus on the sparse phase retrieval problem, which aims to recover a sparse signal from a limited number of phaseless measurements. It has been established that the minimal sample complexity required to ensure s -sparse phase retrievability in the real case is only 2 s for generic sensing vectors [23]. Several algorithms have been proposed to address the sparse phase retrieval problem [24, 25, 26, 27, 28]. These approaches have been demonstrated to effectively reconstruct the ground truth using O ( s 2 log n ) Gaussian measurements. While this complexity is not optimal, it is significantly smaller than that in general phase retrieval.

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

1.1 Contributions

Existing algorithms for sparse phase retrieval primarily employ first-order methods with linear convergence. Recent work [28] introduced a second-order method, while it fails to obtain a quadratic convergence rate. The main contributions of this paper can be summarized in three key points:

We propose a second-order algorithm based on Newton projection for sparse phase retrieval that maintains the same per-iteration computational complexity as popular first-order methods. To ensure fast convergence, we integrate second-order derivative information from intensitybased empirical loss into the search direction; to ensure computational efficiency, we restrict the Newton update to a subset of variables, setting others to zero in each iteration.

We establish a non-asymptotic quadratic convergence rate for our proposed algorithm and provide the iteration complexity. Specifically, we prove that the algorithm converges to the ground truth (up to a global sign) at a quadratic rate after at most O ( log( | x ♮ | /x ♮ min ) ) iterations, provided a sample complexity of O ( s 2 log n ) . To the best of our knowledge, this is the first algorithm to establish a quadratic convergence rate for sparse phase retrieval.

Numerical experiments demonstrate that the proposed algorithm achieves a significantly faster convergence rate in comparison to state-of-the-art methods. Furthermore, the experiments reveal that our algorithm attains a higher success rate in exact signal recovery from noise-free measurements and provides enhanced signal reconstruction performance in noisy scenarios, as evidenced by the improved Peak Signal-to-Noise Ratio (PSNR).

Notation: The p -norm ∥ x ∥ p := ( ∑ n i =1 | x i | p ) 1 /p for p ≥ 1 . ∥ x ∥ 0 denotes the number of nonzero entries of x , and ∥ x ∥ denotes the 2-norm. For a matrix A ∈ R m × n , ∥ A ∥ is the spectral norm of A . For any q 1 ≥ 1 and q 2 ≥ 1 , ∥ A ∥ q 2 → q 1 denotes the induced operator norm from the Banach space ( R n , ∥ · ∥ q 2 ) to ( R m , ∥ · ∥ q 1 ) . λ min ( A ) and λ max ( A ) denote the smallest and largest eigenvalues of the matrix A . |S| denotes the number of elements in S . a ⊙ b denotes the entrywise product of a and b . For functions f ( n ) and g ( n ) , we write f ( n ) ≲ g ( n ) if f ( n ) ≤ cg ( n ) for some constant c ∈ (0 , + ∞ ) . For x , x ♮ ∈ R n , the distance between x and x ♮ is defined as dist( x , x ♮ ) := min { ∥ x -x ♮ ∥ , ∥ x + x ♮ ∥ } . x ♮ min denotes the smallest nonzero entry in magnitude of x ♮ .

2 Problem Formulation and Related Works

We first present the problem formulation for sparse phase retrieval, and then review related works.

2.1 Problem formulation

The standard sparse phase retrieval problem can be concisely expressed as finding x that satisfies

where { a i } i m =1 are known sensing vectors and { y i } i m =1 represent phaseless measurements with 76 y i = |⟨ a i , x ♮ ⟩| 2 , where x ♮ is the ground truth signal ( ∥ x ♮ ∥ 0 ≤ s ). While sparsity level s is assumed 77 known a priori for theoretical analysis, our experiments will also explore cases with unknown s . 78

86

87

To address Problem (2), various problem reformulations have been explored. Convex formulations, 79 such as the ℓ 1 -regularized PhaseLift method [24], often use the lifting technique and solve the problem 80 in the n × n matrix space, resulting in high computational costs. To enhance computational efficiency, 81 nonconvex approaches [25, 26, 28, 29] are explored, which can be formulated as: 82

Both the loss function f ( x ) and the ℓ 0 -norm constraint in Problem (3) are nonconvex, making it 83 challenging to solve. Two prevalent loss functions are investigated: intensity-based empirical loss 84

and amplitude-based empirical loss 85

where z i = √ y i , i = 1 , . . . , m . The intensity-based loss f I ( x ) is smooth, while the amplitude-based loss f A ( x ) is non-smooth because of the modulus.

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

2.2 Related works

Existing nonconvex sparse phase retrieval algorithms can be broadly classified into two categories: gradient projection methods and alternating minimization methods. Gradient projection methods, such as ThWF [25] and SPARTA [26], employ thresholded gradient descent and iterative hard thresholding, respectively. On the other hand, alternating minimization methods, including CoPRAM [27] and HTP [28], alternate between updating the signal and phase. When updating the signal, formulated as a sparsity-constrained least squares problem, CoPRAM leverages the cosamp method [30], while HTP applies the hard thresholding pursuit algorithm [31]. In this paper, we introduce a Newton projection-based algorithm that incorporates second-order derivative information, resulting in a faster convergence rate compared to gradient projection methods, and, unlike alternating minimization methods, it eliminates the need for separate signal and phase updates. We note that ThWF and our algorithm utilize intensity-based loss as the objective function, while SPARTA, CoPRAM, and HTP employ amplitude-based loss. All these algorithms require a sample complexity of O ( s 2 log n ) under Gaussian measurements for successful recovery.

The majority of sparse phase retrieval algorithms, such as ThWF, SPARTA, and CoPRAM, are firstorder methods with linear convergence rates. While HTP is a second-order method that converges in a finite number of iterations, it fails to establish a quadratic convergence rate. We propose a second-order algorithm that attains a non-asymptotic quadratic convergence rate and exhibits lower iteration complexity compared to HTP. Our algorithm maintains the same computational complexity per iteration as popular first-order methods when s ≲ √ n . This condition is always assumed to hold true; otherwise, the established sample complexity for sparse phase retrieval algorithms, O ( s 2 log n ) , would be reduced to that of general phase retrieval methods. Table 1 presents a comparative overview of the previously discussed methods and our proposed method.

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

3 Main Results

In this section, we present our proposed algorithm for sparse phase retrieval. Generally, nonconvex methods comprise two stages: initialization and refinement. The first stage generates an initial guess close to the target signal, while the second stage refines the initial guess using various methods, such as ThWF, SPARTA, CoPRAM, and HTP. Our proposed algorithm adheres to this two-stage strategy. In the first stage, we employ an existing effective method to generate an initial point. Our primary focus is on the second stage, wherein we propose an efficient second-order algorithm based on Newton projection to refine the initial guess.

Before delving into the details of our proposed algorithm, we present a unified algorithmic framework for addressing the sparsity-constrained optimization problem in Eq. (3), as summarized in [32]. Given the k -th iterate x k , the next iterate x k +1 can be obtained through the following steps:

Step 1 (Hard thresholding):

Step 2 (Debiasing):

where ϕ ( x k ) is typically chosen as either ∇ f ( x k ) or x k -η ∇ f ( x k ) , ψ k ( x ) is designed based on the objective function f ( x ) and the iterate x k , and S k +1 is usually defined as the support of u k +1 . The hard-thresholding operator, denoted by H s , is defined with a sparsity level of s as follows:

A variety of well-known algorithms for solving sparsity-constrained optimization problems adhere to the three-step algorithmic framework mentioned earlier. For instance, the Iterative Hard Thresholding (IHT) algorithm solely performs Step 1 using H s ( x k -η ∇ f ( x k )) with η the stepsize; the Hard Thresholding Pursuit (HTP) implements the first two steps by computing u k +1 via one-step IHT in Step 1, and then solving the support-constrained problem in Step 2 with S k +1 = supp( u k +1 ) ; the Compressive Sampling Matching Pursuit (CoSaMP) executes all three steps, calculating u k +1 = H 2 s ( ∇ f ( x k )) in Step 1, performing Step 2 with S k +1 = supp( u k +1 ) ∪ supp( x k ) , and pruning the result in Step 3 to ensure an s -sparse level.

Several state-of-the-art methods for sparse phase retrieval share strong connections with the previously described popular algorithms for sparsity-constrained optimization, and thus relate closely to the algorithmic framework. SPARTA combines IHT with gradient truncation to eliminate erroneously estimated signs. HTP merges hard thresholding pursuit with alternating minimization, updating the signal and phase alternately. CoPRAM integrates CoSaMP with alternating minimization. Our proposed algorithm will also be presented using this algorithmic framework.

3.1 Proposed algorithm

In this subsection, we introduce our proposed algorithm for sparse phase retrieval, which utilizes the intensity-based loss f I defined in Eq. (4) as the objective function. The algorithm incorporates the first two steps of the previously discussed algorithmic framework.

Our algorithm is developed based on the Newton projection method. It is worth mentioning that Newton-type methods typically require solving a linear system at each iteration to determine the Newton direction. This generally results in a computational cost of O ( n 3 ) for our problem, rendering it impractical in high-dimensional situations. To address this challenge, we categorize variables into two groups at each iteration: free and fixed , updating them separately. The free variables, consisting of at most s variables, are updated according to the (approximate) Newton direction, while the fixed variables are set to zero. This strategy requires solving a linear system of size s × s , substantially decreasing the computational expense from O ( n 3 ) to O ( s 3 ) .

Step 3 (Pruning):

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

In the first step, we identify the set of free variables using one-step IHT of the loss f A ( x ) in (5):

where η is the stepsize. Since f A is non-smooth, we adopt the generalized gradient [33] as ∇ f A . The s -sparse hard thresholding limits |S k +1 | to s , implying that there are at most s free variables. We only update free variables along the approximate Newton direction and set others to zero.

In the second step, we update the free variables in S k +1 by solving a support-constrained problem in Eq. (7). Note that we adopt the intensity-based loss f I as the objective function. To accelerate convergence, we choose function ψ k ( x ) in (7) as the second-order Taylor expansion of f I at x k :

Let x ⋆ denote the minimizer of Problem (7). For notational simplicity, define g k S k +1 = [ ∇ f I ( x k ) ] S k +1 , which denotes the sub-vector of ∇ f I ( x k ) indexed by S k +1 , H k S k +1 = [ ∇ 2 f I ( x k ) ] S k +1 , which represents the principle sub-matrix of the Hessian indexed by S k +1 , and H k S k +1 , S c k +1 = [ ∇ 2 f I ( x k ) ] S k +1 , S c k +1 , denoting the sub-matrix of the Hessian whose rows and columns are indexed by S k +1 and S c k +1 , respectively. Following from the first-order optimality condition of Problem (7), we obtain that x ⋆ S c k +1 = 0 and x ⋆ S k +1 satisfies

As a result, we obtain the next iterate x k +1 by

where p k S k +1 represents the approximate Newton direction over S k +1 , which can be calculated by

where J k +1 := S k \ S k +1 with | J k +1 | ≤ s . In contrast to Eq. (10), we replace x k S c k +1 with x k J k +1 in (12), as J k +1 captures all nonzero elements in x k S c k +1 as follows:

where operator G arranges all nonzero elements of a vector to appear first, followed by zero elements. The first equality in (13) follows from the fact that supp( x k ) ⊆ S k . By calculating H k S k +1 ,J k +1 rather than H k S k +1 , S c k +1 as in (12), the computational cost is substantially reduced from O ( smn ) to O ( s 2 m ) . The costs for computing H k S k +1 and solving the linear system in (12) are O ( s 2 m ) and O ( s 3 ) , respectively. Therefore, the overall computational cost for Step 2 is O ( s 2 m ) , while the cost for Step 1 amounts to O ( mn ) , which involves calculating ∇ f A ( x k ) .

In summary, the computational costs for Steps 1 and 2 are O ( mn ) and O ( s 2 m ) , respectively, making the total cost per iteration O ( n + s 2 ) m , with m ∼ O ( s 2 log n ) that is required for successful recovery. Since s ≲ √ n is always assumed to hold true as discussed in Section 2.2, the per-iteration computational complexity of our algorithm is equivalent to that of popular first-order methods, which is O ( ns 2 log n ) . The pruning step is omitted as x k +1 in (11) is already s -sparse.

Algorithm 1 Proposed algorithm

Input: Data { a i , y i } i m =1 , sparsity s , initial estimate x 0 , and stepsize η .

1: for k = 0 , 1 , 2 , . . . do

2: Identify the set of free variables S k +1 = supp( H s ( x k -η ∇ f A ( x k ))) ;

3: Compute the approximate Newton direction p k S k +1 over S k +1 by solving (12).

4: Update x k +1 :

5:

end for

Output: x k +1 .

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

3.2 Initialization

The nonconvex nature of phase retrieval problems often requires a well-designed initial guess to find a global minimizer. Spectral initialization is a common approach [6]. In this paper, we adopt a sparse variant of the spectral initialization method to obtain a favorable initial guess for Algorithm 1.

Assuming { a i } i m =1 are independently drawn from a Gaussian distribution N ( 0 , I n ) , the expectation of the matrix 1 m ∑ m i =1 y i a i a T i is M := ∥ x ♮ ∥ 2 I n +2 x ♮ ( x ♮ ) T . The leading eigenvector of M is precisely ± x ♮ . Hence, the leading eigenvector of 1 m ∑ m i =1 y i a i a T i can be close to ± x ♮ [6]. However, this method requires the sample complexity of at least O ( n ) , which is excessively high for sparse phase retrieval. Leveraging the sparsity of x ♮ is crucial to lower this complexity.

We adopt the sparse spectral initialization method proposed in [27]. Specifically, we first collect the indices of the largest s values from { 1 m ∑ m i =1 y i [ a i ] 2 j } n j =1 and obtain the set ˆ S , which serves as an estimate of the support of the true signal x ♮ . Next, we construct the initial guess x 0 as follows: x 0 ˆ S is the leading eigenvector of 1 m ∑ m i =1 y i [ a i ] ˆ S [ a i ] T ˆ S , and x 0 ˆ S c = 0 . Finally, we scale x 0 such that ∥ x 0 ∥ 2 = 1 m ∑ m i =1 y i , ensuring the power of x 0 closely aligns with the power of x ♮ .

The study in [27] demonstrates that, given a sample complexity m ∼ O ( s 2 log n ) , the aforementioned sparse spectral initialization method can produce an initial estimate x 0 that is sufficiently close to the ground truth. Specifically, it holds dist( x 0 , x ♮ ) ≤ γ ∥ x ♮ ∥ for any γ ∈ (0 , 1) , with a probability of at least 1 -8 m -1 .

3.3 Theoretical results

Given the nonconvex nature of both the objective function and the constraint set in the sparse phase retrieval problem, a thorough theoretical analysis is essential for ensuring the convergence of our algorithm to the ground truth. In this subsection, we provide a comprehensive analysis of the convergence of our algorithm for both noise-free and noisy scenarios.

3.3.1 Noise-free case

We begin by the noise-free case, in which each measurement y i = |⟨ a i , x ♮ ⟩| 2 . Starting with an initial guess obtained via the sparse spectral initialization method, the following theorem shows that our algorithm exhibits a quadratic convergence rate after at most O ( log( ∥ x ♮ ∥ /x ♮ min ) ) iterations.

Theorem 3.1. Let { a i } i m =1 be i.i.d. random vectors distributed as N ( 0 , I n ) , and x ♮ ∈ R n be any signal with ∥ x ♮ ∥ 0 ≤ s . Let { x k } k ≥ 1 be the sequence generated by Algorithm 1 with the input measurements y i = |⟨ a i , x ♮ ⟩| 2 , i = 1 , . . . , m , and the initial guess x 0 generated by the sparse spectral initialization method mentioned earlier. There exists positive constants ρ, η 1 , η 2 , C 1 , C 2 , C 3 , C 4 , C 5 such that if the stepsize η ∈ [ η 1 , η 2 ] and m ≥ C 1 s 2 log n , then with probability at least 1 -( C 2 K + C 3 ) m -1 , the sequence { x k } k ≥ 1 converges to the ground truth x ♮ at a quadratic rate after at most O ( log( ∥ x ♮ ∥ /x ♮ min ) ) iterations, i.e.,

where K ≤ C 4 log ( ∥ x ♮ ∥ /x ♮ min ) + C 5 , and x ♮ min is the smallest nonzero entry in magnitude of x ♮ .

The proof of Theorem 3.1 is available in Appendix B.2.

Remark 3.2 . Theorem 3.1 establishes the non-asymptotic quadratic convergence rate of our algorithm as it converges to the ground truth, leading to an iteration complexity of O ( log(log(1 /ϵ )) + log( ∥ x ♮ ∥ /x ♮ min ) ) for achieving an ϵ -accurate solution. This convergence rate is significantly faster than those of state-of-the-art methods such as ThWF [25], SPARTA [26], and CoPRAM [27], which, as first-order methods, exhibit only linear convergence. Although HTP [28] is a secondorder approach, it fails to establish a quadratic convergence rate, and its iteration complexity, O ( log(log( n s 2 )) + log( ∥ x ♮ ∥ /x ♮ ) ) , is higher than that of our algorithm.

min

Remark 3.3 . It is worth emphasizing that while the superlinear convergence is extensively established for Newton-type methods in existing literature, it often holds only asymptotically: the ratio of the distance to the optimal solution at ( k +1) -th and k -th iterations tends to zero as k goes to infinity. Consequently, the overall iteration complexity cannot be explicitly characterized. This fact highlights the significance of establishing a non-asymptotic superlinear convergence rate.

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

3.3.2 Noisy case

In real-world scenarios, observations are frequently affected by noise. In what follows, we demonstrate the robustness of our proposed algorithm in the presence of noise within phaseless measurements. Building upon [25, 34], we assume that the noisy measurements are given by:

where ϵ represents a vector of stochastic noise that is independent of { a i } i m =1 . Throughout this paper, we assume, without loss of generality, that the expected value of ϵ is 0 .

Theorem 3.4. Let { a i } i m =1 be i.i.d. random vectors distributed as N ( 0 , I n ) , and x ♮ ∈ R n be any signal with ∥ x ♮ ∥ 0 ≤ s . Let { x k } k ≥ 1 be the sequence generated by Algorithm 1 with noisy input y i = |⟨ a i , x ♮ ⟩| 2 + ϵ i , i = 1 , . . . , m . There exists positive constants η 1 , η 2 , C 6 , C 7 , C 8 , and γ ∈ (0 , 1 / 8] , such that if the stepsize η ∈ [ η 1 , η 2 ] , m ≥ C 6 s 2 log n and the initial guess x 0 obeys dist( x 0 , x ♮ ) ≤ γ ∥ x ♮ ∥ with ∥ x 0 ∥ 0 ≤ s , then with probability at least 1 -( C 7 K ′ + C 8 ) m -1 ,

where ρ ′ ∈ (0 , 1) , υ ∈ (0 , 1) , and K ′ is a positive integer.

The proof of Theorem 3.4 is provided in Appendix B.3. Theorem 3.4 validates the robustness of our algorithm, demonstrating its ability to effectively recover the signal from noisy measurements.

4 Experimental Results

In this section, we present a series of numerical experiments designed to validate the efficiency and accuracy of our proposed algorithm. All experiments were conducted on a 2 GHz Intel Core i5 processor with 16 GB of RAM, and all compared methods were implemented using MATLAB.

Unless explicitly specified, the sensing vectors { a i } i m =1 were generated by the standard Gaussian distribution. The true signal x ♮ has s nonzero entries, where the support is selected uniformly from all subsets of [ n ] with cardinality s , and their values are independently generated from the standard Gaussian distribution N (0 , 1) . In the case of noisy measurements, we have:

where { ε i } i m =1 follow i.i.d standard Gaussian distribution, and σ > 0 determines the noise level.

0

20

40

60

80

100

Iterations

10 -4

10 -3

10 -2

10 -1

10 0

Relative error

CoPRAM

HTP

ThWF

SparTA

Proposed

(a) Noise free, sparsity

s

= 80

0

20

40

60

80

100

Iterations

10 -4

10 -3

10 -2

10 -1

10 0

Relative error

CoPRAM

HTP

ThWF

SparTA

Proposed

(b) Noise free, sparsity

s

= 100

0

20

40

60

80

100

Iterations

10 -3

10 -2

10 -1

10 0

Relative error

CoPRAM

HTP

ThWF

SparTA

Proposed

(c) Noise level

σ

= 0

.

03

, sparsity

s

= 80

0

20

40

60

80

100

Iterations

10 -3

10 -2

10 -1

10 0

Relative error

CoPRAM

HTP

ThWF

SparTA

Proposed

(d) Noise level

σ

= 0

.

03

, sparsity

s

= 100

We compare our proposed algorithm with state-of-the-art methods, including ThWF [25], SPARTA 253 [26], CoPRAM [27], and HTP [28]. For ThWF, we set parameters as recommended in [25]. For 254 SPARTA, we set parameters as follows: µ = 1 and |I| = ⌈ m/ 6 ⌉ . Both HTP and our algorithm use 255 a step size η of 0.95. The maximum number of iterations for each algorithm is 100. The Relative 256 Error (RE) between the estimated signal ˆ x and the ground truth x ♮ is defined as RE := dist(ˆ x , x ♮ ) ∥ x ♮ ∥ . 257 A recovery is deemed successful if RE < 10 -3 . We provide additional experimental results on 258 robustness to noise levels and phase transition with varying sparsity levels in Appendix A. 259

260

261

262

263

264

265

266

267

268

269

270

271

272

Figure 1 compares the number of iterations required for convergence across various algorithms. In these experiments, we set the number of measurements to m = 3000 , the dimension of the true signal to n = 5000 , and the sparsity levels to s = 80 and 100 . We consider both noise-free measurements and noisy measurements with a noise level of σ = 0 . 03 . As depicted in Figure 1, all five algorithms perform well under both noise-free and noisy conditions; however, our algorithm converges with significantly fewer iterations compared to state-of-the-art methods.

Table 2 presents a comparison of the convergence running times for various algorithms, corresponding to the experiments depicted in Figure 1. For noise-free measurements, all algorithms are set to terminate when the iterate satisfies the following condition: dist( x k , x ♮ ) ∥ x ♮ ∥ < 10 -3 , which indicates a successful recovery. In the case of noisy measurements, the termination criterion is set as dist( x k +1 , x k ) ∥ x k ∥ < 10 -3 . As evidenced by the results in Table 2, our algorithm consistently outperforms state-of-the-art methods in terms of running time, for both noise-free and noisy cases, highlighting its superior efficiency for sparse phase retrieval applications.

0

500

1000

1500

2000

2500

3000

Measurements (m)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of success

CoPRAM

HTP

ThWF

SPARTA

Proposed

(a)

s

= 25

0

500

1000

1500

2000

2500

3000

Measurements (m)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of success

CoPRAM

HTP

ThWF

SPARTA

Proposed

(b)

s

= 50

Figure 2 depicts the phase transitions of different algorithms, with the true signal dimension fixed 273 at n = 3000 and sparsity levels set to s = 25 and 50 . The phase transition graph is generated by 274 evaluating the successful recovery rate of each algorithm over 100 independent trial runs. Figure 2 275 shows that the probability of successful recovery for each algorithm transitions from zero to one as 276 the sample size m increases. Furthermore, our algorithm consistently outperforms state-of-the-art 277 methods, achieving a higher successful recovery rate across various measurement counts. 278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

In practical applications, natural signals may not be inherently sparse; however, their wavelet coefficients often exhibit sparsity. Figure 3 illustrates the reconstruction performance of a signal from noisy phaseless measurements, where the true signal, with a dimension of 30 , 000 , exhibits sparsity and contains 208 nonzero entries under the wavelet transform, using 20 , 000 samples. The sampling matrix A ∈ R 20 , 000 × 30 , 000 is constructed from a random Gaussian matrix and an inverse wavelet transform generated using four levels of Daubechies 1 wavelet. The noise level is set to σ = 0 . 03 . To evaluate recovery accuracy, we use the Peak Signal-to-Noise Ratio (PSNR), defined as:

where V represents the maximum fluctuation in the ground truth signal, and MSE denotes the mean squared error of the reconstruction. A higher PSNR value generally indicates better reconstruction quality. As depicted in Figure 3, our proposed algorithm outperforms state-of-the-art methods in terms of both reconstruction time and PSNR. It achieves a higher PSNR while requiring considerably less time for reconstruction. In the experiments, the sparsity level is assumed to be unknown, and the hard thresholding sparsity level is set to 300 for various algorithms.

(a) Original signal

(b)

ThWF:

PSNR = 71,

Time(s) = 11.586

(c)

SPARTA:

PSNR = 66,

Time(s) = 38.379

(d)

CoPRAM:

PSNR = 61,

Time(s) = 117.685

(e)

HTP:

PSNR = 62,

Time(s) = 6.689

(f)

Proposed:

PSNR = 78,

Time(s) = 2.629

5 Conclusions and Discussions

In this paper, we have introduced an efficient Newton projection-based algorithm for sparse phase retrieval. Our algorithm attains a non-asymptotic quadratic convergence rate while maintaining the same per-iteration computational complexity as popular first-order methods, which exhibit linear convergence limitations. Empirical results have demonstrated a significant improvement in the convergence rate of our algorithm. Furthermore, experiments have revealed that our algorithm excels in attaining a higher success rate for exact signal recovery with noise-free measurements and provides superior signal reconstruction performance when dealing with noisy data.

Finally, we discuss the limitations of our paper, which also serve as potential avenues for future research. Both our algorithm and state-of-the-art methods share the same sample complexity of O ( s 2 log n ) for successful recovery; however, our algorithm requires this complexity in both the initialization and refinement stages, while state-of-the-art methods require O ( s 2 log n ) for initialization and O ( s log n/s ) for refinement. Investigating ways to achieve tighter complexity in our algorithm's refinement stage is a worthwhile pursuit for future studies.

Currently, the initialization stage exhibits a sub-optimal sample complexity of O ( s 2 log n ) . A key challenge involves reducing its quadratic dependence on s . Recent work [27] attained a complexity of O ( s log n ) , closer to the information-theoretic limit, but relied on the strong assumption of power law decay for sparse signals. Developing an initialization method that offers optimal sample complexity for a broader range of sparse signals is an engaging direction for future research.

311

312

313

314

315

316

317

318

319

320

321

References

Andrew M Maiden and John M Rodenburg. An improved ptychographical phase retrieval algorithm for diffractive imaging. Ultramicroscopy , 109(10):1256-1262, 2009.

Rick P Millane. Phase retrieval in crystallography and optics. Journal of the Optical Society of America A , 7(3):394-411, 1990.

Jianwei Miao, Tetsuya Ishikawa, Qun Shen, and Thomas Earnest. Extending x-ray crystallography to allow the imaging of noncrystalline materials, cells, and single protein complexes. Annual Review of Physical Chemistry , 59(1):387-410, 2008.

Yoav Shechtman, Yonina C Eldar, Oren Cohen, Henry Nicholas Chapman, Jianwei Miao, and Mordechai Segev. Phase retrieval with application to optical imaging: a contemporary overview. IEEE Signal Processing Magazine , 32(3):87-109, 2015.

Matthew Fickus, Dustin G Mixon, Aaron A Nelson, and Yang Wang. Phase retrieval from very 322 few measurements. Linear Algebra and its Applications , 449:475-499, 2014. 323

Emmanuel J Candes, Xiaodong Li, and Mahdi Soltanolkotabi. Phase retrieval via wirtinger 324 flow: Theory and algorithms. IEEE Transactions on Information Theory , 61(4):1985-2007, 325 2015. 326

Emmanuel J Candes, Thomas Strohmer, and Vladislav Voroninski. Phaselift: Exact and stable 327 signal recovery from magnitude measurements via convex programming. Communications on 328 Pure and Applied Mathematics , 66(8):1241-1274, 2013. 329

330

331

Irene Waldspurger, Alexandre d'Aspremont, and Stéphane Mallat. Phase recovery, maxcut and complex semidefinite programming. Mathematical Programming , 149(1):47-81, 2015.

332

333

Tom Goldstein and Christoph Studer. Phasemax: Convex phase retrieval via basis pursuit. IEEE Transactions on Information Theory , 64(4):2675-2689, 2018.

Paul Hand and Vladislav Voroninski. An elementary proof of convex phase retrieval in the 334 natural parameter space via the linear program phasemax. arXiv preprint arXiv:1611.03935 , 335 2016. 336

Praneeth Netrapalli, Prateek Jain, and Sujay Sanghavi. Phase retrieval using alternating 337 minimization. Advances in Neural Information Processing Systems , 26, 2013. 338

Gang Wang, Georgios B Giannakis, and Yonina C Eldar. Solving systems of random quadratic 339 equations via truncated amplitude flow. IEEE Transactions on Information Theory , 64(2):773340 794, 2017. 341

Jian-Feng Cai and Ke Wei. Solving systems of phaseless equations via riemannian optimization 342 with optimal sampling complexity. Journal of Computational Mathematics , 2018. 343

Bing Gao and Zhiqiang Xu. Phaseless recovery using the gauss-newton method. IEEE 344 Transactions on Signal Processing , 65(22):5885-5896, 2017. 345

Chao Ma, Xin Liu, and Zaiwen Wen. Globally convergent levenberg-marquardt method for 346 phase retrieval. IEEE Transactions on Information Theory , 65(4):2343-2359, 2018. 347

Ke Wei. Solving systems of phaseless equations via Kaczmarz methods: A proof of concept 348 study. Inverse Problems , 31(12):125008, 2015. 349

Yuejie Chi and Yue M Lu. Kaczmarz method for solving quadratic equations. IEEE Signal 350 Processing Letters , 23(9):1183-1187, 2016. 351

Ju Sun, Qing Qu, and John Wright. A geometric analysis of phase retrieval. Foundations of 352 Computational Mathematics , 18(5):1131-1198, 2018. 353

Zhenzhen Li, Jian-Feng Cai, and Ke Wei. Toward the optimal construction of a loss function 354 without spurious local minima for solving quadratic equations. IEEE Transactions on 355 Information Theory , 66(5):3242-3260, 2019. 356

357

358

359

Yuxin Chen, Yuejie Chi, Jianqing Fan, and Cong Ma. Gradient descent with random initialization: Fast global convergence for nonconvex phase retrieval. Mathematical Programming , 176(1):5-37, 2019.

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

Irene Waldspurger. Phase retrieval with random gaussian sensing vectors by alternating projections. IEEE Transactions on Information Theory , 64(5):3301-3312, 2018.

Aldo Conca, Dan Edidin, Milena Hering, and Cynthia Vinzant. An algebraic characterization of injectivity in phase retrieval. Applied and Computational Harmonic Analysis , 38(2):346-356, 2015.

Yang Wang and Zhiqiang Xu. Phase retrieval for sparse signals. Applied and Computational Harmonic Analysis , 37(3):531-544, 2014.

Henrik Ohlsson, Allen Yang, Roy Dong, and Shankar Sastry. Cprl-an extension of compressive sensing to the phase retrieval problem. Advances in Neural Information Processing Systems , 25, 2012.

T Tony Cai, Xiaodong Li, and Zongming Ma. Optimal rates of convergence for noisy sparse phase retrieval via thresholded wirtinger flow. The Annals of Statistics , 44(5):2221-2251, 2016.

Gang Wang, Liang Zhang, Georgios B Giannakis, Mehmet Akçakaya, and Jie Chen. Sparse phase retrieval via truncated amplitude flow. IEEE Transactions on Signal Processing , 66(2):479491, 2017.

Gauri Jagatap and Chinmay Hegde. Sample-efficient algorithms for recovering structured signals from magnitude-only measurements. IEEE Transactions on Information Theory , 65(7):44344456, 2019.

Jian-Feng Cai, Jingzhi Li, Xiliang Lu, and Juntao You. Sparse signal recovery from phaseless measurements via hard thresholding pursuit. Applied and Computational Harmonic Analysis , 56:367-390, 2022.

Mahdi Soltanolkotabi. Structured signal recovery from quadratic measurements: Breaking sample complexity barriers via nonconvex optimization. IEEE Transactions on Information Theory , 65(4):2374-2400, 2019.

Deanna Needell and Joel A Tropp. Cosamp: Iterative signal recovery from incomplete and inaccurate samples. Applied and computational harmonic analysis , 26(3):301-321, 2009.

Simon Foucart. Hard thresholding pursuit: an algorithm for compressive sensing. SIAM Journal on Numerical Analysis , 49(6):2543-2563, 2011.

Shenglong Zhou. Gradient projection newton pursuit for sparsity constrained optimization. Applied and Computational Harmonic Analysis , 61:75-100, 2022.

Huishuai Zhang, Yi Zhou, Yingbin Liang, and Yuejie Chi. A nonconvex approach for phase retrieval: Reshaped wirtinger flow and incremental algorithms. Journal of Machine Learning Research , 18, 2017.

Yuxin Chen and Emmanuel J Candès. Solving random quadratic systems of equations is nearly as easy as solving linear systems. Communications on pure and applied mathematics , 70(5):822-883, 2017.

Emmanuel J Candes and Terence Tao. Decoding by linear programming. IEEE Transactions on 396 Information Theory , 51(12):4203-4215, 2005. 397

Simon Foucart and Holger Rauhut. An invitation to compressive sensing. In A Mathematical 398 Introduction to Compressive Sensing , pages 1-39. Springer, 2013. 399

Thomas Blumensath and Mike E Davies. Iterative hard thresholding for compressed sensing. 400 Applied and Computational Harmonic Analysis , 27(3):265-274, 2009. 401

Peer reviews:
comments: SUMMARY:
The authors introduce a novel second-order method for sparse phase retrieval. Compared to previous algorithms, it exhibits faster convergence and better recovery. The method leverages sparsity to reduce the size of the linear system that needs to be solved at each iteration in order to determine the approximate Newton direction (reduced from n^3 to s^3), and a second-order approximation of the intensity-based objective.

STRENGTHS:
This paper presents strong results and a theoretical analysis of the algorithm in both the noisy and noise-free case.

WEAKNESSES:
The sample complexity required for initialization and refinement is sub-optimal. The experiments are only on toy data.

QUESTIONS:
What are the practical implications of the sub-optimal sample complexity required for initializing the algorithm and for the refinement stage? Does it limit the applicability of the method on real-world signals?

Would the authors be able to show experimental comparisons on real-world phase recovery examples?

LIMITATIONS:
The authors discuss the limitations of their method.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a second-order algorithm based in Newton projection for the sparse phase retrieval algorithm. The proposed algorithm is similar to Hard Thresholding Pursuit, where the free variables (i.e. the support) is first identified by a hard thresholding step, followed by an update on the free variables via a Newton projection step.\nAs is standard for approaches to phase retrieval, the proposed method first performs an initialisation stage to ensure that the initial guess is sufficiently close to the true signal, then applies the proposed second-order method to obtain global convergence. There is are theoretical results proving quadratic convergence for the proposed method.\n\nSTRENGTHS:\nThe performance show substantial gains compared to previous methods, moreover, it establishes a quadratic convergence rate.\n\nWEAKNESSES:\nThis work is incremental compared to HTP of [28]. HTP can already to be interpreted as a second order method. In terms of per-iteration complexity, the proposed method is the same as HTP. The comparison in ‘iteration complexity’ is somewhat unclear because the complexity given in HTP is for exact recovery, whereas the rate given in Table 1 for the proposed method is to obtain accuracy \\epsilon — is it just that [28] does not prove a quadratic rate, or do we expect [28] to have worse convergence behaviour in general? Moreover, [28] proves finite convergence for their method, does the proposed method also achieve finite convergence?\n\nIn terms of practical performance, the convergence plots show that the proposed method has faster convergence compared to HTP, but the performance for HTP here is worse than the performance reported in [28]. Perhaps it would be useful to replicate the exact experiments in [28] so that a clear comparison can be given? In general, it would be useful to have a discussion on the differences with HTP and an explanation as to why the performance is superior to HTP, given that both are second-order methods. I also had a look at the proof and it is again similar to the proof given in [28], so it would be useful again to have a discussion on the differences and novelty over [28].\n\nQUESTIONS:\nPlease clarify on the differences in convergence results between [28] and the proposed method. It is also unclear to me why the proposed method has superior performance both in terms of convergence and in terms of the sparse solutions recovered, when both are second-order methods.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the sparse phase retrieval problem and introduces an efficient second-order algorithm based on Newton‘s method. The algorithm aims to recover sparse signals and offers a quadratic convergence rate while maintaining the same per-iteration computational complexity as first-order methods. Experimental results demonstrate that the proposed algorithm outperforms popular first-order methods in terms of convergence rate and success rate in recovering the true sparse signal.\n\nSTRENGTHS:\n1. The authors' algorithm exhibits a lower complexity per iteration and a higher convergence rate compared to popular first-order methods. It is noteworthy that this is the first algorithm to establish a quadratic convergence rate.\n2. The experimental results clearly illustrate the superiority of the proposed algorithm.\n3. The paper effectively communicates the motivation behind the development of the second-order algorithm and highlights the complexity reduction achieved by restricting Newton's step to a subset of variables.\n\nWEAKNESSES:\n1. The authors mention two prevalent loss functions but do not provide an explanation regarding the difference between these functions in the numerical experiments. It would be beneficial if the authors clearly explain the distinction between the two functions, particularly why the first function is used for initialization and the second one is used in Newton's update.\n2. Equation 12 introduces J_{k+1}, which seems to be highly dependent on the choice of S_0, the initial support. This raises concerns about the algorithm's sensitivity to the initial point. It would be valuable for the authors to address this issue and discuss the potential impact of the initial point on the algorithm's performance. \n3. Regarding the overall contribution, this paper focuses on approximating the objective function using a quadratic function, which can be limited. Also, this paper may be interested to only a few people attending this conference.\n\nQUESTIONS:\nMore extensive experiments would help.  E.g., when designing the experiments for unknown sparsity, it would be better to try different inputs for the sparsity levels. How important is the initialization step?\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes a new algorithm for phase retrieval of sparse signals. \nSpecifically, it focuses on a faster algorithm targeting quadratic convergence with the same number of measurements that are also needed in other algorithms. A proof of a quadratic convergence rate is established and experments illustrate the benefit also in experiments.\n\nSTRENGTHS:\nThe paper focuses on aspects in phase retrieval that are often ignored. In particular, a proofable faster convergence rate has not been the focus of other works so far.\nIt is well-written and easy to follow. \nIt may well become a new standard for phase retrieval (or a starting point for other similar algorithms) if other researchers can reproduce the excellent performance.\n\nWEAKNESSES:\nThe novelty is limited in the sense that second order algorithms are well known. However, adaptation and convergence proof for the phase retrieval setting are indeed novel and interesting.\nIt is not clear why this subset of existing algorithms has been used for the experiments.\n\nQUESTIONS:\nAre there any existing second order algorithms for phase retrieval (maybe even without convergence proof)?\nWhy were these specific existing algorithms used for comparsion?\nDoes increasing the maximum number of iterations in the definition of \"successful recovery\" change the performance of the various algorithms in Figure 2?\n\nLIMITATIONS:\nThe authors present no drawbacks of their method compared to the existing algorithms. In particular, the fact that existing algoritms need fewer measurements for refinement but perfom worse in the phase transition Figure 2 is surprising. Eventually, this is an artifact of the restriction to maximally 100 iterations for success (if the initialization is bad, significantly more iterations might be needed and could still make an algorithm successful, although for a significant computational cost). \nIt is strange that a faster algorithm is in this sense also more \"robust\".", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}