Title: Outlier-Robust Phase Retrieval in Nearly-Linear Time
Venue: NeurIPS 2024
Accepted: no
Keywords: phase retrieval

high-dimensional robust statistics

learning theory
Abstract: Phase retrieval is a fundamental problem in signal processing, where the goal is to recover a (complex-valued) signal from phaseless intensity measurements. It is well-known that natural nonconvex formulations of phase retrieval do not have spurious local optima. However, the theoretical analyses of such landscape results often rely on strong assumptions, such as the sampled measurements are (complex) Gaussian. In this paper, we propose and study the problem of outlier robust phase retrieval. We focus on the real-valued case, where we seek to recover a vector $x \in \mathbb{R}^d$ from $n$ intensity measurements $y_i = (a_i^\top x )^2$, under the assumption that the $a_i$'s are initially i.i.d. Gaussian but a small fraction of the $(y_i, a_i)$ pairs are adversarially corrupted. Our main result is a near sample-optimal nearly-linear time algorithm that provably recovers the ground-truth vector $x$ in the presence of outliers. We first solve a lightweight convex program to find a vector close to the ground truth. We then run robust gradient descent starting from this initial solution, leveraging recent advances in high-dimensional robust statistics. Our approach is conceptually simple and provides a framework for developing robust algorithms for tractable nonconvex problems.

Abstract

Phase retrieval is a fundamental problem in signal processing, where the goal is to recover a (complex-valued) signal from phaseless intensity measurements. It is known that natural non-convex formulations of phase retrieval do not have spurious local optima. However, the theoretical analyses of such landscape results often rely on strong assumptions, such as the sampling vectors being Gaussian distributed.

In this paper, we propose and study the problem of outlier robust phase retrieval.

We seek to recover a vector x ∈ R d from n intensity measurements y i = ( a ⊤ i x ) 2 , where the sampling vectors a i 's are initially i.i.d. Gaussian, but a small fraction of the ( a i , y i ) pairs are adversarially corrupted.

Our main result is a near-sample-optimal and nearly-linear-time algorithm that provably recovers the ground-truth x in the presence of adversarial corruptions. We first solve a lightweight convex program to find a vector close to the ground truth. We then run robust gradient descent starting from this initial solution, leveraging recent advances in high-dimensional robust statistics. Our approach is conceptually simple and provides a framework for developing robust algorithms for other tractable non-convex problems.

1 Introduction

Phase retrieval is a fundamental problem in signal processing with applications in various fields, in18 cluding electron microscopy [32], crystallography [33, 36], astronomy [11], and optical imaging [37]. 19 In these applications, one often has access to only the magnitudes of the Fourier transforms of a 20 complex signal. This is because measuring magnitude (e.g., by aggregating energy over time) is 21 much easier than measuring phase (which requires detecting rapid changes). We refer the reader to 22 the survey articles [37, 26] for more details about the theory and applications of phase retrieval. 23

24

25

In this paper, we focus on the real-valued generalized phase retrieval problem, where the Fourier transform is replaced by a general linear operator. We first give a formal definition of this problem.

26

27

28

29

30

31

32

33

34

35

36

37

Definition 1.1 (Phase Retrieval) . Let x ∈ R d be the ground-truth vector. Let a 1 . . . a n ∈ R d be n sampling vectors and let y i = ⟨ a i , x ⟩ 2 ∈ R be the corresponding intensity measurements. Given ( a i , y i ) n i =1 as input, the task is to recover x .

Note that it is impossible to distinguish between x and -x , so it is sufficient to recover either one. Under certain assumptions (e.g., when the a i 's are Gaussian distributed), the phase retrieval problem in Definition 1.1 can be solved in polynomial time with provable recovery guarantees. This was first achieved via approaches based on semidefinite programming (SDP) relaxations (see, e.g., Candès et al. [5]). In practice, the problem is often solved using first-order optimization algorithms such as gradient descent. It is well-established that, although many natural formulations of phase retrieval have nonconvex objectives, all local optima are globally optimal under certain assumptions [34, 3, 40]. An example of such objective function is the following:

However, existing analyses of such landscape results often rely on strong assumptions, such as the 38 sampling vectors a i 's are i.i.d. Gaussian. Our work is motivated by the following questions: Can we 39 relax the assumptions used in proving landscape results in many tractable nonconvex problems? In the 40 context of phase retrieval, what happens if a small fraction of the ( a i , y i ) 's are changed adversarially? 41 42

Definition 1.2 ( ϵ -Corruption) . An algorithm first specifies the number of samples n , and n samples are drawn independently from some unknown distribution D . The adversary is allowed to replace up to ϵn samples with arbitrary points. The modified set of n samples is then given to the algorithm as input. We say that a set of samples is ϵ -corrupted if it is generated by the above process. 1

We focus on the following strong contamination model (see, e.g., [13]). 43 44 45 46

47

48

49

50

51

52

Under the ϵ -corruption model for high-dimensional data, a common goal is to design efficient algorithms that can achieve dimension-independent error guarantees. Early work in robust statistics [42, 23, 25] provided sample-efficient estimators for various tasks, but with runtimes exponential in the dimension. A recent line of work, initiated by [13, 28], has developed computationally efficient robust algorithms for many fundamental high-dimensional tasks. There has been significant progress in the algorithmic aspects of robust high-dimensional statistics (see, e.g., [12]).

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

We now formally define the main problem that we pose and study in this paper.

Problem 1.3 (Outlier-Robust Phase Retrieval) . Let ϵ > 0 . Let x ∈ R d be the ground-truth vector with ∥ x ∥ 2 = 1 . First, n sampling vectors ( a i ) n i =1 are drawn i.i.d. from N (0 , I ) ∈ R d . Let y i = ⟨ a i , x ⟩ 2 be the corresponding intensity measurements. Then, an adversary arbitrarily corrupts an ϵ -fraction of the ( a i , y i ) 's. Finally, the corrupted ( a i , y i ) 's are given to the algorithm as input. The task is to find a vector z ∈ R d such that min {∥ z -x ∥ 2 , ∥ z + x ∥ 2 } ≤ ∆ for some precision parameter ∆ > 0 .

Note that we allow corruption in both the sampling vectors a i ∈ R d and the intensity measurements y i ∈ R . We would like to answer the following algorithmic question:

Can we design a provably robust and near sample-optimal algorithm for the

ϵ

-corrupted phase retrieval problem (Problem 1.3) that runs in nearly-linear time?

1.1 Our Results and Contributions

In this paper, we answer the above question affirmatively. We first state the main result of our paper. Theorem 1.4 (Main, Informal) . Consider the outlier-robust phase retrieval problem (Problem 1.3). Let ∆ > 0 . Given an ϵ -corrupted set of n = ˜ Ω( d log 2 (1 / ∆)) samples, we can compute z ∈ R d in time O ( nd ) such that min( ∥ z -x ∥ , ∥ z + x ∥ ) ≤ ∆ with probability at least 0 . 8 .

˜ 2 2

Our algorithm has near-optimal sample complexity, because even without corruption, recovering the ground-truth vector x in general requires Ω( d ) samples because there are d degrees of freedom in x . Moreover, our algorithm runs in time nearly-linear in the size of the input, and provably recovers the ground-truth vector x with arbitrary precision ∆ . The formal version of Theorem 1.4 is stated as Theorem 3.1 in Section 3.

We remark that the success probability of Theorem 1.4 can be boosted to 1 -δ for any δ > 0 by incurring an additional factor of T = O (log(1 /δ )) in the sample complexity and runtime. We can randomly partition the input into T equal-sized disjoint sets and run our algorithm on each set to obtain T solutions Z = { z 1 , . . . , z T } . If we output a solution z ⋆ that has the maximum number of points in Z within distance 2∆ , we can show that r ( z ⋆ ) ≤ 3∆ with probability at least 1 -δ .

Our main conceptual contribution is to propose and study the outlier-robust phase retrieval problem, where a small fraction of the input data is adversarially corrupted. Note that we allow arbitrary corruption in both the sampling vectors a i ∈ R d and the intensity measurements y i ∈ R . The nonconvex optimization landscape of phase retrieval is well understood when the a i 's are Gaussian distributed, but the adversarial robustness of such landscape results is largely unexplored.

Our main technical contributions include the design and analysis of a near sample-optimal and nearly83 linear time algorithm that provably solves the phase retrieval problem in the presence of outliers. 84 Our approach provides a conceptually simple two-step framework for developing outlier-robust 85 algorithms for tractable nonconvex problems that combines the robustness of spectral initialization 86 and the efficiency of the subsequent robust gradient descent. 87

1.2 Our Approach and Techniques 88

When there are infinite samples and no corruption, the objective function f ( z ) can be simplified as 89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

Even though f ( z ) is nonconvex, we know that it has no spurious local optima [34, 3, 40].

Our approach follows the general structure of Candès et al. [3], which uses a two-step procedure.

The first step uses spectral techniques to find an initial guess that is close enough to the ground truth.

The second step applies gradient descent to converge to the final solution. However, both steps are susceptible to adversarial corruption. We develop nearly-linear time and provably robust algorithms for both steps and combine them to get our main result.

Step 1: Robust Spectral Initialization. When there is no adversarial corruption, the empirical second-moment matrix Y = (1 /n ) ∑ n i =1 y i a i a ⊤ i has expectation E [ Y ] = I + 2 xx ⊤ , so its top eigenvector is close to x . However, the adversary can arbitrarily change the top eigenvector.

To circumvent this issue, we assign a (nonnegative) weight w i to each sample, and let Y w denote the weighted intensity-based second-moment matrix Y w = ∑ n i =1 w i y i a i a ⊤ i . Ideally, if the weights w are uniformly distributed on the remaining clean samples, the top eigenvector of Y w will align with x . We propose a novel optimization problem that can be used to find a weighting w such that Y w must be close to the unknown unbiased expectation I +2 xx ⊤ . Moreover, we show that such a weight w can be computed in nearly-linear time.

Step 2: Approximate Gradient Descent. Starting with the initial guess z 1 ∈ R d produced by the robust spectral initialization, we want to apply gradient descent to recover the ground truth x ∈ R d . Without corruption, if the initialization is close enough to x , each iteration will bring z closer to x by a constant factor. This convergence guarantee can be compromised by the corrupted samples.

At a high level, approximating the gradient at a specific point amounts to a robust mean estimation problem (for the underlying distribution of the gradients). When the input data is ϵ -corrupted, the gradients of the n samples can be viewed as an ϵ -corrupted set of vectors. We can approximate the true gradient using this ϵ -corrupted set of n gradients using robust mean estimation algorithms.

1.3 Related and Prior Works

Phase Retrieval. The problem of phase retrieval arises in many areas of science and engineering [11, 33]. Early research on this problem proposes error-reduction algorithms [22, 17, 18]. Convex and nonconvex optimization with various objective functions were later proposed and achieved exact recovery [43, 3-5, 38]. Follow-up works generalize to robust phase retrieval where the observations are subject to perturbations [45, 27, 7, 6, 31].

Nonconvex Optimization. Even though optimizing a nonconvex function is NP-Hard in general, recent works showed that many nonconvex functions are locally optimizable due to discrete or rotational symmetry. Besides phase retrieval, it is known that all local optima are globally optimal for natural nonconvex formulations of a wide range of machine learning problems, such as matrix completion [21], matrix sensing [2], phase synchronization [1], dictionary learning [39], and tensor decomposition [20] (see also Chapter 7 of [44]). Closely related to our work, a recent line of work explored the robustness of these landscape results: [30] studied matrix sensing in the ϵ -corrupted model and [8, 19] studied matrix completion and matrix sensing in semi-random models.

High-Dimensional Robust Statistics. Recent works in high-dimensional robust statistics developed nearly-linear time algorithms for the problem of robust mean estimation [9, 16, 29]. Prior works [35, 14] developed meta-algorithms for finding first-order stationary points with dimension-independent accuracy guarantees, which is closely related to the robust gradient descent procedure that we use.

1.4 Roadmap

We first introduce notations and background in Section 2. Then we give an overview of our approach in Section 3. Next, we focus on how to get an initialization that is close enough to the ground truth x in Section 4. After the initialization, we use robust mean algorithms to estimate gradients to converge to the desired accuracy in Section 5. Finally, we conclude in Section 6 and discuss open problems.

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

2 Preliminary and Background

Notation. We write [ n ] for the set of integers { 1 , . . . , n } . We use { e 1 , . . . , e d } for the standard unit vector basis in R d and I for the identity matrix. For a vector x , we denote its ℓ 1 , ℓ 2 and ℓ ∞ norm as ∥ x ∥ 1 , ∥ x ∥ 2 and ∥ x ∥ ∞ , respectively, and write the i th coordinate in x as x i . For vectors x, y ∈ R d , we denote its inner product as ⟨ x, y ⟩ = x ⊤ y . For a matrix A , we use ∥ A ∥ 2 , ∥ A ∥ ∗ , and ∥ A ∥ F as its operator norm, nuclear norm, and Frobenius norm, respectively. We write λ k ( A ) as the k th-largest eigenvalues of A , and λ k ( A ) as the sum of the k largest eigenvalues. A symmetric n × n matrix A is said to be positive semidefinite (PSD) if for all vectors x ∈ R n , x ⊤ Ax ≥ 0 . For two symmetric matrices A and B , we write A ⪯ B when B -A is positive semidefinite.

Packing SDP. We will use nearly-linear time solvers for the following packing SDP.

Lemma 2.1 ([10]) . Given an instance of optimization (*) with semi-positive definite matrices A i ∈ R d 1 × d 1 and B i ∈ R d 2 × d 2 with A i = C i C ⊤ i , B i = D i D ⊤ i for all i = 1 , 2 , · · · , m , together with integer k > 0 , error tolerance ϵ 0 ≥ 1 /m 2 , and failure probability δ 0 , there is an algorithm that runs in time ˜ O (( t C + t D + d 1 + d 2 ) poly(1 /ϵ 0 , log 1 /δ 0 )) , where t C i and t D i are the time take to perform a matrix product with C i and D i respectively and t C = ∑ n i =1 t C i and t D = ∑ n i =1 t D i , and outputs w ′ with ∥ w ′ ∥ 1 ≥ (1 -ϵ 0 ) OPT where OPT is optimal value, with probability at least 1 -δ 0 .

Computing the Top Eigenvector. We use power method to compute the top eigenvector of a matrix.

Lemma 2.2 (Power Method for Top Eigenvector, e.g., [41]) . Let A ∈ R d × d and let λ 1 be its largest eigenvalue. For any δ ∈ (0 , 1) , there exists an algorithm that takes A and outputs a unit vector x ∈ R d in time O ( t log( d ) /δ ) such that x T Ax ≥ (1 -δ ) λ 1 with probability at least 0 . 99 , where t is the time required to compute Av for an arbitrary v ∈ R d .

Robust Mean Estimation. Another tool we use is robust mean estimation in the ϵ -corruption model for distributions with bounded covariance. We use robust mean estimation algorithms to approximate the true gradient under adversarial corruption.

Lemma 2.3 (Robust Mean Estimation, e.g., [15]) . Let D be a distribution on R d with unknown mean µ and unknown covariance matrix Σ where Σ ⪯ σ 2 I . Let ϵ 0 be a sufficiently small universal constant. Let 0 < ϵ < ϵ 0 and δ > 0 . Given an ϵ -corrupted set of n samples drawn from D , we can output a vector µ ∈ R d in time O ( nd log(1 /δ )) such that, with probability at least 1 -δ -exp( -nϵ )

̂ ˜ , ( ) .

3 Overview

We first state a formal version of our main result.

Theorem 3.1 (Main) . Consider the setting of Problem 1.3. Let 0 < ϵ < ϵ ′ for some universal constant ϵ ′ and let ∆ > 0 . Given an ϵ -corrupted set of n = ˜ Ω( d log 2 (1 / ∆)) samples, we can compute a vector z ∈ R d in time ˜ O ( nd log(1 / ∆)) such that r ( z ) = min {∥ z -x ∥ 2 , ∥ z + x ∥ 2 } ≤ ∆ with probability at least 0 . 8 .

Theorem 3.1 requires two key technical lemmas: the robust spectral initialization (Lemma 3.2) and the approximate gradient descent (Lemma 3.3).

We first show that the spectral initialization can be done in nearly linear time with high probability, the proof of which can be found in Section 4.

′

Lemma 3.2 (Robust Spectral Initialization) . Under the setting of Problem 1.3, for any 0 < ϵ < ϵ for some universal constant ϵ ′ > 0 , given an ϵ -corrupted set of n = ˜ Ω( d ) samples, we can compute a vector z 0 ∈ R d of the ground truth x in time ˜ O ( nd ) such that r ( z 1 ) = min {∥ z 1 -x ∥ 2 , ∥ z 1 + x ∥ 2 } ≤ 1 8 with probability at least 0 . 95 .

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

Then, with such initialization results, we can proceed to show that an approximate gradient descent algorithm can be used to find an arbitrary approximation of the ground truth in Section 5.

Lemma 3.3 (Robust Gradient Descent) . Consider the setting of Problem 1.3. Let ∆ > 0 be the desired precision. Let 0 < ϵ < ϵ 0 for some sufficiently small universal constant ϵ 0 . Given an ϵ -corrupted set of n = ˜ Ω( d log 2 (1 / ∆)) samples and an initial guess z 1 such that r ( z 1 ) = min( ∥ z 1 -x ∥ 2 , ∥ z 1 + x ∥ 2 ) ≤ 1 / 8 , we can compute a vector z ∈ R d in time ˜ O ( nd ) such that r ( z ) ≤ ∆ with probability at least 0 . 95 .

For technical reasons, we cannot use the same set of samples for both the robust spectral initialization and the approximate gradient descent. Therefore, we partition the ϵ -corrupted set of 2 n samples into two equally sized disjoint sets, using one set for each algorithm.

Proof of Theorem 3.1. Let 2 n = ˜ Ω( d log 2 (1 / ∆)) be a set of ϵ/ 2 -corrupted samples. We partition the input into two disjoint sets of n samples. Both sets are ϵ -corrupted. By Lemmas 3.2 and 3.3, for any ϵ ∈ [0 , ϵ ′ ] and ∆ > 0 , our algorithm takes the first set of samples and output a vector z ′ in time ˜ O ( nd ) such that r ( z ′ ) ≤ 1 / 8 with probability at least 0 . 95 . Then, using z ′ and the second set of samples, our algorithm can output z ∈ R d in time ˜ O ( nd ) such that r ( z ) ≤ ∆ with probability at least 0 . 95 . The overall success probability is at least 0 . 8 , and the combined running time is ˜ O ( nd ) .

4 Robust Spectral Initialization

Wededicate this section to proving Lemma 3.2: Given an ϵ -corrupted set of ( a i , y i ) 's, we can compute an initial guess z 1 ∈ R d that is close to the ground truth x , where min( ∥ z 1 -x ∥ 2 , ∥ z 1 + x ∥ 2 ) ≤ 1 / 8 . To build some intuition, consider the following intensity-based covariance matrix Y = 1 n ∑ n i =1 y i a i a ⊤ i , where each a i is drawn independently from N (0 , I ) and y i = ⟨ a i , x ⟩ 2 . The expectation of this matrix is E [ Y ] = I +2 xx ⊤ . In other words, when there are enough samples and no adversarial corruption, we can obtain a good guess of the ground truth x (or -x ) by computing the top eigenvector of Y . However, we cannot rely on this approach in adversarial settings.

To tackle this issue, we propose a nearly-linear time preprocessing step (Algorithm 1) that can recover the true expectation of Y under adversarial corruptions. Algorithm 1 assigns a non-negative weight to each sample. For a weight vector w ∈ R n and a set of indices S ⊆ [ n ] , the weighted intensity-based covariance matrix is defined as Y S,w . = ∑ i ∈ S w i y i a i a ⊤ i , and we omit S when S = [ n ] . The feasible region for the weight vector is: ∆ n,ϵ := { w ∈ R n : ∥ w ∥ 1 = 1 and ∀ i ∈ [ n ] , 0 ≤ w i ≤ 1 (1 -ϵ ) n } .

A weight w defines an empirical distributions over the samples ( a i , y i ) n i =1 , where the largest probability assigned to any point is 1 (1 -ϵ ) n . Ideally, we would like to find a weight vector w ∗ ∈ ∆ n,ϵ that assigns its weight uniformly to all the uncorrupted samples, i.e., w ∗ i = 1 (1 -ϵ ) n · 1 i ∈ G . To find a suitable weighting w , we use the following optimization problem (**) in which λ 2 returns the sum of the top two eigenvalues (commonly known as the Ky Fan k norm for k = 2 ).

At a high level, our main observation is that y i a i a ⊤ i is always a positive semidefinite matrix as y i ≥ 0 . Consequently, the adversary can only add extra directions with large eigenvalues, but will not be able to remove the eigendirection of x . By minimizing the Ky Fan 2 norm, we can remove any directions added by the adversary and make sure that the only remaining large eigendirection is close to x .

Let δ ≥ 0 be some constant to be determined. We show that we can obtain a robust spectral initialization by solving the packing SDP problem (*), which can be solved efficiently using Lemma 2.1. In particular, to fit the reweighting problem of (**) into the framework of the generalized packing problem (*), we define the following constraint matrices for all i ∈ [ n ] :

The matrices ( A i ) n i =1 are used to implement the constraint that w ∈ ∆ n,ϵ . The matrices ( B i ) n i =1 help make sure the sum of the top two eigenvalues of Y w must be at most roughly 4 , because λ 2 ( Y w ∗ ) ≈ 4 .

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

Algorithm 1 Robust Spectral Initialization

Input: ϵ -corrupted samples ( a i , y i ) i ∈ [ n ]

Output: The initial guess z ′ ∈ R d

1: function ROBUSTINIT( { ( a i , y i ) } i ∈ [ n ] )

2: { A i , B i } ← Constraint matrices as defined in Equation (2)

3: w ′′ ← Solution to Optimization (*) with constraints { A i , B i } , k = 2 , and precision ϵ 0 = 0 . 9 as in Lemma 2.1

4: w ′ ← w ′′ / ∥ w ′′ ∥ 1

5: z 1 ← TOPEIGENVECTOR ( Y w ′ ) as in Lemma 2.2 with sufficiently small constant δ .

6: return z 1

7: end function

First, we show that the weight w ′ computed by Algorithm 1 can ensure the weighted intensity-based covariance matrix Y w ′ is close enough to the unbiased expectation I +2 xx ⊤ .

Lemma 4.1. With probability at least 0 . 98 , the w ′ outputted by Algorithm 1 satisfies:

In order to show Lemma 4.1, we need the following auxiliary Lemma 4.2, the proof of which can be found in Section A. Intuitively, Lemma 4.2 suggests that any weight w in the feasible region ∆ n, 2 ϵ will not have a huge impact on the properties of uncorrupted measurements.

Lemma 4.2. For any δ 0 > 0 , and sufficiently small ϵ ≥ 0 , given a set of n ϵ -corrupted samples with n > ˜ Ω( d ) , with probability at least 0 . 98 , we have ∥ ∥ Y G,w -( I +2 xx ⊤ ) ∥ ∥ 2 ≤ δ 0 for all w ∈ ∆ n, 2 ϵ .

Using Lemma 4.2, we provide a proof sketch for Lemma 4.1, and defer the details to Section A.

Proof. We condition on the fact that the event of Lemma 4.2 holds (with probability at least 0 . 98 ) for δ 0 = δ . Thus, for the remaining of the proof, we assume that for all w ∈ ∆ n, 2 ϵ , it holds that ∥ ∥ Y G,w -( I +2 xx ⊤ ) ∥ ∥ ≤ δ .

Let λ 1 and λ 2 be the top two eigenvalues of Y w ′ , with v 1 and v 2 to be their corresponding eigenvectors.

Note that the largest eigenvalue of I +2 xx ⊤ is 3 , and the rest of the eigenvalues are all 1 . In the proof, we show that the eigenvalues of Y w ′ are also close to the ones of I +2 xx ⊤ . Our proof consists of two parts. We first establish lower bounds for λ 1 and λ 2 , and then find an upper bound for λ 1 + λ 2 .

Lower Bound. Since y i a i a ⊤ i ⪰ 0 for any i ∈ [ n ] , for any positive weight vector w ∈ ∆ n,ϵ , we have Y G,w ⪯ Y w . Thus a lower bound on eigenvalues of Y G,w ′ will also be a lower bound on Y w ′ .

For the top eigenvalue λ 1 of Y w ′ , it holds

Similarly, for the second largest eigenvalue λ 2 of Y w ′ , we have:

Upper Bound. Through the optimization problem (*), a weight w ′′ is calculated such that Y w ′′ are operator-norm upper-bounded by the constraint parameters. Let OPT be the value of the optimal solution of the optimization problem (*). The desired uniform weight vector over the good samples w ∗ ∈ ∆ n, 2 ϵ is also a feasible solution to this optimization problem because Y w ∗ satisfy the optimization constraints due to Lemma 4.2. Since w ′′ is an ϵ 0 -approximation to the problem, we have

By optimization constraints, the Ky Fan 2 -norm of ∑ i w ′′ i B i = 1 2 (1 -δ ) Y w ′′ ≤ 2 , and consequently,

By combining inequalities (4), (5), and (6), we have shown that the top two eigenvalues of Y w ′ are close to 3 and 1 . Since the rest of the eigenvalues of Y w ′ can also be bounded, we can conclude that ∥ ∥ Y w ′ -( I +2 xx ⊤ ) ∥ ∥ 2 = O ( δ ) .

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

We can now show the closeness between the top eigenvector of Y w ′ and the ground truth.

Lemma 4.3. There exists an universal constant ϵ ′ such that if 0 ≤ ϵ ≤ ϵ ′ , and Algorithm 1 receives in input an ϵ -corrupted set of samples, then it outputs z 1 ∈ R d such that with probability at least 0 . 95 it holds r ( z 1 ) ≤ 1 8 .

Proof. We condition on the fact that the event of Lemma 4.1 holds (with probability at least 0 . 98 ). Let the eigendecomposition of Y w ′ be Y w ′ = ∑ i ∈ [ d ] λ i v i v ⊤ i , where λ 1 ≥ . . . ≥ λ d . Under the basis { v 1 , · · · , v d } , the ground truth x can be represented as x = ∑ i ∈ [ d ] α i v i . Note that ∥ x ∥ 2 2 = ∑ i ∈ [ d ] α 2 i = 1 . By Lemma 4.1, we have ∥ ∥ Y w ′ -( I +2 xx ⊤ ) ∥ ∥ 2 = O ( δ ) . Thus, we have x ⊤ Y w ′ x ≥ 3 -O ( δ ) and x ⊤ Y w ′ x ≤ λ 1 α 2 1 + λ 2 (1 -α 2 1 ) ≤ (3 + O ( δ )) α 2 1 +(1 + O ( δ ))(1 -α 2 1 ) ≤ 1 + 2 α 2 + O ( δ ) .

This implies α 2 1 ≥ 1 -O ( δ ) . As a result,

The last inequality holds as long as δ is sufficiently small. Let z 1 = ∑ i ∈ [ d ] β i v i be the unit vector approximating v 1 returned by the algorithm. By Lemma 2.2, we have that z ⊤ 1 Y w ′ z 1 ≥ (1 -δ ) λ 1 with probability at least 0 . 99 . Thus, we have:

Again, this implies that β 2 1 ≥ 1 -O ( δ + δ ) . We can show that min {∥ v 1 -z 1 ∥ 2 2 , ∥ v 1 + z 1 ∥ 2 2 } = O ( δ + δ ) . By the triangle inequality, we can conclude that r 2 ( z 1 ) = O ( δ + δ ) ≤ 1 / 64 , where the last inequality is obtained by choosing sufficiently small δ and δ . Therefore, there exists an universal constant ϵ ′ ≥ 0 such that for all 0 ≤ ϵ ≤ ϵ ′ , Algorithm 1 takes n = ˜ Ω( d ) samples and outputs z 1 such that r ( z 1 ) ≤ 1 / 8 with probability at least 0 . 95 .

Lemma 4.4. Algorithm 1 runs in time ˜ O ( nd ) .

Proof of Lemma 4.4. Since we have the factorization of the rank-two matrices A i and rank-one matrices B i for all i = 1 , 2 , . . . , n , and the time to perform a matrix-vector product with C i and D i is O ( d ) . Therefore, by Lemma 2.1, with t C and t D to be ˜ O ( nd ) , Line 3 runs in ˜ O ( nd ) time. In Line 5, by Lemma 2.2, the top eigenvector of Y w ′ can be computed in ˜ O ( n log d ) time using power method. Scaling in Line 4 runs in O ( n ) time. As a result, Algorithm 1 runs in ˜ O ( nd ) time.

We can directly combine Lemma 4.3 and Lemma 4.4 to finish the proof of Lemma 3.2.

5 Robust Gradient Descent

After the robust spectral initialization in Section 4, we have an initial guess z 1 ∈ R d that is close to the ground truth x or -x . Without loss of generality, we can assume that z 1 is closer to x than to -x . In this section, we prove Lemma 3.3: Given an initial guess z 1 with ∥ z 1 -x ∥ 2 ≤ 1 / 8 , we can use a robust gradient descent algorithm (Algorithm 2) to recover x to any desire precision ∆ > 0 . It is well-known that gradient descent can achieve geometric convergence rates in non-adversarial settings. We show that Algorithm 2 achieves a similar convergence rate even when the input is ϵ -corrupted.

Consider the natural nonconvex formulation: min z ∈ R d ∑ n i =1 f i ( z ) where f i ( z ) . = ( ⟨ a i , z ⟩ 2 -y i ) 2 . Let g i denote the gradient of f i with respect to z . Let D z denote the distribution of g i ( z ) ∈ R d when there is no adversarial corruption. Formally, g ( z ) ∼ D z is distributed as

To run gradient descent, we want to estimate the expected true gradient µ z . = E g ( z ) using samples. 286 The challenge is that the input samples { ( a i , y i ) } i ∈ [ n ] are ϵ -corrupted, and consequently the gradients 287 { g i ( z ) } i ∈ [ n ] is an ϵ -corrupted set of vectors drawn from D z . To address this, we use robust mean 288 estimation algorithms (e.g., [16]) to approximate µ z , the true mean of D z . 289

Algorithm 2 Robust Gradient Descent

The error guarantee of robust mean estimation algorithms depends on the covariance matrix Σ z of the distribution D z . The next lemma upper bounds the spectral norm of Σ z .

Lemma 5.1. Let D z be the distribution of gradients at z as defined in Equation (7) . For any z ∈ R d with ∥ z -x ∥ 2 ≤ 1 , the covariance matrix Σ z of D z satisfies Σ z ⪯ O ( ∥ z -x ∥ 2 2 ) I .

The proof of Lemma 5.1 is deferred to Appendix B. Given Lemma 5.1, we can show that robust mean estimation algorithms can approximate µ z with small error. For technical reasons, we randomly partition the input samples ( a i , y i ) into T subsets, and use one subset in each iteration. With high probability, each partition has at most (2 ϵ ) -fraction of corrupted samples

Lemma 5.2. Consider any z ∈ R d with ∥ z -x ∥ 2 ≤ 1 . Let µ z be the mean of D z as defined in Equation (7) . Fix universal constants c > 0 and ϵ 0 = Θ( c 2 ) . Let 2 ϵ < ϵ 0 and δ > 0 . Given a (2 ϵ ) -corrupted set of m = Ω( d log d/δ ) samples drawn from D z , we can compute ̂ µ z in time ˜ O ( md log(1 /δ )) such that ∥ ̂ µ z -µ z ∥ 2 ≤ c ∥ z -x ∥ 2 with probability at least 1 -O ( δ ) .

Proof of Lemma 5.2. Since 2 ϵ < ϵ 0 , we can view the (2 ϵ ) -corrupted set of m samples as ϵ 0 -corrupted. We need to replace 2 ϵ with ϵ 0 to reduce the failure probability of Lemma 2.3. This weakens the error guarantee of Lemma 2.3, but the resulting ̂ µ z is still accurate enough for our algorithm.

Lemma 5.2 shows that even with a (2 ϵ ) -corrupted set of gradients, the true gradient µ z can be estimated up to an additive error proportional to the distance between z and x . The next lemma shows that such an approximate gradient is sufficient for gradient descent to converge, reducing the distance to the ground truth x by a constant factor in each iteration.

Lemma 5.3. Suppose in iteration t of Algorithm 2, the current solution z t satisfies ∥ z t -x ∥ 2 ≤ 1 / 8 , and the estimated gradient ̂ µ z t ∈ R d satisfies ∥ ̂ µ z t -µ z t ∥ 2 ≤ c ∥ z t -x ∥ 2 for c = 4 . Then, we have ∥ z t +1 -x ∥ 2 2 ≤ 0 . 99 ∥ z t -x ∥ 2 2 .

Proof Sketch of Lemma 5.3. We provide a proof sketch and defer the full proof to Appendix B. Our objective function is nonconvex (even with infinitely many samples and no corruption). However, when the starting point z 1 is close to a global optimum, it is well-known that gradient descent is

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

well-behaved. More specifically, for any z close to the ground truth x , we can show that the (expected) true gradient µ z aligns with the direction toward x :

which is sufficient for proving geometric convergence. We can immediately see that this argument is robust to additive error in µ z that is proportional to ∥ z -x ∥ 2 . When ∥ ̂ µ z -µ z ∥ 2 ≤ c ∥ z -x ∥ 2 ,

When c < 7 . 5 , we can choose an appropriate step size η such that the distance between z t and x decreases by a constant factor in each iteration.

We are now ready to prove Lemma 3.3, which states the performance guarantee, sample complexity, runtime, and success probability of Algorithm 2. We restate Lemma 3.3 before proving it.

Lemma 3.3 (Robust Gradient Descent) . Consider the setting of Problem 1.3. Let ∆ > 0 be the desired precision. Let 0 < ϵ < ϵ 0 for some sufficiently small universal constant ϵ 0 . Given an ϵ -corrupted set of n = ˜ Ω( d log 2 (1 / ∆)) samples and an initial guess z 1 such that r ( z 1 ) = min( ∥ z 1 -x ∥ 2 , ∥ z 1 + x ∥ 2 ) ≤ 1 / 8 , we can compute a vector z ∈ R d in time ˜ O ( nd ) such that r ( z ) ≤ ∆ with probability at least 0 . 95 .

Proof of Lemma 3.3. First, we prove the success probability of Algorithm 2. Algorithm 2 can fail in two ways: (i) if some N t has more than (2 ϵ ) -fraction of corrupted samples, or (ii) if Lemma 5.2 fails in some iteration t . The probability of event (i) is at most 0 . 01 for our choice of n , which follows from a standard application of Hoeffding's inequality and a union bound over T iterations. For event (ii) , we choose a sufficiently small δ = O (1 /T ) in Lemma 5.2, so each robust gradient estimation fails with probability at most O ( δ ) = 0 . 01 /T , and overall the probability of event (ii) is at most 0 . 01 . For the rest of the proof, we assume these bad events do not happen.

Next, we show the correctness of Algorithm 2. Without loss of generality, we can assume that z 1 is closer to the ground truth x than to -x , which implies ∥ z 1 -x ∥ 2 ≤ 1 / 8 . By Lemma 5.2, we can obtain an approximation ̂ µ z 1 of the true gradient µ z 1 at z 1 such that ∥ ̂ µ z 1 -µ z 1 ∥ 2 ≤ c ∥ z 1 -x ∥ 2 . Then by Lemma 5.3, we know that ∥ z 2 -x ∥ 2 ≤ 0 . 99 ∥ z 1 -x ∥ 2 after one step of gradient descent. We can iteratively apply these two lemmas to show that, after T = O (log(1 / ∆)) iterations, we have ∥ z T +1 -x ∥ 2 ≤ ∆ . One technical issue is that in iteration t , we need to use a fresh subset of samples N t . By the principle of deferred decisions, we can view ( a i , y i ) i ∈ N t as being generated (and corrupted) after z t is chosen, which forms a (2 ϵ ) -corrupted set of gradients at z t .

Finally, we analyze the sample complexity and runtime of Algorithm 2. Algorithm 2 requires in total n = mT = Ω( d log d log 2 (1 / ∆)) samples. A random partition can be computed in O ( n ) time by shuffling the input. In each iteration, the m gradients in N t can be computed using Equation (7) in time O ( md ) , and z t can be updated in time O ( d ) . By Lemma 5.2, the true gradient can be robustly estimated in time ˜ O ( md log T ) = ˜ O ( md log log(1 / ∆)) . The overall runtime of the algorithm is ˜ O ( n +( md log log(1 / ∆)) T ) = ˜ O ( nd log log(1 / ∆)) = ˜ O ( nd ) .

6 Conclusions and Future Directions

In this paper, our main conceptual contribution is to propose and study the outlier-robust phase retrieval problem, where a constant fraction of the input data is corrupted. Notably, we allow corruption in both the sampled frequencies a i ∈ R d and the intensity measurements y i ∈ R . Our main technical contribution is the design and analysis of a near-sample-optimal and nearly-linear-time algorithm that solves this problem with provably guarantees.

An immediate technical question is whether our sample complexity can be tightened by removing some log(1 / ∆) factors. One potential approach is to open robust mean estimation algorithms instead of using them in a black-box manner. One could examine the stability conditions that these algorithms require, and see if these stability conditions can be proved without partitioning the samples and using fresh samples in each iteration. More broadly, we believe our framework can be used to develop outlier-robust algorithms for other tractable nonconvex problems, by first finding an initial solution in a saddle-free region near a global optimum and then converging to this global optimum using robust gradient descent.

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

References

A. S. Bandeira, N. Boumal, and V. Voroninski. On the low-rank approach for semidefinite programs arising in synchronization and community detection. In Conference on learning theory , pages 361-382. PMLR, 2016.

S. Bhojanapalli, B. Neyshabur, and N. Srebro. Global optimality of local search for low rank matrix recovery. Advances in Neural Information Processing Systems , 29, 2016.

E. J. Candès, X. Li, and M. Soltanolkotabi. Phase retrieval via wirtinger flow: Theory and algorithms. IEEE Trans. Inf. Theory , 61(4):1985-2007, 2015a.

E. J. Candès, X. Li, and M. Soltanolkotabi. Phase retrieval from coded diffraction patterns. Applied and Computational Harmonic Analysis , 39(2):277-299, Sept. 2015b. ISSN 1063-5203.

E. J. Candès, Y. C. Eldar, T. Strohmer, and V. Voroninski. Phase retrieval via matrix completion. SIAM Rev. , 57(2):225-251, 2015c.

J. Chen, L. Wang, X. Zhang, and Q. Gu. Robust Wirtinger Flow for Phase Retrieval with Arbitrary Corruption, Jan. 2018.

Y. Chen and E. Candes. Solving Random Quadratic Systems of Equations Is Nearly as Easy as Solving Linear Systems. In Advances in Neural Information Processing Systems , volume 28. Curran Associates, Inc., 2015.

Y. Cheng and R. Ge. Non-convex matrix completion against a semi-random adversary. In Conference On Learning Theory , pages 1362-1394. PMLR, 2018.

Y. Cheng, I. Diakonikolas, and R. Ge. High-dimensional robust mean estimation in nearly-linear time. In Proceedings of the 30th ACM-SIAM Symposium on Discrete Algorithms (SODA) , pages 2755-2771. SIAM, 2019.

Y. Cherapanamjeri, S. Mohanty, and M. Yau. List decodable mean estimation in nearly linear time. In S. Irani, editor, 61st IEEE Annual Symposium on Foundations of Computer Science, FOCS , pages 141-148. IEEE, 2020.

C. Dainty and J. R. Fienup. Phase retrieval and image reconstruction for astronomy. Image 393 recovery: theory and application , 231:275, 1987. 394

I. Diakonikolas and D. M. Kane. Algorithmic High-Dimensional Robust Statistics . Cambridge 395 University Press, 2023. 396

397

398

399

400

I. Diakonikolas, G. Kamath, D. M. Kane, J. Li, A. Moitra, and A. Stewart. Robust estimators in high dimensions without the computational intractability. In 57th Annual IEEE Symposium on Foundations of Computer Science-FOCS 2016 , pages 655-664. IEEE Computer Soc., Los Alamitos, CA, 2016.

401

402

403

404

405

406

407

408

409

410

411

412

I. Diakonikolas, G. Kamath, D. Kane, J. Li, J. Steinhardt, and A. Stewart. Sever: A robust meta-algorithm for stochastic optimization. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning , volume 97 of Proceedings of Machine Learning Research , pages 1596-1606. PMLR, 09-15 Jun 2019.

Y. Dong, S. Hopkins, and J. Li. Quantum Entropy Scoring for Fast Robust Mean Estimation and Improved Outlier Detection. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc., 2019.

Y. Dong, S. B. Hopkins, and J. Li. Quantum entropy scoring for fast robust mean estimation and improved outlier detection. In Proc. 33rd Advances in Neural Information Processing Systems (NeurIPS) , 2019.

J. R. Fienup. Reconstruction of an object from the modulus of its Fourier transform. Optics Letters , 3(1):27-29, July 1978. ISSN 1539-4794.

J. R. Fienup. Phase retrieval algorithms: A comparison. Applied Optics , 21(15):2758-2769, 413 Aug. 1982. ISSN 2155-3165. 414

X. Gao and Y. Cheng. Robust matrix sensing in the semi-random model. Proceedings of the 415 37th Conference on Neural Information Processing Systems (NeurIPS) , 2023. 416

R. Ge, F. Huang, C. Jin, and Y. Yuan. Escaping from saddle points-online stochastic gradient 417 for tensor decomposition. In Conference on Learning Theory , pages 797-842, 2015. 418

R. Ge, J. D. Lee, and T. Ma. Matrix completion has no spurious local minimum. In Advances in 419 Neural Information Processing Systems , pages 2973-2981, 2016. 420

R. W. Gerchberg. A practical algorithm for the determination of phase from image and 421 diffraction plane pictures. Optik , Jan. 1972. 422

423

424

F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw, and W. A. Stahel. Robust statistics. The approach based on influence functions . Wiley New York, 1986.

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

P. Hand and V. Voroninski. Corruption robust phase retrieval via linear programming. CoRR , abs/1612.03547, 2016.

P. J. Huber and E. M. Ronchetti. Robust statistics . Wiley New York, 2009.

K. Jaganathan, Y. C. Eldar, and B. Hassibi. Phase retrieval: An overview of recent developments. Optical Compressive Imaging , pages 279-312, 2016.

R. Kolte and A. Özgür. Phase Retrieval via Incremental Truncated Wirtinger Flow, June 2016.

K. A. Lai, A. B. Rao, and S. Vempala. Agnostic estimation of mean and covariance. In focs2016 , pages 665-674, 2016.

G. Lecué, M. Lerasle, and T. Mathieu. Robust classification via MOM minimization. Mach. Learn. , 109(8):1635-1665, 2020.

S. Li, Y. Cheng, I. Diakonikolas, J. Diakonikolas, R. Ge, and S. Wright. Robust second-order nonconvex optimization and its application to low rank matrix sensing. In Proc. 37th Advances in Neural Information Processing Systems (NeurIPS) , 2023.

J.-W. Liu, Z.-J. Cao, J. Liu, X.-L. Luo, W.-M. Li, N. Ito, and L.-C. Guo. Phase Retrieval via Wirtinger Flow Algorithm and Its Variants. In 2019 International Conference on Machine Learning and Cybernetics (ICMLC) , pages 1-9, July 2019.

J. Miao, T. Ishikawa, B. Johnson, E. H. Anderson, B. Lai, and K. O. Hodgson. High resolution 3D X-ray diffraction microscopy. Physical review letters , 89(8):088303, 2002.

R. P. Millane. Phase retrieval in crystallography and optics. JOSA A , 7(3):394-411, 1990.

P. Netrapalli, P. Jain, and S. Sanghavi. Phase retrieval using alternating minimization. In Proc. 27th Advances in Neural Information Processing Systems (NeurIPS) , pages 2796-2804, 2013.

A. Prasad, A. S. Suggala, S. Balakrishnan, and P. Ravikumar. Robust estimation via robust gradient estimation. Journal of the Royal Statistical Society. Series B. Statistical Methodology , 82(3):601-627, 2020.

W. H. Robert. Phase problem in crystallography. JOSA a , 10(5):1046-1055, 1993.

Y. Shechtman, Y. C. Eldar, O. Cohen, H. N. Chapman, J. Miao, and M. Segev. Phase retrieval with application to optical imaging: a contemporary overview. IEEE signal processing magazine , 32(3):87-109, 2015.

M. Soltanolkotabi. Structured signal recovery from quadratic measurements: Breaking sample complexity barriers via nonconvex optimization. IEEE Trans. Inf. Theory , 65(4):2374-2400, 2019.

J. Sun, Q. Qu, and J. Wright. Complete dictionary recovery over the sphere i: Overview and the geometric picture. IEEE Trans. Inf. Theor. , 63(2):853-884, 2 2017.

J. Sun, Q. Qu, and J. Wright. A geometric analysis of phase retrieval. Found. Comput. Math. , 458 18(5):1131-1198, 2018. 459

L. Trevisan. Lecture notes on graph partitioning, expanders and spectral methods. University of 460 California, Berkeley, https://people. eecs. berkeley. edu/luca/books/expanders-2016. pdf , 2017. 461

J. Tukey. Mathematics and picturing of data. In Proceedings of ICM , volume 6, pages 523-531, 462 1975. 463

V. Voroninski. PhaseLift: A Novel Methodology for Phase Retrieval . PhD thesis, UC Berkeley, 464 2013. 465

J. Wright and Y. Ma. High-dimensional data analysis with low-dimensional models: Principles, 466 computation, and applications . Cambridge University Press, 2022. 467

H. Zhang, Y. Chi, and Y. Liang. Provable non-convex phase retrieval with outliers: Median 468 truncated wirtinger flow. In International conference on machine learning , pages 1022-1031. 469 PMLR, 2016. 470

A Omitted Proofs in Section 4 471

472

473

474

475

476

477

478

Lemma A.1. [Lemma 4.2, Formal]. For any δ 0 > 0 , there exists constants ϵ 0 , c > 0 such that when n > c · d log d and we are given a set of n ϵ -corrupted samples, where 0 ≤ ϵ ≤ ϵ 0 , then with probability at least 0 . 98 , it holds

Proof of Lemma A.1. We recall the definition of Y G,w = ∑ i ∈ G w i y i a i a ⊤ i . Let ℓ = ϵ · n and let { ( a n + i , y n + i ) } ℓ i =1 be the set of samples that were removed by the ϵ -corruption adversary. Let G ′ = G ∪ { n +1 , . . . , n + ℓ } , n ′ = n + ℓ , and ϵ ′ = ϵ/ (1 + ϵ ) . Note that without loss of generality, we can assume that | G | = (1 -ϵ ) n and | G ′ | = (1 -ϵ ′ ) n ′ = n .

We define a mapping σ : ∆ n, 2 ϵ → ∆ n ′ , 3 ϵ ′ such that 479

In other words, all the weights are the same for the samples with index in the set [ n ] , and are equal to 0 for the samples removed by the adversary. We can verify that σ ( w ) ∈ ∆ n ′ , 3 ϵ ′ for all w ∈ ∆ n, 2 ϵ since σ ( w ) i ≤ w i ≤ 1 / (1 -2 ϵ ) n = 1 / (1 -3 ϵ ′ ) n ′ for all i ∈ [ n ′ ] , and ∥ σ ( w ) ∥ 1 = ∥ w ∥ 1 = 1 . Furthermore, we have Y G,w = Y G ′ ,σ ( w ) for all w ∈ ∆ n, 2 ϵ . We denote with w ∗ ∈ ∆ n ′ , 3 ϵ ′ the desired uniform weighting of the samples with index in G ′ , i.e., w ∗ i = 1 (1 -ϵ ′ ) n ′ 1 i ∈ G ′ .

It suffices to show both ∥ ∥ Y G ′ ,w ∗ -( I +2 xx ⊤ ) ∥ ∥ 2 ≤ δ 0 / 2 and ∥ ∥ Y G ′ ,σ ( w ) -w ∗ ∥ ∥ 2 ≤ δ 0 / 2 . By triangle inequality, for any w ∈ ∆ n, 2 ϵ , it holds

480

481

482

483

484

485

486

Thus, it suffices to show both ∥ ∥ Y G ′ ,w ∗ -( I +2 xx ⊤ ) ∥ ∥ 2 ≤ δ 0 / 2 and ∥ ∥ Y G ′ ,σ ( w ) -w ∗ ∥ ∥ 2 ≤ δ 0 / 2 . 487

We upper bound the first term. By using the definition of w ∗ , note that 488

Since E ( y i a i a ⊤ i ) = I +2 xx ⊤ for any i ∈ G ′ , we can use a concentration inequality to upper bound 489 this term. By Lemma A.2, as long as n ≥ c 1 ( δ 0 / 2) · d log d , with probability at least 0 . 995 , we have 490

It remains to show a high-probability upper bound to the second term ∥ ∥ Y G ′ ,w ∗ -σ ( w ) ∥ ∥ 2 ≤ δ 0 / 2 . 491 The first observation is that the weighting w ∗ and σ ( w ) for any w ∈ ∆ n, 2 ϵ cannot too different. In 492 particular, we can show the following upper bound: 493

We observe that ∆ n ′ , 3 ϵ ′ can be seen as the convex combination of all possible uniform weighting 494 over subsets of n ′ (1 -3 ϵ ′ ) samples. Thus, the maximum distance will be between two points of the 495 convex hull, and we can upper bound (11) as: 496

497

For ease of notation, let β i . = | w ∗ i -σ ( w ) i | . Observe that 0 ≤ β i ≤ 1 (1 -2 ϵ ) n for all i , and ∑ i β i ≤ 6 ϵ 498 due to (11). We have: 499

Inequality (13) follows by assigning the maximum possible β i to the largest entries of the sum until 500 we hit the budget 6 ϵ due to (11). 501

We bound max L ∑ i ∈ L ⟨ a i , x ⟩ 4 first. Let X i = ⟨ a i , x ⟩ ∼ N (0 , 1) for i ∈ G ′ , and define the 502 threshold function h r ( z ) = { 0 , z ≤ r z, z > r for r = C 2 · ln 2 (1 /ϵ ) with constant C > 0 to be determined. 503

Note that z ≤ r + h r ( z ) for all z > 0 . Therefore, 504

Then, we consider to bound exp (∑ i ∈ G ′ c · h r ( X 4 i ) ) for some c > 0 to be determined. For any 505 i ∈ G ′ and z ≥ 1 , with C = 6 , for all ϵ > 0 , we have 506

At the same time, with c < 1 / 200 , for all z ≥ 1 , we have 507

Therefore, Pr [ exp ( c · h r ( X 4 i ) ) ≥ z ] ≤ min { z -3 , ϵ 3 } , and for all ϵ < 1 / 2 , we have 508

Since { X i } i ∈ G ′ are independent, we have 509

Finally, by Markov's inequality, for any constant δ 1 > 0 , as long as ϵ ≤ √ δ 1 c , we have 510

As a result, with sufficiently large n ≥ c 2 ( δ 1 ) · d log d and sufficiently small ϵ such that 6 ϵr ≤ δ 1 , 511

Also, max L ∑ i ∈ L ⟨ a i , x ⟩ 2 ⟨ a i , u ⟩ 2 has a similar tail bound and therefore can be bounded in the same 512 way. We can then bound the operator norm via an epsilon-net argument. Set δ 1 = δ 0 / 24 . By an 513 1 / 4 -net on S d -1 with |N| ≤ 9 d , we have that 514

By combining the above inequality with Equation (10), we can conclude that, for any δ 0 > 0 , there 515 exists ϵ 0 > 0 , such that when n ≥ max { c 1 ( δ 0 ) , c 2 ( δ 0 ) } · d log d and 0 ≤ ϵ ≤ ϵ 0 , with probability at 516 least 0 . 98 , 517

518

519

520

521

522

523

524

Therefore, with probability at least 0 . 98 , we have ∥ ∥ Y G,w -( I +2 xx ⊤ ) ∥ ∥ 2 ≤ δ 0 for all w ∈ ∆ n, 2 ϵ .

A.1 Concentration Inequalities

For the undisturbed samples, we have the following concentration result.

Lemma A.2 ([3] Section A.4.2) . Let x ∈ R d . For any δ > 0 , there exists a constant C ( δ ) > 0 such that when n > C · d log d and we are given a set of n samples { ( a i , y i ) } n i =1 with a i ∼ N (0 , I ) independently and y i = ⟨ a i , x ⟩ 2 for all i ∈ [ n ] , then with probability at least 0 . 99 , it holds

B Omitted Proofs in Section 5 525

526

527

Lemma 5.1. Let D z be the distribution of gradients at z as defined in Equation (7) . For any z ∈ R d with ∥ z -x ∥ 2 ≤ 1 , the covariance matrix Σ z of D z satisfies Σ z ⪯ O ( ∥ z -x ∥ 2 2 ) I .

Proof of Lemma 5.1. Recall that g ∼ D z is distributed as 528

Because E g ∼D z [ g ] = µ z , the spectral norm of Σ z can be upper bounded as follows: 529

Consequently, it suffices to upper bound ∥ ∥ E g ∼D z [ gg ⊤ ]∥ ∥ 2 . Let h = z -x . 530

531

532

533

534

535

536

The first inequality is due to Cauchy-Schwarz inequality. The last step uses the fact that ∥ x ∥ 2 = 1 and ∥ h ∥ 2 ≤ 1 .

Lemma 5.3. Suppose in iteration t of Algorithm 2, the current solution z t satisfies ∥ z t -x ∥ 2 ≤ 1 / 8 , and the estimated gradient ̂ µ z t ∈ R d satisfies ∥ ̂ µ z t -µ z t ∥ 2 ≤ c ∥ z t -x ∥ 2 for c = 4 . Then, we have ∥ z t +1 -x ∥ 2 2 ≤ 0 . 99 ∥ z t -x ∥ 2 2 .

Proof of Lemma 5.3. Recall that g ∼ D z is distributed as

We can compute the mean of D z using moments of Gaussian: 537

Consider one step of gradient descent in Algorithm 2: z t +1 = z t -η ̂ µ z t , where ̂ µ z t is an approximate 538 gradient. We have 539

To prove convergence, we need to lower bound ⟨ ̂ µ z t , z t -x ⟩ and upper bound ⟨ ̂ µ z t , ̂ µ z t ⟩ . 540

We write z = z t and h = z -x to simplify notation. We can substitute z = x + h in Equation (14): 541

Recall the assumptions of this lemma: ∥ x ∥ 2 = 1 , ∥ h ∥ 2 ≤ 1 / 8 , and ∥ ̂ µ z -µ z ∥ 2 ≤ c ∥ h ∥ 2 . 542

First we lower bound ⟨ ̂ µ z , h ⟩ . 543

The first inequality uses the fact that 16 ⟨ x, h ⟩ 2 + 36 ⟨ x, h ⟩ ∥ h ∥ 2 2 is a second-order polynomial of 544 ⟨ x, h ⟩ , which has minimum value -81 4 ∥ h ∥ 4 2 for all ⟨ x, h ⟩ ∈ R . 545

Next we upper bound ⟨ ̂ µ z , ̂ µ z ⟩ using the triangle inequality. 546

Putting everything together, we have 547

Choosing c = 4 and η = 1 / 300 gives that ∥ z t +1 -x ∥ 2 2 ≤ 0 . 99 ∥ z t -x ∥ 2 2 .

C Counter-examples

Prior robust phase retrieval algorithms [24, 45] focus on the setting where the observations y i 's are subject to adversarial perturbation while the measuring vectors a i 's are independently sampled from the Gaussian distribution. The Median Truncated Wirtinger Flow Algorithm [45] first initialize z (0) by the spectral method, calculating z (0) as the top eigenvector of Y := 1 m ∑ m i =1 y i a i a ⊤ i 1 | y i |≤ α 2 med( { y i } i m =1 ) using a truncated set of samples, where the threshold is determined by med( { y i } i m =1 ) , the median over all y i 's. As long as the fraction of of outliers is not too large and the sample complexity is large enough, the initialization is guaranteed to be within a small neighborhood of the ground truth.

In this section, we present a counter-example where robust phase retrieval algorithms [24, 45] can be insufficient when directly applied to the ϵ -corruption phase retrieval problem.

Let x ∈ S d -1 be the ground truth unit vector. Here we construct an ϵ -corruption adversary that can manipulate the top eigenvector of the empirical covariance matrix Y = ∑ n i =1 y i a i a ⊤ i , even when y i are accurately calculated as y i = ( a ⊤ i x ) 2 .

Let u ∈ S d -1 be a unit vector such that x ⊤ u = 0 . Suppose the adversary changes 1% of the a i 's to a i = √ d -1 / 25 · u +1 / 5 x , and suppose all the y i 's are accurate. In particular, the length of the corrupted a i 's is comparable to Gaussian vectors, and the corresponding y i = ( a ⊤ i x ) 2 = 1 / 25 . Consequently, the median-truncated initialization in [45] will not be able to filter out such y i . However, the top eigenvector of E [ Y ] = E [∑ n i =1 y i a i a ⊤ i ] = O ( d ) uu ⊤ + O ( √ d )( ux ⊤ + xu ⊤ ) + O (1)( I + 2 xx ⊤ ) will be manipulated to u , which is far from the ground truth x .

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

607

608

609

610

611

612

613

614

615

616

617

618

619

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We provide a formal discussion of our theoretical claim in the first section of the paper, and include a formal statement of our resul in Section 3.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We point out to all the assumptions used in our paper. There are a discussion of the limitations in the conclusion. Our algorithm can only handle a corruption ϵ < ϵ 0 for some universal constant ϵ 0 . Additionally, it is possible that the dependency on ∆ for the sample complexity can be removed.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We formally describe our problem setting and assumption used. All the proofs are in the paper, either in the main content pages or in the appendix.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [NA]

Justification: The paper is a theory paper, and it has no experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

707

708

709

710

711

712

713

714

715

716

717

718

719

720

721

722

723

724

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [NA]

Justification: The paper is a theory paper, and it has no experiments.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [NA]

Justification: The paper is a theory paper, and it has no experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [NA]

Justification: The paper is a theory paper, and it has no experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

725

726

727

728

729

730

731

732

733

734

735

736

737

738

739

740

741

742

743

744

745

746

747

748

749

750

751

752

753

754

755

756

757

758

759

760

761

762

763

764

765

766

767

768

769

770

771

772

773

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [NA]

Justification: The paper is a theory paper, and it has no experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The paper conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: The paper provides a theoretical result. We do not believe there are potential societal consequences of our work, aside from advancing the field of Machine Learning.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper only contains theoretical results.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [NA]

Justification: The paper only contains theoretical results. Previous work is properly cited.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not introduce any new asset.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This point does not apply to our paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This point does not apply to our paper.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the challenge of achieving outlier robustness in phase retrieval, specifically focusing on the recovery of real-valued signals from intensity measurements that have been corrupted by adversarial outliers. The contribution of this work is the development of a nearly-linear time algorithm that is nearly sample-optimal and can accurately recover the true vector despite the presence of outliers. This is done through a two-step process that involves robust spectral initialization and robust gradient descent, utilizing recent results in high-dimensional robust statistics.\n\nSTRENGTHS:\n+A two-stage algorithm achieves nearly-linear time complexity consisting of an initial spectral initialization phase and a gradient descent refinement phase.\n\n+Theoretical analysis showing the algorithm can recover the ground truth signal despite the presence of outliers, while maintaining near-optimal sample efficiency.\n\nWEAKNESSES:\n- The paper's theoretical results assume the corruption level $\\epsilon$ as a constant in Theorem 3.1, despite it being initially introduced as a variable in Definition 1.2 to denote the extent of sample corruption. This constant treatment affects the robustness of the results by neglecting the influence of $\\epsilon$ on the sample complexity. A thorough analysis that explicitly considers the variability of $\\epsilon$ would significantly strengthen the theoretical foundation.\n- Lack of numerical validations for the theoretical claims\n- The idea and design of the two-stage robust phase retrieval algorithm is not novel, by adapting existing reweighting phase retrieval algorithms (throught different design of the reweights) to handle outliers through robust statistics. The contamination model is also from existing works, which has been recent studied in a number of different contexts; e.g., robust linear/nonlinear estimation by e.g., Diakonikolas and coauthors.\n- Lack of numerical validation and comparison of the proposed algorithm with respect to existing (robust) Gaussian phase retrieval algorithms. It is difficult to judge if the proposed algorithm is of practical interest (or only of theoretical interest).\n- Quite a lot statements in the paper are rather confusing, e.g., \"we propose the problem of outlier robust phase retrieval\"; robust phase retrieval has been long studied in the literature; as far as I understand, the paper studies a new robust phase retrieval problem by considering also adversarial a_i's on top of existing formualtions. ii) \"It is well-known that natural nonconvex formulations of phase retrieval do not have spurious local optima.\" which is not precise enough and is true under very stringent assumptions. iii) \"This is first achieved via approaches based on semidefinite programming (SDP) relaxations (see, e.g., Cand`es et al. (2015c)).\" I guess the first Gaussian phase retrieval algorithm was the AltMin algorithm (Netrapalli, et al 2013. Phase retrieval using alternating minimization. Advances in Neural Information Processing Systems, 26.) which was interestingly not cited in the submission. iv) \"Similar landscape results are known for other natural nonconvex formulations of phase retrieval as well (e.g., min f(z) = P i( √ yi − |⟨ai, z⟩|)2 (Soltanolkotabi, 2019)).\" The first work to study the Gaussian phase retrieval based on the magnitude-based least-squares nonconvex formulation and achieve provable guarantees is (Wang et al (2017). Solving systems of random quadratic equations via truncated amplitude flow. IEEE Transactions on Information Theory. 64(2):773-94.) Please be careful and fair in stating related results.\n\nQUESTIONS:\ni) The main motivation of the paper is to study a new robust phase retrieval (RPR) problem where in addition to the existing well-studied RPR (earlier in e.g., Hand, et al. 2016. Corruption robust phase retrieval via linear programming. arXiv preprint arXiv:1612.03547.), the sampling vectors a_i's can also be adversarially manipuated. Nonetheless, since PR is quite \"simple\" model, it can be imagined some of the effect of the adversarial a_i's can also be lumped into the adversarial y_i's. I am not sure whether the problem here is indeed more challenging than existing corruption pr considering only adversarial y_i's. If not rigorously theoretically analyzed, numerical comparisons with these existing rpr algorithms will be needed and appreciated. \nii) The designed algorithm is a two-stage reweighting algorithm that consists of a reweighting spectral initialization and reweighting gradient descent algorithm; nonetheless, this unique design and use of reweighting to improve nonconvex PR performance has been initially documented in [Yuan, et al (2017). Phase retrieval via reweighted Wirtinger flow. Applied Optics, 56(9), 2418-2427; Wang, et al (2018). Phase retrieval via reweighted amplitude flow. IEEE Transactions on Signal Processing. 66(11):2818-33.]; the form of using the weights here is exactly the same as in the two papers]; The only difference is the specific option/design of the values of the weights, certainly for considerations of handling the outliers here. It is very interesting to understand what is the original motivation of using and where does the reweighting technique come from. \niii) Last but not the least, from a practical perspective, say e.g., the Fourier/modulated phase retrieval problem using random masks, it makes sense that possiblem (adversarial) noise may be present in measuring the intensity measurements; but it is not clear why the sampling vectors that correspond to rows of DFT matrices (or multiplied by random masking +1 or -1 values) could also be manupulated experimentally. Please provide a more practical setup validating the motivation of studying such a RPR problem. \niv) A lot of typos or inaccurate statements. e.g., \"we propose and study...\"; the claim \"into a saddle-free region\" is not theoretically and rigorously established.\n\nLIMITATIONS:\nNA", "TITLE": "Official Review", "RECOMMENDATION": 2, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on the problem of outlier robust phase retrieval, whose goal is to recover a vector $x \\in \\mathbb{R}^d$ from $n$ intensity measurements $y_i = (a_i^\\top x)^2$ when a small fraction of the samples are adversarially corrupted. The authors propose and study this problem, providing a nearly sample-optimal and nearly linear-time algorithm to recover the ground-truth vector $x$ in the presence of outliers.\n\nSTRENGTHS:\nThis paper provides an analysis of the practically interesting problem of outlier robust phase retrieval. The algorithm and framework might have implications for various applications that are relevant to phase retrieval.\n\nWEAKNESSES:\n1. The analysis appears to be more incremental in nature compared to those in previous theoretical works related to phase retrieval. More precisely, the key novelty of the spectral initialization step resides in the assignment of a nonnegative weight to each sample. Regarding the gradient descent step, the problem seems to be simplified to the analysis of robust mean estimation algorithms.\n\n2. Important references are missing. For instance, the authors ought to cite the works related to robust compressed sensing (and it would be better to discuss in more detail the disparities between the analysis for the robust gradient descent step in this work and the analysis in these relevant works), such as\n\n- Liu, Liu, Yanyao Shen, Tianyang Li, and Constantine Caramanis. \"High dimensional robust sparse regression.\" In International Conference on Artificial Intelligence and Statistics, pp. 411-421. PMLR, 2020.\n- Liu, Liu, Tianyang Li, and Constantine Caramanis. \"High Dimensional Robust $ M $-Estimation: Arbitrary Corruption and Heavy Tails.\" arXiv preprint arXiv:1901.08237 (2019).\n\n3. The authors solely consider the scenario of noiseless intensity measurements and fail to take into account the noisy case.\n\n4. The paper does not include experimental results, which could limit the confidence in the practical effectiveness of the proposed approach.\n\nQUESTIONS:\nWhy in Theorem 1.4 (or its formal version in Theorem 3.1), there is no dependence on $\\epsilon$ in both the sample complexity and the upper bound of $\\min (\\\\|z-x\\\\|_2,\\\\|z+x\\\\|_2)$?\n\nLIMITATIONS:\nNo experimental validations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies a classical problem called phase retrieval. The goal is to obtain unknown $d$-dimensional vector $x$ from $n$ datapoints $(a_i, \\langle a_i, x \\rangle^2)$. This work assumes that $a_i$ are iid Gaussian vectors, but also that a small $\\varepsilon$ fraction of the data is corrupted. The authors suggest a two-stage process to identify the vector up to a small error. First, a spectral based algorithm is used to have a small constant error. Further, a robust gradient descent is used to approximate the initial guess up to a small error.\n\nSTRENGTHS:\n1. Paper is well-written and provides a good overview of the problem and of the techniques.\n2. Phase retrieval is a traditional non-convex problem, which was largely studied before, and understanding how robust algorithms perform on it is important.\n3. Paper uses prior technique in a simple way, and it is possible that this two-stage approach can be applied to other problems.\n\nWEAKNESSES:\n1. Results are limited to the Gaussian setting.\n2. The method for RME that is used assumes that variance $\\sigma$ is known? But in the way it is used here, $\\sigma$ depends on the distance between current solution and the true vector. Authors do not comment on this issue.\n3. $\\tilde O, \\tilde \\Omega$ notation is not defined.\n4.  Intuition in line 98 in my interpretation contradicts more exact version in line 214 (In the end, if I understand correctly, the crucial reason why the spectral initialization algorithm works is that the adversary cannot change the top eigendirection, but can only add new directions).\n\nQUESTIONS:\n1. How applicable are the techniques beyond Gaussian iid setting?\n2. Why using $k = 1$ is not sufficient for the optimization problem in Algorithm 1?\n\nLIMITATIONS:\nThere are no ethical limitations of this work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
The authors study the phase retrieval problem for retrieving a real signal under the influence of arbitrary corruption. The corruption is allowed to be present in labels or features. They propose a two-step solution. First, they ensure that the initialization is robust to the corruption and second, they show that the gradient descent updates can be made resilient. Authors claim that their method can recover the true signal (with possibly sign mismatch) to an arbitrary precision.

STRENGTHS:
The ideas presented in the paper are certainly interesting. If resilience to corruption can be achieved in individual steps, then it makes sense that it might lead to a good overall recovery.

WEAKNESSES:
1. The authors claim that corruption level up to some universal constant $\epsilon'$ can be handled through their method. Although, to the best of my understanding, this quantity is not characterized in the main paper. What is the maximum value for $\epsilon'$? 
2. There is no discussion on the dependency of $\epsilon'$ on $n$ or $d$.   
3. The claim of signal recovery to an arbitrary precision puzzles me. It is known that for $\epsilon$-corrupted vectors, the robust mean estimation can only be done up to $\Omega(\sqrt{\epsilon})$ error. Despite that, the authors claim signal recovery (with possibly a flipped sign) to an arbitrary precision. Can the authors comment on how this is achieved?   
4. The claim in line 213 says that $y_i$ is always greater than $0$. Why is that true when the adversary can corrupt $y_i$ arbitrarily? As far as I can tell, the algorithm does not discard negative $y_i$s.

QUESTIONS:
Please see above.

LIMITATIONS:
Please see above.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-16