Polynomial time guarantees for the Burer-Monteiro method
Abstract
The Burer-Monteiro method is one of the most widely used techniques for solving large-scale semidefinite programs (SDP).
The basic idea is to solve a nonconvex program in Y , where Y is an n × p matrix such that X =
Y Y T .
We show that this method can solve SDPs in polynomial time in a smoothed analysis setting.
More precisely, we consider an SDP whose domain satisfies some compactness and smoothness assumptions, and slightly perturb the cost matrix and the constraints.
We show that if p ≳ (cid:112) 2(1+ η ) m , where m is the number of constraints and η > 0 is any fixed constant, then the Burer-Monteiro method can solve SDPs to any desired accuracy in polynomial time, in the setting of smooth analysis.
The bound on p approaches the celebrated Barvinok-Pataki bound in the limit as η goes to zero, beneath which it the nonconvex program can be suboptimal.
Our main technical contribution, which is key for our tight bound on p , is to connect spurious approximately critical points of the nonconvex program to tubular neighborhoods of certain algebraic varieties, and then estimate the volume of such tubes.
Introduction
Let S n be the set of n × n symmetric matrices.
Given matrices A 1 , . . .
, A m , C ∈ S n and a vector b ∈ R m , consider the semidefinite program (SDP):
[Equation]
where A : S n → R m , X (cid:55)→ ( A 1 • X, . .
A m • X ) is a linear map and the notation X ⪰ 0 in indicates that X is a positive semidefinite (PSD) matrix.
Also consider the least squares SDP
[Equation]
Though interior point methods solve ( SDP ) and ( SDP ls ) in polynomial time, they often have memory problems for large values of n .
The Burer-Monteiro method [11,12] is one of the most widely used procedures for large scale problems.
Several papers have worked in understanding the practical success of this method, e.g., [9,10,12].
Although various results have been shown, they all fall short of showing that one can reach an approximately optimal solution of ( SDP )/
( SDP ls ) in polynomial time.
This paper provides the first polynomial time guarantees for the Burer-Monteiro method.
The Burer-Monteiro method consists in writing X
= Y Y T for some Y ∈ R n × p , and solving a nonconvex optimization problem in terms of Y .
For ( SDP ) we get the problem
[Equation]
and for ( SDP ls ) we get the unconstrained problem
[Equation]
We focus now on problem ( SDP ), though our results apply to ( SDP ls ) as well.
Let τ ( k ) :
= (cid:0) k +12 (cid:1) be the k -th triangular number.
It is known that problems ( SDP ) and ( BM ) have the same optimal value for any p such that τ ( p ) ≥ m ; this is known as the Barvinok-Pataki bound [3, 29].
But due to nonconvexity, local optimization methods may converge to a critical point of ( BM ) which is not globally optimal [34].
In this paper we are mainly interested in 2nd-order critical points (abbreviated as 2-critical points).
Problem ( BM ) has been well studied, most remarkably in the sequel of papers [5,9,10,30].
Boumal et al.
[9,10] showed that ( BM ) has no spurious 2-critical points when τ ( p ) >m , assuming that the feasible set M is a smooth manifold and that the cost matrix C is generic .
Hence, the Burer-Monteiro method converges to the global optimal solution as long as the cost C is outside a certain “bad” set, which has measure zero.
However, this analysis only applies to the limit points.
The follow up works [5] and [30] provide finite-iteration guarantees, though for values of p larger than the
Barvinok-Pataki bound.
They introduce a smoothed analysis [32] setting in which the cost matrix
C is subject to a small random perturbation of magnitude σ .
Such a perturbation ensures that
C stays outside the “bad” region, so the limit points are globally optimal.
They then argue that certain notions of approximate criticality for ( BM ) guarantee approximate optimality for ( SDP ) when τ ( p ) ≳ 92 m log(Ω( √ n/σ )) .
Despite the earlier work, the question of whether the Burer-Monteiro method can be used to completely solve problem ( SDP ) in polynomial time has not been yet resolved.
The works [5,30] do not provide end-to-end polynomial time guarantees and make structural assumptions on the problem, as will be further discussed in the related work section.
In particular, finding a feasible solution to ( SDP ) that could serve as initial point is a complicated and practically relevant problem, but it has not been addressed before.
In addition, the lower bound on p is qualitatively wrong, as it is not only larger than the Barvinok-Pataki bound, but also gets worse as the magnitude σ of the perturbation decreases.
In order to fully solve ( SDP ), we propose a 2-stage procedure.
First, solve ( BM ls ) to either get a point Y 0 that is approximately feasible, i.e,.
A ( Y 0 Y T 0 )
≈ b , or to prove that no such point exists.
Second, solve ( BM ) to get an approximately optimal point, starting with point Y 0 from the first stage.
We prove that this scheme is successful
when τ ( p ) >
(1+ η ) m , for an arbitrary constant η > 0 .
The next theorem provides polynomial time guarantees for the second stage.
The complexity is with respect to the setting of a smoothed analysis, in which the cost matrix C is perturbed.
Theorem 1 (( SDP ) in polytime) .
Let p such that τ ( p ) >
(1+ η ) m , for a fixed η > 0 .
Apply a random perturbation of magnitude σ to the cost matrix C .
Assume that M is compact and smooth (LICQ holds).
Solve ( BM ) using a constrained optimization method with 2nd-order guarantees (e.g., Theorem 4) initialized at an approximately feasible point Y 0 .
Then, after poly( n, σ − 1 ) iterations, we get a point Y such that Y Y T is approximately optimal for ( SDP ) with high probability.
A more detailed version of this result, with explicit optimality and complexity constants, appears in Theorem 9.
Note that our bound on p does not depend on the magnitude σ of the perturbation, as opposed to earlier works.
This improved bound relies on a technical novelty of this paper, namely, using sophisticated tools from real algebraic geometry for concentration bounds.
Indeed, we characterize the presence of spurious approximately critical points through tubular neighborhoods of algebraic varieties, and make use of effective bounds for the volume of such tubes [4,23,27].
On the other hand, [9,10] uses simple dimension arguments, while [5,30] rely on random matrix theory.
The following theorem provides polynomial time guarantees for the first stage.
We again consider a smoothed analysis setting, but this time perturbing the matrices A i that define the linear map A .
Theorem 2 (( SDP ls ) in polytime) .
Let p such that τ ( p ) >
(1+ η ) m , for a fixed η > 0 .
Apply a random perturbation of magnitude σ to the constraint map A .
Assume that the sublevel sets { Y : ∥A ( Y Y T ) − b ∥≤ α } are compact.
Solve ( BM ls ) using an unconstrained optimization method with 2nd-order guarantees (e.g., [15,16]).
Then, after poly( n, σ − 1 ) iterations, we get a point Y such that Y Y T is approximately optimal for ( SDP ls ) with high probability.
A more detailed version of this result, with explicit optimality and complexity constants, appears in Theorem 10.
[Equation]
Theorems 1 and 2 together provide end-to-end guarantees for solving ( SDP ), with polynomial smoothed analysis complexity.
The main assumptions are that p is slightly above the Barvinok-Pataki bound, while the feasible set is compact and smooth.
In the general case, we require that both C, A are slightly perturbed.
If a feasible point of the ( SDP ) is known, then Theorem 1 alone is enough.
We point out that problem ( SDP ls ) is a matrix sensing problem which is of interest in its own right, independent of its connection to SDP feasibility.
The guarantees from Theorem 2 apply even if the optimal value of ( SDP ls ) is strictly positive (i.e., there is no X ⪰ 0 with A ( X )= b ).
There are earlier works [6,22,26] proving global optimality guarantees for the nonconvex problem ( BM ls ), but they all rely on the restricted isometry property (RIP).
To the best of our knowledge, Theorem 2 provides the first global guarantees for ( BM ls ) that do not rely on RIP.
Related work
We first discuss ( SDP ).
Early work by Burer and Monteiro [12] and by Journée et al.
[25] gave strong indications that ( BM ) has no spurious critical points above the Barvinok-Pataki bound.
A formal proof was given by Boumal et al.
[5,9] under a genericity assumption.
Subsequent work by
Bhojanapalli et al.
[5] and Pumir et al.
[30] investigated the case of approximately critical points under smoothed analysis.
The Barvinok-Pataki bound was recently shown to be optimal up to lower order terms for general SDPs [34], though for structured families of SDPs a smaller rank might suffice [2,28].
The paper [19] extended the results from [9] to SDPs with multiple PSD constraints.
The works [5, 30] are the closest to this paper, but they do not provide end-to-end polynomial guarantees for solving general SDPs (nor it is claimed).
In particular, neither of them discuss initialization or feasibility.
The paper [30] relies on Riemannaian optimization, which is only practical for very special SDPs (like the maxcut SDP).
In the general case, the retraction operator is computationally intractable.
The paper [5] relies on a penalized version of ( SDP ).
It has the crucial catch that they require a preprocessing step that in general is as hard as solving another SDP.
Since solving an auxiliary SDP is required, it fails to explain why the BM method improves over general
SDP solvers.
In addition, the optimality guarantees in [5] are with respect to the solution of the penalized problem.
Translating their results into polynomial time guarantees with respect to the original SDP is nontrivial due to interdependence between the penalty parameter, the perturbation magnitude, and the target precision 1 .
In contrast, our paper does provide end-to-end polynomial time guarantees, and even better, our results hold down to the Barvinok-Pataki bound.
Problem ( SDP ls ) has been well studied in the matrix sensing community.
Bhojanapalli et al.
[6] showed that ( BM ls ) has no spurious local minima under the RIP condition, and they also provided polynomial time guarantees.
Similar results have been derived later, e.g., [22, 26].
Note that RIP is a very strong assumption about the condition number of the linear map A , particularly since the
RIP constant needs to be small [35].
In contrast, our result in Theorem 2 simply assumes a small perturbation around a worst-case instance A .
The comparison of smoothed analysis with RIP is akin to assuming that a matrix has a nonzero smallest singular value (and getting bounds that depend on inverse polynomials on this quantity), rather than assuming its condition number is close to one.
Critical points in nonlinear programming
Unconstrained case.
Consider the optimization problem
[Equation]
with f : R n → R twice continuously differentiable.
A vector y ∈ R n is a 2nd-order critical point for ( P un ), abbreviated 2-critical, if ∇ f ( y ) = 0 , ∇ 2 f ( y ) ⪰ 0 .
Practical optimization algorithms cannot obtain a solution satisfying these equations exactly.
Hence, we relax these conditions.
Definition 1.
Given ε 1 , ε 2 ∈ R + , a vector y is ( ε 1 , ε 2 ) - approximately 2-critical (AC) for ( P un ) if:
[Equation]
Several algorithms for unconstrained optimization with provable convergence guarantees are known.
Recent work has focused on deriving algorithms with finite-time guarantees.
In particular, the trust region method computes an ( ε 1 , ε 2 ) -AC point in O (max { ε − 2
ε − 1 2 , ε − 3 2 } ) iterations [16], and the adaptive regularization with cubics (ARC) method takes O (max { ε − 2 1 , ε − 3 2 } ) iterations [15].
We formally state the result for the ARC method.
Theorem 3 ( [15]) .
Assume that there exists α > 0 such that a point y 0 with f ( y 0 )
≤ α is known, and the functions f, ∇ f, ∇ 2 f are uniformly bounded and Lipschitz continuous on the set { y : f ( y ) ≤ α } .
The ARC method initialized at y 0 requires O (max { ε − 2 1 , ε − 3 2 } ) iterations to produce an ( ε 1 , ε 2 )
-AC point y .
Furthermore, each iteration requires O (1) evaluations of f and its derivatives.
Constrained case.
Consider the nonlinear program
[Equation]
with f : R n → R , h : R n → R m twice continuously differentiable.
The Lagrangian function is
L ( y, λ )=
f ( y )+ λ · h ( y ) .
A vector y ∈ R n is a 2-critical point if there are multipliers λ ∈ R m satisfying some first order and second order optimality conditions.
As before, for practical optimization we need to consider a notion of approximate criticality.
Definition 2.
Given ε = ( ε 0 , ε 1 , ε 2 ) ∈
R 3+ , γ ∈ R + , a pair ( y, λ ) is ( ε , γ ) - approximately feasible approximately 2-critical (AFAC) for ( P con )
if:
[Equation]
In the constrained case, for a local minima to be a critical point we need some regularity conditions.
One such condition is the linear independence constraint qualification (LICQ), that states that ∇ h ( y ) is full rank.
This is equivalent to M being smooth at y for the case of complete intersections (i.e., codim M = m ).
We next introduce a quantitative version of LICQ.
Definition 3.
For ϱ> 0 , say that ϱ -LICQ holds at y if the singular values of ∇ h ( y ) are at least ϱ .
Several local optimization methods for ( P con ) with provable convergence guarantees are known.
In particular, augmented Lagrangians [1] and trust-region methods [20, §15.4] converge to 2-critical points.
Recent work has focused on finding algorithms with finite-time guarantees.
The complexity of computing approximately 1-critical points was studied in, e.g., [7,17,18,21].
As for approximately 2-critical points, we are only aware of [14,31].
But both papers use a different 2nd-order condition, which is not easy to translate into our setting.
Nonetheless, in Theorem 4 below we show that AFAC points can be computed in polynomial time.
The proof of this theorem is in
Appendix A, and relies on a variant of the method from [14].
To the best of our knowledge, this is the first polynomial time bound for computing 2-critical points using the standard notions of criticality.
Theorem 4.
Assume that there exist β, ϱ ∈ R + such that a point y 0 in the set M β :=
{ y : ∥ h ( y ) ∥≤ β } is known, the functions f, ∇ f, ∇ 2 f , ∇ h
i , ∇ 2 h
i ( i ∈ [ m ] ) are uniformly bounded and Lipschitz continuous on M β , and ϱ -LICQ holds at all y ∈ M β .
Let ε = ( ε 0 , ε 1 , ε 2 )
∈
R 3+ , γ ∈ R + such that
[Equation]
) where R λ := 2+2 ϱ − 1 L f , L f := max y ∥∇ f ( y ) ∥ .
There is an algorithm that, when initialized at y 0 , requires O (max { ε − 2 0 ε − 2 1 , ε − 3 0 ε − 3 2 } ) evaluations of f, h and their derivatives to produce an ( ε , γ )
-AFAC
pair ( y, λ ) , with ∥ λ ∥ ≤ R λ .
Remark.
Theorem 4 assumes that ϱ -LICQ holds everywhere in M β , an assumption which can only be guaranteed when β is very small.
Hence the initial point y 0 must be approximately feasible.
Optimality of critical points of ( BM )
In this section we will show that problem ( BM ) has no spurious approximately feasible approximately critical (AFAC) points with high probability.
This means that any AFAC point of ( BM ) is approximately optimal for ( SDP ).
We consider a smoothed analysis setting in which the constraint variables A , b are fixed, and the cost matrix C is subject to a small random perturbation.
We will restrict our attention to AFAC pairs ( Y, λ ) of bounded norm.
Hence, we assume that ∥ Y ∥≤ R Y and ∥ λ ∥≤ R λ for some fixed constants R Y , R λ .
A crucial step toward our theorem is a geometric characterization of the spurious AFAC points in terms of tubes around algebraic varieties.
We then take advantage of known effective bounds for the volume of such tubes [4,27].
From now on, we use the Frobenius norm for all matrices.
3.1 Spurious approximately critical points
The optimality conditions for ( SDP ) are well known:
A ( X )=
b, S ( λ ) X =0 , X ⪰ 0 , S ( λ ) ⪰ 0 , where
S ( λ ) is the following slack matrix
[Equation]
and A ∗ : R m → S n , λ (cid:55)→ (cid:80) i λ i
A i is the adjoint of A .
We now relax the optimality conditions.
Definition 4.
Let ε =( ε 0 , ε 1 , ε 2 ) ∈ R 3+ .
A pair ( X, λ ) is ε - approximately optimal for ( SDP )
if:
[Equation]
It is known that an ε -approximately optimal solution is at distance O ( ∥ ε ∥ ) from an optimal solution under nondegeneracy assumptions [33].
We can also give a simple bound on the optimality gap.
Lemma 1.
If ( ¯ X, ¯ λ ) is ε -approximately optimal for ( SDP ) then
[Equation]
Proof.
The lemma follows from the following equations:
[Equation]
We proceed to problem ( BM ).
This is a special instance of the nonlinear program ( P con ) with f ( Y ) =
C • Y Y T and h ( Y )
=
A ( Y Y T ) − b .
The Lagrangian function is L ( y, λ ) =
S ( λ ) • Y Y T + b T λ .
The criticality conditions for ( BM ) are obtained by specializing (2).
Definition 5.
Let ε =( ε 0 , ε 1 , ε 2 ) ∈ R 3+ , γ ∈ R + .
A pair ( Y, λ ) is ( ε , γ
[Equation]
-AFAC for ( BM ) if:
[Equation]
We are ready to formalize the concept of spurious critical points.
Definition 6.
Let R Y , R λ ∈ R + be fixed and let ( Y, λ )
such that ∥ Y ∥ ≤ R Y and ∥ λ ∥ ≤ R λ .
Given ε = ( ε 0 , ε 1 , ε 2 ) ∈
R 3+ , γ ∈ R + , ( Y, λ ) is spurious ( ε , γ ) -AFAC
if ( Y, λ ) is ( ε , γ ) -AFAC for ( BM ) but ( Y Y T , λ ) is not ε
′ -approx.
optimal for ( SDP ) for ε ′ :=(
ε 0 , R Y ε 1 , ε 2 ) .
A pair ( Y, λ ) is spurious exactly critical if the above holds for ε = 0 , γ = 0 .
3.2 Statement of the theorem
We present the main result of this section.
Let A , b be fixed, and let C be obtained from a random perturbation of magnitude σ around some fixed ¯ C .
Concretely, C is uniformly distributed on the
Frobenius ball B σ ( ¯ C ) ⊂
S n of radius σ centered at ¯ C .
Consider the set C ε ,γ ⊂ S n , consisting of all cost matrices for which there is a spurious AFAC point:
[Equation]
We show that if τ ( p ) > m , then the probability Pr[ C ∈ C ε ,γ ] → 0 as the ratio ε 1 /γ → 0 .
Theorem 5 (critical ⇒ optimal) .
Let p such that τ ( p ) >m .
Let ε ∈ R 3+ , γ ∈ R + .
Let C be uniformly distributed on the Frobenius ball B σ ( ¯ C ) .
Then
[Equation]
where δ := ε 1 ∥A∥ /γ and κ := R λ ∥A∥ , provided that δ < σ/ 4 n 3 .
The following corollary shows that when the stronger condition τ ( p ) > (1+ η ) m holds, where η is a fixed constant, then we can derive a high probability bound while maintaining δ polynomially bounded.
Its proof is a straightforward manipulation.
Corollary 1.
Consider the setup from Theorem 5.
Assume that τ ( p ) ≥ (1+ η ) m + ηt and δ ≤ (1 / 3 κ ) 1 /η ( σ/ 4 n 3 ) 1+1 /η for some η, t > 0 .
Then Pr[ C ∈ C ε ,γ ] ≤ 4 e ( σ/ 12 κn 3 ) t .
3.3 Tubes around varieties
Our proof of Theorem 5 relies on a geometric characterization of the set C ε ,γ .
Such characterization is known for the case ε = 0 , γ = 0 , corresponding to exactly critical points.
It was shown in [10], see also [19], that the existence of a spurious exactly critical point implies that C lies in a certain algebraic variety of S n , as follows:
[Equation]
where S nn − p :
= { X : rank X ≤ n − p } is a variety of bounded rank matrices, and Im A ∗ is the linear space spanned by A 1 , . . .
, A m .
Hence, we have that C 0 , 0 ⊂ S nn − p
+ Im A ∗ .
When τ ( p ) > m , the variety S nn − p
+ Im A ∗ is properly contained in S n .
It follows that C 0 , 0 has measure zero and hence, for generic C , there are no spurious exactly critical points.
We show below that for approximately critical points the situation is analogous, except that we need to consider a tubular neighborhood around the variety S nn − p
+Im A ∗ .
Definition 7.
Given W ⊂ S n , δ ∈ R + , let tube δ W := { X ∈ S n : ∃ W ∈ W s.t.
∥
X − W ∥ ≤ δ } .
Proposition 1.
Let δ :=
ε 1 ∥A∥ /γ and B λ := { λ ∈ R m : ∥ λ ∥≤ R λ } .
Then
C ε ,γ ⊂ tube δ ( S nn − p )
+ A ∗ ( B λ ) .
The next lemma is the key ingredient for Proposition 1.
It shows that if Y is a spurious AFAC point, then its smallest singular value σ p ( Y ) is bounded from below.
This is a generalization of a previous result by Burer and Monteiro [12] about exactly critical points.
An analogue of this lemma is found in [30, Lem.3.2] for the case where ε 0 = 0 .
Lemma 2.
Let Y be an ( ε , γ ) -AFAC point of ( BM ) .
If σ p ( Y ) ≤ γ/ ∥A∥ , then Y Y T is ε ′ - approximately optimal for ( SDP ) , with ε ′ :
= ( ε 0 , R Y ε 1 , ε 2 ) .
Proof.
Let ( Y, λ ) satisfy (5), and let us show that ( Y Y T , λ ) satisfies (4).
The first three conditions in (4) are easy to check.
We proceed to show the last one: S ( λ ) ⪰− ε 2 I n .
Given a unit vector u ∈ R n , we need to show that u T S ( λ ) u ≥− ε 2 .
There is a unit vector z ∈ R p such that ∥ Y z ∥ = σ p ( Y ) .
The matrix U := uz T satisfies ∥ U ∥
=1 and
[Equation]
Then ∥A ( UY T ) ∥≤ γ , so by (5b)
we have
[Equation]
Proof of Proposition 1.
Let C ∈ C ε ,γ , so there is a spurious ( ε , γ )
-AFAC pair ( Y, λ ) .
By Lemma 2, we must have σ p ( Y ) >γ/ ∥A∥ .
Let S :
= S ( λ ) , and note that ∥ S Y ∥≤ ε 1 = γ δ/ ∥A∥ by (5a).
Then dist
[Equation]
6) so that C = S ( λ )+
A ∗ ( λ ) ∈ tube δ ( S nn − p )
+ A ∗ ( B λ ) .
By Proposition 1, the set of “bad” cost matrices C ε ,γ is contained in a tube around a variety.
To prove Theorem 5, we need an upper bound on the probability mass of this tube.
The computation of integrals over tubes has a long history in differential geometry [23].
Effective bounds for these integrals were shown in [4,13,27]; they were used for smooth analysis in the first reference.
We will use the following recent bound.
Theorem 6 ( [4, Thm.1.1]) .
Let V ⊂ R k be a real variety of codimension c defined by polynomials of degree at most D .
Let x be uniformly distributed on the Euclidean ball B σ (¯ x ) ⊂ R k .
If the ratio σ/δ ≥ (4 D +1)( k − c ) , then we have that Pr[ x ∈ tube δ ( V )] ≤ 4 e (4 kDδ/σ ) c ,
Proof of Theorem 5.
Let B λ ⊂
R m be the ball of radius R λ centered at zero.
Consider an ϵ -net N of B λ , where ϵ :=
δ/ ∥A∥ .
It is known that (3 R λ /ϵ ) m
= (3 κ/δ ) m points suffices.
Observe that
[Equation]
Recall that S nn − p is a variety of codimension τ ( p ) defined by equations of degree n − p
+1 .
For any ℓ ∈ N , Theorem 6 gives
[Equation]
Finally, the union bound gives
[Equation]
Optimality of critical points of ( BM ls )
In this section we will show that ( BM ls ) has no spurious approximately critical (AC) points with high probability.
This means that any AC point of ( BM ls ) is approximately optimal for ( SDP ls ).
Hence, we can solve the least squares problem ( SDP ls ) to global optimality using the Burer-Monteiro method.
Note that if ( SDP ) is feasible, then the optimal value of ( SDP ls ) is zero.
Nevertheless, the results of this section apply to an arbitrary instance of ( SDP ls ), even if the optimal value is nonzero.
Appendix B contains the proofs of the lemmas, propositions, and theorems from this section.
We proceed to formalize the notion of spurious critical points.
The optimality conditions for the convex problem ( SDP ls ) are: S ( X ) X
= 0 , X ⪰ 0 , S ( X ) ⪰ 0 , where S ( X ) is the gradient of the least squares objective function f ( X ) =
∥A ( X )
− b ∥ 2 :
[Equation]
We call X approximately optimal if either A ( X ) ≈ b or the above conditions are almost satisfied.
Definition 8.
Let ε =( ε 0 , ε 1 , ε 2 ) ∈ R 3+ .
A matrix X ⪰ 0 is ε - approximately optimal for ( SDP ls ) if ∥A ( X ) − b ∥ ≤ εor ( ∥ S ( X ) X ∥ ≤ ε 1 and S ( X ) ⪰ − ε 2 I n ) .
The following lemma bounds the optimality gap for the second case in (8).
Lemma 3.
Let ¯ X ∈ S n + such that ∥ S ( ¯ X )
¯ X ∥ ≤ ε 1 , S ( ¯ X ) ⪰ − ε 2 I n .
Then
[Equation]
The Burer-Monteiro problem ( BM ls ) is a special instance of the unconstrained optimization problem ( P un ).
The criticality conditions for ( BM ls ) are obtained by specializing (1).
Definition 9.
Let ( ε 1 , ε 2 ) ∈
R 2+ .
Y ∈ R n × p is ( ε 1 , ε 2 ) - approximately 2-critical (AC) for ( BM ls ) if
[Equation]
We are ready to formalize the concept of spurious critical points.
Definition 10.
Let R Y ∈ R
+ be fixed and let Y with ∥ Y ∥ ≤ R Y .
Given ε =
( ε 0 , ε 1 , ε 2 ) ∈ R 3+ , Y is spurious ε -AC
if Y is ( ε 1 , ε 2 ) -AC for ( BM ls ) but Y Y T is not ε
′ -approx.
optimal for ( SDP ls ) for ε
= ( ε 0 , R Y ε 1 , 5 ε 2 ) .
Y is spurious exactly critical if the above holds for ε = 0 .
We assume here that b ∈ R m is fixed, and we vary the linear map A .
We identify A with the tuple of matrices ( A 1 , . . .
, A m ) , and hence view it as an element of the Euclidean space ( S n ) m .
We assume that A is uniformly distributed on the Euclidean ball B σ ( ¯ A ) ⊂ ( S n ) m of radius σ centered at ¯ A .
Consider the set A ε ⊂ ( S n ) m , consisting of all A for which there is a spurious AC point:
A ε := {A ∈ ( S n ) m : ∃ Y a spurious ε -AC point } .
We show that if τ ( p ) > m , then the probability Pr[ A ∈
A ε ] → 0 as the ratio ε 1 / √ ε 2 → 0 .
Theorem 7 (critical ⇒ feasible) .
Let p such that τ ( p ) >m , and ε ∈ R 3+ .
Let A be uniformly distributed on the Euclidean ball B σ ( ¯ A ) .
Then
[Equation]
where δ := ε 1 R A / √ ε 2 , κ := 2( R A R 2 Y + ∥
b ∥ ) R A , and R A :=
∥ ¯ A∥ + σ , provided that δ < σε 0 / 4 n 3 m .
As before, we can derive a high probability bound when τ ( p ) >
(1+ η ) m .
Corollary 2.
Consider the setup from Theorem 7.
Assume that τ ( p ) ≥ (1+ η ) m + ηt , ε 0 ≥ ρσ and δ ≤ (1 / 3 κ ) 1 /η ( ρσ 2 / 2 n 3 m ) 1+1 /η for some η, t, ρ > 0 .
Then Pr[ A ∈
A ε ] ≤ 4 e ( ρσ 2 / 6 κn 3 m ) t .
As in Section 3, our proof of Theorem 5 relies on a geometric characterization of the spurious AC points.
First consider the simpler case of spurious exactly critical points ( ε = 0 ).
The following equation is a consequence of our analysis:
[Equation]
A ∗ is nontrivial .
This implies that the set A 0 has measure zero when τ ( p ) >m .
In order to handle the case ε > 0 we rely again on tubular neighborhoods, as stated next.
Proposition 2.
Let δ :=
ε 1 R A / √ ε 2 , D λ := { λ ∈ R m : 2 ε 0 ≤∥ λ ∥≤ R λ } , R λ := 2( R A R 2 Y + ∥
b ∥ ) .
Then for A ∈
A ε we have that tube δ ( S nn − p ) ∩ A ∗ ( D λ ) ̸
= ∅ .
Polynomial time guarantees
We are ready to derive polynomial time guarantees for the Burer-Monteiro method.
The proof of
Theorem 1 relies on two facts: (i) the method from Theorem 4 finds an AFAC point of ( BM ) in polynomial time; (ii) Corollary 1 shows that an AFAC point of ( BM ) is approximately optimal for ( SDP ) with high probability.
Hence, the Burer-Monteiro method solves ( SDP ) in polynomial time in the setting of smooth analysis.
The proof of Theorem 2 is analogous, except that it relies on
Theorem 3 and corollary 2.
Appendix C provides explicit complexity estimates for solving ( SDP ) and ( SDP ls ) using the BurerMonteiro method, indicating the precise optimality guarantees of the output.
Experiments
We present some experimental results to complement our theorems.
We rely on the library NLopt [24] (MIT license) to solve the nonlinear problem ( BM ).
Concretely, we use the augmented Lagrangian method (ALM) implemented in NLopt (which is based on [8]), and we use the preconditioned truncated Newton method as the subroutine.
We also rely on the commercial solver Mosek for SDPs.
For our first experiment we consider a random SDP with a planted solution.
More precisely, we take a matrix X 0 ∈ S 50 , X 0 ⪰ 0 of rank r ∈ { 4 , 7 , 12 } , and generate a random SDP for which X 0 is an optimal solution.
To do so, we generate m := τ ( r ) random constraints that are satisfied at X 0 , and then find a cost matrix C in the normal cone of X 0 .
We generate 100 random SDPs for each r
We solve these SDPs with the Burer-Monteiro method, using different values of p (the rank of Y ) and random initializations.
The initial points are matrices with i.i.d.
normalized Gaussian entries.
Figure 1 shows the percentage of experiments solved correctly for each value of r and p .
We regard an experiment as “correct” if the criticality conditions from (4) are satisfied.
Figure 1 illustrates that there is a sharp phase transition at the Barvinok-Pataki bound p = r .
Above the Barvinok-Pataki bound, the Burer-Monteiro method solves most instances.
Beneath the BarvinokPataki bound, it is not just that our techniques stop working, but that the method itself usually fails.
Note that previous work [5, 30] required p to be larger than 3 r/ √ 2 .
However, we see from experiments that the phase transition is much sharper than this.
Observe that, even for p ≥ r , the number of experiments solved is always below 100% .
Nonetheless, the number of bad instances seems to get smaller for larger values of p .
This agrees with our result from Theorem 5.
For the second experiment we fix the parameters n := 50 , m := 28 , and p :=
r := 7 .
Among the 100 random SDPs considered in Figure 1, we take an instance for which the Burer-Monteiro problem performed badly.
We then perturb this seemingly bad SDP by adding varying amounts of noise σ .
For each noise level we solve 70 random experiments, in which both the perturbations and the initializations are random.
The perturbations consist in adding to each matrix a random matrix with i.i.d.
Gaussian entries scaled by the noise level.
Figure 2 summarizes the results obtained.
[Figure]
Figure 2a shows the percentage of instances for which the Burer-Monteiro method succeeded for each noise level.
The percentage is with respect to the random perturbation and the random initialization.
For the unperturbed problem ( σ =0 ) the method succeeds only for 52% of the random initializations.
This percentage increases as we add noise.
For σ = 0 .
2 the method succeeds 87% of the time.
Figure 2b shows the progress of the algorithm for the cases σ = 0 and σ = 0 .
The progress is measured in terms of the residual of the criticality conditions.
The figure shows the mean and standard deviation of the residual for each iteration of ALM.
Figures 2a and 2b illustrate the advantages of smoothing a badly behaved SDP.
Theorem 9 predicts that the complexity of the algorithm is proportional to σ − d for some exponent d .
So for a fixed number of iterations N , we should set the noise level proportional to N − 1 /d .
However, our bounds were shown for the algorithm from Theorem 4.
We do not know if they also apply to ALM.
Discussion
We study the Burer-Monteiro method for solving the problems ( SDP ) and ( SDP ls ).
We provide the first polynomial time guarantees, using a smoothed analysis setting.
Previous analyses for ( SDP ) did not provide end-to-end guarantees and made structural assumptions on the problem instance, while previous work for ( SDP ls ) relied on the RIP condition, a strong assumption.
The main technical novelty of this paper is the use of volumes of tubular neighborhoods of varieties for deriving concentration bounds.
This allows our results to hold even as p approaches the BarvinokPataki bound arbitrarily close.
Although the Barvinok-Pataki bound is optimal for general SDPs, there are better bounds for some special classes of SDPs.
An important open question is whether our results could be extended to allow for smaller values of p in those special classes.
The Burer-Monteiro method is not a single algorithm, but a family of algorithms, one for each local optimization method for solving ( BM ).
Similarly, our guarantees in Theorems 1 and 2 apply for any local method with 2nd order guarantees.
The only method we are aware that provably computes
AFAC points of ( BM ) efficiently is the one in Theorem 4.
It would be interesting to extend our results to classical optimization methods, such as ALM, which have robust implementations.
Acknowledgments and Funding Disclosure
Ankur Moitra was supported in part by a Microsoft Trustworthy AI Grant, NSF CAREER Award
CCF-1453261, NSF Large CCF-1565235, a David and Lucile Packard Fellowship, an Alfred P. Sloan
Fellowship and an ONR Young Investigator Award.
[-Also note that t 
-]
[-, where we used that ε 1 and γR λ are at most 14 ( R − 1 λ ε 
-]
[-Let λ := 
-]
[-• ∃ β ∈ R + such that: M β is compact, a point Y 
-]
[-+ and a matrix Y 
-]
[-such that M α is compact and Y 
-]
[-where ε ′ 
-]
[-= ρ σ , ε ′ 
-]
[-= ϵ 3 / 2 R Y , ε ′ 
-]
[-This means that ε ′′ 
-]