Non-convex online learning via algorithmic equivalence
Anonymous Author(s)
AffiliationAddress
Abstract
We study an algorithmic equivalence technique between nonconvex gradient descent and convex mirror descent.
We start by looking at a harder problem of regretminimization in online non-convex optimization.
We show that under certain geometric and smoothness conditions, online gradient descent applied to non-convexfunctions is an approximation of online mirror descent applied to convex functionsunder reparameterization.
In continuous time, the gradient flow with this reparameterization was shown to be exactly equivalent to continuous-time mirror descent by
Amid and Warmuth [4], but theory for the analogous discrete time algorithms isleft as an open problem.
We prove an O ( T 23 ) regret bound for non-convex onlinegradient descent in this setting, answering this open problem.
Our analysis is basedon a new and simple algorithmic equivalence method.
Introduction
Gradient descent is probably the simplest and most popular algorithm in convex optimization, withnumerous and extensive studies on its convergence properties.
For non-convex objectives, it is knownthat gradient descent does not necessarily converge to the global optimum, which is in general NPhard.
Thus recent research focuses on alternate objectives, such as finding first-order stationary pointsefficiently, or the importance of higher order stationary points, e.g. [19, 22, 1].
However, the study of global convergence of gradient descent for non-convex objectives is increasinglyimportant due to the fact that in practice, gradient descent and its variants can achieve zero error on ahighly non-convex loss function of a deep neural network.
To explain the success of modern deep learning it is thus important to understand why gradientdescent can converge to a global minimum for some highly non-convex functions.
Several importantresearch directions have stemmed from this motivation, including the study of optimization for deeplinear networks [20, 21, 6], nonconvex matrix factorization [7], provable convergence for neuralnetworks in the linearization regime of the neural tangent kernel [25, 12, 13, 2, 31, 18, 9], and more.
A recent promising approach, proposed in [4, 3], is to consider reparameterizing mirror descent asgradient descent.
In particular, [4] shows that a continuous-time online mirror descent with a convexloss can be written in an equivalent form of a continuous-time online gradient descent whose lossisn’t necessarily convex after reparameterization, but has significant structure.
[3] further explores this idea showing that after reparameterization, online gradient descent has the same worst-caseregret bound as the exponentiated gradient algorithm.
Both [4] and [3] provide a new prospect onunderstanding the convergence of non-convex gradient descent.
1.1 The Amid-Warmuth Question
The works of [4] [3] are limited in two respects: either they are restricted to the continuous-timegradient-flow setting, which is impractical, or they address specific algorithms (Winnow, Hedge,
EG for linear regression) with a relative entropy divergence.
Amid and Warmuth pose the questionof extending the reparameterization idea to the general online convex optimization as well as theconditions where this can be obtained.
Solving the question posed in [4] would give an answer torealistic scenarios and algorithms, with precise regret and running time bounds, for certain non-convexoptimization problems that can be solved to global optimality using gradient descent.
1.2 Statement of Results
Our study starts from the more general problem of regret minimization in online convex optimization.
Specifically, we study the reparameterization of online mirror descent (OMD) as online gradientdescent (OGD) [29] for general online convex optimization.
We show that under certain geometricand smoothness conditions, a non-convex reparameterized OGD algorithm closely match the originalconvex OMD algorithm.
Quantitative bounds on this matching allow us to prove that the OGDalgorithm after reparameterization achieves an O ( T 23 ) regret bound.
This bound is discrete-time andfairly general, applying broadly to smooth, bounded mirror descent regularizers rather than anythingmore specific.
We also provide sufficient conditions for nonconvex losses such that OGD implicitlyhas an equivalent mirror descent regularizer and hence will have O ( T 23 ) regret.
Our analysis relies on a simple and new algorithmic equivalence technique, which may be ofindependent interest.
The key step is to show that, from the same starting point, the outputs of
OMD and reparameterized OGD are very close to each other.
Considering the OGD update as aperturbed version of OMD, and combining with the fact that OMD can tolerate bounded (adversarial)perturbation per trial, allows us to prove an O ( T 23 ) regret bound for the OGD update.
This answers the open question raised in [4], generalizing from continuous-time gradient flow andspecialized algorithms, to general online convex optimization.
The regularization term of MD isalso generalized from relative entropy to any strongly convex function.
Further, for discrete-timereparameterization, our result extend to arbitrary Lipschitz convex loss functions, as opposed tocustom analysis for specific settings of expert prediction, linear prediction, and linear regression [3].
1.3 Paper Structure
Section 2 introduces the setting and sketches out background work in the continuous setting.
Section 3provides our main result, which is subsequently proven in Section 4.
Section 5 provides analysis inthe opposite direction, showing the existence of an equivalent mirror descent regularizer for somenonconvex gradient descent problems.
Some analysis is deferred to the Appendix.
1.4 Related Works
Gradient descent is the simplest and one of the most popular algorithms in convex optimization.
Extensive studies have been conducted to show the convergence of GD to the global minimizer inboth the stochastic optimization setting [23], and the online convex optimization setting [30].
For thenon-convex setting, however, finding the global minimum is NP-hard and the study mostly focuses onthe convergence to first-order stationary points instead.
The well-known descent lemma guaranteesthe convergence of non-convex GD when the objective is smooth, and dropping smoothness ispossible if other conditions are introduced [10].
Many works consider avoiding saddle points; [22]shows that GD converges to local minimizers
a.s. if all saddle points are strict while [19] proposesperturbed gradient descent to escape saddle points.
It’s also shown that even in the online setting,chasing first-order stationary points is possible [17].
Mirror descent, first introduced by [24], generalizes the gradient descent method in the sense that itcan adapt to the ’geometry’ of the optimization problem [11].
Its analogue in the online setting, onlinemirror descent also achieves tight regret bounds [26] and without projection is shown to be equivalentto the classical regularized follow-the-leader (RFTL) method with constant step-size.
Though it’snatural to think of mirror descent as changing the geometry of the optimization objective in gradient descent [15], the idea to explain non-convex GD by a corresponding convex MD is explored onlyrecently.
The most relevant works to ours are [4] and [3], which prove the equivalence between continuous time
OMD and OGD after reparameterization, and special discrete-time algorithms when the regularizationis the relative entropy, respectively.
Our results extend [4] and [3] to general discrete-time OCO withgeneral regularization.
Preliminaries2.1 Notation
For a function f : R d → R d , we use the notation J f ( x ) :
R d → R d to denote the Jacobian off at x .
We use the notaion ⊙ to represents an element-wise product of vectors.
Given a strictlyconvex function R : R d → R d , we denote the Bregman divergence as D R ( x ∥ y ) :
= R ( x ) −
R ( y ) − ∇ R ( y ) ⊤ ( x − y ) .
For a convex set K and strictly convex regularizer R , we use Π R K ( x ) :=
arg min y ∈K D R
( y ∥ x ) to denote Bregman projection, with shorthand Π K :
= Π ∥·∥ 2 K for Euclideanprojection.
Given a positive-definite matrix M ∈ R d × d , we define the norm ∥ x ∥ M := √ x ⊤ Mx .
Weuse the notation, B p :=
{ x ∈ R d : ∥ x ∥ p ≤ 1 } for an ℓ p ball and B + p to denote the intersection of theℓ p ball and the positive orthant.
2.2 Online Convex Optimization
We consider the online convex optimization (OCO) problem.
At each round t the player
A chooses x t ∈ K
where K ⊂ R d is some convex domain, then the adversary reveals loss function f t ( x ) and player suffers loss f t ( x t ) .
The goal is to minimize regret:
[Equation]
A [Equation] f
[Equation]
The player is allowed to get access to the (sub-)gradient ∇ x f t ( x t ) as well.
2.3 Reparameterizing continuous-time mirror flow as gradient flow
In the continuous-time setting, we have exact conditions from [4] where the trajectory of mirror-flowexactly coincides with that of gradient descent in a reparameterized space.
Concretely, mirror flow onf : R d → R is defined by the following ordinary differential equation (ODE):
[Equation]
In Theorem 2 of [4], this update is shown to be equivalent to the following gradient flow ODE withx ( t ) =
q ( u ( t )) :
[Equation]
if the mirror descent regularizer R and reparameterization function q satisfy [ ∇ 2 R ( q ( u ))] − 1 =
J q ( u ) J q ( u ) ⊤ .
This relationship between the Hessian of the OMD regularizer and the reparameterization function assures that, up to second order factors in ∥ u − v ∥
[Equation]
(3) so the geometry induced by R is approximately transformed into a Euclidean geometry by q − 1 .
Because higher order factors vanish in continuous time, this assures that mirror flow and thisreparameterized gradient flow coincide as desired.
2.4 Discretizing the updates
Unfortunately, continuous-time updates cannot be implemented in practice so we must rely ondiscretization.
Using the forward Euler scheme for discretization of (1), yields the mirror descentalgorithm.
After discretization, the exact equivalence of mirror descent and reparameterized gradientdescent no longer holds as higher order factors in (3) are relevant.
This motivates the open problem of[4] of finding general conditions under which the discretizations of these continuous time trajectoriesclosely track each other.
We tackle extending these results through the online setting described above.
The Online Mirror Descent (OMD) algorithm has the following update on x t :
[Equation]
where R : K → R is a 1-strongly convex regularization over the domain 1 , D R is the Bregmandivergence, and y 1 is initialized to satisfy ∇ R ( y 1 )
= 0 .
Another equivalent interpretation of OMD is
[Equation]
The most important special case is online gradient descent (OGD) x
t +1 =
Π K ( x t − η ∇ f t ( x t )) , whichis OMD with D R ( x || x t ) = 12 ∥ x − x t ∥ 22 .
In this paper we consider reparameterizing general OMDupdate as a simple OGD update.
We introduce assumptions of the regularizer, reparameterization,and losses in the sequel.
2.5 Assumptions
Assumption 1.
There exists a convex domain K ′ ⊂ R d and a bijective reparameterization functionq : K ′
→ K satisfying [ ∇ 2 R ( x )] − 1 =
J q ( u ) J q ( u ) ⊤ , with x = q ( u ) .
We also make the following smoothness/Lipschitz assumptions:
Assumption 2.
Let G > 1 be a constant.
We assume q is G -Lipschitz, and the 1 - strongly convexregularization R is smooth with its first and third derivatives upper bounded by some constant G .
The first and second derivatives of q − 1 are bounded by G .
Furthermore, assume that for all x ∈ K D R ( x ∥· ) is G -Lipschitz over K .
Finally, we assume loss functions have bounded gradients and the convex domain has boundeddiameter with respect to the Bregman divergence.
Assumption 3.
The loss functions f t have gradients bounded by ∥∇ f t ( x ) ∥ 2 ≤ G F for all x ∈ K .
The diameter of K , sup x,y ∈K D R ( x ∥ y ) ≤ D .
Algorithm
We now provide our main result, a regret bound for Algorithm 2.
We note, Algorithm 1 and
Algorithm 2 are the (projected) forward euler discretizations of (1) and (2).
It’s tempting to directlyanalyze the regret of OGD with losses ˜ f t ( u ) =
f t ( q ( u )) .
The main barrier in doing so is that ˜ f
isn’tnecessarily convex.
However, we show that the OMD and OGD updates are in fact O ( η 32 ) -close toeach other, therefore the OGD update still suffers only sublinear regret.
1: Input: Initialization x 1 ∈ K , regularizer R .
2: for t = 1 , . . .
, T do 3:
Predict x t 4:
Observe ∇ f t ( x t ) 5:
Update y t
+1 = ( ∇ R ) − 1 ( ∇ R ( x t ) − η ∇ f t ( x t {+)) x t +
+}
= Π R K ( y t +1 ) 6: end for
1: Input: Initialization u 1 ∈ K ′ =
q − 1 ( K ) .
2: for t = 1 , . . .
, T do 3:
Predict u t 4:
Observe ∇ ˜ f t ( u t ) = ∇ f t ( q ( u t ))) 5:
Update v t +1 =
u t − η ∇ ˜
f t ( u t ) u t
+
=
Π K ′ ( v t +1 ) 6: end for
Remark 1.
The gradient oracle ∇ ˜
f t can also be calculated from an oracle of ∇ f t if we know thereparameterization function q .
Remark 2.
The reparameterized gradient descent uses Euclidean projection rather than Bregmanprojection.
This can potentially be an advantage if Euclidean projection can be computed efficientlyon q − 1 ( K ) (eg. via method like [28]), while Bregman projection is less efficient on K .
Theorem 3.
Under Assumptions 1-3, by setting η = T − 2 / 3 D 2 / 3 G − 10 / 3 G − 1 F the regret of Algorithm2 is upper bounded by O ( T 2 / 3 D 1 / 3 G 10 / 3 G F ) .
Proof of the theorem is in Section 4.
3.1 Examples
We now provide a few example settings.
Note that in all three settings, q are such that ˜ f ( u ) =
f ( q ( u ))can be nonconvex for a convex f .
Exponentiated Gradient using Quadratic Reparameterization One of the most popular algorithms for online learning over the filled-in simplex 2 K
= B +1 is the Exponentiated Gradient (EG)[8].
In this case, we show that OGD with quadratic reparameterization q ( u ) =
14 u ⊙ u has vanishingaverage regret.
We empirically verify that reparameterized GD iterates and EG iterates stay close ona toy problem in Figure 1.
In [3] custom analysis is provided for linear losses that achieves optimalregret, but our approach has a simpler and more general analysis.
If R ( x ) = (cid:80) di
=1 x i log( x i ) then
OMD is EG.
In this case, if x = q ( u ) , then
[Equation]
We note that for this example, Assumptions 2 will not hold as relative entropy is not sufficientlybehaved near the boundary of the simplex.
This can be handled by modifying K to be the smoothedsimplex where all weights are at least ε for some ε .
This is a fairly standard technique.
ε can bechosen such that the full regret in sublinear (see App. B).
Log Barrier with Exponential Reparameterization Consider the case of a log-barrier regularization R ( x ) = − (cid:80) di
=1 log( x i ) with K =
[ ε, 1] d .
It can readily be shown that the box constraint mapsto a box constraint.
Consider x = q ( u ) =
exp( u ) , where the exponential is elementwise.
We see
Assumption 1 is satisfied as
[Equation]
Tempered Bregman Divergences with Power Reparameterization A more recently studiedfamily of mirror descent regularizers interpolate between ℓ 2 regularization and negative entropyregularization [5].
Regularization for this family uses link function ∇ R ( x ) = log τ ( x ) = 11 − τ ( x 1 − τ − 1) , where τ is a temperature that can range between 0 and 1 .
As τ approaches 1 , the link functionapproaches the natural logarithm, and hence the update approaches EG, while τ = 0 corresponds toℓ 2 regularization.
Such divergences are amenable to reparameterization when K = B + p .
To see this, ignoring constant factors [ ∇ 2 R ( x )] − 1 = x τ for τ ∈ [0 , 1) .
Now if q ( u ) = u 22 − τ wherethe power is elementwise, then J q ( u ) =
diag ( u τ 2 − τ ) =
diag ( q ( u ) τ/ 2 ) =
diag ( x τ/ 2 ) as desired.
Now, we note that the reparameterization is a power, so B + p would get mapped to the B + 2 p 2 − τ , which isconvex as long as p > 1 − τ/ 2 .
3.2 Challenges
In some situations, we do not have a closed form solution to the differential equation describedby J q .
For example with the β -hypentropy regularizer of [14], the reparameterization involvesq − 1 ( x ) = (cid:82) ( x 2 + β 2 ) − 1 / 4 dx , and this integral does not have a known closed form.
Furthermore,other cases of interest, like von Neumann Entropy regularization for Matrix Multiplicative Weights[27] seem difficult to fit into this framework as Assumption 1 is challenging to use with spectralfunctions.
Reparameterization Analysis
We now move on to the proof of Theorem 3.
The proof idea is to show that the OMD and OGDoutput iterates which are close to each other after a single update step starting from the same initialpoint.
Then we can view the OGD update a perturbed version of OMD, and combine it with the factthat the OMD algorithm can tolerate bounded noise per trial.
We begin with the following key lemma showing that the updates x t +1 and q ( u t +1 ) created by OMDand OGD respectively, are close to each other from the same initial point x t
= q ( u t ) .
To do this,we carefully analyze the errors that occur due to the approximation in (3) starting from a proximalformulation of mirror descent.
We can show that the two updates are solutions to approximately thesame strongly-convex objective, hence the solutions must be close.
Lemma 4.
Suppose Assumptions 1-3 hold and x t = q ( u t ) , then we have that ∥ x t +1 − q ( u t +1 )
∥ 2 =
O ( G 4 G 3 / 2 F η 3 / 2 ) .
Proof.
We consider the following forms of Algorithms 1 and 2.
[Equation]
We observe that both objectives we are minimizing can be written as the sum of a linear function plusa strongly convex function.
In particular, if ∥ x − x t ∥ 2 > 2 ηG F , then the objective takes positive valuewhich can’t be minimal because taking x = x t gives 0.
Define K r,x t = K ∩ { x : ∥ x − x t ∥ 2 ≤ r } ,then the OMD iterate x t +1 ∈ K t :=
K 2 ηG F ,x t .
Likewise, u t +1 ∈ K ′ t := K ′ 2 ηG F · G,u t .
Using Taylor expansion with Assumption 2, for D R ( x || x t ) with ∥ x − x t ∥ 2 ≤ 2 ηG F , we rewrite it as
[Equation]
where ε ( x ) =
O ( G · G 3 F η 3 ) for x ∈ K t .
And the OMD update becomes
[Equation]
We now use Taylor expansion to relate ∇ ˜ f t ( u t ) ⊤ ( u − u t ) to ∇ f t ( x t ) ⊤ ( x − x t ) .
Let x = q ( u ) foru ∈ K ′ t , hence x ∈ ¯ K t := K 2 ηG F · G 2 ,x t , then we have∇ ˜ f ⊤ t [Equation] u
[Equation]
⊤
[Equation]
where ˜ ε ( x ) =
O ( G 5 G 3 F η 2 ) .
Using the above approximation, we can rewrite the OGD update as
[Equation]
On the other hand, we can rewrite 12 ∥ q − 1 ( x ) − q − 1 ( x t ) ∥ 22 as
[Equation]
Then by Assumption 1, ∇ 2 R ( x t ) =
J − 1 q ( x t )
J − 1 q ( x t ) ⊤ , so we conclude that 1 η B R ( x || x t ) and 12 η ∥ q − 1 ( x ) − q − 1 ( x t ) ∥ 22 are O ( G 8 G 3 F η 2 ) -close.
We can then rewrite the OGD update as
[Equation]
) , where ¯ ε ( x ) =
O ( G 8 G 3 F η 2 ) for x ∈ ¯ K t .
We note that K t ⊆
¯ K t as needed.
Comparing both OMD and OGD updates, the objective functions are only off by an O ( G 8 G 3 F η 2 )term, therefore the minimizers are also O ( G 4 G 3 / 2 F η 32 ) -close
since the objective functions are bothΘ( 1 η ) strongly convex.
Remark 5.
It can be shown that if projection is not required, ∥ x t +1 − q u t +1 ∥ 2 =
O ( η 2 ) in Lemma 4.
Furthermore, if the projection operation that Algorithms 1 and 2 perform are the same, the distanceonly gets closer.
Such a bound would result in an O ( √ T ) regret bound.
For the case of EG over thesimplex, the projection operations in both are the same weight scaling operations, so this could bea part of making O ( √ T ) regret possible, though this still requires a better approach for handlingexploding Lipschitz constants (see App. B).
Next, we show that the OMD algorithm is actually robust to noise: with a (potentially non-stochastic)bounded perturbation on the output x t per round, we can still get vanishing regret.
Lemma 6.
Suppose Assumptions 2 and 3 hold and Algorithm A does the following update:
[Equation]
where ∥ r t ∥ 2 ≤ C
The regret of Algorithm A can be upper bounded by
[Equation]
This follows from a slightly modified analysis of OMD from [16], where the effect of the perturbationis bounded due to the Bregman divergence being Lipschitz.
In particular, we end up with the standardregret bound for mirror descent that follows from a telescoping argument with an additional errorterm
[Equation]
where x Rt is the counterfactual mirror descent iterate with r t = 0 .
Full proof of Lemma 6 can be found in Appendix A. Combining both lemmas, we complete our proofof Theorem 3:
Proof.
Algorithm 2 gives a perturbed version of OMD with perturbation bounded by C
= O ( η 32 )
due to Lemma 4.
Plugging C
=
O ( G 4 G 3 / 2 F η 3 / 2 ) into the bound of Lemma 6, the regret is upperbounded by O ( G 5 G 3 / 2 F √ ηT + Dη ) ignoring the lower order term.
Optimizing this and all constantsgives the choice of η = T − 2 / 3 D 2 / 3 G − 10 / 3 G − 1 F and regret bound O
( T 2 / 3 D 1 / 3 G 10 / 3 G F ) .
Implicit OMD Reparameterization
We have shown that a general convex OMD can be reparameterized as a (potentially) non-convex
OGD, with vanishing O ( T 23 ) regret.
The other direction from OGD to OMD is even more interesting:given a non-convex OGD, can we show its global convergence by showing the existence of a convex
OMD which corresponds to OGD implicitly?
In other words, given a convex and compact domain K ′ , what do we need to know about the domain
K ′ and the (not necessarily convex) loss ˜ f t to ensure the existence of such equivalence?
Fullycharacterizing the necessary and sufficient condition seems hard, and we provide here a simplesufficient condition which covers some known scenarios.
Assumption 4.
We assume the following properties about q .
• There exists a function q such that ˜ f t ( u ) can be written as f t ( q ( u )) where f t is convex.
• q is a C 3 -diffeomorphism, and J q ( u ) is diagonal.
• q ( K ′ ) is convex and compact.
We argue that once the conditions above are satisfied, there exists a regularization R and a corresponding OMD with the desired equivalence.
The only thing we need to verify is the existence ofa regularization R which satisfies Assumption 1, and is strongly convex.
The rough idea is to first show that J q ( u ) always has non-zero (in fact, with the same sign because of continuity) determinantdue to the fact that q is a diffeomorphism, then reconstruct R by integrating twice its Hessians whichwe compute from the equation in Assumption 1.
R is also strongly convex as the Hessians are strictly PSD.
We note that such conditions may not be tight, but are general, succinct and may cover someinteresting examples.
Theorem 7.
Given a convex and compact domain K ′ , and not necessarily convex loss ˜ f t satisfying
Assumption 3.
When Assumption 4 is met, there exists an OMD object with convex loss f t , aconvex domain and a strongly convex regularization R satisfying Assumption 1.
As a result, running
Algorithm 2 on loss ˜ f t ( u ) has regret upper bound
˜ O
( T 23 ) .
Proof.
The first two properties are satisfied by the assumption.
We verify the third property byconstructing a regularization R , which is strongly convex and satisfies Assumption 1. The restfollows Theorem 3.
By the fact that q is a diffeomorphism, J q ( u ) is an invertible matrix.
The fact that J q ( u ) is an invertible matrix implies that J q ( u ) J q ( u ) ⊤ is invertible, and further J q ( u ) J q ( u ) ⊤ ≻ 0 .
Denote H ( u ) =
[ J q ( u ) J q ( u ) ⊤ ] − 1 ≻ 0 which is well defined by the above argument, we want to construct R such that ∇ R 2 ( q ( u ))
=
H ( u ) .
By the assumption that J q ( u ) is diagonal, we can write q ( u ) as
[Equation]
where each q i is a scalar function, such that H ( u ) is diagonal with H ( u ) ii
= 1 / ( q ′ i ( u i )) 2 , which only depends on u i .
We set R ( q ( u )) =
(cid:80) di
=1 R i ( q i ( u i ))) and denote the function
˜ R i ( u i ) =
R i ( q i ( u i )) with variable u i , then by the chain rule we have that [Equation] Plugging the first formula into the second one, we eliminate the R ′ i
( x i ) term:
[Equation]
Recall, we want to find R such that R ′′
i ( x i ) =
1 / ( q ′ i ( u i )) 2 .
Plugging it back to the equation above, the problem is reduced to find R i such that it satisfies the ODE
[Equation]
It’s known that there exists a solution to any linear and continuous second-order ODE.
In fact, using the standard variation of constant method, one example solution ˜
R i ′ ( u i ) can be
[Equation]
where c i , C i are constants and C i =
min u ∈K ′
u i .
It’s well-defined because q ′ i ( u i ) always has the same sign.
To verify the correctness, we calculate that
[Equation]
which solves the ODE together with the solution of ˜ R i
′ ( u i ) .
Next, plugging ˜ R
i ( u i ) =
(cid:82) u i C
i ˜
R i ′ ( u ) du and u = q − 1 ( x ) into the expression of R gives a solutionof the regularization.
Its strong-convexity is implied by the fact that ∇ 2 R ( x ) =
H ( u ) ≻ 0 and iscontinuous over a compact domain , thus there exists a constant c > 0 such that ∇ 2 R ( x ) ≻ cI .
Finally, q being a C 3 -diffeomorphism means that both q and q − 1 are 3 times continuously differentiable.
By the compactness of domains and the continuity of the derivatives, there exists a constant Gsatisfying Assumption 2.
A similar argument holds for R .
Conclusion
We study the convergence of non-convex gradient descent in this paper, by showing (approximate)algorithmic equivalence between gradient descent and mirror descent in the discrete setting.
Weprove that under certain geometric and smoothness conditions, running OGD on non-convex lossesobtained by reparameterizing a convex OMD has regret bound O
( T 23 ) , answering an open problemin [4].
Our analysis is based on a new algorithmic equivalence technique, combining the one-stepcloseness between OMD and the reparameterized OGD and the robustness of OMD.
We furtherextend our result to the other direction, providing sufficient conditions for a non-convex OGD tohave an implicit corresponding OMD and thus converge well.
We leave several questions as futuredirections.1.
Is the O ( T 23 ) regret bound improvable to O ( √ T ) in general?
The decay in the bound inthis work seems to be caused by differences in projection in the reparameterized space andthe original Bregman projection.
Perhaps a different analysis technique can produce optimalbounds.2.
Can assumptions for reparameterization be relaxed?
For example, it is not clear that
Assumption 1 is a necessary condition.
In the implicit mirror descent direction, it may bepossible to lift the diagonal J q ( u ) assumption in Theorem 7 with more refined conditionson PDEs.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes]
(b) Did you include complete proofs of all theoretical results?
[Yes]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[N/A](b)
Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A [Table]
(b) Did you describe any potential participant risks, with links to Institutional Review [Table] Board (IRB) approvals, if applicable?
[N/A
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amount [Table] spent on participant compensation?
[N/A]
[Table]