Original: On the Theoretical Properties of Noise Correlation in Stochastic Optimization
Revised: On the Theoretical Properties of Noise Correlation in Stochastic Optimization

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: Studying the properties of stochastic noise to optimize complex non-convex functions has been an active area of research in the ﬁeld of machine learning.
Revised: Studying the properties of stochastic noise to optimize complex non-convex functions has been an active area of research in the field of machine learning.

Original: Priorwork [51, 46] has shown that the noise of stochastic gradient descent improvesoptimization by overcoming undesirable obstacles in the landscape.
Revised: Prior work [55, 50] has shown that the noise of stochastic gradient descent improves optimization by overcoming undesirable obstacles in the landscape.

Original: Moreover,injecting artiﬁcial Gaussian noise has become a popular idea to quickly escapesaddle points.
Revised: Moreover, injecting artificial Gaussian noise has become a popular idea to quickly escape saddle points.

Original: Indeed, in the absence of reliable gradient information, the noise isused to explore the landscape, but it is unclear what type of noise is optimal in termsof exploration ability.
Revised: Indeed, in the absence of reliable gradient information, the noise is used to explore the landscape, but it is unclear what type of noise is optimal in terms of exploration ability.

Original: In order to narrow this gap in our knowledge, we study ageneral type of continuous-time non-Markovian process, based on fractional Brownian motion, that allows for the increments of the process to be correlated.
Revised: In order to narrow this gap in our knowledge, we study a general type of continuous-time non-Markovian process, based on fractional Brownian motion, that allows for the increments of the process to be correlated.

Original: Thisgeneralizes processes based on Brownian motion, such as the Ornstein-Uhlenbeckprocess.
Revised: This generalizes processes based on Brownian motion, such as the Ornstein-Uhlenbeck process.

Original: We demonstrate how to discretize such processes which gives rise to thenew algorithm “fPGD”.
Revised: We demonstrate how to discretize such processes which gives rise to the new algorithm “fPGD”.

Original: This method is a generalization of the known algorithms
Revised: This method is a generalization of the known algorithms

Original: PGD and Anti-PGD [33].
Revised: PGD and Anti-PGD [36].

Original: We study the properties of fPGD both theoretically andempirically, demonstrating that it possesses exploration abilities that, in some cases,are favorable over PGD and Anti-PGD.
Revised: We study the properties of fPGD both theoretically and empirically, demonstrating that it possesses exploration abilities that, in some cases, are favorable over PGD and Anti-PGD.

Original: These results open the ﬁeld to novel waysto exploit noise for training machine learning models.
Revised: These results open the field to novel ways to exploit noise for training machine learning models.

Original:  Introduction
Revised:  Introduction

Original: Injecting random noise to the parameters of a function has been a commonly used technique inoptimization, dating back to at least [28, 2].
Revised: Injecting random noise to the parameters of a function has been a commonly used technique in optimization, dating back to at least [31, 2].

Original: More recently, [29] demonstrated the empirical beneﬁtsof injecting Gaussian noise (a.k.a.
Revised: More recently, [32] demonstrated the empirical benefits of injecting Gaussian noise (a.k.a.

Original: Brownian motion noise) to the gradients while training deepnetworks.
Revised: Brownian motion noise) to the gradients while training deep networks.

Original: Injecting Gaussian noise has also been a common technique used to show faster escapefrom saddle points [14].
Revised: Injecting Gaussian noise has also been a common technique used to show faster escape from saddle points [15].

Original: Importantly, the noise of the optimizer also has an effect on the typeof minima that can be reached, raising some important connections to ﬂatness and generalizationproperties [21].
Revised: Importantly, the noise of the optimizer also has an effect on the type of minima that can be reached, raising some important connections to flatness and generalization properties [24].

Original: A distinctive property of noise is its exploration ability which is crucial when thegradient of the objective function is not reliable, either due to saddles, valleys, plateaus, or localminima.
Revised: A distinctive property of noise is its exploration ability which is crucial when the gradient of the objective function is not reliable, either due to saddles, valleys, plateaus, or local minima.

Original: Despite a large body of work debating the distribution of the intrinsic noise of stochasticgradient descent (SGD) [40, 50, 42], there is a large gap in our knowledge regarding the role of noiseto escape undesirable regions.
Revised: Despite a large body of work debating the distribution of the intrinsic noise of stochastic gradient descent (SGD) [44, 54, 46, 13], there is a large gap in our knowledge regarding the role of noise to escape undesirable regions.

Original: In order to address this question, we investigate the behavior of a general type of non-Markovianstochastic process known as fractional Brownian motion (fBM).
Revised: In order to address this question, we investigate the behavior of a general type of non-Markovian stochastic process known as fractional Brownian motion (fBM).

Original: The latter is a generalization of
Revised: The latter is a generalization of

Original: Brownian motion where the increments need not be independent but can instead be positively ornegatively correlated.
Revised: Brownian motion where the increments need not be independent but can instead be positively or negatively correlated.

Original: Interestingly, the processes that can be constructed from fBM recover knowninstances in the literature, including perturbed gradient descent (PGD) [51, 20], and anticorrelated
Revised: Interestingly, the processes that can be constructed from fBM recover known instances in the literature, including perturbed gradient descent (PGD) [55, 22], and anticorrelated

Original: PGD (Anti-PGD) [33].
Revised: PGD (Anti-PGD) [36, 37].

Original: As we will see, the correlation between increments has a drastic effect of thebehavior of the optimizer, and neither PGD or Anti-PGD are optimal in all cases.
Revised: As we will see, the correlation between increments has a drastic effect of the behavior of the optimizer, and neither PGD or Anti-PGD are optimal in all cases.

Original: Contribution.
Revised: Contribution.

Original: The ﬁrst question we address is what is the effect of the noise correlation on theoverall magnitude of the process and its ability to explore the landscape?
Revised: The first question we address is what is the effect of the noise correlation on the overall magnitude of the process and its ability to explore the landscape?

Original: In many applications, thisis typically captured by ensuring a uniform control on the evolution of the random process.
Revised: In many applications, this is typically captured by ensuring a uniform control on the evolution of the random process.

Original: Thiscan be achieved by bounding its expected supremum, which is connected to the concept of exit time,see Rmk. 2.
Revised: This can be achieved by bounding its expected supremum, which is connected to the concept of exit time, see Rmk. 2.

Original: To conduct such an analysis, we will consider a well-known type of continuous-timestochastic process with drift known as the fractional Ornstein-Uhlenbeck (fOU) process, whichis connected to SGD (see Section 3).
Revised: To conduct such an analysis, we will consider a well-known type of continuous-time stochastic process with drift known as the fractional Ornstein-Uhlenbeck (fOU) process, which is connected to SGD (see Section 3).

Original: We will also demonstrate how to discretize such a processto obtain a practical optimization algorithm that we name fPGD (for fractional PGD).
Revised: We will also demonstrate how to discretize such a process to obtain a practical optimization algorithm that we name fPGD (for fractional PGD).

Original: Concretely,our main result will be to provide upper and lower bounds on the expected supremum of the fOUprocess.
Revised: Concretely, our main result will be to provide upper and lower bounds on the expected supremum of the fOU process.

Original: These bounds will give a precise characterization of the effect of the noise correlation onthe magnitude of the process.
Revised: These bounds will give a precise characterization of the effect of the noise correlation on the magnitude of the process.

Original: Second, we experimentally investigate the practical behavior of fBMnoise injection:
Revised: Second, we experimentally investigate the practical behavior of fBM noise injection:

Original: In Section 5.1, we demonstrateon a bi-stable potential that fBM perturbations indeedhelp optimizers to move faster from one local minimum to another – as suggested by our theory.
Revised: In Section 5.1, we demonstrate that fBM perturbations indeed help optimizers to move faster from one local minimum to another – as suggested by our theory.

Original: In Section 5.2, we demonstrate the existence of an optimization problem where non-Markovianversions of fPGD indeed outperform the known Markovian special cases (PGD and Anti-PGD) – aﬁrst proof-of-concept for the practical utility of non-Markovian noise injection.
Revised: In Section 5.2, we demonstrate the existence of an optimization problem where non-Markovian versions of fPGD indeed outperform the known Markovian special cases (PGD and Anti-PGD) – a first proof-of-concept for the practical utility of non-Markovian noise injection.

Original:  Related work
Revised:  Related work

Original: Non-Markovian stochastic processes.
Revised: Non-Markovian stochastic processes.

Original: The vast majority of the work done in stochastic optimization for machine learning focuses on Markovian processes.
Revised: The vast majority of the work done in stochastic optimization for machine learning focuses on Markovian processes.

Original: Yet, non-Markovian processes areknown to appear naturally in many situations.
Revised: Yet, non-Markovian processes are known to appear naturally in many situations.

Original: They for instance describe the widths of consecutiveannual rings of a tree or the values of the log return of a stock [6].
Revised: They for instance describe the widths of consecutive annual rings of a tree or the values of the log return of a stock [6].

Original: In this paper, we are especiallyinterested in fractional Brownian Motion (fBM) as an example of a non-Markovian process.
Revised: In this paper, we are especially interested in fractional Brownian Motion (fBM) as an example of a non-Markovian process.

Original: fBM is ageneralization of Brownian motion where the increments need not be independent but can instead bepositively or negatively correlated.
Revised: fBM is a generalization of Brownian motion where the increments need not be independent but can instead be positively or negatively correlated.

Original: The amount of correlation is captured by the Hurst parameter
Revised: The amount of correlation is captured by the Hurst parameter

Original: H 2 (0 , 1) (a formal introduction to fBM is discussed in Section 3).
Revised: H ∈ (0 , 1) (a formal introduction to fBM is discussed in Section 3).

Original: Mathematical techniques for non-Markovian processes are largely underdeveloped compared to their
Revised: Mathematical techniques for non-Markovian processes are largely underdeveloped compared to their

Original: Markovian counterparts.
Revised: Markovian counterparts.

Original: There are two sets of results that are of relevance to the problem discussedin this work.
Revised: There are two sets of results that are of relevance to the problem discussed in this work.

Original: The ﬁrst set of results are escape times for fBM which are studied by [25, 4,37, 41, 43],but they are restricted to simple 1-dimensional processes, and they are often applicable to speciﬁcproblems such as Kramer’s problem, instead of more general problems of interest.
Revised: The first set of results are escape times for fBM which are studied by [28, 4, 41, 45, 47], but they are restricted to simple 1-dimensional processes, and they are often applicable to specific problems such as Kramer’s problem, instead of more general problems of interest.

Original: Extending theseresults to multiple dimensions is difﬁcult, and we are not aware of any existing result.
Revised: Extending these results to multiple dimensions is difficult, and we are not aware of any existing result.

Original: The second results of relevance are bounds on the expected supremum of the stochastic process,sometimes referred to as maximal inequalities.
Revised: The second results of relevance are bounds on the expected supremum of the stochastic process, sometimes referred to as maximal inequalities.

Original: There is a large body of work focusing on the expectedsupremum of "pure" fBM (i.e., without any drift term), including [31, 27, 36].
Revised: There is a large body of work focusing on the expected supremum of "pure" fBM (i.e., without any drift term), including [34, 30, 40].

Original: For instance, [8]establish both upper and lower bounds on the expected supremum of fBM scaling as O ⇣ 1 / p H ⌘ .
Revised: For instance, [8] establish both upper and lower bounds on the expected supremum of fBM scaling as O (cid:16) 1 / √ H (cid:17) .

Original: Some limited results exist with time-dependent drift terms in the 1-dimensional case, e.g. [35] givesan asymptotic result for H 12 , .
Revised: Some limited results exist with time-dependent drift terms in the 1-dimensional case, e.g. [39] gives an asymptotic result for H ∈ (cid:0) 12 , 1 (cid:1) .

Original: In the multi-dimensional case, [23] derive lower and upperbound for the distribution of fBM in the multi-dimensional case, see Rmk. 4.
Revised: In the multi-dimensional case, [26] derive lower and upper bound for the distribution of fBM in the multi-dimensional case, see Rmk. 4.

Original: In this work, our goalwill be to provide sharp bounds for stochastic processes with more general drift terms than prior workand in the multi-dimensional case.
Revised: In this work, our goal will be to provide sharp bounds for stochastic processes with more general drift terms than prior work and in the multi-dimensional case.

Original: Brownian motion and Ornstein-Uhlenbeck process.
Revised: Brownian motion and Ornstein-Uhlenbeck process.

Original: A well-studied type of stochastic processis Brownian motion (BM), which is a special case of fBM (corresponding to H = 12 ) where theincrements are not correlated.
Revised: A well-studied type of stochastic process is Brownian motion (BM), which is a special case of fBM (corresponding to H = 12 ) where the increments are not correlated.

Original: When also considering a stochastic process with a drift, we obtainthe well-known Ornstein-Uhlenbeck process, which has been the subject of a large body of work,including both results on the exit time [1, 9] as well as on the expected supremum [15, 19].
Revised: When also considering a stochastic process with a drift, we obtain the well-known Ornstein-Uhlenbeck process, which has been the subject of a large body of work, including both results on the exit time [1, 9, 23] as well as on the expected supremum [16, 21].

Original: Finally,[10] also derived a maximal inequality for a multi-dimensional Ornstein-Uhlenbeck process, althoughtheir result is restricted to the radial case.
Revised: [Figure] Finally, [10] also derived a maximal inequality for a multi-dimensional Ornstein-Uhlenbeck process, although their result is restricted to the radial case.

Original: SGD & Noise Injection.
Revised: SGD & Noise Injection.

Original: The distribution of the noise of SGD is often a subject of debate in theliterature.
Revised: The distribution of the noise of SGD is often a subject of debate in the literature.

Original: In some cases, the noise of SGD appears to be heavy-tailed [34, 40, 16], which yields fasterexit from sharp to ﬂat minima [30].
Revised: In some cases, the noise of SGD appears to be heavy-tailed [38, 44, 17], which yields faster exit from sharp to flat minima [33].

Original: Noise is also often added to (stochastic) gradient descent.
Revised: Noise is also often added to (stochastic) gradient descent.

Original: Forinstance, Perturbed Gradient Descent (PGD) is a version of GD where artiﬁcial noise is added to theparameters after every step.
Revised: For instance, Perturbed Gradient Descent (PGD) is a version of GD where artificial noise is added to the parameters after every step.

Original: Multiple PGD methods have been shown to help quickly escape spuriouslocal minima [51] and saddle points [20].
Revised: Multiple PGD methods have been shown to help quickly escape spurious local minima [55] and saddle points [22].

Original: Another way to add perturbations to SGD is to add noise tothe labels of the data used for training.
Revised: Another way to add perturbations to SGD is to add noise to the labels of the data used for training.

Original: Recent work has demonstrated that such perturbations areindeed beneﬁcial for generalization by implicitly regularizing the loss [7, 17, 12].[Figure]
Revised: Recent work has demonstrated that such perturbations are indeed beneficial for generalization by implicitly regularizing the loss [7, 18, 12].

Original:  Background
Revised:  Background

Original: Fractional Brownian Motion.
Revised: Fractional Brownian Motion.

Original: Fractional Brownian motion (fBM) is a generalization of Brownianmotion where the increments of the stochastic process need not be independent.
Revised: Fractional Brownian motion (fBM) is a generalization of Brownian motion where the increments of the stochastic process need not be independent.

Original: This family of
Revised: This family of

Original: Gaussian processes was developed for some hydrological modelling by [22].
Revised: Gaussian processes was developed for some hydrological modelling by [25].

Original: [18] later studied thelong term water ﬂow characteristics of the Nile River that was affected by long periods of drought.
Revised: [19] later studied the long term water flow characteristics of the Nile River that was affected by long periods of drought.

Original: Insuch a scenario, the self similarity of the process is related to a long term dependence in the waterlevels.
Revised: In such a scenario, the self similarity of the process is related to a long term dependence in the water levels.

Original: Since then, fBM has also found applications in ﬁnancial modeling [32].
Revised: Since then, fBM has also found applications in financial modeling [35].

Original: We now give a formaldeﬁnition of fBM, and discuss its main properties.
Revised: We now give a formal definition of fBM, and discuss its main properties.

Original: Deﬁnition 1.
Revised: Definition 1.

Original: Fractional Brownian motion (fBM) is a centered Gaussian process ( B H ( t ))
Revised: Fractional Brownian motion (fBM) is a centered Gaussian process ( B H ( t ))

Original: t 2 [0 ,T ] for
Revised: t ∈ [0 ,T ] for

Original: T > 0 with covariance function
Revised: T > 0 with covariance function

Original: [Equation]
Revised: [Equation]

Original: where H 2 (0 , 1) is called the Hurst parameter .
Revised: where H ∈ (0 , 1) is called the Hurst parameter .

Original: Hence, fBM is a generalization of Brownian motion, which is recovered for H = 12 .
Revised: Hence, fBM is a generalization of Brownian motion, which is recovered for H = 12 .

Original: If H > 12 , thenthe increments are positively correlated, while they are negatively correlated if H < 12 .
Revised: If H > 12 , then the increments are positively correlated, while they are negatively correlated if H < 1 2 .

Original: For H 6 = 12 ,the increments of fBM are not independent, although they are stationary, with variance
Revised: For H ̸ = 1 2 , the increments of fBM are not independent, although they are stationary, with variance

Original: [Equation]
Revised: [Equation]

Original: A crucial property of negatively correlated increments is their ability to ﬁll in the space, as illustratedin Fig. 1; see Rmk. 2.
Revised: A crucial property of negatively correlated increments is their ability to fill in the space, as illustrated in Fig. 1; see Rmk. 2.

Original: Fractional Ornstein-Uhlenbeck (OU) process.
Revised: Fractional Ornstein-Uhlenbeck (OU) process.

Original: We conduct our analysis on the fractional
Revised: We conduct our analysis on the fractional

Original: Ornstein-Uhlenbeck process, which can be thought of as a generalization of the well-known
Revised: Ornstein-Uhlenbeck process, which can be thought of as a generalization of the well-known

Original: Ornstein-Uhlenbeck process that has already found many applications in the ﬁeld of machine learning [7, 24, 44].
Revised: Ornstein-Uhlenbeck process that has already found many applications in the field of machine learning [7, 27, 48].

Original: Formally, this process is a d -dimensional stochastic process X Ht 2 R d deﬁned by
Revised: Formally, this process is a d -dimensional stochastic process X Ht ∈ R d defined by

Original: [Equation]
Revised: [Equation]

Original: where A 2 R d ⇥ d and B Ht =
Revised: where A ∈ R d × d and B Ht =

Original: ( B H, 1 t , . . .
Revised: ( B H, 1 t , . . .

Original: B H,dt ) is a fBM with H 2 (0 , 1) .
Revised: B H,dt ) is a fBM with H ∈ (0 , 1) .

Original: For further informationregarding the fractional Ornstein-Uhlenbeck process, we refer the reader to [11, 26, 3].
Revised: For further information regarding the fractional Ornstein-Uhlenbeck process, we refer the reader to [11, 29, 3].

Original: We will forinstance later make use of the covariance of this process that has been derived by [26, 3].
Revised: We will for instance later make use of the covariance of this process that has been derived by [29, 3].

Original: The fractional Ornstein-Uhlenbeck process is relevant for optimization purposes as it corresponds tooptimizing a quadratic function of the form f ( x ) = 12 x > Ax .
Revised: The fractional Ornstein-Uhlenbeck process is relevant for optimization purposes as it corresponds to optimizing a quadratic function of the form f ( x ) = 12 x ⊤ Ax .

Original: Indeed, the gradient of the functionr f ( x ) =
Revised: Indeed, the gradient of the function ∇ f ( x ) =

Original: Ax then corresponds to the drift term in Eq.
Revised: Ax then corresponds to the drift term in Eq.

Original: We also note that when the stochasticprocess reaches stationarity, the stationary distribution of the iterates is known to be approximated bya quadratic function [24, 39].
Revised: We also note that when the stochastic process reaches stationarity, the stationary distribution of the iterates is known to be approximated by a quadratic function [27, 43].

Original: This implies that such processes can be used to analyze the behavior ofcomplex functions at stationarity, but also close to saddle points where quadratic functions often playa crucial role to analyze the escape behavior of gradient descent methods [20].
Revised: This implies that such processes can be used to analyze the behavior of complex functions at stationarity, but also close to saddle points where quadratic functions often play a crucial role to analyze the escape behavior of gradient descent methods [22].

Original: Discretization by fractional Perturbed Gradient Descent (fPGD).
Revised: Discretization by fractional Perturbed Gradient Descent (fPGD).

Original: For standard Brownian motion
Revised: For standard Brownian motion

Original: B t (i.e. for Hurst parameter H = 1 / 2 ), the discretization of the continuous-time SDE dX t =r
Revised: B t (i.e. for Hurst parameter H = 1 / 2 ), the discretization of the continuous-time SDE dX t = −∇ f

Original: f ( X ) dt +  dB t of any optimization problem argmin x f ( x ) , is written
Revised: ( X ) dt + σdB t of any optimization problem argmin x f ( x ) , is written

Original: [Equation]
Revised: [Equation]

Original: with step size ⌘ > 0 .
Revised: with step size η > 0 .

Original: This method is called perturbed gradient descent (PGD) [20].
Revised: This method is called perturbed gradient descent (PGD) [22].

Original: Here, byinstead discretizing the fractional SDE dX t = r
Revised: Here, by instead discretizing the fractional SDE dX t = −∇ f

Original: f ( X ) dt +  dB Ht for any H 2 (0 , 1) , we obtain ageneralized version of PGD, called fractional PGD (fPGD):
Revised: ( X ) dt + σdB Ht for any H ∈ (0 , 1) , we obtain a generalized version of PGD, called fractional PGD (fPGD):

Original: [Equation]
Revised: [Equation]

Original: which is equal to PGD if H = 1 / 2 .
Revised: which is equal to PGD if H = 1 / 2 .

Original: The parameter H determines the correlation of noise in fPGD:anticorrelated noise for H 2 (0 , 1 / 2) , positively correlated noise for H 2 (1 / 2 , 1) , and uncorrelated(independent) noise for H = 1 / 2 .
Revised: The parameter H determines the correlation of noise in fPGD: anticorrelated noise for H ∈ (0 , 1 / 2) , positively correlated noise for H ∈ (1 / 2 , 1) , and uncorrelated (independent) noise for H = 1 / 2 .

Original: Importantly – since B Ht converges to a white-noise process (indistribution) as H ! 0 ; see Lemma 4.1 by [8] – the method Anti-PGD [33, Section 2] can beconsidered as the limit case of fPGD as H # 0 .
Revised: Importantly – since B Ht converges to a white-noise process (in distribution) as H → 0 ; see Lemma 4.1 by [8] – the method Anti-PGD [36, Section 2] can be considered as the limit case of fPGD as H ↓ 0 .

Original: Hence, fPGD with H 2 (0 , 1 / 2) interpolates between
Revised: Hence, fPGD with H ∈ (0 , 1 / 2) interpolates between

Original: PGD ( H = 1 / 2 ) and Anti-PGD ( H = 0 ).
Revised: PGD ( H = 1 / 2 ) and Anti-PGD ( H = 0 ).

Original: Positive correlations ( H 2 (1 / 2 , 1) ) have not been shownto be useful yet, and indeed the below theorems and experiments will indicate that a small H yieldsquicker escapes from difﬁcult regions such as saddle points and local minima.
Revised: Positive correlations ( H ∈ (1 / 2 , 1) ) have not been shown to be useful yet, and indeed the below theorems and experiments will indicate that a small H yields quicker escapes from difficult regions such as saddle points and local minima.

Original: While the next sectionwill analyze the continuous-time model based on fBM, we will return to the optimizer fPGD in theexperiments (Section 5) where we will illustrate how the theoretical properties relate to optimization,and why H might, in some settings, be a useful tuning parameter for perturbed-gradient methods.
Revised: While the next section will analyze the continuous-time model based on fBM, we will return to the optimizer fPGD in the experiments (Section 5) where we will illustrate how the theoretical properties relate to optimization, and why H might, in some settings, be a useful tuning parameter for perturbed-gradient methods.

Original:  Analysis 
Revised:  Analysis 4.1 Preliminaries

Original: 4.1 Preliminaries

Original: Our analysis relies on a technique known as chaining that provides a general framework that can beused to prove uniform bounds on a random process.
Revised: Our analysis relies on a technique known as chaining that provides a general framework that can be used to prove uniform bounds on a random process.

Original: The main tools we use are Dudley’s inequalityand Sudakov’s inequality, which give upper and lower bounds on E [sup t 2 T X t ] in terms of the metricentropy of T denoted by log N ( T, d, ✏ ) .
Revised: The main tools we use are Dudley’s inequality and Sudakov’s inequality, which give upper and lower bounds on E [sup t ∈ T X t ] in terms of the metric entropy of T denoted by log N ( T, d, ϵ ) .

Original: Both inequalities are stated in Appendix A.1.
Revised: Both inequalities are stated in Appendix A.1.

Original: We will consider processes that have sub-Gaussian increments as deﬁned below.
Revised: We will consider processes that have sub-Gaussian increments as defined below.

Original: For that, werecall the deﬁnition of the sub-Gaussian norm of a random variable X , denoted by k X k 2 , ask X k 2 = inf { t > 0 : E [exp( X 2 /t 2 )]  2 } .
Revised: For that, we recall the definition of the sub-Gaussian norm of a random variable X , denoted by ∥ X ∥ ψ 2 , as ∥ X ∥ ψ 2 = inf { t > 0 : E [exp( X 2 /t 2 )] ≤ 2 } .

Original: Deﬁnition 2.
Revised: Definition 2.

Original: A random process ( X t ) t 2 T deﬁned on a metric space ( T, d ) has sub-Gaussian increments if there exists K  0{+≥ 
+} such that k X t  X s k 2 
Revised: A random process ( X t ) t ∈ T defined on a metric space ( T, d ) has sub-Gaussian increments if there exists K {+≥ 
+} such that ∥ X t − X s ∥

Original:  Kd ( t, s ) 8 t, s 2 T .
Revised: ψ 2 ≤ Kd ( t, s ) ∀ t, s ∈ T .

Original: 4.2 One-dimensional case
Revised: 4.2 One-dimensional case

Original: In this section, we derive both upper and lower bounds on the expected supremum of the stochasticprocess deﬁned in Eq.
Revised: In this section, we derive both upper and lower bounds on the expected supremum of the stochastic process defined in Eq.

Original: Since the analysis is rather technical, we ﬁrst present the main ideas in the1-dimensional case.
Revised: Since the analysis is rather technical, we first present the main ideas in the 1-dimensional case.

Original: We then present the general result for the d -dimensional case.
Revised: We then present the general result for the d -dimensional case.

Original: Theorem 1 (Upper bound, 1-D case) .
Revised: Theorem 1 (Upper bound, 1-D case) .

Original: Consider the stochastic process
Revised: Consider the stochastic process

Original: X Ht deﬁned by
Revised: X Ht defined by

Original: [Equation]
Revised: [Equation]

Original: with X 0 = 0 ,  2 R and |  | < 1 .
Revised: with X 0 = 0 , σ ∈ R and | β | < 1 .

Original: Then, for t 2 [0 , 1] ,
Revised: Then, for t ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: where C  = e 2   and C are positive constants (independent of H ).
Revised: where C β = e 3 β and C are positive constants (independent of H ).

Original: Theorem 2 (Lower bound, 1-D case) .
Revised: Theorem 2 (Lower bound, 1-D case) .

Original: Consider the stochastic process
Revised: Consider the stochastic process

Original: X Ht deﬁned by
Revised: X Ht defined by

Original: [Equation]
Revised: [Equation]

Original: with X 0 = 0 ,  2 R and |  | < 1 .
Revised: with X 0 = 0 , σ ∈ R and | β | < 1 .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where C  = e  2 |  | and c are positive constants (independent of H ).
Revised: where C β = e − 2 | β | and c are positive constants (independent of H ).

Original: Note that both upper and lower bounds scale with O ⇣ 1 p H ⌘ , which implies the bound is tight andcan not be improved in terms of H .
Revised: Note that both upper and lower bounds scale with O (cid:16) 1 √ H (cid:17) , which implies the bound is tight and can not be improved in terms of H .

Original: However, the lower bound requires  > e 2 |  |  1 / 2 (large-noiseregime) to be non-vacuous.
Revised: However, the lower bound requires σ > e 2 | β |− 1 / 2 (large-noise regime) to be non-vacuous.

Original: We can also derive a lower bound for the small-noise regime as follows.
Revised: We can also derive a lower bound for the small-noise regime as follows.

Original: Corollary 3 (Small noise regime) .
Revised: Corollary 3 (Small noise regime) .

Original: In the case where 0 <  < e 2 |  |  12 , we observe that X Ht =
Revised: In the case where 0 < σ < e 2 | β |− 12 , we observe that X Ht =

Original:  Y
Revised: σY

Original: Ht ,where dY Ht =   Y
Revised: Ht , where dY Ht = − βY

Original: Ht dt + dB
Revised: Ht dt + dB

Original: Ht , Y H 

Revised: Ht , Y H

Original: = 0 .
Revised: = 0 .

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: We thus obtain a lower bound for |  |  14 .
Revised: We thus obtain a lower bound for | β | ≤ 14 .

Original: In the general case 14 < |  | < 1 , one gets principally a161 lower bound in terms of entropy numbers (Thm. 7) or by means of so called Talagrand functionals (seee.g.
Revised: In the general case 14 < | β | < 1 , one gets principally a lower bound in terms of entropy numbers (Thm. 7) or by means of so called Talagrand functionals (see e.g. [ 45 ] ).

Original: However, the latter quantities cannot be computed explicitly, in general.
Revised: However, the latter quantities cannot be computed explicitly, in general.

Original: In Appendix B.1we however provide experiments that demonstrate the 1 / p H in this case, for the example of  = 12 .
Revised: In Appendix B. we however provide experiments that demonstrate the 1 / √ H in this case, for the example of β = 12 .

Original: Remark 1 (Extension to any ﬁnite interval) .
Revised: Remark 1 (Extension to any finite interval) .

Original: Although the theorems are stated for the interval
Revised: Although the theorems are stated for the interval

Original: T = [0 , 1] , one can extend the results to an arbitrary ﬁnite interval using the self-similarity property offBM, namely B H at d = a H B H t for a > 0 .
Revised: T = [0 , 1] , one can extend the results to an arbitrary finite interval using the self-similarity property of fBM, namely B Hat d = a H B Ht for a > 0 .

Original: This implies that E 
Revised: This implies that E (cid:104) sup s ∈ [0 ,t ] B Hs (cid:105) =

Original: [Equation]
Revised: t H E (cid:104) sup s ∈ [0 , 1] B Hs (cid:105) .

Original: In the multi-dimensional case, the same extension works by the independence of the dimensions.
Revised: In the multi-dimensional case, the same extension works by the independence of the dimensions.

Original: 4.3 Multi-dimensional case
Revised: 4.3 Multi-dimensional case

Original: We now generalize the bounds to the multi-dimensional case.
Revised: We now generalize the bounds to the multi-dimensional case.

Original: The quantity of interest becomes theexpected supremum of the norm, or more precisely E h sup t 2 [0 , 1] k X
Revised: The quantity of interest becomes the expected supremum of the norm, or more precisely E (cid:104) sup t ∈ [0 , 1] ∥ X

Original: Ht  E
Revised: Ht − E

Original: [ X Ht ] k i .
Revised: [ X Ht ] ∥ (cid:105) .

Original: Similarly tothe 1-dimensional case, we apply Sudakov’s inequality to the Gaussian component of the multidimensional fOU process
Revised: Similarly to the 1-dimensional case, we apply Sudakov’s inequality to the Gaussian component of the multidimensional fOU process

Original: X Ht 1 .
Revised: X Ht 2 .

Original: Theorem 4 (Upper bound, d -dimensional case) .
Revised: Theorem 4 (Upper bound, d -dimensional case) .

Original: Consider the d -dimensional stochastic process X Ht 2 R d deﬁned by
Revised: Consider the d -dimensional stochastic process X Ht ∈ R d defined by

Original: [Equation]
Revised: [Equation]

Original: where A 2 R d ⇥ d and B Ht =
Revised: where A ∈ R d × d and B Ht =

Original: ( B H, 1 t , . 
Revised: ( B H, 1 t , . . .

Original: [Equation]

Original: B H,dt ) is a fBM with H 2 (0 , 1) .
Revised: B H,dt ) is a fBM with H ∈ (0 , 1) .

Original: Denote by a ij ( t ) the matrix entries of exp( tA ) A .
Revised: Denote by a ij ( t ) the matrix entries of exp( tA ) A .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Revised: where C 2 = max di

Revised: [Equation]

Revised: j =

Original: where C 2 = max di,j =1 sup t 2 [0 , 1] | a ij ( t ) | 2 + max di,j =1 sup t 2 [0 , 1] | a ij ( t ) | + 1 and C > 0 is a constant independent of H .
Revised: [Equation] ∈ [Equation] ) | + 1 and C > 0 is a constant independent of H .

Original: Theorem 5 (Lower bound, d -dimensional case) .
Revised: Theorem 5 (Lower bound, d -dimensional case) .

Original: Consider the d -dimensional stochastic process X Ht 2 R d deﬁned by
Revised: Consider the d -dimensional stochastic process X Ht ∈ R d defined by

Original: [Equation]
Revised: [Equation]

Original: where A 2 R d ⇥ d and B Ht =
Revised: where A ∈ R d × d and B Ht =

Original: ( B H, 1 t , . . .
Revised: ( B H, 1 t , . . .

Original: B H,dt ) is a fBM with H 2 (0 , 12 ) .
Revised: B H,dt ) is a fBM with H ∈ (0 , 12 ) .

Original: Denote by a ij ( t ) the matrix entries of exp( tA )
Revised: Denote by a ij ( t ) the matrix entries of exp( tA )

Original: A and assume that there exists i 0 2 { 1 , . . .
Revised: A and assume that there exists i 0 ∈ { 1 , . . .

Original: d } such that min t 2 [0 , 1] a i 0 i 0 ( t )   1 + C 1 , where 0 < C 1 < 1 .
Revised: d } such that min t ∈ [0 , 1] a i 0 i 0 ( t ) ≥ − 1 + C 1 , where 0 < C 1 < 1 .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: where c is a constant independent of H .
Revised: where c is a constant independent of H .

Original: We obtain an upper bound of the form O ⇣ 1 p H ⌘ with a matching lower bound, although we dorequire a condition on the entries of the matrix A , which we discuss further in Example 1 below.
Revised: We obtain an upper bound of the form O (cid:16) 1 √ H (cid:17) with a matching lower bound, although we do require a condition on the entries of the matrix A , which we discuss further in Example 1 below.

Original: Example 1 (Assumption lower bound) .
Revised: Example 1 (Assumption lower bound) .

Original: Let A = diag (  1 , ...,  d ) .
Revised: Let A = diag ( λ 1 , ..., λ d ) .

Original: Then e tA A = diag ( e t  1  1 , ...,e t  d  d ) .
Revised: Then e tA A = diag ( e tλ 1 λ 1 , ..., e tλ d λ d ) .

Original: If some diagonal element of A is non-negative, then the assumptions of the above theoremare satisﬁed.
Revised: If some diagonal element of A is non-negative, then the assumptions of the above theorem are satisfied.

Original: Suppose that all entries  1 , ...,  d are negative and that  i 0 >  1 , for some i 0 .
Revised: Suppose that all entries λ 1 , ..., λ d are negative and that λ i 0 > − 1 , for some i 0 .

Original: Thenmin t 2 [0 , 1] a i 0{+
+} i 0 ( t ) =
Revised: Then min t ∈ [0 , 1] a i {+
+} i 0 ( t ) =

Original: min t 2 [0 , 1] ( e t  i 
 i 0 )
Revised: min t ∈ [0 , 1] ( e tλ i

Original:   i 0 >  1 + C 1 for some C 1 2 (0 , 1) .
Revised: λ i 0 ) ≥ λ i 0 > − 1 + C 1 for some C 1 ∈ (0 , 1) .

Original: Hence, if some181 diagonal element  i 0 >  1 exists, then the conditions of the previous theorem are fulﬁlled.
Revised: Hence, if some diagonal element λ i 0 > − 1 exists, then the conditions of the previous theorem are fulfilled.

Original: We make a few comments regarding the meaning of the bound, as well as some extensions.
Revised: We make a few comments regarding the meaning of the bound, as well as some extensions.

Original: First of all, the theorems discussed above bound the expected supremum of the process.
Revised: First of all, the theorems discussed above bound the expected supremum of the process.

Original: We nowexplain how these are connected to what we call "exploration ability".
Revised: We now explain how these are connected to what we call "exploration ability".

Original: Remark 2 (Exploration ability) .
Revised: Remark 2 (Exploration ability) .

Original: We give three different justiﬁcations for the concept of exploration.
Revised: We give three different justifications for the concept of exploration.

Original: 1) Expected behavior: Based on Theorem 5, we have
Revised: 1) Expected behavior: Based on Theorem 5, we have

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where c, C 1 are constant independent of H .
Revised: where c, C 1 are constant independent of H .

Original: Since E X
Revised: Since E X

Original: Ht solves the ODE  ( t ) =
Revised: Ht solves the ODE ϕ ( t ) =

Original: ⇠ + R t 

Revised: ξ + (cid:82) t

Original: A  ( s ) ds , the quantity a := sup t 2 [0 , 1] k E [ X Ht ] k < 1is independent of H .
Revised: Aϕ ( s ) ds , the quantity a := sup t ∈ [0 , 1] ∥ E [ X Ht ] ∥ < ∞ is independent of H .

Original: Thus E h [Equation]
Revised: Thus E (cid:104) sup t ∈ [0 , 1] ∥

Original:  ce 
Revised: X Ht ∥ (cid:105) ≥ ce −(cid:114)

Original: [Equation] e  12 ⌘  a .
Revised: H log (cid:16) 2 C 1 e − 12 (cid:17) − a .

Original: Hence, for a given
Revised: Hence, for a given

Original: H 
Revised: H 1 ∈ (cid:0) 0 , 12 (cid:1) , there is an H < H 1 such that E (cid:104) sup t ∈ [0 , 1] ∥ X Ht ∥

Original: [Equation]
Revised: [Equation]

Original:  0 , , there is an H < H 1 such that E
Revised: > [Equation] (cid:104) sup

Original: [Equation]
Revised: [Equation]

Revised: ∈

Revised: [Equation]

Original:  We conclude that the average maximal distance of X Ht from the origin is larger than that of X H 1 t .
Revised: ∥ [Equation] (cid:105) . We conclude that the average maximal distance of X Ht from the origin is larger than that of X H 1 t .

Original: This is what we mean by the process having a better exploration ability.
Revised: This is what we mean by the process having a better exploration ability.

Original: Better exploration, in thissense, can help to escape suboptimal local minima more quickly (experiments in Section 5.1) or toexplore the space around a saddle point for an exit (experiments in Section 5.2).2)
Revised: Better exploration, in this sense, can help to escape suboptimal local minima more quickly (experiments in Section 5.1) or to explore the space around a saddle point for an exit (experiments in Section 5.2).

Original:  Exit time: First, note that E [Equation] ⌘ ds , therefore
Revised: 2) Exit time: First, note that E (cid:104) sup t ∈ [0 , 1] ∥ X Ht ∥ (cid:105) =

Original: [Equation]{+(cid:82) ∞ 
 P (cid:16) sup t ∈ [0 , 1] ∥ X Ht ∥ > s (cid:17) ds , therefore (cid:82) ∞ 0 (cid:110) P (cid:16) sup t ∈ [0 , 1] ∥ X Ht ∥ > s (cid:17) − P (cid:16) sup t ∈ [0 , 1] ∥ X H 1 t ∥ > s (cid:17)(cid:111) ds > 0 .+}
Revised: {+(cid:82) ∞ 
 P (cid:16) sup t ∈ [0 , 1] ∥ X Ht ∥ > s (cid:17) ds , therefore (cid:82) ∞ 0 (cid:110) P (cid:16) sup t ∈ [0 , 1] ∥ X Ht ∥ > s (cid:17) − P (cid:16) sup t ∈ [0 , 1] ∥ X H 1 t ∥ > s (cid:17)(cid:111) ds > 0 .+}

Original: Thus, we can ﬁnd a s 0 > 0 such that P 
Revised: Thus, we can find a s 0 > 0 such that P (cid:16) sup t ∈ [0 , 1] ∥

Original: [Equation] ⌘ >0 and, since probability distributions are right-continuous, there exists ✏ > 0 such that for alls 2 [ s 0 , s 0 + ✏ ) , P 
Revised: X Ht ∥ > s 0 (cid:17) − P (cid:16) sup t ∈ [0 , 1] ∥ X H 1 t ∥ > s 0 (cid:17) > 0 and, since probability distributions are right-continuous, there exists ϵ > 0 such that for all s ∈ [ s 0 , s 0 + ϵ ) , P (cid:16) sup t ∈ [0 , 1] ∥

Original: [Equation] sup t 2 [0 , 1] [Equation] ⌘ > 0 .
Revised: X Ht ∥ > s (cid:17) > P (cid:16) sup t ∈ [0 , 1] ∥ X H 1 t ∥ > s (cid:17) > 0 .

Original: On the other hand,
Revised: On the other hand,

Original: P [Equation] Hs < 1) where ⌧ Hs denotes the ﬁrst exit time of X H from theball B s (0) of radius s .
Revised: P (cid:16) sup t ∈ [0 , 1] ∥ X Ht ∥ > s (cid:17) = P ( τ Hs < 1) where τ Hs denotes the first exit time of X H from the ball B s (0) of radius s .

Original: We conclude that P ( ⌧ Hs < 1) > P ( ⌧ H 1 s < 1) 8 s 2 [ s 0 , s 0 + ✏ ) , i.e. theprobability of an exit before time s of H is larger than that of H 1 .3)
Revised: We conclude that P ( τ Hs < 1) > P τ H 1 s < 1) ∀ s ∈ [ s 0 , s 0 + ϵ ) , i.e. the probability of an exit before time s of H is larger than that of H 1 .

Original: Space-ﬁlling property: We note that fractional Brownian motion with low Hurst parameter H ﬁllsthe space more densely.
Revised: 3) Space-filling property: We note that fractional Brownian motion with low Hurst parameter H fills the space more densely.

Original: This behavior comes from the fact that the graph of the fBM has a Hausdorffdimension and box-counting dimension given by d = 2  H .
Revised: This behavior comes from the fact that the graph of the fBM has a Hausdorff dimension and box-counting dimension given by d = 2 − H .

Original: The latter implies that N (1 /n ) ⇡
Revised: The latter implies that N (1 /n ) ≈

Original: Cn dfor large n , where N ( " ) denotes the number of boxes with side length " , which is needed to cover thegraph of the fBM.
Revised: Cn d for large n , where N ( ε ) denotes the number of boxes with side length ε , which is needed to cover the graph of the fBM.

Original: Therefore, the graph of a (1-dimensional) fBM "behaves" like a 2  dimensionalsmooth manifold for small H .
Revised: Therefore, the graph of a (1-dimensional) fBM "behaves" like a 2 − dimensional smooth manifold for small H .

Original: See [38].
Revised: See [42].

Original: The above claim about the Hausdorff dimension and thebox-ﬁlling dimension of fBM is proved by [47].
Revised: The above claim about the Hausdorff dimension and the box-filling dimension of fBM is proved by [51].

Original: Further related articles are [5] and [48].
Revised: Further related articles are [5] and [52].

Original: Remark 3 (More general version) .
Revised: Remark 3 (More general version) .

Original: Let us mention that the results in Theorems5 and 4  can be alsogeneralized to the case of linear SDE‘s of the form
Revised: Let us mention that the results in Theorems 4 and 5 can be also generalized to the case of linear SDE‘s of the form

Original: [Equation]
Revised: [Equation]

Original: where A : [0 , T ] ! R d ⇥ d is locally bounded and measurable, ⌃ : [0 , T ] ! R d ⇥ d is continuousdifferentiable and H < 12 .
Revised: where A : [0 , T ] −→ R d × d is locally bounded and measurable, Σ : [0 , T ] −→ R d × d is continuous differentiable and H < 12 .

Original: Here the stochastic integral with respect to B H · is given by the Young integral.
Revised: Here the stochastic integral with respect to B H · is given by the Young integral.

Original: In fact, one can show that 
Revised: In fact, one can show that (cid:82) t 0 Σ( s )

Original: [Equation]{+dB Hs = Σ( t ) B Ht − (cid:82) t 
+}
Revised: {+dB Hs = Σ( t ) B Ht − (cid:82) t 
+}

Original: s ⌃ ( s ) B Hs ds (see e.g. [6]).
Revised: ∂∂s Σ( s ) B Hs ds (see e.g. [6]).

Original: Byusing the linear transformation Y t :=
Revised: By using the linear transformation Y t :=

Original: X t  [Equation] ⌃ ( s ) dBHs , we observe that Y t = x + R
Revised: X t − (cid:82) t 0 Σ( s ) dB

Original: [Equation] 0 ( A ( s ) Y s +a ( s ))
Revised: Hs , we observe that Y t = x + (cid:82) t 0 ( A ( s ) Y s + a ( s ))

Original: ds , where a ( t ) :
Revised: ds , where a ( t ) :

Original: = A ( t ) R t 0 ⌃ ( s ) dB Hs .
Revised: = A ( t ) (cid:82) t 0 Σ( s ) dB Hs .

Original: Denote by  ( t ) , 0  t  T the fundamental solutionassociated with A ( t ) , that is the unique (absolutely continuous) solution to the equation ˙  ( t ) =
Revised: Denote by Φ( t ) , 0 ≤ t ≤ T the fundamental solution associated with A ( t ) , that is the unique (absolutely continuous) solution to the equation ˙Φ( t ) =

Original: A ( t )  ( t ) ,  (0) =
Revised: A ( t )Φ( t ) , Φ(0) =

Original: I d ⇥ d , the d ⇥ d identity matrix.
Revised: I d × d , the d × d identity matrix.

Original: Hence,
Revised: Hence,

Original: [Equation]
Revised: [Equation]

Original: Although the solution X t , 0  t  T is in general not stationary any longer, we can use similararguments as in the proofs of Theorems5 and 4 and  the latter representation of the process to derivea lower and upper bound for E [Equation]
Revised: Although the solution X t , 0 ≤ t ≤ T is in general not stationary any longer, we can use similar arguments as in the proofs of Theorems 4 and 5 and the latter representation of the process to derive a lower and upper bound for E (cid:2) sup 0 ≤ t ≤ 1 ∥ X t ∥ (cid:3) .

Original: Remark 4 (Known bounds for distribution of fBM in multi-dimensional case) .
Revised: Remark 4 (Known bounds for distribution of fBM in multi-dimensional case) .

Original: In Theorems 5 and 4we obtained a lower and upper bound for the expected supremum of the norm of a (centered) multidimensional fOU process.
Revised: In Theoremsand 5 we obtained a lower and upper bound for the expected supremum of the norm of a (centered) multi-dimensional fOU process (without a drift term).

Original: In comparison to the latter we also mention the following correspondingresult for distributions with respect to the fBm (see Theorem 4.6 by [23]): Let B Ht , 0  t  1be a fractional Brownian motion in R d for H 2 (0 , 1) .
Revised: In comparison to the latter we also mention the following corresponding result for distributions with respect to the fBm (see Theorem 4.6 by [26]): Let B Ht , 0 ≤ t ≤ 1 be a fractional Brownian motion in R d for H ∈ (0 , 1) .

Original: Then the following lower and upperbound of the distribution of sup 0  t 
Revised: Then the following lower and upper bound of the distribution of sup 0 ≤ t ≤ 1 (cid:13)(cid:13)

Original: B Ht  ( k · k maximum norm) hold: There exist constants0 < K 1 ( H )  K 2 ( H ) <
Revised: B Ht (cid:13)(cid:13) ( ∥·∥ maximum norm) hold: There exist constants 0 < K 1 ( H ) ≤ K 2 ( H ) <

Original: 1 such that for all 0 < "  1 :
Revised: ∞ such that for all 0 < ε ≤ 1 :

Original: [Equation]
Revised: [Equation]

Original: Remark 5 (Tightness of the bound) .
Revised: Remark 5 (Tightness of the bound) .

Original: As in the 1-D case, the bound is right in terms of H , which isthe most important quantity for the argument made in this paper.
Revised: As in the 1-D case, the bound is right in terms of H , which is the most important quantity for the argument made in this paper.

Original: Regarding the dimension d , notethat the lower bound does not depend on it (and thus it is not improvable) while the upper boundmight not be tight.
Revised: Regarding the dimension d , note that the lower bound does not depend on it (and thus it is not improvable) while the upper bound might not be tight.

Original:  Experiments
Revised:  Experiments

Original: By bounding the expected supremum of the stochastic process X Ht , our analysis reveals some insightsinto its exploration capability.
Revised: The above analysis reveals some insights into the exploration capability of a stochastic process X Ht driven by a fracional Brownian motion B Ht with Hurst parameter H .

Original: We have seen that the correlation between increments directly affectsthe supremum.
Revised: We have seen that the correlation between increments directly affects the expected supremum of X Ht .

Original: Concretely, lower values of the Hurst parameters are expected to yield processes withbetter exploration abilities, as we detailed in Rmk. 2.
Revised: Concretely, lower values of the Hurst parameters are expected to yield processes with better exploration abilities, as we detailed in Remark 2.

Original: To demonstrate the relevance of our theoreticalﬁndings for optimization, we run a series of experiments with the optimizer fPGD from Eq.
Revised: To demonstrate the relevance of our theoretical findings for optimization, we run a series of experiments with the optimizer fPGD from Eq.

Original: In Section 5.1, we will illustrate how fPGD behaves on bi-stable optimization landscapes, where thereare two adjacent minima of different desirability (also studied by [51, 49]).
Revised: In Section 5.1, we show that fPGD with small H better explores a non-convex loss landscapes with multiple local minima better, as predicted by our theory (Rmk. 2).

Original: In Section 5.2, we willdemonstrate on an embedded saddle point that different choices of H might be beneﬁcial in differentoptimization phases – thus justifying the introduction of fPGD as a compromise between PGD and Anti-PGD.
Revised: In Section 5.2, we will demonstrate on an embedded saddle point that different choices of H might be beneficial in different optimization phases – thus justifying the introduction of fPGD as a compromise between PGD and Anti-PGD.

Original: In  Appendix B.3, we present another experiment on  the Styblinski–Tang function in twodimensions which conﬁrms our ﬁnding that fPGD with small H has a better exploration ability. 5.1 Moving to better local minima in non-convex optimization
Revised: In the Appendix B.3, we add experiments on how fPGD behaves on bi-stable optimization landscapes, where there are two adjacent minima of different desirability (also studied by [55, 53]; the results of these experiments confirm that a small H yields faster escapes from local minima. 5.1 Demonstration of exploration ability on Styblinski–Tang function

Original: We consider the behavior of fPGD  in a bi-stable optimization landscape, given by
Revised: In this section, we test the ability of fPGD to escape local minima and explore the landscape. To this end, we consider the Styblinski–Tang function in two dimensions, which is written

Original: [Equation]
Revised: [Equation]

Original: where  is  the indicator function  and the parameters ( v 0 , v 1 , v 2 , a, c, m ) determine the position andshape of two adjacent local minima.
Revised: A contour plot of this function is depicted on the upper left of Fig. This function has four minima: one shallow one (upper right), two medium ones (upper left and lower right), and one global deep minimum (lower left); see upper right subfigure of Fig.

Original: Similar landscapes have been studied in the context ofescapingspurious local minima [51] and in the context of generalization in machine learning by selecting ﬂatminima [30, 49].
Revised: Since the direct way between the shallow and deep minimum is blocked by a large hill at zero, gradient-based optimizers have to go through the medium ones, to get to the shallow one. It is therefore a suitable test problem to test the validity of the alleged exploration ability of fPGD with small H (as explained in Remark 2). We thus run experiments, where fPGD is initialized in the shallow minimum at x 0 = (2 . 7 , 2 . 7) . After enough iterations, fPGD will eventually take the following steps: First, it will escape the shallow minimum.

Original: We here study the former case, but we obtain analogous results in the latter (see
Revised: Second, it will reach either of the medium minima. Third, it will exit the medium minimum. Fourth, it will reach the deep minimum. (This might, of course, take a long time and the budget might not suffice.)

Original: App. B).
Revised: Each of these steps can be interpreted as stopping times whose distribution we can compare for different H to assess how good the exploration is: the lower each of these stopping times, the better the exploration for a given H . The precise experimental setup is the following. For H ∈ { 0 . 1 , 0 . 3 , 0 . 5 , 0 . 7 , 0 . 9 } , we run fPGD 10times with a budget of 10 5 steps. We chose a generic step size of η = 0 . 01 .

Original: For the former case, we set ( v 0 , v 1 , v 2 , a, c, m ) = 
Revised: To zoom in on the effect, the perturbation scale σ is set to the highest level of noise admissible without divergence on this landscape, i.e. σ = 1 . 25 . (Note that we here again re-scale the final time to t = 1 so that the final amount of variance is equal for all Hurst parameters.) For each run, we determined after each step if fPGD is in the shallow, one of the medium or the deep minimum (or outside of all of them). From this information, we extracted the below summary statistics to study the exploration ability.

Original: (0 . 0 , 45 . 0,  12 . 5 , 3 . 5 , 13 . 9  , 18 . 0)
Revised: As a start, the lower row of Fig. 2 shows for H ∈ { 0 . 1 , 0 . 5 , 0 . 9 } , where fPGD ends up after 10 4 and after all 10 5 steps. It is clearly visible that the smaller the Hurst parameter, the faster fPGD moves to better local minima. (For other values of H the same tendency would be visible, as the below figures will confirm.)

Original: ; forthe corresponding landscape see the upper left subplot of Fig. 
Revised: To give a more quantitative assessment of this effect, we provide a bar chart of the position of fPGD for all H after 10 4 and 10 5 steps in the upper row of Fig. It clearly shows that a small value of H is less likely to remain in the shallow minimum and is more likely to end up in the deep minimum. This is a strong effect: after 10% of steps only H ∈ { 0 . 1 , 0 . 3 } have exited at all. Only H = 0 .

Original: On this landscape, we run fPGD with different choices of  H  . 
Revised: 1 has a chance of reaching the deep minimum by then. While H ∈ { 0 . 3 , 0 . 5 } can eventually reach the deep minimum after an order of magnitude longer, H ∈ { 0 . 7 , 0 . 9 } cannot reach it at all in 10 5 steps. The beneficial exploration of H is confirmed by the shape of the cumulative distribution functions (cdfs) in the lower row of Fig. 5.2 Non-Markovian fPGD as a trade-off between PGD and Anti-PGD

Original: Note that, in the two minima (and thesaddle in between at ( a + c ) / 2 ) fPGD is indeed a discretization of a fractional Ornstein–Uhlenbeck process, like the one considered in Theorems 1 and 2.
Revised: As discussed earlier, fPGD interpolates between two known Markovian methods: PGD ( H = 1 / 2) and Anti-PGD ( H = 0) . Both PGD and Anti-PGD have unique advantages; see [55] or [22] for PGD and [36] for Anti-PGD. It is thus natural to ask whether there are cases where non-Markovian fPGD ( H ∈ (0 , 1 / 2) ) is beneficial.

Original: We thus expect that a small Hurst parameterleads to  a faster escape from the shallow minimum (left) to the deep minimum (right).
Revised: While we do not have a complete answer to this question, we here present a setting that demonstrates that H ∈ (0 , 1 / 2) can indeed outperform both PGD ( H = 1 / 2 ) and Anti-PGD ( H = 0 ) simultaneously. We consider the following regularized quadratic: f ( x ) = 12 x ⊺ Mx + λ (cid:80) di =1 x 4 i , where x i is the i-th coordinate of x ∈ R d , M ∈ R d × d , d = 400 , and λ = 0 . 0001 .

Revised: If M has negative eigenvalues, this landscape has a d -dim.

Revised: saddle point at x = 0 .

Revised: If a stochastic optimizer can find the negative eigenvectors, it will reach a global minimum away from zero – while gradient descent remains stuck at zero.

Revised: The problem gets increasingly hard as the number of negative eigendirections decreases [22].

Revised: Indeed, [14] showed that vanilla gradient descent takes in the worst case exponential time (in the number of dimensions) to escape such saddle points.

Revised: We choose M as a diagonal matrix whose diagonal entries are uniformly sampled from [0 , 1] , and – to make it a challenging saddle – we multiply the 10 lowest of these entries by − 1 .

Revised: We test fPGD on this problem.

Original: We run fPGDfor N = 1000 iterations with a ﬁxed step size of ⌘ = 1 .  0 , initialised at x = 0 (the bottom of theshallow minimum).
Revised: We initialise at the saddle x = 0 and run fPGD with σ = 0 . 005 and η = 0 . 005 for 10 6 iterations with different Hurst parameters. (While in general σ and H are tuning parameters that can be jointly tuned, here σ = 0 .

Revised: 005 is a good choice for all values of H which we demonstrate in Appendix B.2.)

Revised: The results are presented in Fig.

Original: To stay in the setting of Theorems 1 and 2, we set  = 80 . [-
-]
Revised: In this problem, there is indeed a trade-off between uncorrelated ( H = 0 . 5) and anticorrelated perturbations ( H < 0 . 5 ): While a large Hurst parameter ( H = 0 . [-
-]5) moves away from the saddle quickly, the non-Markovian fPGD [Figure]

Revised: ( H = 0 . 4 ) finds the direction with negative curvature most quickly (in the sense that it gets below zero).

Revised: Once they have exited, all H perform similarly for a while.

Revised: Eventually, however, smaller Hurst parameters converge better to the bottom of the final minimum – thereby eventually catching up and surpassing the at first faster higher Hurst parameters.

Revised: Which H is best, will therefore depend on how many iterations the computational budget allows for.

Revised: Here, the non-Markovian H = 0 . 3 and

Revised: H = 0 .

Revised: 4 yield the best final accuracy.

Revised: This suggests that, if all other parameters are the same, different choices of Hurst parameters are beneficial in different situations.

Revised: It can thus be advantageous to tune and adapt the correlation of noise by choosing H .

Revised: It is especially interesting that both PGD ( H = 1 / 2 ) and Anti-PGD

Revised: ( H = 0 ) have shortcomings in this experiment: PGD has to increase the loss significantly in the beginning before exiting the saddle and does not converge to the bottom of the minimum in the end.

Revised: Anti-PGD takes a long time to exit the saddle in the beginning.

Revised: In contrast, non-Markovian fPGD (with H = 0 .or

Revised: H = 0 . 4 ) stays clear of these two undesirable extremes.

Revised: We stress, however, that the above experiment is a proof of concept and only shows that there exist landscapes where non-Markovian fPGD performs best.

Revised: On many other landscapes, PGD or Anti-PGD will still be better.

Revised: To make fPGD efficient on general landscapes, further work is needed to better understand the interplay between the scale σ and the correlation parameter H , which interact according to Eq.

Revised: Using this knowledge, one could design an adaptive method that jointly tunes σ and H to maximize the efficiency of perturbed-gradient methods.

Revised: For the above experiments, we again emphasize that our constant choice of σ is suitable for all H , as justified in Appendix B.2.

Revised:  Conclusion

Revised: We analyzed the behavior of a new type of stochastic noise with correlated increments.

Revised: We found that the amount of correlation, captured by the Hurst parameter H , has a direct effect on the overall magnitude of the process and its ability to explore.

Revised: This insight opens the doors to a new area of research where the noise of stochastic processes could be manipulated to speed up the convergence of an optimization method, as well as target specific minima (e.g., depending on their flatness properties).

Revised: Our experimental results validate our theoretical analysis.

Revised: We showed, on some non-convex potentials, that injection of fractional noise with small H speeds up the escape from a suboptimal minimum to a better one.

Revised: Also, we showed that there are indeed settings where non-Markovian (somewhat anti-correlated) noise outperforms the standard Markovian (uncorrelated or perfectly anti-correlated) noise.

Revised: Finally, one direction of interest would be to extend the analysis to more complex objective functions, which could potentially be achieved using the comparison theorem for SDEs [20].

Revised: Acknowledgement We would like to thank Nacira Agram and Bernt Øksendal for some helpful discussions in the early stage of this project, as well as the reviewers for their feedback that greatly helped us improve this manuscript.

Revised: Frank Proske acknowledges the financial support of the Center for

Revised: International Cooperation in Education, project No CPEA-LT-2016/10139, Norway.

Revised: Hans Kersting and Francis Bach acknowledge support from the French government under the management of the

Revised: Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference

Revised: Checklist

Revised: The checklist follows the references.

Revised: Please read the checklist guidelines carefully for information on how to answer these questions.

Revised: For each question, change the default [TODO] to [Yes] , [No] , or [N/A] .

Revised: You are strongly encouraged to include a justification to your answer , either by referencing the appropriate section of your paper or providing a brief inline description.

Revised: For example:

Revised: • Did you include the license to the code and datasets?

Revised: [Yes] • Did you include the license to the code and datasets?

Revised: [No] The code and the data are proprietary.

Revised: • Did you include the license to the code and datasets?

Revised: [N/A]

Revised: Please do not modify the questions and only use the provided macros for your answers.

Revised: Note that the

Revised: Checklist section does not count towards the page limit.

Revised: In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/answers below.

Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Revised: [Yes] (b) Did you describe the limitations of your work?

Revised: [Yes] See discussion at the end of the experiment section.

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [N/A]

Revised: Our work is more of theoretical nature.

Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Revised: [Yes] 2.

Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Revised: [Yes] (b) Did you include complete proofs of all theoretical results?

Revised: [Yes] Detailed proofs are available in the appendix.

Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Revised: [Yes] Code for experiments are provided as supplementary material.

Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Revised: [Yes] (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Revised: [N/A] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Revised: [No] All code were run on a Macbook 2020 with 2,3 GHz Quad-Core Intel Core i7.

Revised: We did not consider this info interesting to readers, but are happy to include it, if desired.

Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Revised: [N/A] (b) Did you mention the license of the assets?

Revised: [N/A] (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [N/A] (d) Did you discuss whether

Revised: and how consent was obtained from people whose data you’re using/curating?

Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Revised: [N/A] 5.

Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Revised: [N/A]

Revised: A Analysis A.1 Preliminaries

Revised: We present two well-known inequalities that connect the concept of expected supremum to the metric entropy of T denoted by log N ( T, d, ϵ ) .

Revised: First, we state Dudley’s inequality, which as stated by [49] gives the supremum of X t in terms of the metric entropy of T denoted by log N ( T, d, ϵ ) .

Revised: Theorem 6 (Dudley’s integral inequality) .

Revised: Let ( X t ) t ∈ T be a mean-zero random process on a metric space ( T, d ) with sub-Gaussian increments.

Revised: Then, there exists a constant C > 0 such that

Revised: [Equation]

Revised: We refer the reader to [49] for a detailed discussion and a proof of the theorem.

Revised: One can also obtain a lower bound on E [sup t ∈ T X t ] by Sudakov’s inequality (see Theorem 7.4.

Revised: by [49]) stated below.

Revised: Theorem 7 (Sudakov’s inequality) .

Revised: Let ( X t ) t ∈ T be a mean-zero Gaussian process on a metric space ( T, d ) .

Revised: Then, for any ϵ > 0 , we have

Revised: [Equation]

Revised: where d is the canonical metric d ( t, s )

Revised: =

Revised: ( E ( X t − X s ) 2 ) 1 / 2 .

Revised: A.2 One-dimensional case A.2.1 Upper bound

Revised: We first derive an upper bound on the expected supremum of X Ht introduced in Eq.

Revised: Theorem 1 (Upper bound, 1-D case) .

Revised: Consider the stochastic process

Revised: X Ht defined by

Revised: [Equation]

Revised: with X 0 = 0 , σ ∈ R and | β | < 1 .

Revised: Then, for t ∈ [0 , 1] ,

Revised: [Equation]

Revised: where C β = e 3 β and C are positive constants (independent of H ).

Revised: Proof.

Revised: We will use the bound on the supremum of a stochastic process X t discussed in Theoremthat requires defining a metric δ on [0 , 1] such that

Revised: [Equation]

Revised: First, we note that Z t,s :=

Revised: X Ht − X Hs is known to be randomly distributed [3].

Revised: For X 0 , we have E [ Z t,s ] = 0 since E [ X Ht ]

Revised: =

Revised: X 0 e βt

Revised: =

Revised: The variance of the increments of the process is derived in [3, 29] as

Revised: [Equation]

Revised: Next, we upper bound the two integrals.

Revised: We assume w.l.o.g. that β is positive.

Revised: Else, we can simply exchange the bound on the two integrals (since we have e − βz in one integral and e βz in the other integral).

Revised: Then we obtain

Revised: [Equation]

Revised: where we used e βz ≤ e β in the first inequality since ( t − s ) ≤ 1 (because t ∈ [0 , 1] ) and ¯ C β = e β in the last inequality.

Revised: For the other integral, (cid:90)

Revised: [Equation]

Revised: Therefore, for s ≤ t ,

Revised: [Equation]

Revised: where we used ¯ C β ≥ 1 and C β :

Revised: = e 3 β .

Revised: Let Z ∼ N (0 , 1) and f Z is the probability density function of Z .

Revised: We have

Revised: E

Revised: [Equation]

Revised: Recall that the integral of the Gaussian function f ( x ) =

Revised: e − ax 2 for a > 0 is (cid:82) ∞

Revised: −∞ e − ax 2 dx =

Revised: (cid:112) π a , therefore

Revised: E

Revised: [Equation]

Revised: [Equation]

Revised: where we need (cid:16) 12 − C β σ 2 ( t − s ) 2 H δ 2 ( t,s ) (cid:17) > 0 .

Revised: Next, we choose δ such that

Revised: [Equation]

Revised: [Equation]

Revised: Then

Revised: [Equation]

Revised: for r :=

Revised: [Equation]

Revised: bracket)

Revised: [Equation]

Revised: Hence

Revised: [Equation]

Revised: Then, using Theorem

Revised: [Equation]

Revised: we obtain that there exists a constant C 2 such that

Revised: [Equation]

Revised: Finally

Revised: [Equation]

Revised: we obtain an analytical form for the integral using (cid:82) c 0 (cid:112) log( cx ) dx = c √ π 2 (which can be derived using (cid:82) 1 0 (cid:113) log( 1 x ) dx = √ π 2 combined with a change of variables).

Revised: A.2.2 Lower bound

Revised: We now turn our attention to a lower bound.

Revised: This will demonstrate that the result obtained in the upper bound is actually sharp in terms of the dependency to the Hurst parameter H .

Revised: Theorem 2 (Lower bound, 1-D case) .

Revised: Consider the stochastic process

Revised: X Ht defined by

Revised: [Equation]

Revised: with X 0 = 0 , σ ∈ R and | β | < 1 .

Revised: Then

Revised: [Equation]

Revised: where C β = e − 2 | β | and c are positive constants (independent of H ).

Revised: Proof.

Revised: We will use Sudakov’s inequality that relies on the canonical metric

Revised: [Equation]

Revised: In order to get an expression for δ ( t, s )

Revised: [Equation]

Revised: we first note that Z t,s =

Revised: X

Revised: Ht − X Hs is known to be randomly distributed [3].

Revised: For X 0 , we have E [ Z t,s ] = 0 since E [ X Ht ]

Revised: =

Revised: X 0 e βt

Revised: =

Revised: The variance of the increments of the process is given in [3, 29] as

Revised: [Equation]

Revised: Next, we lower bound the two integrals.

Revised: We assume w.l.o.g. that β is positive.

Revised: Else, we can simply exchange the bound on the two integrals (since we have e − βz in one integral and e βz in the other integral).

Revised: Then we obtain

Revised: [Equation]

Revised: For the other integral,

Revised: [Equation]

Revised: Therefore, for s ≤ t ,

Revised: [Equation]

Revised: where C β = e − 2 | β | .

Revised: We therefore obtain the following canonical metric

Revised: [Equation]

Revised: Then

Revised: [Equation]

Revised: for r := (cid:16) εσC β (cid:17) 1 H .

Revised: So

Revised: N δ

Revised: [Equation]

Revised: ε

Revised: [Equation]

Revised: By Sudakov’s inequality (Theorem 7), we have

Revised: E

Revised: [Equation]

Revised: From our previous calculation, we have

Revised: [Equation]

Revised: Since arg max ε ∈ (0 , 1] ε (cid:112) − log( ε ) =

Revised: e − 12 , we get

Revised: [Equation]

Revised: A.3 Multi-dimensional case A.3.1 Lower bound

Revised: We first derive a lower bound on the expected supremum of X Ht introduced in Eq.

Revised: (2) in the multidimensional case.

Revised: Theorem 5 (Lower bound, d -dimensional case) .

Revised: Consider the d -dimensional stochastic process X H t ∈ R d defined by

Revised: [Equation]

Revised: where A ∈ R d × d and B Ht =

Revised: ( B H, 1 t , . . .

Revised: B H,dt ) is a fBM with H ∈ (0 , 12 ) .

Revised: Denote by a ij ( t ) the matrix entries of exp( tA )

Revised: A and assume that there exists i 0 ∈ { 1 , . . .

Revised: d } such that min t ∈ [0 , 1] a i 0 i 0 ( t ) ≥ − 1 + C 1 , where 0 < C 1 < 1 .

Revised: Then

Revised: [Equation]

Revised: where c is a constant independent of H .

Revised: Proof.

Revised: The solution X t (we omit the superscript H for simplicity) is given by

Revised: [Equation]

Revised: where ϕ ( t ) is the fundamental solution, that solves the matrix equation

Revised: [Equation]

Revised: [Equation]

Revised: In the case where the matrix A is constant with time, the fundamental solution is given by

Revised: [Equation]

Revised: By centering X t , we get

Revised: [Equation]

Revised: Using the stationarity of X t

Revised: [Equation]

Revised: we find that for t 1 ≤ t 2 ,

Revised: [Equation]

Revised: Denote by a ij ( t ) the matrix entries of exp( tA )

Original: · N  H . 
Revised: A . We have ∥

Revised: [Equation]

Revised: (cid:88)

Revised: j = a ij ( t 2 − t 1 − s ) B H,js ds 

Revised:  2 

Revised:  + E [( B H,it 2 − t 1 ) 2 ] + 2 (cid:90) t 2 − t d (cid:88)

Revised: j = a ij ( t 2 − t 1 − s ) E [ B H,js B H,it 2 − t 1 ] ds (cid:111) ( i ) =

Revised: d (cid:88)

Revised: i =

Revised: (cid:110)

Revised: E      (cid:90) t 2 − t d (cid:88)

Revised: j = a ij ( t 2 − t 1 − s ) B H,j s ds 

Revised:  2    (cid:124) (cid:123)(cid:122) (cid:125) =

Revised: A + | t 2 − t 1 | 2 H + (cid:90) t 2 − t d (cid:88)

Revised: j = a ij ( t 2 − t 1 − s ) (cid:0) | t 2 − t 1 | 2 H + | s | 2 H − | t 2 − t 1 − s |

Revised: 2 H (cid:1) ds (cid:124) (cid:123)(cid:122) (cid:125) =: B (cid:111) ≥ (cid:90) t 2 − t a ii

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: t

Revised: [Equation]

Original: Thus, for all
Revised: t [Equation] | [Equation]

Revised: H +

Revised: [Equation]

Revised: s

Revised: [Equation]

Revised: H

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: (cid:1) ds

Revised: [Equation]

Revised: t

Revised: [Equation]

Revised: − t 1 | 2 H where ( i ) uses

Revised: [Equation]

Revised: and

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: 2 H (cid:1) .

Revised: Next, assume that there exists i 0 ∈ { 1 , . . .

Revised: d } such that

Revised: [Equation]

Revised: where 0 ≤ C 1 ≤ 1 .

Revised: Then

Revised: [Equation]

Revised: Taking δ ( t 2 , t 1 )

Revised: =

Revised: C 1 | t 2 − t 1 | H as the metric on [0 , 1] , we obtain

Revised: [Equation]

Revised: for r := (cid:16) εC 1 (cid:17) 1 H .

Revised: So

Revised: [Equation]

Revised: By Sudakov’s inequality (Theorem 7), we have

Revised: [Equation]

Revised: where C 2 is a constant independent of H .

Revised: From our previous calculation, we have

Revised: [Equation]

Revised: =

Revised: [Equation]

Revised: [Equation]

Revised: ε

Revised: [Equation]

Revised: Since arg max ε ∈ (0 , 1] ε (cid:112) − log( ε ) =

Revised: e − 12 , we get sup 0 <ε ≤ 2 C 1 ε (cid:112) log N δ ( ε ) ≥ e −(cid:115)

Revised: [Equation]

Revised: (cid:18)

Revised: [Equation]

Revised: e

Revised: [Equation]

Revised: (cid:19) .

Revised: We conclude that

Original: H 2 (0 , 1) , the employed fBM B Ht , deﬁned on the time interval t 2 [0 , N ] , is by Rmk.
Revised: [Equation] Alternative proof for lower bound Recall we have shown that [Equation] Let’s first focus on the term A :

Revised: [Equation]

Revised: Then, note that

Original: [-1 distributedlike 80 . 
-]
Revised: [-1 distributedlike 80 . 
-][Equation] Therefore E [Equation]

Revised: For the term B

Original: · B Ht on t 2 [0 , 1] – in the sense that the discrete steps { BHk ; k = 0 , . . .
Revised: [Equation] we have [Equation] Combining the lower bounds on A and B

Original: [-, N } are distributedlike { 80 . 
-]
Revised: [-, N } are distributedlike { 80 . 
-][Equation] we obtain [Equation]

Original: · B Hk/N ; k = 0 ,  . . . , N } .
Revised: Let C 1 = min t ∈ [0 , 1] ( a 1 ( i ) t 2 + a 2 ( i ) t + 1) > 0 . We can then choose the metric on [0 , 1] as δ ( t 2 , t 1 ) = C 1 | t 2 − t 1 | H . The rest of the proof is identical but we obtain a different constant C 1 that covers different scenarios of the entries of the matrix A .

Original: By this H -dependent choice of  , we give all fBMs the same ﬁnalvariance of 80 . 0 2 which lets us zoom inon the impact of correlation.
Revised: We now turn our attention to an upper bound. This will demonstrate that the result obtained in the lower bound is actually sharp in terms of the dependency to the Hurst parameter H .

Original: (Otherwise the perturbationswould have different variances; this way we disentangle the correlation from the variance.)
Revised: Theorem 4 (Upper bound, d -dimensional case) .

Original: We notethat here the experimental outcomes are not sensitive to the precise value of  .
Revised: Consider the d -dimensional stochastic process X Ht ∈ R d defined by [Equation] where A ∈ R d × d and B Ht = ( B H, 1 t , . . .

Original: Fig.
Revised: B H,dt ) is a fBM with H ∈ (0 , 1) . Denote by a ij ( t ) the matrix entries of exp( tA ) A . Then [Equation]

Revised: where C 2 = max di,j =1 sup t ∈ [0 , 1] | a ij ( t ) | 2 + max di,j =1 sup

Revised: [Equation]

Revised: ∈

Revised: [Equation]

Revised: ] | a ij ( t ) | + 1 and C > 0 is a constant independent of H .

Revised: Proof.

Revised: The solution X t (we omit the superscript H for simplicity) is given by

Revised: [Equation]

Revised: where ϕ ( t ) is the fundamental solution, that solves the matrix equation

Revised: [Equation]

Revised: In the case where the matrix A is constant with time, the fundamental solution is given by

Revised: [Equation]

Revised: By centering X t , we get

Revised: [Equation]

Revised: Using the stationarity of X t

Revised: [Equation]

Revised: we find that for t 1 ≤ t 2 ,

Revised: [Equation]

Revised: [Equation]

Revised: Denote by a ij ( t ) the matrix entries of exp( tA )

Revised: A .

Revised: We have

Revised: [Equation]

Revised: where ( i ) uses

Revised: E

Revised: [Equation]

Revised: and

Revised: E

Revised: [Equation]

Revised: We will once again build on the proof in the 1-dimensional case.

Revised: To do so, we first observe that, given an interval T = [0 , 1] ,

Revised: [Equation]

Revised: thus

Revised: [Equation]

Revised: Focusing on the i -th coordinate, we first get for the term A that

Revised: [Equation]

Revised: Then, note that

Revised: [Equation]

Revised: Therefore

Revised: E

Revised: [Equation]

Revised: For the term B

Revised: [Equation]

Revised: we have

Revised: [Equation]

Revised: Combining the bound on the terms A and B, we obtain the following

Revised: [Equation]

Revised: where

Revised: [Equation]

Revised: Taking δ 2 ( t 2 , t 1 ) =

Revised: dC 2 | t 2 − t 1 | 2 H as the metric on [0 , 1] , we obtain

Revised: [Equation]

Revised: for r

Revised: [Equation]

Revised: Hence

Revised: [Equation]

Revised: Then, using Theorem 6 applied to each centered component process (with positive and negative sign)

Revised: [Equation]

Revised: we obtain that there exists a constant C such that

Revised: [Equation]

Revised: Finally, we obtain an analytical form for the integral using (cid:82) c 0 (cid:112) log( cx ) dx = c √ π 2 (which can be derived using (cid:82) 1 0 (cid:113) log( 1 x ) dx = √ π 2 combined with a change of variables).

Revised: B Additional experiments B.1 Demonstration of the 1 / √ H scaling from Theorems 1 and

Revised: In this section, we demonstrate the 1 / √ H scaling of the expected supremum of the one-dimensional fOUP.

Revised: Recall that Theorem 1 proved that

Revised: [Equation]

Revised: for C > 0 .

Revised: Theorem 2 and Corollary 3 also provide a lower bound

Revised: [Equation]

Revised: although they either require 1) σ > e 2 | β | 12 and | β | < 1 or 2) 0 < σ < e 2 | β |− 12 and | β | ≤ 14 .

Revised: Here, we want to verify whether the lower bound E (cid:104) sup t ∈ [0 , 1] ∥

Revised: X Ht ∥ (cid:105) ∼ 1 √ H holds generally, even outside the range required for σ and β .

Revised: To this end, we ran a Monte Carlo simulation with 10simulations of this expectation and plotted it next to a function of the form f ( x ) =

Revised: w 0 + w 1 1 √ H , where we fitted ( w 0 , w 1 ) by ordinary least squares.

Revised: For the discretization, we used 10 5 steps (i.e. a step size η of 10 − 5 ).

Revised: For H , we chose an evenly spaced grid of mesh size 0 .

Revised: from 0 .

Revised: 2 to 0 .

Revised: We did not include H = 0 .

Revised: 1 here because the Monte-Carlo approximation is known not be a good approximation for such small values of H .

Revised: This issue is described in more detail in [8] (see e.g. Figure 1 and description in the text).

Revised: We remark that we still observe a relatively good

Revised: The resulting plots are depicted in Fig.

Revised: B.1 for noise scales σ ∈ { 10 − 3 , 10 − 2 , 10 − 1 , 10 0 } .

Revised: In each plot, we used β = 1 / 2 such that all values of σ violate both conditions: 1) σ > e 2 | β |− 1 / 2 and | β | <or 2) 0 < σ < e 2 | β |− 12 and | β | ≤ 14 . .

Revised: Nonetheless, the fit between the simulation and the theoretical scaling is almost perfect.

Revised: This confirms our theoretical findings and shows that the scaling also holds outside the regime of Theorem 2 and Corollary 3.

Revised: B.2 Justification for choice of σ in Section 5.

Revised: In this section, we justify the use of σ = 0 .

Revised: 005 in all experiments from Section 5.2.

Revised: There, we used σ = 0 .

Revised: 005 for all H ∈ { 0 . 0 , 0 . 1 , 0 . 2 , 0 . 3 , 0 .

Revised: 4 , 0 . 5 } in Figure 4, i.e. in both the Markovian { 0 . 0 , 0 . 5 } and the non-Markovian cases { 0 . 1 , 0 .

Revised: Here, we explain why σ is not chosen dependent on H .

Original: 2 depicts our ﬁndings.
Revised: Our justification is simple: The choice of σ = 0 . 005 is a good choice for all H . To see this, consider Figure 6.

Revised: For a fixed H ∈ { 0 . 0 , 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 } and all σ ∈ { 0 . 05 , 0 .

Revised: } , they depict the performance on the regularized quadratic from Section 5.2, analogous to Figure 4.

Revised: We observe that indeed, for all choices of H , the curve for σ = 0 .

Revised: 005 is best – in the sense that it exits the saddle fastest (by reaching a negative value) and converges quickly to the final minimum.

Revised: Clearly, for all H , there is no point in picking an even smaller or larger order of magnitude for σ .

Revised: Hence, we conclude that the choice of σ = 0 .

Revised: 005 is close to the optimal choice for all H .

Revised: While we acknowledge that grid-searching the optimal σ at a finer scale would maybe uncover small differences in the optimal σ , such precision would be beyond our goal of a proof-of-concept and would distract from the bigger picture.

Revised: In fact, the constant σ also ensures that the increments of fBM with different

Revised: H are distributed the same (but correlated differently).

Revised: Thus, this experimental set-up allows us to zoom in on the effect of the Hurst parameter H – with the result that the non-Markovian H = 0 .performs best.

Revised: B.3 Moving to better local minima in bi-stable potentials

Revised: We consider the behavior of fPGD in a bi-stable optimization landscape, given by f ( x ) =

Revised: [Equation]

Revised: (27) where χ is the indicator function and the parameters ( v 0 , v 1 , v 2 , a, c, m ) determine the position and shape of two adjacent local minima.

Revised: Similar landscapes have been studied in the context of escaping spurious local minima [55] and in the context of generalization in machine learning by selecting flat minima [33, 53], which are by some believed to generalized better.

Revised: We here study both cases.

Revised: For the former case, we set ( v 0 , v 1 , v 2 , a, c, m ) =

Revised: (0 . 0 , 45 . 0 , − 12 .

Revised: 5 , 3 . 5 , 13 . 9 , 18 . 0) ; for the corresponding landscape see the upper left subplot of Fig.

Revised: For the latter case, we set ( v 0 , v 1 , v 2 , a, c, m ) =

Revised: (0 . 0 , 30 . 0 , 0 . 0 , 1 . 2 , 8 . 7 , 15 . 0) ; for the for the corresponding landscape see the upper left subplot of Fig. 8.

Revised: On these landscapes, we run fPGD with different choices of H .

Revised: Note that, in the two minima (and the saddle in between at ( a + c ) / 2 ) fPGD is indeed a discretization of a fractional Ornstein–Uhlenbeck process, like the one considered in Theorems 1 and 2.

Revised: We thus expect that a small Hurst parameter leads to a faster escape from the shallow minimum (left) to the deep minimum (right) in Fig.

Revised: 7, or from the sharp to the flat minimum in Fig.

Revised: In both cases, we run fPGD for N = 1000 iterations with a fixed step size of η = 1 . 0 , initialised at x = 0 (the bottom of the shallow/sharp minimum).

Original: {+To stay in the setting of Theorems 1 and 2, we set σ = 80 . 
+}
Revised: {+To stay in the setting of Theorems 1 and 2, we set σ = 80 . 
+}

Revised: · N − H .

Revised: Thus, for all H ∈ (0 , 1) , the employed fBM B Ht , defined on the time interval t ∈ [0 , N ] , is by Rmk.

Original: {+1 distributed like 80 . 
+}
Revised: {+1 distributed like 80 . 
+}

Original: {+· B Ht on t ∈ [0 , 1] – in the sense that the discrete steps { B Hk ; k = 0 , . . . , N } are distributed like { 80 . 
+}
Revised: {+· B Ht on t ∈ [0 , 1] – in the sense that the discrete steps { B Hk ; k = 0 , . . . , N } are distributed like { 80 . 
+}

Revised: · B Hk/N ; k = 0 , . . . , N } .

Revised: By this H -dependent choice of σ , we give all fBMs the same final variance of 80 . 0 2 which lets us zoom in on the impact of correlation.

Revised: (Otherwise the perturbations would have different variances; this way we disentangle the correlation from the variance.)

Original: We  say that fPGD is in the shallow minimum at iteration k if its iterate x kis smaller than a , and in the deep minimum if it is larger than c ; both a and c are plotted as dashedlines in the optimization landscape (top row) and in the trajectories (bottom row).
Revised: We note that here the experimental outcomes are not sensitive to the precise value of σ . Figs. 7 and 8 depict our findings. We say that fPGD is in the shallow/sharp minimum at iteration k if its iterate x k is smaller than a , and in the deep/flat minimum if it is larger than c ; both a and c are plotted as dashed lines in the optimization landscape (top row) and in the trajectories (bottom row).

Revised: [Figure]

Original: In the bottom row,for all H , we start below the lower dashed line a (shallow minimum) and then cross above the higherdashed line c (deep minimum).
Revised: In the bottom row, for all H , we start below the lower dashed line a (shallow/sharp minimum) and then cross above the higher dashed line c (deep/flat minimum).

Original: It is evident that a smaller H enables faster crossings.
Revised: It is evident that a smaller H enables faster crossings.

Original: We thus escapefaster from the spurious shallow minimum to the desirable deep one.
Revised: We thus escape faster from the spurious shallow/sharp minimum to the desirable deep/flat one.

Original: (Note, however, if the noise isnot cooled down, one might oscillate back.)
Revised: (Note, however, if the noise is not cooled down, one might oscillate back.)

Original: The upper right plot depicts the cumulative distributionfunction of the ﬁrst-exit time (i.e. the ﬁrst time fPGD reaches the deep minimum, above c ).
Revised: The upper right plot depicts the cumulative distribution function of the first-exit time (i.e. the first time fPGD reaches the deep/flat minimum, above c ).

Original: Again,we can observe that for a small Hurst parameter the probability of a fast exit is signiﬁcantly higher.
Revised: Again, we can observe that for a small Hurst parameter the probability of a fast exit is significantly higher.

Original: Overall, these ﬁndings for fPGD match the continuous-time result of Theorems 1 and 2.
Revised: Overall, these findings for fPGD match the continuous-time result of Theorems 1 and 2.

Original: Anotherbi-stable landscape (with a sharp and a ﬂat minimum) is studied in Appendix B with the same ﬁndings.

Original: Note that physicists [41] have also observed faster escapes for smaller Hurst parameters on a related
Revised: Note that physicists [45] have also observed faster escapes for smaller Hurst parameters on a related

Original: Kramer-like escape problem.
Revised: Kramer-like escape problem.

Original: 5.2 Non-Markovian fPGD as a trade-off between PGD and Anti-PGD

Original: As discussed earlier, fPGD interpolates between two known Markovian methods: PGD ( H = 1 / 2)and Anti-PGD ( H = 0) .

Original: Both PGD and Anti-PGD have unique advantages; see [51] or [20] for

Original: PGD and [33] for Anti-PGD.

Original: It is thus natural to ask whether there are cases where non-MarkovianfPGD ( H 2 (0 , 1 / 2) ) is beneﬁcial.

Original: While we do not have a complete answer to this question,we here present a setting that demonstrates that H 2 (0 , 1 / 2) can indeed outperform both PGD( H = 1 / 2 ) and Anti-PGD ( H = 0 ) simultaneously.

Original: We consider the following regularized quadratic:f ( x ) =

Original: 12 x | Mx +

Original: P di =1 x 4 i , where x i is the i-th coordinate of x 2 R d , M 2 R d ⇥ d , d = 400 ,and  = 0 .

Original: If M has negative eigenvalues

Original: If a stochastic optimizer can ﬁnd the negative eigenvectors, it will reach a global minimum away

Original: [Figure]
Revised: [Figure]

Original: from zero – while gradient descent remains stuck at zero.

Original: The problem gets increasingly hard asthe number of negative eigendirections decreases [20].

Original: Indeed, [13] showed that vanilla gradientdescent takes in the worst case exponential (in the number of dimensions) time to escape such saddlepoints.

Original: We choose M as a diagonal matrix whose diagonal entries are uniformly sampled from [0 , 1] ,and – to make it a challenging saddle – we multiply the 10 lowest of these entries by  1 .

Original: We test fPGD on this problem.

Original: We initialise at the saddle x = 0 and run fPGD with  = 0 .

Original: 005 and⌘ = 0 .

Original: 005 for 10 6 iterations with different Hurst parameters.

Original: (While in general  and H are tuningparameters that can be jointly tuned, here  = 0 .

Original: 005 is a good choice for all values of H which wedemonstrate in Appendix B.4.)

Original: The results are presented in Fig.

Original: In this problem, there is indeeda trade-off between uncorrelated ( H = 0 . 5) and anticorrelated perturbations

Original: ( H < 0 .

Original: 5 ): While alarge Hurst parameter ( H = 0 . 5) moves away from the saddle quickly, the non-Markovian fPGD( H = 0 . 4 ) ﬁnds the direction with negative curvature most quickly (in the sense that it gets belowzero).

Original: Once they have exited, all H perform similar for a while.

Original: Eventually, however, smaller Hurstparameters converge better to the bottom of the ﬁnal minimum – thereby eventually catching up andsurpassing the at-ﬁrst faster higher Hurst parameters.

Original: Which H is best, will therefore depend onhow many iterations the computational budget allows for.

Original: Here, the non-Markovian H = 0 . 3 and

Original: H = 0 .

Original: 4 yield the best ﬁnal accuracy.

Original: This suggests that, if all other parameters are the same, different choices of Hurst parameters arebeneﬁcial in different situations.

Original: It can thus be advantageous to tune and adapt the correlation ofnoise by choosing H .

Original: It is especially interesting that both PGD ( H = 1 / 2 ) and Anti-PGD

Original: ( H = 0 )have shortcomings in this experiment: PGD has to increase the loss signiﬁcantly in the beginningbefore exiting the saddle and does not converge to the bottom of the minimum in the end.

Original: Anti-PGDtakes a long time to exit the saddle in the beginning.

Original: In contrast, non-Markovian fPGD (with H = 0 .

Original: 3or H = 0 . 4 ) stays clear of these two undesirable extremes.

Original: We stress, however, that the aboveexperiment is a proof of concept and only shows that there exist landscapes where non-MarkovianfPGD performs best.

Original: On many other landscapes, PGD and Anti-PGD will still be best.

Original: To make fPGD efﬁcient on general landscapes, further work is needed to better understand theinterplay between the scale  and the correlation parameter H , which interact according to Eq.

Original: Using this knowledge, one could design an adaptive method that jointly tunes  and H to maximizethe efﬁciency of perturbed-gradient methods.

Original: For the above experiments, we again emphasize thatour constant choice of  is suitable for all H , as justiﬁed in Appendix B.4.

Original: Conclusion

Original: We analysed the behavior of a new type of stochastic noise with correlated increments.

Original: We foundthat the amount of correlation, captured by the Hurst parameter H , has a direct effect on the overallmagnitude of the process and its ability to explore.

Original: This insight opens the doors to a new area ofresearch where the noise of stochastic processes could be manipulated to speed up the convergence ofan optimization method, as well as target speciﬁc minima (e.g., depending on their ﬂatness properties).

Original: Our experimental results validate our theoretical analysis.

Original: We showed that, in bi-stable potentials,injection of fractional noise with small H speeds up the escape from a suboptimal minimum to a betterone.

Original: Also, we showed that there are indeed settings where non-Markovian (somewhat anti-correlated)noise outperforms the standard Markovian (uncorrelated or perfectly anti-correlated) noise.

Original: Checklist

Original: The checklist follows the references.

Original: Please read the checklist guidelines carefully for information onhow to answer these questions.

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Original: You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Original: For example:

Original: • Did you include the license to the code and datasets?

Original: [Yes]

Original: • Did you include the license to the code and datasets?

Original: [No] The code and the data are

Original: proprietary.

Original: • Did you include the license to the code and datasets?

Original: [N/A]

Original: Please do not modify the questions and only use the provided macros for your answers.

Original: Note that the

Original: Checklist section does not count towards the

Original: page

Original: limit.

Original: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Original: For all authors...

Original: (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s

Original: contributions and scope?

Original: [Yes]

Original: (b) Did you describe the limitations of your work?

Original: [Yes] See discussion at the end of the

Original: experiment section.

Original: (c) Did you discuss any potential negative societal impacts of your work?

Original: [N/A]

Original: Our work

Original: is more of theoretical nature.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?

Original: [Yes]

Original: If you are including theoretical results...

Original: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes]

Original: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] Detailed proofs are

Original: available in the appendix.

Original: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main ex-

Original: perimental results (either in the supplemental material or as a URL)?

Original: [Yes] Code for

Original: experiments are provided as supplementary material.

Original: (b) Did you specify all the training details (e

Original: g., data splits, hyperparameters, how theywere chosen)?

Original: [Yes]

Original: (c) Did you report error bars (e

Original: g., with respect to the random seed after running experiments multiple times)?

Original: [N/A]

Original: (d) Did you include the total amount of compute and the type of resources used (e

Original: g., typeof GPUs, internal cluster, or cloud provider)?

Original: [No] All code were run on a Macbook

Original: 2020 with 2,3 GHz Quad-Core Intel Core i7.

Original: We did not consider this info interesting

Original: to readers, but are happy to include it, if desired.

Original: If you are using existing assets (e

Original: g., code, data, models) or curating/releasing new assets...

Original: (a) If your work uses existing assets, did you cite the creators?

Original: [N/A]

Original: (b) Did you mention the license of the assets?

Original: [N/A]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [N/A]

Original: [Table]

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: [Table]

Original: using/curating?

Original: [N/A]

Original: [Table]

Original: (e) Did you discuss whether the data you are using/curating contains personally identiﬁable

Original: [Table]

Original: information or offensive content?

Original: [N/A]

Original: [Table]

Original: If you used crowdsourcing or conducted research with human subjects...

Original: [Table]

Original: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: [Table] applicable? [N/A] [Table]
Revised: [Figure]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: [Table]

Original: Board (IRB) approvals, if applicable?

Original: [N/A]

Original: [Table]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: [Table]

Original: spent on participant compensation?

Original: [N/A]

Original: [Table]