Original: A CCELERATING N ONCONVEX L EARNING VIA R EPLICA E XCHANGE L ANGEVIN D IFFUSION
Revised: A CCELERATING N ONCONVEX L EARNING VIA R EPLICA E XCHANGE L ANGEVIN D IFFUSION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Langevin diffusion is a powerful method for nonconvex optimization, which enables the escape from local minima by injecting noise into the gradient.
Revised: Langevin diffusion is a powerful method for nonconvex optimization, which enables the escape from local minima by injecting noise into the gradient.

Original: In particular, the temperature parameter controlling the noise level gives rise to a tradeoff between “global exploration” and “local exploitation”, which correspond to high and low temperatures.
Revised: In particular, the temperature parameter controlling the noise level gives rise to a tradeoff between “global exploration” and “local exploitation”, which correspond to high and low temperatures.

Original: To attain the advantages of both regimes, we propose to use replica exchange, which swaps between two Langevin diffusions with different temperatures.
Revised: To attain the advantages of both regimes we propose to use replica exchange, which swaps between two Langevin diffusions with different temperatures.

Original: We theoretically analyze the acceleration effect of replica exchange from two perspectives: (i) the convergence in (cid:31) 2 -divergence, and (ii) the large deviation principle.
Revised: We theoretically analyze the acceleration effect of replica exchange from two perspectives: (i) the convergence in (cid:31) 2 -divergence and (ii) the large deviation principle.

Original: Such an acceleration effect allows us to faster approach the global minima.
Revised: Such an acceleration effect allows us to faster approach the global minima.

Original: Furthermore, by discretizing the replica exchange Langevin diffusion, we obtain a discrete-time algorithm.
Revised: Furthermore, by discretizing the replica exchange Langevin diffusion, we obtain a discrete-time algorithm.

Original: For such an algorithm, we quantify its discretization error in theory and demonstrate its acceleration effect in practice.
Revised: For such an algorithm, we quantify its discretization error in theory and demonstrate its acceleration effect in practice

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: We consider the problem of minimizing a nonconvex objective function, which arises from numerous machine learning problems such as training neural networks.
Revised: We consider the problem of minimizing a nonconvex objective function, which arises from numerous machine learning problems such as training neural networks.

Original: However, due to the existence of spurious local minima, nonconvex optimization remains challenging in both theory and practice.
Revised: However, due to the existence of spurious local minima, nonconvex optimization remains challenging in both theory and practice.

Original: To overcome this difﬁculty, one idea is to construct a diffusion process whose invariant distribution concentrates around the global minima (Chiang et al., 1987).
Revised: To overcome this difﬁculty, one idea is to construct a diffusion process whose invariant distribution concentrates around the global minima (Chiang et al., 1987).

Original: If we run such a diffusion process for a sufﬁciently long time, we expect to draw from its invariant distribution a point that is close to a global minimum with high probability.
Revised: If we run such a diffusion process for a sufﬁciently long time, we expect to draw from its invariant distribution a point that is close to a global minimum with high probability.

Original: One commonly used diffusion process is the Langevin diffusion, which is closely related to ﬁrstorder optimization (Gidas, 1985).
Revised: One commonly used diffusion process is the Langevin diffusion, which is closely related to ﬁrstorder optimization (Gidas, 1985).

Original: In speciﬁc, the Langevin diffusion can be viewed as gradient ﬂow with injected noise, where the noise is scaled by a temperature parameter.
Revised: In speciﬁc, the Langevin diffusion can be viewed as gradient ﬂow with injected noise, where the noise is scaled by a temperature parameter.

Original: Such a temperature parameter gives rise to a tradeoff between two opposite effects, namely “global exploration” and “local exploitation”.
Revised: Such a temperature parameter gives rise to a tradeoff between two opposite effects, namely “global exploration” and “local exploitation”.

Original: More speciﬁcally, a higher temperature results in a larger injected noise, which increases the probability of escaping from spurious local minima and facilitates the global exploration of the whole domain.
Revised: More speciﬁcally, a higher temperature results in a larger injected noise, which increases the probability of escaping from spurious local minima and facilitates the global exploration of the whole domain.

Original: On the other hand, with a lower temperature, the Langevin diffusion behaves more like the gradient ﬂow, which focuses more on exploiting the local geometry to decrease the objective value, yielding a more concentrated invariant distribution.
Revised: On the other hand, with a lower temperature, the Langevin diffusion behaves more like the gradient ﬂow, which focuses more on exploiting the local geometry to decrease the objective value, yielding a more concentrated invariant distribution.

Original: In this paper, we aim to bridge the gap between “global exploration” and “local exploitation” in nonconvex optimization with replica exchange, which originates from parallel tempering Markov chain Monte Carlo methods for sampling from a multimodal distribution (Earl & Deem, 2005).
Revised: In this paper, we aim to bridge the gap between “global exploration” and “local exploitation” in nonconvex optimization with replica exchange, which originates from parallel tempering Markov chain Monte Carlo methods for sampling from a multimodal distribution (Earl & Deem, 2005).

Original: In contrast to the standard Langevin diffusion driven by a single temperature, the replica exchange
Revised: In contrast to the standard Langevin diffusion driven by a single temperature, the replica exchange

Original: Langevin diffusion consists of two Langevin diffusions with low and high temperatures, which locally exploits the geometry and globally explores the domain, respectively.
Revised: Langevin diffusion consists of two Langevin diffusions with low and high temperatures, which locally exploits the geometry and globally explores the domain, respectively.

Original: In particular, the two
Revised: In particular, the two

Original: Langevin diffusions exchange their positions following a speciﬁc swapping scheme, which ensures that the invariant distribution concentrates around the global minima.
Revised: Langevin diffusions exchange their positions following a speciﬁc swapping scheme, which ensures that the invariant distribution concentrates around the global minima.

Original: See Figures 1.(a)-(b) for an illustration of swapping.
Revised: See Figures 1.(a)-(b) for an illustration of swapping.

Original: Compared with the standard Langevin diffusion, replica exchange accelerates the rate of convergence to the invariant distribution.
Revised: Compared with the standard Langevin diffusion, replica exchange accelerates the rate of convergence to the invariant distribution.

Original: As a result, with replica exchange we[Figure] obtain a more accurate solution to the nonconvex optimization problem within a shorter time.
Revised: As a result, with replica exchange we obtain a more accurate solution to the nonconvex optimization problem within a shorter time.

Original: It is worth mentioning that, although in this paper we focuses on the case of swapping two Langevin diffusions, our theory and method naturally extend to multiple Langevin diffusions.
Revised: It is worth mentioning that, although in this paper we focuses on the case of swapping two Langevin diffusions, our theory and method naturally extend to multiple Langevin diffusions.

Original: Our theory quantiﬁes the acceleration effect of replica exchange from two perspectives.
Revised: Our theory quantiﬁes the acceleration effect of replica exchange from two perspectives.

Original: First, we use the theory of Markov semigroup, particularly the Dirichlet form, to characterize the evolution of the (cid:31) 2 -divergence between the distribution at time t and the invariant distribution.
Revised: First, we use the theory of Markov semigroup, particularly the Dirichlet form, to characterize the evolution of the (cid:31) 2 -divergence between the distribution at time t and the invariant distribution.

Original: In speciﬁc, we show that, compared with the standard Langevin diffusion, replica exchange boosts the Dirichlet form with a nonnegative term, which results in a faster decay of the (cid:31) 2 -divergence.
Revised: In speciﬁc, we show that, compared with the standard Langevin diffusion, replica exchange boosts the Dirichlet form with a nonnegative term, which results in a faster decay of the (cid:31) 2 -divergence.

Original: Further combined with the Poincar´e inequality, we establish the exponential rate of convergence of the (cid:31) 2 -divergence to zero, where the acceleration effect of replica exchange is characterized by the Poincar´e constant.
Revised: Further combined with the Poincar´e inequality, we establish the exponential rate of convergence of the (cid:31) 2 -divergence to zero, where the acceleration effect of replica exchange is characterized by the Poincar´e constant.

Original: We illustrate such an acceleration effect in Figure 1.(c).
Revised: We illustrate such an acceleration effect in Figure 1.(c).

Original: Second, we use the large deviation principle (LDP) to characterize the exponential decay of the probability that the empirical measure of the sample path deviates from the invariant distribution.
Revised: Second, we use the large deviation principle (LDP) to characterize the exponential decay of the probability that the empirical measure of the sample path deviates from the invariant distribution.

Original: In particular, we show that replica exchange boosts the LDP rate function, which implies a faster exponential decay rate.
Revised: In particular, we show that replica exchange boosts the LDP rate function, which implies a faster exponential decay rate.

Original: See Figure 1.(d) for an illustration.
Revised: See Figure 1.(d) for an illustration.

Original: Both perspectives are built on the inﬁnitesimal generator of the replica exchange
Revised: Both perspectives are built on the inﬁnitesimal generator of the replica exchange

Original: Langevin diffusion.
Revised: Langevin diffusion.

Original: We show that essentially the acceleration effects are the consequences of an additional term in the inﬁnitesimal generator, which is introduced by replica exchange.
Revised: We show that essentially the acceleration effects are the consequences of an additional term in the inﬁnitesimal generator, which is introduced by replica exchange.

Original: In addition to quantifying the acceleration effect of replica exchange, we use the Euler scheme to discretize the replica exchange Langevin diffusion and propose a discrete-time algorithm.
Revised: In addition to quantifying the acceleration effect of replica exchange, we use the Euler scheme to discretize the replica exchange Langevin diffusion and propose a discrete-time algorithm.

Original: We establish an upper bound of the discretization error via Gr¨onwall’s inequality.
Revised: We establish an upper bound of the discretization error via Gr¨onwall’s inequality.

Original: We also discuss several issues related to the parameters of the replica exchange Langevin diffusion, especially the swapping intensity and temperatures.
Revised: We also discuss several issues related to the parameters of the replica exchange Langevin diffusion, especially the swapping intensity and temperatures.

Original: We defer these discussions to x A of the appendix.
Revised: We defer these discussions to x A of the appendix.

Original: Related Work:
Revised: Related Work:

Original: The idea of nonconvex optimization via diffusion process dates back to simulated annealing ( ˇCern`y, 1985) and is systematically studied in Chiang et al.
Revised: The idea of nonconvex optimization via diffusion process dates back to simulated annealing ( ˇCern`y, 1985) and is systematically studied in Chiang et al.

Original: For the application of the Langevin diffusion in nonconvex optimization, see, for example, Gidas (1985) and Geman & Hwang (1986) and the followup works.
Revised: For the application of the Langevin diffusion in nonconvex optimization, see, for example, Gidas (1985) and Geman & Hwang (1986) and the followup works.

Original: More recently, Dalalyan & Tsybakov (2009);
Revised: More recently, Dalalyan & Tsybakov (2009);

Original: Bubeck et al.
Revised: Bubeck et al.

Original: ; Dalalyan (2017) analyze the nonasymptotic rate of convergence of the Langevin diffusion for strongly convex objective functions.
Revised: ; Dalalyan (2017) analyze the nonasymptotic rate of convergence of the Langevin diffusion for strongly convex objective functions.

Original: Moreover, Durmus et al.
Revised: Moreover, Durmus et al.

Original: Zhang et al.
Revised: Zhang et al.

Original: ; Raginsky et al.
Revised: ; Raginsky et al.

Original: (2017) further extend the analysis to handle nonconvexity and stochastic gradient.
Revised: (2017) further extend the analysis to handle nonconvexity and stochastic gradient.

Original: See also, for example, Belloni et al.
Revised: See also, for example, Belloni et al.

Original: ; Ge et al.
Revised: ; Ge et al.

Original: ; Hazan et al.
Revised: ; Hazan et al.

Original: ; Cheng et al.
Revised: ; Cheng et al.

Original: (2017) for related works on the extensions of the Langevin diffusion and other diffusions.
Revised: (2017) for related works on the extensions of the Langevin diffusion and other diffusions.

Original: However, all these works focus on the setting with a single temperature.
Revised: However, all these works focus on the setting with a single temperature.

Original: In comparison, we study the acceleration effect of replica exchange, which utilizes more than one temperature.
Revised: In comparison, we study the acceleration effect of replica exchange, which utilizes more than one temperature.

Original: Hence, our work is orthogonal to these existing works and can potentially be applied to accelerate other diffusion processes. 
Revised: Hence, our work is orthogonal to these existing works and can potentially be applied to accelerate other diffusion processes. [Figure]

Original: In a parallel line of works, the idea of replica exchange is studied in the context of parallel tempering Markov chain Monte Carlo for sampling from a multimodal distribution.
Revised: In a parallel line of works, the idea of replica exchange is studied in the context of parallel tempering Markov chain Monte Carlo for sampling from a multimodal distribution.

Original: See, for example, Earl & Deem (2005); Sindhikara et al.
Revised: See, for example, Earl & Deem (2005); Sindhikara et al.

Original: (2008) and the references therein.
Revised: (2008) and the references therein.

Original: More recently,
Revised: More recently,

Original: Woodard et al.
Revised: Woodard et al.

Original: (2009) study the mixing time of parallel tempering, while Dupuis et al.
Revised: (2009) study the mixing time of parallel tempering, while Dupuis et al.

Original: (2012) extend the swapping rate of replica exchange to inﬁnity and establish the corresponding LDP.
Revised: (2012) extend the swapping rate of replica exchange to inﬁnity and establish the corresponding LDP.

Original: In addition, another well-studied method is simulated tempering, a technique similar to replica exchange, which also aims at accelerating the convergence of diffusion processes (Marinari & Parisi, 1992;
Revised: In addition, another well-studied method is simulated tempering, a technique similar to replica exchange, which also aims at accelerating the convergence of diffusion processes (Marinari & Parisi, 1992;

Original: Geyer & Thompson, 1995).
Revised: Geyer & Thompson, 1995).

Original: The main difference between replica exchange and simulated tempering is that the former tracks multiple diffusion processes driven by various temperatures and accelerates the convergence by swapping, while the latter tracks only one diffusion process but treats the temperature as a stochastic process.
Revised: The main difference between replica exchange and simulated tempering is that the former tracks multiple diffusion processes driven by various temperatures and accelerates the convergence by swapping, while the latter tracks only one diffusion process but treats the temperature as a stochastic process.

Original: See, for example, Zheng (2003) and the references therein for a detailed discussion on the difference.
Revised: See, for example, Zheng (2003) and the references therein for a detailed discussion on the difference.

Original: More recently, Ge et al.
Revised: More recently, Ge et al.

Original: (2017) study the convergence of simulating tempering for nonconvex objective functions from the perspective of mixing time, which is also orthogonal to our work.
Revised: (2017) study the convergence of simulating tempering for nonconvex objective functions from the perspective of mixing time, which is also orthogonal to our work.

Original: The LDP techniques in our work may potentially be applied to the analysis of simulated tempering as well.
Revised: The LDP techniques in our work may potentially be applied to the analysis of simulated tempering as well.

Original: Main Contribution:
Revised: Main Contribution:

Original: Despite the broad application in sampling, to the best of our knowledge, this paper is the ﬁrst attempt to apply the replica exchange Langevin diffusion in the context of nonconvex optimization.
Revised: Despite the broad application in sampling, to the best of our knowledge, this paper is the ﬁrst attempt to apply the replica exchange Langevin diffusion in the context of nonconvex optimization.

Original: In summary, our contribution is twofold: (i)
Revised: In summary, our contribution is twofold: (i)

Original: We quantify the acceleration effect of the replica exchange Langevin diffusion from two perspectives, that is, the convergence of the (cid:31) 2 -divergence and the LDP for empirical measures.
Revised: We quantify the acceleration effect of the replica exchange Langevin diffusion from two perspectives, that is, the convergence of the (cid:31) 2 -divergence and the LDP for empirical measures.

Original: (ii)
Revised: (ii)

Original: We propose a nonconvex optimization algorithm based on the discretization of the replica exchange Langevin diffusion and establish an upper bound for the discretization error.
Revised: We propose a nonconvex optimization algorithm based on the discretization of the replica exchange Langevin diffusion and establish an upper bound for the discretization error.

Original:  B ASIC I DEA
Revised:  B ASIC I DEA

Original: We consider the following unconstrained optimization problem, 
Revised: We consider the following unconstrained optimization problem, minimize x 2 R d

Original: [Equation]
Revised: U ( x ) : (2.1)

Original: When U ( x ) is nonconvex, it is difﬁcult to obtain its global minima.
Revised: When U ( x ) is nonconvex, it is difﬁcult to obtain its global minima.

Original: A commonly used algorithm is the Langevin diffusion, which is deﬁned by the stochastic differential equation
Revised: A commonly used algorithm is the Langevin diffusion, which is deﬁned by the stochastic differential equation

Original: [Equation]
Revised: [Equation]

Original: Here f W t g t (cid:21) 0 is a standard d -dimensional Brownian motion and (cid:28) > 0 is the temperature parameter.
Revised: Here f W t g t (cid:21) 0 is a standard d -dimensional Brownian motion and (cid:28) > 0 is the temperature parameter.

Original: Under mild regularity conditions, the Langevin diffusion f X t g t (cid:21) 0 has a unique invariant distribution that is absolutely continuous with respect to the Lebesgue measure with density
Revised: Under mild regularity conditions, the Langevin diffusion f X t g t (cid:21) 0 has a unique invariant distribution that is absolutely continuous with respect to the Lebesgue measure with density

Original: [Equation]
Revised: [Equation]

Original: Here ∫
Revised: Here ∫

Original: R d e (cid:0) U ( x ) =(cid:28)
Revised: R d e (cid:0) U ( x ) =(cid:28)

Original: d x is the normalization constant.
Revised: d x is the normalization constant.

Original: In particular, (cid:25) (cid:28) is the limiting distribution of f X t g t (cid:21) 0 , which means that with any initialization X 0 , the distribution of X t converges to (cid:25) (cid:28) (Bakry et al., 2013).
Revised: In particular, (cid:25) (cid:28) is the limiting distribution of f X t g t (cid:21) 0 , which means that with any initialization X 0 , the distribution of X t converges to (cid:25) (cid:28) (Bakry et al., 2013).

Original: We observe from (2.3) that with (cid:28) ! 

Revised: We observe from (2.3) that with (cid:28) !

Original: the probability measure (cid:25) (cid:28) concentrates around the global minima of U ( x ) (Hwang, 1980).
Revised: the probability measure (cid:25) (cid:28) concentrates around the global minima of U ( x ) (Hwang, 1980).

Original: In other words, if we choose a sufﬁciently small temperature parameter (cid:28) and run the Langevin diffusion in (2.2) for a sufﬁciently long time, we expect to obtain a solution that falls into the neighborhood of a global minimum of U ( x ) with high probability.
Revised: In other words, if we choose a sufﬁciently small temperature parameter (cid:28) and run the Langevin diffusion in (2.2) for a sufﬁciently long time, we expect to obtain a solution that falls into the neighborhood of a global minimum of U ( x ) with high probability.

Original: Here the temperature parameter (cid:28) plays a crucial role.
Revised: Here the temperature parameter (cid:28) plays a crucial role.

Original: In practice, we can only run the Langevin diffusion for a ﬁnite time, giving rise to a tradeoff between “global exploration” and “local exploitation”, which correspond to high and low temperatures.
Revised: In practice, we can only run the Langevin diffusion for a ﬁnite time, giving rise to a tradeoff between “global exploration” and “local exploitation”, which correspond to high and low temperatures.

Original: In speciﬁc, when the temperature parameter (cid:28) is small, the convergence of X t is slow and the particle can be trapped by a local minimum for a long time without globally exploring the whole domain.
Revised: In speciﬁc, when the temperature parameter (cid:28) is small, the convergence of X t is slow and the particle can be trapped by a local minimum for a long time without globally exploring the whole domain.

Original: Consequently, within a ﬁnite time, we can only obtain a sample from a distribution that is far away from the invariant distribution (cid:25) (cid:28) .
Revised: Consequently, within a ﬁnite time, we can only obtain a sample from a distribution that is far away from the invariant distribution (cid:25) (cid:28) .

Original: In contrast, when the temperature parameter (cid:28) is large, the convergence is accelerated due to better global exploration.
Revised: In contrast, when the temperature parameter (cid:28) is large, the convergence is accelerated due to better global exploration.

Original: The distribution of X t converges faster to the invariant distribution (cid:25) (cid:28) globally, but locally (cid:25) (cid:28) is less concentrated around the global minima of U ( x ) .
Revised: The distribution of X t converges faster to the invariant distribution (cid:25) (cid:28) globally, but locally (cid:25) (cid:28) is less concentrated around the global minima of U ( x ) .

Original: To bridge the gap between high and low temperatures, in this paper we study an adaptive algorithm called replica exchange Langevin diffusion.
Revised: To bridge the gap between high and low temperatures, in this paper we study an adaptive algorithm called replica exchange Langevin diffusion.

Original: In detail, we consider a pair of particles driven by two
Revised: In detail, we consider a pair of particles driven by two

Original: Langevin diffusions as deﬁned in (2.2) with temperatures (cid:28) 1 < (cid:28) 2 , respectively.
Revised: Langevin diffusions as deﬁned in (2.2) with temperatures (cid:28) 1 < (cid:28) 2 , respectively.

Original: We use Z t =
Revised: We use Z t =

Original: ( Z (1) t ; Z (2) t ) to denote the positions of the two particles at time t .
Revised: ( Z (1) t ; Z (2) t ) to denote the positions of the two particles at time t .

Original: In other words, we have
Revised: In other words, we have

Original: [Equation]
Revised: [Equation]

Original: According to (2.3), the invariant distribution of f Z t g t (cid:21) 0 is absolutely continuous with respect to the
Revised: According to (2.3), the invariant distribution of f Z t g t (cid:21) 0 is absolutely continuous with respect to the

Original: Lebesgue measure on R 2 d and its density is proportional to
Revised: Lebesgue measure on R 2 d and its density is proportional to

Original: [Equation]
Revised: [Equation]

Original: The marginal invariant distribution of the low-temperature particle concentrates around the global minima of U ( x ) .
Revised: The marginal invariant distribution of the low-temperature particle concentrates around the global minima of U ( x ) .

Original: Hence, the low-temperature particle is of particular interest for the purpose of nonconvex optimization.
Revised: Hence, the low-temperature particle is of particular interest for the purpose of nonconvex optimization.

Original: The key idea of replica exchange is to enable the low-temperature particle to achieve better global exploration by swapping its position with the high-temperature particle.
Revised: The key idea of replica exchange is to enable the low-temperature particle to achieve better global exploration by swapping its position with the high-temperature particle.

Original: In speciﬁc, at time t and positions ( x 1 ; x 2 ) , the two particles swap with rate
Revised: In speciﬁc, at time t and positions ( x 1 ; x 2 ) , the two particles swap with rate

Original: [Equation]
Revised: [Equation]

Original: which means
Revised: which means

Original: [Equation]
Revised: [Equation]

Original: Here a (cid:21) 0 is a constant called swapping intensity.
Revised: Here a (cid:21) 0 is a constant called swapping intensity.

Original: As is shown in Lemma 3.2, the speciﬁc form of s ( x 1 ; x 2 ) in (2.6) ensures that for any a , the invariant distribution of the replica exchange Langevin diffusion
Revised: As is shown in Lemma 3.2, the speciﬁc form of s ( x 1 ; x 2 ) in (2.6) ensures that for any a , the invariant distribution of the replica exchange Langevin diffusion

Original: f Z t g t (cid:21) 0 is the same as (2.5), that is, as if the two particles are independent.
Revised: f Z t g t (cid:21) 0 is the same as (2.5), that is, as if the two particles are independent.

Original: Corresponding to the continuous-time process in (2.4)-(2.6), in x 3.4 of the appendix we consider the replica exchange stochastic gradient descent algorithm, which corresponds to the discretization of f Z t g t (cid:21)using Euler scheme.
Revised: Corresponding to the continuous-time process in (2.4)-(2.6), in x 3.4 of the appendix we consider the replica exchange stochastic gradient descent algorithm, which corresponds to the discretization of f Z t g t (cid:21)using Euler scheme.

Original: To better understand the intuition behind replica exchange, note that we use two particles driven by low and high temperatures to achieve “local exploitation” and “global exploration”, respectively.
Revised: To better understand the intuition behind replica exchange, note that we use two particles driven by low and high temperatures to achieve “local exploitation” and “global exploration”, respectively.

Original: For the purpose of optimization, we only need to track the trajectory of the ﬁrst particle.
Revised: For the purpose of optimization, we only need to track the trajectory of the ﬁrst particle.

Original: By plugging (2.5) in (2.6), we obtain
Revised: By plugging (2.5) in (2.6), we obtain

Original: [Equation]
Revised: [Equation]

Original: Since (cid:28) 1 < (cid:28) 2 , the rate a (cid:1) s ( x 1 ; x 2 ) is monotone increasing with respect to the difference between the objective values at x 1 and x 2 , which denote the positions of the two particles.
Revised: Since (cid:28) 1 < (cid:28) 2 , the rate a (cid:1) s ( x 1 ; x 2 ) is monotone increasing with respect to the difference between the objective values at x 1 and x 2 , which denote the positions of the two particles.

Original: Hence, the two particles are more likely to swap when the ﬁrst one has a larger objective value.
Revised: Hence, the two particles are more likely to swap when the ﬁrst one has a larger objective value.

Original: In other words, swapping tends to move the ﬁrst particle, which we are interested in, to a position corresponding to a lower objective value.
Revised: In other words, swapping tends to move the ﬁrst particle, which we are interested in, to a position corresponding to a lower objective value.

Original: An extreme case of the replica exchange Langevin diffusion is (cid:28) 1 = 0 and (cid:28) 2 = 1 .
Revised: An extreme case of the replica exchange Langevin diffusion is (cid:28) 1 = 0 and (cid:28) 2 = 1 .

Original: In this case, the ﬁrst equation in (2.4) reduces to an ordinary differential equation charactering the deterministic gradient descent, while the second one in (2.4) corresponds to an approximated uniform exploration of the whole domain R d .
Revised: In this case, the ﬁrst equation in (2.4) reduces to an ordinary differential equation charactering the deterministic gradient descent, while the second one in (2.4) corresponds to an approximated uniform exploration of the whole domain R d .

Original: According to (2.8), the particles never swap if the objective value of the ﬁrst particle is smaller than the second.
Revised: According to (2.8), the particles never swap if the objective value of the ﬁrst particle is smaller than the second.

Original: Hence, roughly speaking, the replica exchange Langevin diffusion reduces to the deterministic gradient descent with uniformly randomized restarts.
Revised: Hence, roughly speaking, the replica exchange Langevin diffusion reduces to the deterministic gradient descent with uniformly randomized restarts.

Original: In contrast to this extreme case, for 0 < (cid:28) 1 < (cid:28) 2 < 1 , the second particle globally explores the whole domain more adaptively with not only noise but also gradient information.
Revised: In contrast to this extreme case, for 0 < (cid:28) 1 < (cid:28) 2 < 1 , the second particle globally explores the whole domain more adaptively with not only noise but also gradient information.

Original: In particular, its stationary distribution has a larger density around the local minima, which leads to better restarts that adapt to the global geometry for the ﬁrst particle.
Revised: In particular, its stationary distribution has a larger density around the local minima, which leads to better restarts that adapt to the global geometry for the ﬁrst particle.

Original:  T HEORETICAL A NALYSIS
Revised:  T HEORETICAL A NALYSIS

Original: In this section, we lay out the theoretical analysis of replica exchange Langevin diffusion introduced in x 2, which demonstrates the acceleration effect of swapping.
Revised: In this section, we lay out the theoretical analysis of replica exchange Langevin diffusion introduced in x 2, which demonstrates the acceleration effect of swapping.

Original: Due to space constraint, we defer the necessary background and detailed proofs to x B and x C of the appendix, respectively.
Revised: Due to space constraint, we defer the necessary background and detailed proofs to x B and x C of the appendix, respectively.

Original: Throughout the following analysis, we assume that the objective function U ( (cid:1) ) satisﬁes the following assumption.
Revised: Throughout the following analysis, we assume that the objective function U ( (cid:1) ) satisﬁes the following assumption.

Original: Assumption 3.1 (Smoothness and Dissipativity) .
Revised: Assumption 3.1 (Smoothness and Dissipativity) .

Original: The function U ( (cid:1) ) is L -smooth, that is, there exists a positive constant L such that for all x; y 2 R d(cid:13)
Revised: The function U ( (cid:1) ) is L -smooth, that is, there exists a positive constant L such that for all x; y 2 R d

Original: [Equation]
Revised: [Equation]

Original: The function U ( (cid:1) ) is also ( (cid:11); (cid:12) )
Revised: The function U ( (cid:1) ) is also ( (cid:11); (cid:12) )

Original: -dissipative, that is, there exist positive constants (cid:11) and (cid:12) such that for all x 2 R d
Revised: -dissipative, that is, there exist positive constants (cid:11) and (cid:12) such that for all x 2 R d

Original: [Equation]
Revised: [Equation]

Original: The assumption on smoothness in (3.1) characterizes the Lipschitz continuity of the gradient of objective function, which is commonly used in the optimization literature (Nesterov, 2013).
Revised: The assumption on smoothness in (3.1) characterizes the Lipschitz continuity of the gradient of objective function, which is commonly used in the optimization literature (Nesterov, 2013).

Original: The assumption on dissipativity in (3.2), roughly speaking, characterizes the approximate quadratic growth of the objective function at inﬁnity, which is commonly used in the control and dynamic system literature (Hale, 2010).
Revised: The assumption on dissipativity in (3.2), roughly speaking, characterizes the approximate quadratic growth of the objective function at inﬁnity, which is commonly used in the control and dynamic system literature (Hale, 2010).

Original: This condition is also used in Raginsky et al.
Revised: This condition is also used in Raginsky et al.

Original: It is worth noting that our theory does not require the convexity assumption.
Revised: It is worth noting that our theory does not require the convexity assumption.

Original: 3. I NVARIANCE AND R EVERSIBILITY
Revised: 3. I NVARIANCE AND R EVERSIBILITY

Original: In this section, we show that the invariant distribution of the replica exchange Langevin diffusion f
Revised: In this section, we show that the invariant distribution of the replica exchange Langevin diffusion f

Original: Z t g t (cid:21) 0 has the form in (2.5) rigorously, whose ﬁrst component preserves the concentration.
Revised: Z t g t (cid:21) 0 has the form in (2.5) rigorously, whose ﬁrst component preserves the concentration.

Original: We also show the reversibility of f Z t g t (cid:21) 0 , which is a nice property necessary for the subsequent convergence analysis.
Revised: We also show the reversibility of f Z t g t (cid:21) 0 , which is a nice property necessary for the subsequent convergence analysis.

Original: We consider the replica exchange Langevin diffusion f Z t g t (cid:21) 0 deﬁned by (2.4)- (2.6), where Z t =
Revised: We consider the replica exchange Langevin diffusion f Z t g t (cid:21) 0 deﬁned by (2.4)- (2.6), where Z t =

Original: ( Z (1) t ; Z (2) t ) denotes the positions of the two particles at time t , and a (cid:21) 0 is the swapping intensity that controls the frequency of swapping.
Revised: ( Z (1) t ; Z (2) t ) denotes the positions of the two particles at time t , and a (cid:21) 0 is the swapping intensity that controls the frequency of swapping.

Original: The following auxiliary lemma characterizes the invariant distribution of f Z t g t (cid:21) 0 and its reversibility.
Revised: The following auxiliary lemma characterizes the invariant distribution of f Z t g t (cid:21) 0 and its reversibility.

Original: Restricted to the space, we provide a detailed proof in x C.1 of the appendix.
Revised: Restricted to the space, we provide a detailed proof in x C.1 of the appendix.

Original: Lemma 3.2.
Revised: Lemma 3.2.

Original: The inﬁnitesimal generator of f Z t g t (cid:21) 0 with swapping intensity a , which is deﬁned in (2.4)-(2.6), takes the form
Revised: The inﬁnitesimal generator of f Z t g t (cid:21) 0 with swapping intensity a , which is deﬁned in (2.4)-(2.6), takes the form

Original: [Equation]
Revised: [Equation]

Original: Moreover, f Z t g t (cid:21) 0 is reversible and its invariant distribution (cid:25) has density
Revised: Moreover, f Z t g t (cid:21) 0 is reversible and its invariant distribution (cid:25) has density

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: with respect to the Lebesgue measure on R 2 d , where (cid:22) ( x 1 ; x 2 ) is deﬁned in (2.5).
Revised: with respect to the Lebesgue measure on R 2 d , where (cid:22) ( x 1 ; x 2 ) is deﬁned in (2.5).

Original: In Lemma 3.2, the ﬁrst two terms L a 1 and L a 2 on the right-hand side of (3.3) correspond to the standard Langevin diffusion, while the last term L as arises from swapping.
Revised: In Lemma 3.2, the ﬁrst two terms L a 1 and L a 2 on the right-hand side of (3.3) correspond to the standard Langevin diffusion, while the last term L as arises from swapping.

Original: The replica exchange
Revised: The replica exchange

Original: Langevin diffusion f Z t g t (cid:21) 0 deﬁned in (2.4)-(2.6) is ergodic, which means that, with any initialization, the distribution of Z t converges to the invariant distribution (cid:25) .
Revised: Langevin diffusion f Z t g t (cid:21) 0 deﬁned in (2.4)-(2.6) is ergodic, which means that, with any initialization, the distribution of Z t converges to the invariant distribution (cid:25) .

Original: There are two perspectives to characterize the convergence of the Markov process f Z t g t (cid:21) 0 :
Revised: There are two perspectives to characterize the convergence of the Markov process f Z t g t (cid:21) 0 :

Original: (i) the (cid:31) 2 -divergence between the distribution of Z t and the invariant distribution (cid:25) , and (ii) the convergence of the empirical measure of f Z t g t (cid:21) 0 , which is viewed as a random element in the space of measures.
Revised: (i) the (cid:31) 2 -divergence between the distribution of Z t and the invariant distribution (cid:25) , and (ii) the convergence of the empirical measure of f Z t g t (cid:21) 0 , which is viewed as a random element in the space of measures.

Original: In the following, we quantify the convergence of f Z t g t (cid:21) 0 from both perspectives.
Revised: In the following, we quantify the convergence of f Z t g t (cid:21) 0 from both perspectives.

Original: In x 3.2, we use Poincar´e inequality to quantify (i), while in x 3.3, we apply the large deviation principle (LDP) to characterize (ii).
Revised: In x 3.2, we use Poincar´e inequality to quantify (i), while in x 3.3, we apply the large deviation principle (LDP) to characterize (ii).

Original: In particular, we show that the term L as in (3.3) plays a crucial role in accelerating the convergence.
Revised: In particular, we show that the term L as in (3.3) plays a crucial role in accelerating the convergence.

Original: 3. C ONVERGENCE IN (cid:31) 2 -D IVERGENCE
Revised: 3. C ONVERGENCE IN (cid:31) 2 -D IVERGENCE

Original: Let (cid:22) t be the distribution of the replica exchange Langevin diffusion
Revised: Let (cid:22) t be the distribution of the replica exchange Langevin diffusion

Original: f Z t g t (cid:21) 0 at time t , and (cid:25) be its invariant distribution.
Revised: f Z t g t (cid:21) 0 at time t , and (cid:25) be its invariant distribution.

Original: In the following, we quantify the discrepancy between (cid:22) t and (cid:25) using the (cid:31) 2 -divergence, which is deﬁned as
Revised: In the following, we quantify the discrepancy between (cid:22) t and (cid:25) using the (cid:31) 2 -divergence, which is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where d (cid:22) t = d (cid:25) is the Radon-Nikodym derivative between (cid:22) t and (cid:25) .
Revised: where d (cid:22) t = d (cid:25) is the Radon-Nikodym derivative between (cid:22) t and (cid:25) .

Original: In the following, we characterize the evolution of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) along time t using the Dirichlet form, which is deﬁned as 
Revised: In the following, we characterize the evolution of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) along time t using the Dirichlet form, which is deﬁned as [Equation] a

Original: [Equation]
Revised: [Equation]

Original: where
Revised: f [Equation] =

Original: [Equation]
Revised: [Equation]

Revised: f

Revised: [Equation]

Original:  Here (cid:0) a is called Carr´e du Champ operator and recall that we use (cid:25) and L a to denote the invariant distribution and inﬁnitesimal generator of f Z t g t (cid:21) 0 , respectively.
Revised: where (cid:0) a [Equation] Here (cid:0) a is called Carr´e du Champ operator and recall that we use (cid:25) and L a to denote the invariant distribution and inﬁnitesimal generator of f Z t g t (cid:21) 0 , respectively.

Original: In other words, the Dirichlet form is deﬁned as the integration of the Carr´e du Champ operator under the invariant distribution (cid:25) .
Revised: In other words, the Dirichlet form is deﬁned as the integration of the Carr´e du Champ operator under the invariant distribution (cid:25) .

Original: To characterize the evolution of the (cid:31) 2 -divergence, we take the derivative of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) with respect to time t .
Revised: To characterize the evolution of the (cid:31) 2 -divergence, we take the derivative of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) with respect to time t .

Original: Recall that (cid:22) t is the distribution of Z t , and by (B.2) in the appendix the corresponding semigroup f P t g t (cid:21) 0 is deﬁned as P t ( f ( x )) =
Revised: Recall that (cid:22) t is the distribution of Z t , and by (B.2) in the appendix the corresponding semigroup f P t g t (cid:21) 0 is deﬁned as P t ( f ( x )) =

Original: E [ f ( Z t ) j Z 0 = x ] .
Revised: E [ f ( Z t ) j Z 0 = x ] .

Original: By setting f as d (cid:22) 

Revised: By setting f as d (cid:22)

Original: = d (cid:25) and the deﬁnition of conditional expectation, we have d (cid:22) t = d (cid:25) =
Revised: = d (cid:25) and the deﬁnition of conditional expectation, we have d (cid:22) t = d (cid:25) =

Original: P t (d (cid:22) 

Revised: P t (d (cid:22)

Original: = d (cid:25) ) , which implies 
Revised: = d (cid:25) ) , which implies d

Original: [Equation]
Revised: [Equation]

Original: where the last equation follows from the deﬁnition of the inﬁnitesimal generator in Deﬁnition B. in the appendix.
Revised: where the last equation follows from the deﬁnition of the inﬁnitesimal generator in Deﬁnition B. in the appendix.

Original: Meanwhile, by Deﬁnition B.3 in the appendix and the invariance of (cid:25) , we have ∫ L a ( f 2 )d (cid:25) = 0 , which together with (3.5) implies
Revised: Meanwhile, by Deﬁnition B.3 in the appendix and the invariance of (cid:25) , we have ∫ L a [Equation] [Equation] = 0 , which together with (3.5) implies

Original: [Equation]
Revised: [Equation]

Original: Combining (3.6) and (3.7) we obtain
Revised: Combining (3.6) and (3.7) we obtain

Original: [Equation]
Revised: [Equation]

Original: which shows that the derivative of the (cid:31) 2 -divergence between (cid:22) t and the invariant distribution (cid:25) is exactly negative twice the Dirichlet form of the Radon-Nykodim derivative d (
Revised: which shows that the derivative of the (cid:31) 2 -divergence between (cid:22) t and the invariant distribution (cid:25) is exactly negative twice the Dirichlet form of the Radon-Nykodim derivative d (

Original: cid:22) t = d (cid:25) .
Revised: cid:22) t = d (cid:25) .

Original: In the following theorem, we show that a larger swapping intensity a boosts the Dirichlet form and thus accelerates the convergence.
Revised: In the following theorem, we show that a larger swapping intensity a boosts the Dirichlet form and thus accelerates the convergence.

Original: Recall that, as deﬁned in (3.5), E a is the Dirichlet form of the replica exchange
Revised: Recall that, as deﬁned in (3.5), E a is the Dirichlet form of the replica exchange

Original: Langevin diffusion f Z t g t (cid:21) 0 with swapping intensity a .
Revised: Langevin diffusion f Z t g t (cid:21) 0 with swapping intensity a .

Original: Theorem 3.3.
Revised: Theorem 3.3.

Original: For any ﬁxed function f , E a ( f ) is an increasing nonnegative function with respect to a .
Revised: For any ﬁxed function f , E a ( f ) is an increasing nonnegative function with respect to a .

Original: In particular, we have
Revised: In particular, we have

Revised: [Equation]

Revised: |

Revised: [Equation]

Revised: Acceleration Eﬀect

Original: [Equation]
Revised: [Equation]

Original: where the both terms on the right-hand side are nonnegative.
Revised: where the both terms on the right-hand side are nonnegative.

Original: See x C.2 of the appendix for a detailed proof of Theorem 3.3.
Revised: See x C.2 of the appendix for a detailed proof of Theorem 3.3.

Original: Combined with (3.8), Theorem 3.
Revised: Combined with (3.8), Theorem 3.

Original: shows that swapping accelerates the evolution of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) .
Revised: shows that swapping accelerates the evolution of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) .

Original: In particular, the Dirichlet form E a on the right-hand side of (3.8) decomposes into two terms.
Revised: In particular, the Dirichlet form E a on the right-hand side of (3.8) decomposes into two terms.

Original: The ﬁrst term E 0 in (3.9) corresponds to the replica exchange Langevin diffusion without swapping, that is, a = 0 .
Revised: The ﬁrst term E 0 in (3.9) corresponds to the replica exchange Langevin diffusion without swapping, that is, a = 0 .

Original: More speciﬁcally, the two nonnegative terms in E 0 characterize the individual dynamics of the two particles, respectively.
Revised: More speciﬁcally, the two nonnegative terms in E 0 characterize the individual dynamics of the two particles, respectively.

Original: In particular, larger temperatures (cid:28) 1 and (cid:28) 2 yield a larger E 0 , which implies a faster rate of convergence.
Revised: In particular, larger temperatures (cid:28) 1 and (cid:28) 2 yield a larger E 0 , which implies a faster rate of convergence.

Original: Meanwhile, as shown in the proof of Theorem 3.3, the speciﬁc form of swapping in (2.7) ensures the nonnegativity of the second term in (3.9), which further boosts the Dirichlet form and leads to faster evolution of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) in (3.8).
Revised: Meanwhile, as shown in the proof of Theorem 3.3, the speciﬁc form of swapping in (2.7) ensures the nonnegativity of the second term in (3.9), which further boosts the Dirichlet form and leads to faster evolution of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) in (3.8).

Original: It is worth mentioning that Theorem 3.3 does not rely on
Revised: It is worth mentioning that Theorem 3.3 does not rely on

Original: Assumption 3.3 and holds for all objective functions U ( (cid:1) ) .
Revised: Assumption 3.3 and holds for all objective functions U ( (cid:1) ) .

Original: To better understand Theorem 3.3, note that the symmetry of f , which corresponds to d (cid:22) t
Revised: To better understand Theorem 3.3, note that the symmetry of f , which corresponds to d (cid:22) t

Original: = d (cid:25) ( x 1 ; x 2 ) in (3.8), plays a crucial role in the acceleration effect.
Revised: = d (cid:25) ( x 1 ; x 2 ) in (3.8), plays a crucial role in the acceleration effect.

Original: In speciﬁc, when d (cid:22) t = d (cid:25) ( x 1 ; x 2 ) is asymmetric, the second term of the Dirichlet form in (3.9) is positive.
Revised: In speciﬁc, when d (cid:22) t = d (cid:25) ( x 1 ; x 2 ) is asymmetric, the second term of the Dirichlet form in (3.9) is positive.

Original: Otherwise when d (cid:22) t = d (cid:25) ( x 1 ; x 2 ) is symmetric, the second term in (3.9) vanishes, since d (cid:22) t
Revised: Otherwise when d (cid:22) t = d (cid:25) ( x 1 ; x 2 ) is symmetric, the second term in (3.9) vanishes, since d (cid:22) t

Original: = d (cid:25) ( x 1 ; x 2 ) (cid:0) d (cid:22) t = d (cid:25) ( x 2 ; x 1 ) = 

Revised: = d (cid:25) ( x 1 ; x 2 ) (cid:0) d (cid:22) t = d (cid:25) ( x 2 ; x 1 ) =

Original: for all x 1 and x 2 .
Revised: for all x 1 and x 2 .

Original: In other words, the acceleration effect degenerates due to symmetry.
Revised: In other words, the acceleration effect degenerates due to symmetry.

Original: Intuitively, at time t the two particles are equivalent and swapping does not change their joint distribution, and hence does not affect the convergence of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) .
Revised: Intuitively, at time t the two particles are equivalent and swapping does not change their joint distribution, and hence does not affect the convergence of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) .

Original: Based on (3.8), we further establish the exponential convergence of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) .
Revised: Based on (3.8), we further establish the exponential convergence of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) .

Original: The key ingredient is the Poincar´e inequality (Bakry et al., 2013).
Revised: The key ingredient is the Poincar´e inequality (Bakry et al., 2013).

Original: Furthermore, we show that such an exponential rate of convergence is dictated by the constant in the Poincar´e inequality, which is called the Poincar´e constant.
Revised: Furthermore, we show that such an exponential rate of convergence is dictated by the constant in the Poincar´e inequality, which is called the Poincar´e constant.

Original: In particular, we show how the swapping intensity a affects the Poincar´e constant and accelerates the exponential convergence.
Revised: In particular, we show how the swapping intensity a affects the Poincar´e constant and accelerates the exponential convergence.

Original: For a Markov process, the Poincar´e inequality states that the (cid:31) 2 -divergence between any probability measure and its invariant distribution is uniformly upper bounded by the Dirichlet form of the
Revised: For a Markov process, the Poincar´e inequality states that the (cid:31) 2 -divergence between any probability measure and its invariant distribution is uniformly upper bounded by the Dirichlet form of the

Original: Radon-Nykodim derivative, provided that the probability measure of interest is absolutely continuous with respect to the invariant distribution.
Revised: Radon-Nykodim derivative, provided that the probability measure of interest is absolutely continuous with respect to the invariant distribution.

Original: In speciﬁc, we say that f Z t g t (cid:21) 0 satisﬁes the Poincar´e inequality with Poincar´e constant (cid:20) , if for all probability measures (cid:23) ≪ (cid:25) , the following inequality holds,
Revised: In speciﬁc, we say that f Z t g t (cid:21) 0 satisﬁes the Poincar´e inequality with Poincar´e constant (cid:20) , if for all probability measures (cid:23) ≪ (cid:25) , the following inequality holds,

Original: [Equation]
Revised: [Equation]

Original: Note that the Poincar´e inequality is speciﬁed by the invariant distribution (cid:25) and the Dirichlet form E a .
Revised: Note that the Poincar´e inequality is speciﬁed by the invariant distribution (cid:25) and the Dirichlet form E a .

Original: When the Poincar´e inequality in (3.10) holds, the derivative of the (cid:31) 2 -divergence in (3.8) is upper bounded by itself, which implies the exponential rate of convergence.
Revised: When the Poincar´e inequality in (3.10) holds, the derivative of the (cid:31) 2 -divergence in (3.8) is upper bounded by itself, which implies the exponential rate of convergence.

Original: In particular, we have
Revised: In particular, we have

Original: [Equation]
Revised: [Equation]

Original: ) ; which yields (cid:31)
Revised: which yields (cid:31)

Original: [Equation]
Revised: [Equation]

Original: Establishing the Poincar´e inequality is highly nontrivial.
Revised: Establishing the Poincar´e inequality is highly nontrivial.

Original: However, for the replica exchange
Revised: However, for the replica exchange

Original: Langevin diffusion, the following theorem shows that Poincar´e inequality holds under the smoothness and dissipativity conditions in Assumption 3.1, which implies (3.11).
Revised: Langevin diffusion, the following theorem shows that Poincar´e inequality holds under the smoothness and dissipativity conditions in Assumption 3.1, which implies (3.11).

Original: Theorem 3.4.
Revised: Theorem 3.4.

Original: Under Assumption 3.1, the replica exchange Langevin diffusion speciﬁed in (2.4)- (2.6) satisﬁes the Poincar´e inequality in (3.10).
Revised: Under Assumption 3.1, the replica exchange Langevin diffusion speciﬁed in (2.4)- (2.6) satisﬁes the Poincar´e inequality in (3.10).

Original: Hence, the second inequality in (3.11) holds, which implies the exponential decay of (cid:31) 2 -divergence.
Revised: Hence, the second inequality in (3.11) holds, which implies the exponential decay of (cid:31) 2 -divergence.

Original: The proof of Theorem 3.4 adapts from the proof in Bakry et al.
Revised: The proof of Theorem 3.4 adapts from the proof in Bakry et al.

Original: (2008), where a standard Langevin diffusion with unique driving temperature is considered.
Revised: (2008), where a standard Langevin diffusion with unique driving temperature is considered.

Original: See x C.3 of the appendix for a detailed proof.
Revised: See x C.3 of the appendix for a detailed proof.

Original: The second inequality in (3.11) shows that the rate of convergence of (cid:31) 2 ( (cid:22) t ∥ (cid:25) ) is dictated by the

Original: Poincar´e constant (cid:20) .

Original: In speciﬁc, a smaller Poincar´e constant leads to a faster rate of convergence.

Original: Theorem 3.3 shows that swapping boosts the Dirichlet form and hence requires a smaller Poincar´e constant (cid:20) , which by Theorem 3.4 leads to a faster exponential rate of convergence to the invariant distribution.

Original: 3. L ARGE D EVIATION P RINCIPLE (LDP) A NALYSIS
Revised: 3. L ARGE D EVIATION P RINCIPLE (LDP) A NALYSIS

Original: We focus on the empirical measure of the replica exchange Langevin diffusion f Z t g t (cid:21) 0 deﬁned in (2.4)-(2.6), which is in the same spirit of Polyak averaging (Polyak & Juditsky, 1992) but replaces the iterates with Dirac measures.
Revised: We focus on the empirical measure of the replica exchange Langevin diffusion f Z t g t (cid:21) 0 deﬁned in (2.4)-(2.6), which is in the same spirit of Polyak averaging (Polyak & Juditsky, 1992) but replaces the iterates with Dirac measures.

Original: In speciﬁc, for any ﬁxed time T > 0 , the empirical measure of f
Revised: In speciﬁc, for any ﬁxed time T > 0 , the empirical measure of f

Original: Z t g t (cid:21) 0 is deﬁned as
Revised: Z t g t (cid:21) 0 is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: where (cid:14) Z t denotes the Dirac measure at Z t .
Revised: where (cid:14) Z t denotes the Dirac measure at Z t .

Original: Note that f (cid:23) T g T (cid:21) 0 is a sequence of random measures, which are random elements of P ( R 2 d ) , that is, the space of probability measures on R 2 d .
Revised: Note that f (cid:23) T g T (cid:21) 0 is a sequence of random measures, which are random elements of P ( R 2 d ) , that is, the space of probability measures on R 2 d .

Original: We equip
Revised: We equip

Original: P ( R 2 d ) with the topology of weak convergence, which enables us to deﬁne open and closed sets of
Revised: P ( R 2 d ) with the topology of weak convergence, which enables us to deﬁne open and closed sets of

Original: P ( R 2 d ) .
Revised: P ( R 2 d ) .

Original: Recall that Lemma 3.2 shows that the replica exchange Langevin diffusion f Z t g t (cid:21) 0 is reversible.
Revised: Recall that Lemma 3.2 shows that the replica exchange Langevin diffusion f Z t g t (cid:21) 0 is reversible.

Original: This fact enables us to apply the Donsker-Varadhan theory (Donsker & Varadhan, 1975), which implies that f Z t g t (cid:21) 

Revised: This fact enables us to apply the Donsker-Varadhan theory (Donsker & Varadhan, 1975), which implies that f Z t g t (cid:21)

Original: obeys LDP.
Revised: obeys LDP.

Original: Formally, we have the following theorem.
Revised: Formally, we have the following theorem.

Original: Theorem 3.5.
Revised: Theorem 3.5.

Original: The sequence of empirical measures f (cid:23) T g T (cid:21) 0 of the replica exchange Langevin diffusion f
Revised: The sequence of empirical measures f (cid:23) T g T (cid:21) 0 of the replica exchange Langevin diffusion f

Original: Z t g t (cid:21) 0 deﬁned in (3.12) obeys LDP.
Revised: Z t g t (cid:21) 0 deﬁned in (3.12) obeys LDP.

Original: That is to say, for all open sets O and closed sets F in P ( R 2 d ) , the following inequalities hold, 
Revised: That is to say, for all open sets O and closed sets F in P ( R 2 d ) , the following inequalities hold, [Equation] lim

Original: [Equation]
Revised: [Equation]

Revised: 2F

Revised: I

Revised: [Equation]

Revised: ( (cid:23)

Revised: [Equation]

Original: Here I a ( (cid:1) ) :
Revised: Here I a ( (cid:1) ) :

Original: P ( R 2 d ) !
Revised: P ( R 2 d ) !

Original: [0 ; 1 ] is the rate function, which takes the form
Revised: [0 ; 1 ] is the rate function, which takes the form

Original: I 
Revised: I a ( (cid:23) ) = ∫

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: z
Revised: Acceleration Eﬀect [Equation]

Original: [Equation]
Revised: [Equation]

Original: for all (cid:23) ≪ (cid:25) , and I a ( (cid:23) )
Revised: for all (cid:23) ≪ (cid:25) , and I a ( (cid:23) )

Original: = 1 otherwise.
Revised: = 1 otherwise.

Original: See x C.5 of the appendix for a detailed proof of theorem 3.5.
Revised: See x C.5 of the appendix for a detailed proof of theorem 3.5.

Original: The LDP rate function I a ( (cid:1) ) in
Revised: The LDP rate function I a ( (cid:1) ) in

Original: Theorem 3.5 characterizes the rate of convergence for the empirical measures f (cid:23) T g T (cid:21) 0 towards its population version, which is the invariance distribution of f Z t g t (cid:21) 0 .
Revised: Theorem 3.5 characterizes the rate of convergence for the empirical measures f (cid:23) T g T (cid:21) 0 towards its population version, which is the invariance distribution of f Z t g t (cid:21) 0 .

Original: More speciﬁc, it quantiﬁes the exponential rate of decay of the probability that empirical empirical measures f (cid:23) T g T (cid:21) 

Revised: More speciﬁc, it quantiﬁes the exponential rate of decay of the probability that empirical empirical measures f (cid:23) T g T (cid:21)

Original: deviate from the invariant distribution (cid:25) .
Revised: deviate from the invariant distribution (cid:25) .

Original: Recall that, under weak topology, P ( R 2 d ) is metrizable via the
Revised: Recall that, under weak topology, P ( R 2 d ) is metrizable via the

Original: L´evy-Prokhorov metric d LP ( (cid:1) ; (cid:1) )
Revised: L´evy-Prokhorov metric d LP ( (cid:1) ; (cid:1) )

Original: (Billingsley, 2013).
Revised: (Billingsley, 2013).

Original: We use B r to denote the open ball centered at (cid:25) with radius r in the metric space ( P ( R 2 d ) ; d LP ) .
Revised: We use B r to denote the open ball centered at (cid:25) with radius r in the metric space ( P ( R 2 d ) ; d LP ) .

Original: Then B c r is a closed set and (cid:22) B c r is open.
Revised: Then B c r is a closed set and (cid:22) B c r is open.

Original: Then according to Theorem 3.5, we have
Revised: Then according to Theorem 3.5, we have

Original: [Equation]
Revised: [Equation]

Original: If we ignore the slight difference between B cr and (cid:22) B cr , when T is large, the probability
Revised: If we ignore the slight difference between B cr and (cid:22) B cr , when T is large, the probability

Original: P ( d LP ( (cid:23) T ; (cid:22) ) (cid:21) r ) has approximated scale of exp( (cid:0) T inf (cid:23) 2B
Revised: P ( d LP ( (cid:23) T ; (cid:22) ) (cid:21) r ) has approximated scale of exp( (cid:0) T inf (cid:23) 2B

Original: cr I ( (cid:23) )) .
Revised: cr I ( (cid:23) )) .

Original: That is to say, inf (cid:23) 2B cr I ( (cid:23) ) characterizes the exponential rate of decay of the probability P ( d LP ( (cid:23) T ; (cid:22) )
Revised: That is to say, inf (cid:23) 2B cr I ( (cid:23) ) characterizes the exponential rate of decay of the probability P ( d LP ( (cid:23) T ; (cid:22) )

Original: (cid:21) r ) as T goes to inﬁnity.
Revised: (cid:21) r ) as T goes to inﬁnity.

Original: A larger rate function implies a faster exponential rate of decay.
Revised: A larger rate function implies a faster exponential rate of decay.

Original: Note that LDP includes both upper and lower bounds in Theorem 3.5 and hence, the exponential rate of convergence characterized by LDP is tight.
Revised: Note that LDP includes both upper and lower bounds in Theorem 3.5 and hence, the exponential rate of convergence characterized by LDP is tight.

Original: To see how swapping accelerates the convergence of the empirical measure towards its invariant distribution, note that the LDP rate function in (3.13) decomposes into two terms.
Revised: To see how swapping accelerates the convergence of the empirical measure towards its invariant distribution, note that the LDP rate function in (3.13) decomposes into two terms.

Original: The ﬁrst term corresponds to the LDP rate function of replica exchange Langevin diffusion without swapping, that is, a = 0 .
Revised: The ﬁrst term corresponds to the LDP rate function of replica exchange Langevin diffusion without swapping, that is, a = 0 .

Original: In speciﬁc, it is the sum of two nonnegative terms, which characterize the individual dynamics of the two particles, respectively.
Revised: In speciﬁc, it is the sum of two nonnegative terms, which characterize the individual dynamics of the two particles, respectively.

Original: Meanwhile, the second term in (3.13) is nonnegative and boosts the LDP rate function, which accelerates the convergence of the empirical measure.
Revised: Meanwhile, the second term in (3.13) is nonnegative and boosts the LDP rate function, which accelerates the convergence of the empirical measure.

Original: Similar to the convergence of the (cid:31) 2 -divergence in Theorem 3.3, a larger swapping intensity a yields a more signiﬁcant acceleration effect.
Revised: Similar to the convergence of the (cid:31) 2 -divergence in Theorem 3.3, a larger swapping intensity a yields a more signiﬁcant acceleration effect.

Original: The acceleration effect also degenerates when the Radon-Nykodim derivative d (cid:23)= d (cid:25) is symmetric.
Revised: The acceleration effect also degenerates when the Radon-Nykodim derivative d (cid:23)= d (cid:25) is symmetric.

Original: Finally, it is worth noting that, like Theorem 3.3, Theorem 3.5 does not rely on Assumption 3.3 and holds for all objective functions U ( (cid:1) ) .
Revised: Finally, it is worth noting that, like Theorem 3.3, Theorem 3.5 does not rely on Assumption 3.3 and holds for all objective functions U ( (cid:1) ) .

Original: In summary, Theorems 3.
Revised: In summary, Theorems 3.

Original: and 3.5 together justify the beneﬁt of swapping from two different perspectives.
Revised: and 3.5 together justify the beneﬁt of swapping from two different perspectives.

Original: Note that the LDP rate function given in Theorem 3.5 takes a different form compared with the one obtained in Dupuis et al.
Revised: Note that the LDP rate function given in Theorem 3.5 takes a different form compared with the one obtained in Dupuis et al.

Original: Despite the different the forms, the two results are not contradictory.
Revised: Despite the different the forms, the two results are not contradictory.

Original: Our formula is derived from the Donsker-Varadhan theory, which uses the assumption of reversibility, while theirs is obtained from a dual formulation of the LDP rate function.
Revised: Our formula is derived from the Donsker-Varadhan theory, which uses the assumption of reversibility, while theirs is obtained from a dual formulation of the LDP rate function.

Original: Moreover, one can show that they are equivalent.
Revised: Moreover, one can show that they are equivalent.

Original: 3. D ISCRETIZATION E RROR A NALYSIS
Revised: 3. D ISCRETIZATION E RROR A NALYSIS

Original: In this section, we lay out the discretization error analysis of replica exchange Langevin diffusion.
Revised: In this section, we lay out the discretization error analysis of replica exchange Langevin diffusion.

Original: Our previous discussion in x 3.2 and x 3.3 focuses on the continuous-time process
Revised: Our previous discussion in x 3.2 and x 3.3 focuses on the continuous-time process

Original: f Z t g t (cid:21) 

Revised: f Z t g t (cid:21)

Original: deﬁned in (2.4)-(2.6), which needs to be discretized in practice.
Revised: deﬁned in (2.4)-(2.6), which needs to be discretized in practice.

Original: However, swapping the positions of particles makes it hard to analyze the discretization error, since there is no way to incorporate the swapping of positions into one uniﬁed stochastic differential equation.
Revised: However, swapping the positions of particles makes it hard to analyze the discretization error, since there is no way to incorporate the swapping of positions into one uniﬁed stochastic differential equation.

Original: To overcome this difﬁculty, note that the two particles in the replica exchange Langevin diffusion swap their positions with a speciﬁc rate, which is equivalent in distribution to swapping their temperatures with the same rate (Dupuis et al., 2012).
Revised: To overcome this difﬁculty, note that the two particles in the replica exchange Langevin diffusion swap their positions with a speciﬁc rate, which is equivalent in distribution to swapping their temperatures with the same rate (Dupuis et al., 2012).

Original: We name the later equivalent process as the temperature swapping Langevin diffusion.
Revised: We name the later equivalent process as the temperature swapping Langevin diffusion.

Original: With a slight abuse of notations, we still use f Z t g t (cid:21) 0 to denote the positions of the two particles in the temperature swapping Langevin diffusion.
Revised: With a slight abuse of notations, we still use f Z t g t (cid:21) 0 to denote the positions of the two particles in the temperature swapping Langevin diffusion.

Original: In speciﬁc, f Z t g t (cid:21) 0 is characterized by the following stochastic differential equation,
Revised: In speciﬁc, f Z t g t (cid:21) 0 is characterized by the following stochastic differential equation,

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Here (cid:6) t is a random matrix that switches between the diagonal matrices diag fp 2 (cid:28) 1 (cid:1)
Revised: Here (cid:6) t is a random matrix that switches between the diagonal matrices diag fp 2 (cid:28) 1 (cid:1)

Original: I d ; p 2 (cid:28) 2 (cid:1) I d g and diag fp 2 (cid:28) 2 (cid:1) I d ; p 2 (cid:28) 1 (cid:1) I d g , where I d denotes the d -dimensional identity matrix.
Revised: I d ; p 2 (cid:28) 2 (cid:1) I d g and diag fp 2 (cid:28) 2 (cid:1) I d ; p 2 (cid:28) 1 (cid:1) I d g , where I d denotes the d -dimensional identity matrix.

Original: The matrix (cid:6) t characterizes the switching of temperatures, and the rate of switching is same as the one speciﬁed in (2.6) and (2.7).
Revised: The matrix (cid:6) t characterizes the switching of temperatures, and the rate of switching is same as the one speciﬁed in (2.6) and (2.7).

Original: Note that the temperature swapping Langevin diffusion in (3.14) is equivalent to the replica exchange Langevin diffusion deﬁned in (2.4)-(2.6) in distribution.
Revised: Note that the temperature swapping Langevin diffusion in (3.14) is equivalent to the replica exchange Langevin diffusion deﬁned in (2.4)-(2.6) in distribution.

Original: Furthermore, the stochastic differential equation in (3.14) enables us to deﬁne a continuous-time interpolation of the discrete-time sequence, which is useful in the discretization error analysis.
Revised: Furthermore, the stochastic differential equation in (3.14) enables us to deﬁne a continuous-time interpolation of the discrete-time sequence, which is useful in the discretization error analysis.

Original: Hence, for the purpose of optimization, we only need to discretize and analyze the temperature swapping Langevin diffusion in (3.14).
Revised: Hence, for the purpose of optimization, we only need to discretize and analyze the temperature swapping Langevin diffusion in (3.14).

Original: Note that (3.14) deﬁnes a Markov jump diffusion, which can be discretized using the Euler scheme.
Revised: Note that (3.14) deﬁnes a Markov jump diffusion, which can be discretized using the Euler scheme.

Original: We denote by ( Z (1) ( k ) ; Z (2) ( k )) and ( (cid:28) (1) ( k ) ; (cid:28) (2) ( k ))
Revised: We denote by ( Z (1) ( k ) ; Z (2) ( k )) and ( (cid:28) (1) ( k ) ; (cid:28) (2) ( k ))

Original: the positions and temperatures of the two particles at discrete time k , respectively.
Revised: the positions and temperatures of the two particles at discrete time k , respectively.

Original: Let ( Z (1) (0) ; Z (2) (0))
Revised: Let ( Z (1) (0) ; Z (2) (0))

Original: =
Revised: =

Original: Z 0 and ( (cid:28) (1) (0) ; (cid:28) (2) (0)) =
Revised: Z 0 and ( (cid:28) (1) (0) ; (cid:28) (2) (0)) =

Original: ( (cid:28) 1 ; (cid:28) 2 ) be the initialization.
Revised: ( (cid:28) 1 ; (cid:28) 2 ) be the initialization.

Original: For all integers k (cid:21) 0 , we update the positions of the two particles sequentially as following,
Revised: For all integers k (cid:21) 0 , we update the positions of the two particles sequentially as following,

Original: [Equation]
Revised: [Equation]

Original: and swap temperatures according to the following rule, ((cid:28) (1) ( k + 1) ; (cid:28) (2) ( k [Equation] [Equation]
Revised: and swap temperatures according to the following rule, (

Original: = ( (cid:28)

Original: [Equation]

Original: [Equation]

Original: ( k

Original: [Equation]
Revised: [Equation]

Original:  (cid:28) [Equation] [Equation] ( k 
Revised: (1) ( k ) ) with probability a (cid:1) (cid:17) (cid:1) s ( Z (1) ( k ) ; Z (2) ( k ) ) ; (3.16) ( (cid:28) (1) ( k + 1) ; (cid:28) (2) ( k + 1) ) =

Original: [Equation]

Original: ) with probability [Equation] 16) ( (cid:28) (1) ( k + 1) ; (cid:28) (2) ( k + 1) ) = ( (cid:28) (1) ( k ) ; (cid:28) (2) ( k ) ) with probability 1 (cid:0) a (cid:1) (cid:17) (cid:1) s ( Z (1) ( k ) ; Z (2) ( k ) ) :
Revised: ( (cid:28) (1) ( k ) ; (cid:28) (2) ( k ) ) with probability 1 (cid:0) a (cid:1) (cid:17) (cid:1) s ( Z (1) ( k ) ; Z (2) ( k ) ) :

Original: Here f (cid:24) (1) ( k ) g k (cid:21) 0 and f (cid:24) (2) ( k ) g k (cid:21) 0 are two sequences of independent and identically distributed d -dimensional Gaussian random vectors, (cid:17) > 0 is the stepsize, while a and s ( (cid:1) ; (cid:1) ) are speciﬁed in (2.6).
Revised: Here f (cid:24) (1) ( k ) g k (cid:21) 0 and f (cid:24) (2) ( k ) g k (cid:21) 0 are two sequences of independent and identically distributed d -dimensional Gaussian random vectors, (cid:17) > 0 is the stepsize, while a and s ( (cid:1) ; (cid:1) ) are speciﬁed in (2.6).

Original: Note that (cid:17) and a are chosen such that (cid:17) (cid:1) a < 1 in order to deﬁne a valid probability.
Revised: Note that (cid:17) and a are chosen such that (cid:17) (cid:1) a < 1 in order to deﬁne a valid probability.

Original: Hereafter we denote by f Z (cid:17) ( k ) g k (cid:21) 

Revised: Hereafter we denote by f Z (cid:17) ( k ) g k (cid:21)

Original: the iterates of the algorithm speciﬁed in (3.15) and (3.16).
Revised: the iterates of the algorithm speciﬁed in (3.15) and (3.16).

Original: We use the superscript (cid:17) to emphasize that the iterates depend on the stepsize (cid:17) .
Revised: We use the superscript (cid:17) to emphasize that the iterates depend on the stepsize (cid:17) .

Original: Let f Z (cid:17)t g t (cid:21) 0 be the continuous-time interpolation of f Z (cid:17) ( k ) g k (cid:21) 1 , which is a continuous-time stochastic process deﬁned as
Revised: Let f Z (cid:17)t g t (cid:21) 0 be the continuous-time interpolation of f Z (cid:17) ( k ) g k (cid:21) 1 , which is a continuous-time stochastic process deﬁned as

Original: [Equation]
Revised: [Equation]

Original:  (3.17) where (cid:6) (cid:17)k(cid:17) =
Revised: W s ; (3.17) where (cid:6) (cid:17)k(cid:17) =

Original: diag f [Equation] I d ; √ 2 (cid:28) (2) ( k ) (cid:1) I d g .
Revised: diag f √ 2 (cid:28) (1) ( k ) (cid:1) I d ; √ 2 (cid:28) (2) ( k ) (cid:1) I d g .

Original: Then for all integers k (cid:21) 0 and t = k(cid:17) , we have Z (cid:17)t = Z (cid:17)k(cid:17) =
Revised: Then for all integers k (cid:21) 0 and t = k(cid:17) , we have Z (cid:17)t = Z (cid:17)k(cid:17) =

Original: Z (cid:17) ( k ) .
Revised: Z (cid:17) ( k ) .

Original: In the following theorem, we characterize the discretization error by the mean squared error between the continuous-time temperature swapping Langevin diffusion in (3.14) and the continuous-time interpolation in (3.17) of the discrete-time algorithm in (3.15) and (3.16).
Revised: In the following theorem, we characterize the discretization error by the mean squared error between the continuous-time temperature swapping Langevin diffusion in (3.14) and the continuous-time interpolation in (3.17) of the discrete-time algorithm in (3.15) and (3.16).

Original: In speciﬁc, we consider a ﬁxed time interval [0 ; T ] and upper bound the mean squared error E [ ∥ Z t (cid:0) Z (cid:17)t ∥ 2 ] for all t 2 [0 ; T ] .
Revised: In speciﬁc, we consider a ﬁxed time interval [0 ; T ] and upper bound the mean squared error E [ ∥ Z t (cid:0) Z (cid:17)t ∥ 2 ] for all t 2 [0 ; T ] .

Original: The following theorem shows that this error grows linearly with respect to the stepsize (cid:17) .
Revised: The following theorem shows that this error grows linearly with respect to the stepsize (cid:17) .

Original: Theorem 3.6.
Revised: Theorem 3.6.

Original: Under Assumption 3.1, there exists a constant (cid:13) ( d; (cid:28) 1 ; (cid:28) 2 ; a; L; (cid:11); (cid:12); T ) that only depends on the dimension d , temperature parameters (cid:28) 1 and (cid:28) 2 , swapping intensity a , smoothness constant L and dissipative constants ( (cid:11); (cid:12) ) of U ( (cid:1) ) , and length of the time interval T , such that for all t 2 [0 ; T ] ,
Revised: Under Assumption 3.1, there exists a constant (cid:13) ( d; (cid:28) 1 ; (cid:28) 2 ; a; L; (cid:11); (cid:12); T ) that only depends on the dimension d , temperature parameters (cid:28) 1 and (cid:28) 2 , swapping intensity a , smoothness constant L and dissipative constants ( (cid:11); (cid:12) ) of U ( (cid:1) ) , and length of the time interval T , such that for all t 2 [0 ; T ] ,

Original: [Equation]
Revised: [Equation]

Original: provided that the stepsize (cid:17) satisﬁes 0 < (cid:17) < (cid:11)=L 2 .
Revised: provided that the stepsize (cid:17) satisﬁes 0 < (cid:17) < (cid:11)=L 2 .

Original: The proof of Theorem 3.6 adapts from the proof framework of Theorem 5.13 in Yin & Zhu (2010).
Revised: The proof of Theorem 3.6 adapts from the proof framework of Theorem 5.13 in Yin & Zhu (2010).

Original: The main obstacle is that we establish a result for the high-dimensional case, while the original proof is for one dimension.

Original: We defer the detailed proof to x C of the appendix.
Revised: We defer the detailed proof to x C of the appendix.

Original: A D ISCUSSION AND C ONCLUSION
Revised: A D ISCUSSION AND C ONCLUSION

Original: In this paper, we apply the idea of replica exchange to the Langevin diffusion in the context of nonconvex optimization.
Revised: In this paper, we apply the idea of replica exchange to the Langevin diffusion in the context of nonconvex optimization.

Original: In particular, we quantify the beneﬁts of replica exchange compared with the standard Langevin diffusion from two perspectives, that is, the convergence of the (cid:31) 2 -divergence and the LDP of the empirical measures.
Revised: In particular, we quantify the beneﬁts of replica exchange compared with the standard Langevin diffusion from two perspectives, that is, the convergence of the (cid:31) 2 -divergence and the LDP of the empirical measures.

Original: We show from both perspectives that replica exchange accelerates the convergence towards the invariant distribution.
Revised: We show from both perspectives that replica exchange accelerates the convergence towards the invariant distribution.

Original: In the following, we discuss several related issues.
Revised: In the following, we discuss several related issues.

Original: Flat and sharp minima: The Langevin diffusion yields an invariant distribution that concentrates around the global minima.
Revised: Flat and sharp minima: The Langevin diffusion yields an invariant distribution that concentrates around the global minima.

Original: In particular, as illustrated in Keskar et al.
Revised: In particular, as illustrated in Keskar et al.

Original: (2016), ﬂat minima generally achieves better generalization than sharp ones.
Revised: (2016), ﬂat minima generally achieves better generalization than sharp ones.

Original: Meanwhile, as observed in Zhang et al.
Revised: Meanwhile, as observed in Zhang et al.

Original: (2018), the invariant distribution of the Langevin diffusion biases more towards the ﬂat minima than the sharp minima.
Revised: (2018), the invariant distribution of the Langevin diffusion biases more towards the ﬂat minima than the sharp minima.

Original: Intuitively, this observation follows from locally integrating the density of the invariant distribution, which is proportional to exp f(cid:0) U ( x ) =(cid:28) g .
Revised: Intuitively, this observation follows from locally integrating the density of the invariant distribution, which is proportional to exp f(cid:0) U ( x ) =(cid:28) g .

Original: As speciﬁed in (2.3), the invariant distribution of the replica exchange Langevin diffusion takes the same form, and hence processes the same desired property of biasing towards the ﬂat minima.
Revised: As speciﬁed in (2.3), the invariant distribution of the replica exchange Langevin diffusion takes the same form, and hence processes the same desired property of biasing towards the ﬂat minima.

Original: Choice of swapping intensity: Recall that in x 3 we prove that swapping accelerates the exponential rate of convergence of the (cid:31) 2 -divergence and increases the LDP rate function of the empirical measures.
Revised: Choice of swapping intensity: Recall that in x 3 we prove that swapping accelerates the exponential rate of convergence of the (cid:31) 2 -divergence and increases the LDP rate function of the empirical measures.

Original: In particular, a large swapping intensity a leads to a stronger acceleration effect.
Revised: In particular, a large swapping intensity a leads to a stronger acceleration effect.

Original: In continuous time, the swapping intensity a should be as large as possible.
Revised: In continuous time, the swapping intensity a should be as large as possible.

Original: However, in discrete time, to ensure the existence of a valid swapping probability in (3.16), a can only be as large as 1 =(cid:17) , since by (2.6) we have s ( (cid:1) ; (cid:1) )
Revised: However, in discrete time, to ensure the existence of a valid swapping probability in (3.16), a can only be as large as 1 =(cid:17) , since by (2.6) we have s ( (cid:1) ; (cid:1) )

Original: (cid:20) 1 .
Revised: (cid:20) 1 .

Original: This constraint characterizes the tradeoff between the acceleration effect and the discretization effort, since more ﬁne-grained discretization with a smaller stepsize (cid:17) allows for a larger swapping intensity a .
Revised: This constraint characterizes the tradeoff between the acceleration effect and the discretization effort, since more ﬁne-grained discretization with a smaller stepsize (cid:17) allows for a larger swapping intensity a .

Original: Moreover, intuitively speaking, more frequent swapping makes the particles more volatile, which lowers the accuracy of discretization.
Revised: Moreover, intuitively speaking, more frequent swapping makes the particles more volatile, which lowers the accuracy of discretization.

Original: Choice of temperatures: In the replica exchange Langevin diffusion, we use two particles driven by low and high temperatures to achieve “local exploitation” and “global exploration”, respectively.
Revised: Choice of temperatures: In the replica exchange Langevin diffusion, we use two particles driven by low and high temperatures to achieve “local exploitation” and “global exploration”, respectively.

Original: There is a fundamental tradeoff in the choice of the temperatures, especially the high temperature.
Revised: There is a fundamental tradeoff in the choice of the temperatures, especially the high temperature.

Original: In speciﬁc, a larger (cid:28) 2 results in faster global exploration.
Revised: In speciﬁc, a larger (cid:28) 2 results in faster global exploration.

Original: However, if (cid:28) 2 is too large, by the second equation in (2.4), the gradient term ∇ U ( (cid:1) ) becomes negligible compared with the white noise term p 2 (cid:28) 2 d W t .
Revised: However, if (cid:28) 2 is too large, by the second equation in (2.4), the gradient term ∇ U ( (cid:1) ) becomes negligible compared with the white noise term p 2 (cid:28) 2 d W t .

Original: Consequently, the second particle “blindly” explores the whole domain R d uniformly at random, without adapting the geometry of the objective function.
Revised: Consequently, the second particle “blindly” explores the whole domain R d uniformly at random, without adapting the geometry of the objective function.

Original: As discussed in x 2, most swaps happen only when better positions with smaller objective values are discovered by the second particle.
Revised: As discussed in x 2, most swaps happen only when better positions with smaller objective values are discovered by the second particle.

Original: Thus, if (cid:28) 2 is too large, the global exploration is fast but ineffective.
Revised: Thus, if (cid:28) 2 is too large, the global exploration is fast but ineffective.

Original: B B ACKGROUND
Revised: B B ACKGROUND

Original: In this section, we provide the background of Markov process, which is necessary for the theoretical analysis.
Revised: In this section, we provide the background of Markov process, which is necessary for the theoretical analysis.

Original: We analyze the properties of Markov processes from the viewpoint of Markov semigroup.
Revised: We analyze the properties of Markov processes from the viewpoint of Markov semigroup.

Original: A semigroup is a family of linear operators on Banach space C ( R d ) , the space of bounded continuous functions on R d equipped with the uniform norm.
Revised: A semigroup is a family of linear operators on Banach space C ( R d ) , the space of bounded continuous functions on R d equipped with the uniform norm.

Original: Formally, we have the following deﬁnition.
Revised: Formally, we have the following deﬁnition.

Original: Deﬁnition B.1 (Semigroup of Operators) .
Revised: Deﬁnition B.1 (Semigroup of Operators) .

Original: A family f P t g t (cid:21) 0 of linear operators on C ( R d ) is called a semigroup if and only if it satisﬁes the following conditions:
Revised: A family f P t g t (cid:21) 0 of linear operators on C ( R d ) is called a semigroup if and only if it satisﬁes the following conditions:

Original: (1) For t = 0 , we have P t
Revised: (1) For t = 0 , we have P t

Original: =
Revised: =

Original: I , which is the identical mapping on C ( R d ) .
Revised: I , which is the identical mapping on C ( R d ) .

Original: (2) The map t !
Revised: (2) The map t !

Original: P t is continuous in the sense that, for all f 2 C ( R d ) , t !
Revised: P t is continuous in the sense that, for all f 2 C ( R d ) , t !

Original: P t ( f ) is a continuous map from R + to C ( R d ) .
Revised: P t ( f ) is a continuous map from R + to C ( R d ) .

Original: For all f 2 C ( R d ) and s; t > 0 , we have P s + t ( f ) 
Revised: For all f 2 C ( R d ) and s; t > 0 , we have P s + t ( f ) =

Original: Moreover, a semigroup of operators is Markov if and only if the following additional conditions are satisﬁed:
Revised: Moreover, a semigroup of operators is Markov if and only if the following additional conditions are satisﬁed:

Original: (4) For all t > 0 , we have P t ( 1 ) = 1 , where 1 is the constant function with value one.
Revised: (4) For all t > 0 , we have P t ( 1 ) = 1 , where 1 is the constant function with value one.

Original: For all f (cid:21) 0 , we have P t ( f ) (cid:21) 0 .
Revised: For all f (cid:21) 0 , we have P t ( f ) (cid:21) 0 .

Original: Recall that a Markov process on R d is a stochastic process f X t g t (cid:21) 0 satisfying
Revised: Recall that a Markov process on R d is a stochastic process f X t g t (cid:21) 0 satisfying

Original: [Equation]
Revised: [Equation]

Original: for all measurable sets (cid:0) and times s < t .
Revised: for all measurable sets (cid:0) and times s < t .

Original: Here F s = (cid:27) ( X u ; u (cid:20) s ) , which is the natural ﬁltration of the Markov process f X t g t (cid:21) 0 .
Revised: Here F s = (cid:27) ( X u ; u (cid:20) s ) , which is the natural ﬁltration of the Markov process f X t g t (cid:21) 0 .

Original: Given a Markov process f X t g t (cid:21) 0 , we deﬁne a family of operators f
Revised: Given a Markov process f X t g t (cid:21) 0 , we deﬁne a family of operators f

Original: P t g t (cid:21) 0 on C ( R d ) , the space of bounded continuous functions on R d , as follows
Revised: P t g t (cid:21) 0 on C ( R d ) , the space of bounded continuous functions on R d , as follows

Original: [Equation]
Revised: [Equation]

Original: According to the Markov property in (B.1), we have
Revised: According to the Markov property in (B.1), we have

Original: [Equation]
Revised: [Equation]

Original: and hence, f P t g t (cid:21) 0 indeed forms a semigroup.
Revised: and hence, f P t g t (cid:21) 0 indeed forms a semigroup.

Original: We call f P t g t (cid:21) 

Revised: We call f P t g t (cid:21)

Original: the Markov semigroup associated with the Markov process f X t g t (cid:21) 0 .
Revised: the Markov semigroup associated with the Markov process f X t g t (cid:21) 0 .

Original: Given a Markov semigroup, we can deﬁne its inﬁnitesimal generator, a linear operator that describes the Markov semigroup’s behavior for an inﬁnitesimal t .
Revised: Given a Markov semigroup, we can deﬁne its inﬁnitesimal generator, a linear operator that describes the Markov semigroup’s behavior for an inﬁnitesimal t .

Original: Formally, we have the following deﬁnition.
Revised: Formally, we have the following deﬁnition.

Original: See, e.g., Revuz & Yor (2013) for more details.
Revised: See, e.g., Revuz & Yor (2013) for more details.

Original: Deﬁnition B.2 (Inﬁnitesimal Generator of Markov Semigroup) .
Revised: Deﬁnition B.2 (Inﬁnitesimal Generator of Markov Semigroup) .

Original: The inﬁnitesimal generator L of a
Revised: The inﬁnitesimal generator L of a

Original: Markov semigroup f P t g t (cid:21) 0 is deﬁned by
Revised: Markov semigroup f P t g t (cid:21) 0 is deﬁned by

Original: [Equation]
Revised: [Equation]

Original: for all f 2 D ( L ) .
Revised: for all f 2 D ( L ) .

Original: Here D ( L ) denotes the subset of C ( R d ) such that the above limit exists.
Revised: Here D ( L ) denotes the subset of C ( R d ) such that the above limit exists.

Original: Intuitively, the inﬁnitesimal generator can be viewed as the derivative of the Markov semigroup at time t = 0 .
Revised: Intuitively, the inﬁnitesimal generator can be viewed as the derivative of the Markov semigroup at time t = 0 .

Original: It uniquely determines the Markov semigroup according to the semigroup property (B.3).
Revised: It uniquely determines the Markov semigroup according to the semigroup property (B.3).

Original: In the following, we deﬁne the invariant and reversible measure with respect to a Markov semigroup.
Revised: In the following, we deﬁne the invariant and reversible measure with respect to a Markov semigroup.

Original: Deﬁnition B.3 (Invariance and Reversibility) .
Revised: Deﬁnition B.3 (Invariance and Reversibility) .

Original: For a Markov semigroup f P t g t (cid:21) 

Revised: For a Markov semigroup f P t g t (cid:21)

Original: whose inﬁnitesimal generator is L , a probability measure (cid:25) is invariant with respect to f P t g t (cid:21) 

Revised: whose inﬁnitesimal generator is L , a probability measure (cid:25) is invariant with respect to f P t g t (cid:21)

Original: if and only if ∫
Revised: if and only if ∫

Original: [Equation]
Revised: [Equation]

Original: for all t (cid:21) 0 and f 2 C ( R d ) .
Revised: for all t (cid:21) 0 and f 2 C ( R d ) .

Original: A probability measure (cid:25) is reversible with respect to f P t g t (cid:21) 

Revised: A probability measure (cid:25) is reversible with respect to f P t g t (cid:21)

Original: if and only if
Revised: if and only if

Original: [Equation]
Revised: [Equation]

Original: for all f; g 2 D ( L ) .
Revised: for all f; g 2 D ( L ) .

Original: Note that reversibility implies invariance when we plug g = 1 into the deﬁnition of reversibility.
Revised: Note that reversibility implies invariance when we plug g = 1 into the deﬁnition of reversibility.

Original: We remark that the standard deﬁnitions of invariance and reversibility are based on the Markov semigroup f P t g t (cid:21) 0 instead of its inﬁnitesimal generator L , which is equivalent to the following deﬁnition.
Revised: We remark that the standard deﬁnitions of invariance and reversibility are based on the Markov semigroup f P t g t (cid:21) 0 instead of its inﬁnitesimal generator L , which is equivalent to the following deﬁnition.

Original: The deﬁnitions via inﬁnitesimal generator simplify the proof of our lemmas and theorems.
Revised: The deﬁnitions via inﬁnitesimal generator simplify the proof of our lemmas and theorems.

Original: Deﬁnition B.4 (Equivalent Deﬁnitions of Invariance and Reversibility ) .
Revised: Deﬁnition B.4 (Equivalent Deﬁnitions of Invariance and Reversibility ) .

Original: For a Markov semigroup f
Revised: For a Markov semigroup f

Original: P t g t (cid:21) 0 , a probability measure (cid:25) is invariant with respect to f P t g t (cid:21) 

Revised: P t g t (cid:21) 0 , a probability measure (cid:25) is invariant with respect to f P t g t (cid:21)

Original: if and only if
Revised: if and only if

Original: [Equation]
Revised: [Equation]

Original: for every t (cid:21) 0 and f 2 C ( R d ) .
Revised: for every t (cid:21) 0 and f 2 C ( R d ) .

Original: Reversibility is another important concept in Markov semigroup theory.
Revised: Reversibility is another important concept in Markov semigroup theory.

Original: A probability measure (cid:22) is reversible for f P t g t (cid:21) 

Revised: A probability measure (cid:22) is reversible for f P t g t (cid:21)

Original: if and only if
Revised: if and only if

Original: [Equation]
Revised: [Equation]

Original: for all t (cid:21) 0 and f; g 2 C ( R d ) .
Revised: for all t (cid:21) 0 and f; g 2 C ( R d ) .

Original: Based on the Deﬁnition B.2, we can show that this deﬁnition is equivalent to Deﬁnition B.3.
Revised: Based on the Deﬁnition B.2, we can show that this deﬁnition is equivalent to Deﬁnition B.3.

Original: In the deﬁnitions of Markov semigroup and associated concepts, we restrict the test function f to C ( R d ) , the space of bounded continuous functions.
Revised: In the deﬁnitions of Markov semigroup and associated concepts, we restrict the test function f to C ( R d ) , the space of bounded continuous functions.

Original: However, for (cid:25) being the invariant distribution with respect to f P t g t (cid:21) 0 , we can always (slightly) extend the deﬁnition of Markov semigroup to a larger function space L 2 ( (cid:25) ) .
Revised: However, for (cid:25) being the invariant distribution with respect to f P t g t (cid:21) 0 , we can always (slightly) extend the deﬁnition of Markov semigroup to a larger function space L 2 ( (cid:25) ) .

Original: We use f (cid:22) P t g t (cid:21) 0 and (cid:22) L to denote the corresponding extended Markov semigroup and inﬁnitesimal generator.
Revised: We use f (cid:22) P t g t (cid:21) 0 and (cid:22) L to denote the corresponding extended Markov semigroup and inﬁnitesimal generator.

Original: One can show that, when (cid:25) is reversible, (cid:22) L is a self-adjoint operator in the Hilbert space L 2 ( (cid:25) ) , which is equipped with the inner product ⟨ f; g ⟩ (cid:25) = ∫ fg d (cid:25) .
Revised: One can show that, when (cid:25) is reversible, (cid:22) L is a self-adjoint operator in the Hilbert space L 2 ( (cid:25) ) , which is equipped with the inner product ⟨ f; g ⟩ (cid:25) = ∫ fg d (cid:25) .

Original: Furthermore, one can show that (cid:22) L is positive semideﬁnite.
Revised: Furthermore, one can show that (cid:22) L is positive semideﬁnite.

Original: In this paper, for the convenience of discussion, we do not distinguish the slight difference between L and its extension (cid:22) L , and also the difference of D ( L ) , C ( R d ) , and L 2 ( (cid:25) ) , since C ( R d ) is dense in L 2 ( (cid:25) ) and D ( L ) is dense in
Revised: In this paper, for the convenience of discussion, we do not distinguish the slight difference between L and its extension (cid:22) L , and also the difference of D ( L ) , C ( R d ) , and L 2 ( (cid:25) ) , since C ( R d ) is dense in L 2 ( (cid:25) ) and D ( L ) is dense in

Original: C ( R d ) according to Hille-Yosida theorem (Phillips & Hille, 1957).
Revised: C ( R d ) according to Hille-Yosida theorem (Phillips & Hille, 1957).

Original: When we choose a test function, we also assume that all the related operations are well-deﬁned.
Revised: When we choose a test function, we also assume that all the related operations are well-deﬁned.

Original: C D ETAILED P ROOFS
Revised: C D ETAILED P ROOFS

Original: In this section, we present the detailed proofs for all the theorems and lemmas in this paper.
Revised: In this section, we present the detailed proofs for all the theorems and lemmas in this paper.

Original: C. P ROOF OF L EMMA 3. 
Revised: C. P ROOF OF L EMMA 3.

Original: Proof.
Revised: Proof.

Original: The dynamics in (2.4)-(2.6) deﬁnes a Langevin diffusion with jump.
Revised: The dynamics in (2.4)-(2.6) deﬁnes a Langevin diffusion with jump.

Original: Its generator takes the form
Revised: Its generator takes the form

Original: [Equation]
Revised: [Equation]

Original: and its domain D ( L a ) =
Revised: and its domain D ( L a ) =

Original: C 2c ( R 2 d ) , which is the space of all twice-differentiable functions with compact support.
Revised: C 2c ( R 2 d ) , which is the space of all twice-differentiable functions with compact support.

Original: The ﬁrst two terms L a 1 and L a 2 on the right-hand side of (C.1) correspond to the standard Langevin diffusion, while the last term L as arises from swapping.
Revised: The ﬁrst two terms L a 1 and L a 2 on the right-hand side of (C.1) correspond to the standard Langevin diffusion, while the last term L as arises from swapping.

Original: To prove invariance and reversibility, by Deﬁnition B.3, we only need to show that ∫R d
Revised: To prove invariance and reversibility, by Deﬁnition B.3, we only need to show that ∫

Original: [Equation]
Revised: [Equation]

Original: Note that for the Laplacian term in L a 1 on the right-hand side of (C.1), by integration by parts and the fact that f and g have compact support, we have that for all ﬁxed x 2 2 R d ,[Equation]
Revised: Note that for the Laplacian term in L a 1 on the right-hand side of (C.1), by integration by parts and the fact that f and g have compact support, we have that for all ﬁxed x 2 2 R d ,

Original: [Equation]
Revised: [Equation]

Original: Recall that (cid:22) ( x 1 ; x 2 ) is deﬁned in (2.5).
Revised: Recall that (cid:22) ( x 1 ; x 2 ) is deﬁned in (2.5).

Original: Hence, (C.3) takes the equivalent form ∫
Revised: Hence, (C.3) takes the equivalent form ∫

Revised: [Equation]

Revised: x

Original: [Equation]
Revised: [Equation]

Original: and hence,
Revised: and hence,

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: (C.4)
Revised: (C.4)

Original: By switching the positions of f and g in (C.4), we have ∫
Revised: By switching the positions of f and g in (C.4), we have ∫

Original: [Equation]
Revised: [Equation]

Original: By (C.4) and (C.5) we have ∫
Revised: By (C.4) and (C.5) we have ∫

Original: [Equation]
Revised: [Equation]

Original: By the same derivation, (C.6) also holds for L a 2 .
Revised: By the same derivation, (C.6) also holds for L a 2 .

Original: Thus, it remains to prove that (C.6) holds for L as as well.
Revised: Thus, it remains to prove that (C.6) holds for L as as well.

Original: For notational simplicity, in the following we use f + and f (cid:0) to denote f ( x 1 ; x 2 ) and f ( x 2 ; x 1 ) respectively, and deﬁne g + , g (cid:0) , (cid:22) + , and (cid:22) (cid:0) in a similar way.
Revised: For notational simplicity, in the following we use f + and f (cid:0) to denote f ( x 1 ; x 2 ) and f ( x 2 ; x 1 ) respectively, and deﬁne g + , g (cid:0) , (cid:22) + , and (cid:22) (cid:0) in a similar way.

Original: Then we have ∫
Revised: Then we have ∫

Original: R d ∫
Revised: R d ∫

Original: [Equation]
Revised: [Equation]

Original: + [Equation] =∫
Revised: =

Original: [Equation]
Revised: [Equation]

Original: where the third equality follows from symmetry
Revised: where the third equality follows from symmetry

Original: [Equation]
Revised: [Equation]

Original: Similar to (C.7)
Revised: Similar to (C.7)

Original: [Equation]
Revised: [Equation]

Original: we also have ∫
Revised: we also have ∫

Original: [Equation]
Revised: [Equation]

Original: g + (cid:0) f 
Revised: g + (cid:0) f + g

Revised: [Equation]

Original: + g + (cid:0) f (cid:0) g (cid:0) )
Revised: (cid:0) f (cid:0) [Equation] (cid:0) )

Original: d x 1 d x 2 : (C.8)
Revised: d x [Equation] d x 2 : (C.8)

Original: Combining (C.7) and (C.8), we obtain ∫
Revised: Combining (C.7) and (C.8), we obtain ∫

Original: [Equation]
Revised: [Equation]

Original: which is equivalent to ∫
Revised: which is equivalent to ∫

Original: [Equation]
Revised: [Equation]

Original: by the deﬁnition of L as in (C.1).
Revised: by the deﬁnition of L as in (C.1).

Original: In summary, we obtain that (C.2) holds for (cid:25) deﬁned in (3.4).
Revised: In summary [Equation] we obtain that (C.2) holds for (cid:25) deﬁned in (3.4).

Original: Thus, f Z t g t (cid:21) 0 is a reversible Markov process with the invariant distribution d (cid:25) [Equation] x 1 ; x 2 ) /
Revised: Thus, f Z t g t (cid:21) 0 is a reversible Markov process with the invariant distribution d (cid:25) ( x 1 ; x 2 ) /

Original: [Equation]
Revised: [Equation]

Original: C. P ROOF OF T HEOREM 3. 
Revised: C. P ROOF OF T HEOREM 3.

Original: Proof.
Revised: Proof.

Original: Recall the inﬁnitesimal generator of the replica exchange Langevin diffusion with swapping intensity a deﬁned in (3.3).
Revised: Recall the inﬁnitesimal generator of the replica exchange Langevin diffusion with swapping intensity a deﬁned in (3.3).

Original: According to the properties of ∇ and ∆ , for i = 1 ; 2 , we have
Revised: According to the properties of ∇ and ∆ , for i = 1 ; 2 , we have

Original: [Equation]
Revised: [Equation]

Original: Then by the deﬁnition of the Carr´e du Champ operator (cid:0) a in (3.5)
Revised: Then by the deﬁnition of the Carr´e du Champ operator (cid:0) a in (3.5)

Original: [Equation]
Revised: [Equation]

Original: we have (cid:0)
Revised: we have (cid:0)

Original: [Equation]
Revised: [Equation]

Original: Hence, the Dirichlet form E a is given by
Revised: Hence, the Dirichlet form E a is given by

Original: [Equation]
Revised: [Equation]

Original: which concludes the proof of Theorem
Revised: which concludes the proof of Theorem

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: C. P ROOF OF T HEOREM 3. 
Revised: C. P ROOF OF T HEOREM 3.

Original: Proof.
Revised: Proof.

Original: Note that the replica exchange Langevin diffusion without swapping, that is, a = 0 , shares the same invariant distribution in (2.5) with the ones with swapping, that is, a > 0 .
Revised: Note that the replica exchange Langevin diffusion without swapping, that is, a = 0 , shares the same invariant distribution in (2.5) with the ones with swapping, that is, a > 0 .

Original: Furthermore,
Revised: Furthermore,

Original: Theorem 3.3 shows that E a ( f ) (cid:21) E 0 ( f ) for all a > 0 .
Revised: Theorem 3.3 shows that E a ( f ) (cid:21) E 0 ( f ) for all a > 0 .

Original: Hence it remains to show that the replica exchange Langevin diffusion without swapping satisﬁes the Poincar´e inequality in (3.10).
Revised: Hence it remains to show that the replica exchange Langevin diffusion without swapping satisﬁes the Poincar´e inequality in (3.10).

Original: Recall that the replica exchange Langevin diffusion without swapping is deﬁned as
Revised: Recall that the replica exchange Langevin diffusion without swapping is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: and its inﬁnitesimal generator is given by
Revised: and its inﬁnitesimal generator is given by

Original: [Equation]
Revised: [Equation]

Original: The Poincar´e inequality for the replica exchange Langevin diffusion in (C.9) is a direct consequence of the existence of a Lyapunov function.
Revised: The Poincar´e inequality for the replica exchange Langevin diffusion in (C.9) is a direct consequence of the existence of a Lyapunov function.

Original: In the speciﬁc, we have the following lemma, which is adapted from Theorem 1.4 in Bakry et al.
Revised: In the speciﬁc, we have the following lemma, which is adapted from Theorem 1.4 in Bakry et al.

Original: Lemma C.

Revised: Lemma C.

Original: (Lyapunov implies Poincar´e) .
Revised: (Lyapunov implies Poincar´e) .

Original: Let f Z t g t (cid:21) 0 be the replica exchange Langevin diffusion without swapping on R 2 d , whose inﬁnitesimal generator is L 0 deﬁned in (C.10).
Revised: Let f Z t g t (cid:21) 0 be the replica exchange Langevin diffusion without swapping on R 2 d , whose inﬁnitesimal generator is L 0 deﬁned in (C.10).

Original: We further assume that there exists a function V ( x 1 ; x 2 ) (cid:21) 1 ; and constants (cid:21) > 0 , b (cid:21) 0 , and r > 0 such that
Revised: We further assume that there exists a function V ( x 1 ; x 2 ) (cid:21) 1 ; and constants (cid:21) > 0 , b (cid:21) 0 , and r > 0 such that

Original: [Equation]
Revised: [Equation]

Original: where 1 B r denotes the indicator function of the centered ball with radius r in R 2 d .
Revised: where 1 B r denotes the indicator function of the centered ball with radius r in R 2 d .

Original: Then f Z t g t (cid:21)satisﬁes the Poincar´e inequality.
Revised: Then f Z t g t (cid:21)satisﬁes the Poincar´e inequality.

Original: We call V ( x 1 ; x 
Revised: We call V ( x 1 ; x 2 ) the function.

Original: Proof.
Revised: Proof.

Original: We extend the proof of Theorem 1.4 in Bakry et al.
Revised: We extend the proof of Theorem 1.4 in Bakry et al.

Original: (2008), which characterizes the standard
Revised: (2008), which characterizes the standard

Original: Langevin diffusion with a single temperature, to the setting with two temperatures.
Revised: Langevin diffusion with a single temperature, to the setting with two temperatures.

Original: For completeness, we provide a detailed proof in x C.4.
Revised: For completeness, we provide a detailed proof in x C.4.

Original: When the loss function U ( x ) satisﬁes the ( (cid:11); (cid:12) )
Revised: When the loss function U ( x ) satisﬁes the ( (cid:11); (cid:12) )

Original: -dissipative condition in Assumption 3.1, following
Revised: -dissipative condition in Assumption 3.1, following

Original: Raginsky et al.
Revised: Raginsky et al.

Original: (2017), we construct the following Lyapunov function
Revised: (2017), we construct the following Lyapunov function

Original: [Equation]
Revised: [Equation]

Original: A direct calculation shows that V ( x 1 ; x 2 ) satisﬁes the condition in (C.11).
Revised: A direct calculation shows that V ( x 1 ; x 2 ) satisﬁes the condition in (C.11).

Original: Hence, we apply Lemma
Revised: Hence, we apply Lemma

Original: C.1 to obtain that the replica exchange Langevin diffusion without swapping satisﬁes the Poincar´e inequality.
Revised: C.1 to obtain that the replica exchange Langevin diffusion without swapping satisﬁes the Poincar´e inequality.

Original: In summary, we prove that the Poincar´e inequality holds for the replica exchange
Revised: In summary, we prove that the Poincar´e inequality holds for the replica exchange

Original: Langevin diffusion, which concludes the proof.
Revised: Langevin diffusion, which concludes the proof.

Original: C. P ROOF OF L EMMA C. 
Revised: C. P ROOF OF L EMMA C.

Original: Proof.
Revised: Proof.

Original: We ﬁrst prove the Poincar´e inequality in a more general form.
Revised: We ﬁrst prove the Poincar´e inequality in a more general form.

Original: For all functions f ( x 1 ; x 2 ) : R 2 d !
Revised: For all functions f ( x 1 ; x 2 ) : R 2 d !

Original: R , we show that if there exists a function V ( x 1 ; x 2 ) (cid:21) 1 such that (C.11) holds, then there exists a constant (cid:26) such that
Revised: R , we show that if there exists a function V ( x 1 ; x 2 ) (cid:21) 1 such that (C.11) holds, then there exists a constant (cid:26) such that

Original: [Equation]
Revised: [Equation]

Original: ) ; where (cid:28) 1 and (cid:28) 2 are the temperatures, and (cid:25) ( x 1 ; x 2 ) is the invariant distribution.
Revised: ; where (cid:28) 1 and (cid:28) 2 are the temperatures, and (cid:25) ( x 1 ; x 2 ) is the invariant distribution.

Original: By setting the function f as the Radon-Nykodim derivative d (cid:23)= d (cid:25) , we obtain the Poincar´e inequality for the (cid:31) 2divergence in (3.10).
Revised: By setting the function f as the Radon-Nykodim derivative d (cid:23)= d (cid:25) , we obtain the Poincar´e inequality for the (cid:31) 2divergence in (3.10).

Original: Note that for all constants u 2 R , we have
Revised: Note that for all constants u 2 R , we have

Original: [Equation]
Revised: [Equation]

Original: where Var (cid:25) ( f ) denotes the variance of f ( X 1 ; X 2 ) when ( X 1 ; X 2 ) follows the invariant distribution (cid:25) .
Revised: where Var (cid:25) ( f ) denotes the variance of f ( X 1 ; X 2 ) when ( X 1 ; X 2 ) follows the invariant distribution (cid:25) .

Original: We use f u to denote the function f (cid:0) u .
Revised: We use f u to denote the function f (cid:0) u .

Original: By multiplying f 2 u ( x 1 ; x 2 ) =
Revised: By multiplying f 2 u ( x 1 ; x 2 ) =

Original: ( (cid:21)V ( x 1 ; x 2 )) on the both sides of (C.11) and integrating them over (cid:25) ( x 1 ; x 2 ) , we have ∫
Revised: ( (cid:21)V ( x 1 ; x 2 )) on the both sides of (C.11) and integrating them over (cid:25) ( x 1 ; x 2 ) , we have ∫

Original: [Equation]
Revised: [Equation]

Original: In the sequel, we upper bound the two terms on the right-hand side of (C.12).
Revised: In the sequel, we upper bound the two terms on the right-hand side of (C.12).

Original: For the ﬁrst term on the right-hand side of (C.12), we invoke the divergence theorem, which states that for a scalar-valued function g , whose value at inﬁnity is zero, and a vector-valued function h , for the integration under the measure !
Revised: For the ﬁrst term on the right-hand side of (C.12), we invoke the divergence theorem, which states that for a scalar-valued function g , whose value at inﬁnity is zero, and a vector-valued function h , for the integration under the measure !

Original: we have ∫
Revised: we have ∫

Original: [Equation]
Revised: [Equation]

Original: In (C.13), by setting d ! as the Lebesgue measure d x 1 d x 2 and g = exp ((cid:0) U
Revised: In (C.13), by setting d ! as the Lebesgue measure d x 1 d x 2 and g = exp (

Original: ( x 1 ) =(cid:28) 1 (cid:0) U ( x

Original: [Equation]
Revised: [Equation]

Original: (cid:28) 2 ) and h= f u ( x
Revised: and h

Original: [Equation]
Revised: [Equation]

Original: x
Revised: f

Original: [Equation]
Revised: [Equation]

Original: ( (cid:21)V

Original: [Equation]

Original: x 2 ) ; we have ∫

Original: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: + (cid:28) 2 =(cid:21) (cid:1) ∫⟨ ∇ x [ f u ( x
Revised: ; we have ∫

Original: [Equation]
Revised: [Equation]

Original: x 2 )
Revised: (cid:28)

Original: [Equation]
Revised: [Equation]

Original: V ( x 1 ; x 2 ) ] [Equation] ∇ x2 V x
Revised: (cid:21) (cid:1) ∫ ⟨ ∇ x

Original: [Equation]
Revised: [Equation]

Original: x

Original: [Equation]
Revised: [Equation]

Original: ⟩ (cid:1) d (cid:25) ( x

Original: [Equation]
Revised: [Equation]

Original: x 2 ) : (C.14)
Revised: Meanwhile we

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst equality follows from expanding the gradient of f 2 u ( x 1 ; x 2 )
Revised: where the ﬁrst equality follows from expanding the gradient of f 2 u ( x 1 ; x 2 )

Original: =
Revised: =

Original: V ( x 1 ; x 2 ) with respect to x 1 .
Revised: V ( x 1 ; x 2 ) with respect to x 1 .

Original: By a similar derivation, the same inequality holds for the second term on the right-hand side of (C.14) as well.
Revised: By a similar derivation, the same inequality holds for the second term on the right-hand side of (C.14) as well.

Original: Hence, plugging them into (C.15), for the ﬁrst term on the right-hand side of (C.12) we have
Revised: Hence, plugging them into (C.15), for the ﬁrst term on the right-hand side of (C.12) we have

Original: [Equation]
Revised: [Equation]

Original: For the second term on the right-hand side of (C.12), we restrict our attention to the bounded domain B r .
Revised: For the second term on the right-hand side of (C.12), we restrict our attention to the bounded domain B r .

Original: Based on the assumption that V ( x 1 ; x 2 ) (cid:21) 1 , there exists a positive constant k such that ∫
Revised: Based on the assumption that V ( x 1 ; x 2 ) (cid:21) 1 , there exists a positive constant k such that ∫

Original: [Equation]
Revised: [Equation]

Original: 17) where the second inequality follows from the Poincar´e inequality for measures on a bounded domain (Bakry et al., 2008).
Revised: 17) where the second inequality follows from the Poincar´e inequality for measures on a bounded domain (Bakry et al., 2008).

Original: Since (C.17) holds for all functions f u ( x 1 ; x 2 ) , one can choose a suitable u (cid:3) such that ∫
Revised: Since (C.17) holds for all functions f u ( x 1 ; x 2 ) , one can choose a suitable u (cid:3) such that ∫

Original: B r f u (cid:3) d (cid:25) = 0 .
Revised: B r f u (cid:3) d (cid:25) = 0 .

Original: Then we have ∫
Revised: Then we have ∫

Original: [Equation]
Revised: [Equation]

Original: Finally
Revised: Finally

Original: [Equation]
Revised: [Equation]

Original: by plugging (C.16) and (C.18) into (C.12)
Revised: by plugging (C.16) and (C.18) into (C.12)

Original: [Equation]
Revised: [Equation]

Original: we have that there exists a positive constant (cid:26) such that
Revised: we have that there exists a positive constant (cid:26) such that

Original: [Equation]
Revised: [Equation]

Original: which concludes the proof of Lemma C
Revised: which concludes the proof of Lemma C

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: C. P ROOF OF T HEOREM 3. 
Revised: C. P ROOF OF T HEOREM 3.

Original: According to Lemma 3.2, the replica exchange Langevin diffusion f Z t g t (cid:21) 0 is reversible.
Revised: According to Lemma 3.2, the replica exchange Langevin diffusion f Z t g t (cid:21) 0 is reversible.

Original: Hence, we can apply the Donsker-Varadhan theory, which states that for a continuous-time reversible Markov process with invariant distribution (cid:25) and inﬁnitesimal generator L , LDP holds and its LDP rate function takes the explicit form
Revised: Hence, we can apply the Donsker-Varadhan theory, which states that for a continuous-time reversible Markov process with invariant distribution (cid:25) and inﬁnitesimal generator L , LDP holds and its LDP rate function takes the explicit form

Original: [Equation]
Revised: [Equation]

Original: (cid:25) ; if (cid:23) ≪ (cid:25); 1 ; otherwise : (C.19)
Revised: (cid:25) ; if (cid:23) ≪ (cid:25); 1 ; otherwise : (C.19)

Original: Here p L denotes the square root of (cid:0) L , which is deﬁned as follows.
Revised: Here p L denotes the square root of (cid:0) L , which is deﬁned as follows.

Original: Let A be a positive semideﬁnite self-adjoint operator in the Hilbert space ( H ; ⟨(cid:1) ; (cid:1)⟩ H ) , the square root of A is deﬁned as the self-adjoint operator B such that A = B 2 , which means ⟨ x; Ax ⟩ H = ⟨ x; B 2 x ⟩ H = ⟨ Bx; Bx ⟩ H for every x 2 H .
Revised: Let A be a positive semideﬁnite self-adjoint operator in the Hilbert space ( H ; ⟨(cid:1) ; (cid:1)⟩ H ) , the square root of A is deﬁned as the self-adjoint operator B such that A = B 2 , which means ⟨ x; Ax ⟩ H = ⟨ x; B 2 x ⟩ H = ⟨ Bx; Bx ⟩ H for every x 2 H .

Original: As explained in x B, the inﬁnitesimal generator (cid:0) L can be extended to a positive semideﬁnite self-adjoint operator in the Hilbert space L 2 ( (cid:25) ) .
Revised: As explained in x B, the inﬁnitesimal generator (cid:0) L can be extended to a positive semideﬁnite self-adjoint operator in the Hilbert space L 2 ( (cid:25) ) .

Original: We ignore the slight differences caused by extension.
Revised: We ignore the slight differences caused by extension.

Original: Hence, p(cid:0) L is well-deﬁned here.
Revised: Hence, p(cid:0) L is well-deﬁned here.

Original: Recall that the inﬁnitesimal generator L a of the replica exchange Langevin diffusion is given by (3.3).
Revised: Recall that the inﬁnitesimal generator L a of the replica exchange Langevin diffusion is given by (3.3).

Original: We ﬁrst derive the explicit form of ∥p(cid:0)
Revised: We ﬁrst derive the explicit form of ∥p(cid:0)

Original: L a ( f ) ∥ 2 (cid:25) .
Revised: L a ( f ) ∥ 2 (cid:25) .

Original: Since the square root of a positive semidefinite self-adjoint operator is self-adjoint, we have
Revised: Since the square root of a positive semidefinite self-adjoint operator is self-adjoint, we have

Original: [Equation]
Revised: [Equation]

Original: Since (cid:25) is the invariant distribution, by Deﬁnition B.3 we have ∫ L a
Revised: Since (cid:25) is the invariant distribution, by Deﬁnition B.3 we have ∫ L a

Original: [Equation]
Revised: [Equation]

Original: f
Revised: f

Original: [Equation]
Revised: [Equation]

Original: )d
Revised: )d

Original: [Equation]
Revised: [Equation]

Original: = 0 .
Revised: = 0 .

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: which is exactly the Dirichlet form.
Revised: which is exactly the Dirichlet form.

Original: Then based on (3.3) [Equation] we obtain
Revised: Then based on (3.3), we obtain

Original: [Equation]
Revised: [Equation]

Original: Hence
Revised: Hence

Original: [Equation]
Revised: [Equation]

Original: we have that for probability measures (cid:23) ≪ (cid:25) ,
Revised: we have that for probability measures (cid:23) ≪ (cid:25) ,

Original: [Equation]
Revised: [Equation]

Original: and I ( (cid:23) )
Revised: and I ( (cid:23) )

Original: = 1 , otherwise.
Revised: = 1 , otherwise.

Original: C. P ROOF OF T HEOREM 3. 
Revised: C. P ROOF OF T HEOREM 3.

Original: Proof.
Revised: Proof.

Original: Recall that the temperature swapping Langevin diffusion f Z t g t (cid:21) 

Revised: Recall that the temperature swapping Langevin diffusion f Z t g t (cid:21)

Original: and the continuous-time interpolated process f Z (cid:17)t g t (cid:21) 0 are deﬁned in (3.14) and (3.17).
Revised: and the continuous-time interpolated process f Z (cid:17)t g t (cid:21) 0 are deﬁned in (3.14) and (3.17).

Original: For all t 2 [0 ; T ] , we have
Revised: For all t 2 [0 ; T ] , we have

Original: [Equation]
Revised: [Equation]

Original: By applying the Cauchy-Schwartz inequality and then taking expectation, we have
Revised: By applying the Cauchy-Schwartz inequality and then taking expectation, we have

Original: [Equation]
Revised: [Equation]

Original: In the sequel, we upper bound the two terms on the right-hand side of (C.21).
Revised: In the sequel, we upper bound the two terms on the right-hand side of (C.21).

Original: For the ﬁrst term, by Cauchy-Schwarz inequality, we have
Revised: For the ﬁrst term, by Cauchy-Schwarz inequality, we have

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation] 
Revised: [Equation] [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: C.22) where the second inequality follows from the L -smoothness of U ( (cid:1) ) in Assumption 3.1.
Revised: C.22) where the second inequality follows from the L -smoothness of U ( (cid:1) ) in Assumption 3.1.

Original: In the following, we upper bound the second term on the right-hand side of (C.22).
Revised: In the following, we upper bound the second term on the right-hand side of (C.22).

Original: Note that we have
Revised: Note that we have

Original: [Equation]
Revised: [Equation]

Original: For all integers k (cid:21) 

Revised: For all integers k (cid:21)

Original: and s 2 [ k(cid:17); ( k + 1) (cid:17) ) , based on the deﬁnition of f Z (cid:17)t g t (cid:21) 0 in (3.17), we have
Revised: and s 2 [ k(cid:17); ( k + 1) (cid:17) ) , based on the deﬁnition of f Z (cid:17)t g t (cid:21) 0 in (3.17), we have

Original: [Equation]
Revised: [Equation]

Original: By applying the Cauchy-Schwartz inequality again, we obtain (cid:13)(cid:13)
Revised: By applying the Cauchy-Schwartz inequality again, we obtain (cid:13)(cid:13)

Original: [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation][Table] [Equation] [Table]
Revised: [Equation]

Original: [Equation] [Table] [Equation] [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table][Equation] [Table] [Equation]
Revised: [Table]

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table][Equation] [Table] [Equation]
Revised: [Table]

Original: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]

Original: where x (cid:3) is a stationary point of U ( (cid:1) )
Revised: where x (cid:3) is a stationary point of U ( (cid:1) )

Original: [Table]
Revised: [Table]

Original: and by deﬁnition ∇ U ( x (cid:3) ) = 0 .
Revised: and by deﬁnition ∇ U ( x (cid:3) ) = 0 .

Original: Recall that by Assumption 3.1, U ( (cid:1) ) is L -smooth.
Revised: Recall that by Assumption 3.1, U ( (cid:1) ) is L -smooth.

Original: Based on (C.24), we have
Revised: Based on (C.24), we have

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: ( s (cid:0) k(cid:17) )

Original: 2 + 2 (cid:1) (cid:13)(cid:13)(cid:13) (cid:6) (cid:17)k(cid:17) ∫ s k(cid:17)

Original: d W u (cid:13)(cid:13)(cid:13)(cid:20)

Original: [Equation]
Revised: [Equation]

Original: L

Original: [Equation]
Revised: [Table]

Original: (cid:1) ( ∥
Revised: ( s (cid:0) k(cid:17) )

Original: [Equation] (cid:17)k(cid:17) ∥ [Equation] ∥ 2 ) (cid:1) (
Revised: 2 + 2 (cid:1) (cid:13)(cid:13)(cid:13) (cid:6) (cid:17)k(cid:17) ∫ s k(cid:17)

Original: [Equation] (cid:0) [Equation] [Equation]
Revised: d W u (cid:13)(cid:13)(cid:13)(cid:20) 4 L 2 (cid:1) ( ∥ Z (cid:17)k(cid:17) ∥ 2 + ∥ x (cid:3) ∥ 2 ) (cid:1) ( s (cid:0) k(cid:17) )

Original:  + [Equation] (cid:1) (cid:13)(cid:13)(cid:13) (cid:6) (cid:17)k(cid:17) ∫ s k(cid:17)
Revised: 2 + 2 (cid:1) (cid:13)(cid:13)(cid:13) (cid:6) (cid:17)k(cid:17) ∫ s k(cid:17)

Original: [Equation]

Original:  W u (cid:13)(cid:13)(cid:13) 2 : (C.25)
Revised: d W u (cid:13)(cid:13)(cid:13) 2 : (C.25)

Original: By integrating the second inequality in (C.25) over the interval [ k(cid:17); ( k +1) (cid:17) ] and taking expectation, then plugging it into the right-hand side of (C.23), we obtain
Revised: By integrating the second inequality in (C.25) over the interval [ k(cid:17); ( k +1) (cid:17) ] and taking expectation, then plugging it into the right-hand side of (C.23), we obtain

Original: E [∫ ( k +1) (cid:17) k(cid:17) (cid:13)(cid:13)
Revised: E [∫ ( k +1) (cid:17) k(cid:17) (cid:13)(cid:13)

Original: [Equation]

Original:  (cid:17)s (cid:0) Z (cid:17) [Equation] (cid:17) ⌋ (cid:17) (cid:13)(cid:13) 2 d s ] (cid:20) [Equation]{+4 L 2 (cid:1) ( sup k (cid:21) 
+}
Revised: Z (cid:17)s (cid:0) Z (cid:17) ⌊ s=(cid:17) ⌋ (cid:17) (cid:13)(cid:13) 2 d s ] (cid:20) {+4 L 2 (cid:1) ( sup k (cid:21) 
+}

Original: L
Revised: E

Original: [-[Equation] (cid:1) ( sup k (cid:21) 
 E-] [ ∥ Z (cid:17)k(cid:17) [Equation]
Revised: [-[Equation] (cid:1) ( sup k (cid:21) 
 E-] [ ∥ Z (cid:17)k(cid:17) ∥ 2 ] + ∥ x (cid:3) ∥) (cid:1) (cid:17) 3 = 3 + 2 (cid:1) ∫ ( k +1) (cid:17) k(cid:17)

Original: +

Original: [Equation]

Original: (cid:3) ∥) (cid:1) (cid:17) 3 = 3 + 2 (cid:1) [Equation] ( k +1) (cid:17) k(cid:17) E [(cid:13)(cid:13)(cid:13) (cid:6) (cid:17)k(cid:17) ∫ s k(cid:17)
Revised: E [(cid:13)(cid:13)(cid:13) (cid:6) (cid:17)k(cid:17) ∫ s k(cid:17)

Original: d W u (cid:13)(cid:13)(cid:13)
Revised: d W u (cid:13)(cid:13)(cid:13)

Original: [Equation]
Revised: [Equation]

Original: Note that on the right-hand side of (C.26), (cid:6) (cid:17)k(cid:17) is a diagonal matrix with diagonal entries p 2 (cid:28) 1 or p 2 (cid:28) 2 , then by Itˆo isometry, we have
Revised: Note that on the right-hand side of (C.26), (cid:6) (cid:17)k(cid:17) is a diagonal matrix with diagonal entries p 2 (cid:28) 1 or p 2 (cid:28) 2 , then by Itˆo isometry, we have

Original: [Equation]
Revised: [Equation]

Original: where (cid:6) (cid:17)k(cid:17) ( j ) denotes the j -th diagonal entry of the matrix (cid:6) (cid:17)k(cid:17) , and W ( j ) u is the j -th component of the 2 d -dimensional Brownian motion W u .
Revised: where (cid:6) (cid:17)k(cid:17) ( j ) denotes the j -th diagonal entry of the matrix (cid:6) (cid:17)k(cid:17) , and W ( j ) u is the j -th component of the 2 d -dimensional Brownian motion W u .

Original: Hence, we have ∫
Revised: Hence, we have ∫

Original: [Equation]
Revised: [Equation]

Original: which provides an upper bound for the right-hand side of (C.26)
Revised: which provides an upper bound for the right-hand side of (C.26)

Original: Then by plugging (C.27) into (C.26), we obtain
Revised: Then by plugging (C.27) into (C.26), we obtain

Original: E [∫
Revised: E [∫

Original: [Equation]
Revised: [Equation]

Original: k +
Revised: k +

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: (C.28)
Revised: C.28)

Original: Then by plugging (C.28) into (C.23), we have
Revised: Then by plugging (C.28) into (C.23), we have

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: The following lemma shows that if the discretization stepsize (cid:17) falls into the interval (0 ; (cid:11)=L 2 ) , f Z (cid:17) k(cid:17) g k (cid:21) 0 are uniformly upper bounded in the L 2 sense.
Revised: The following lemma shows that if the discretization stepsize (cid:17) falls into the interval (0 ; (cid:11)=L 2 ) , f Z (cid:17) k(cid:17) g k (cid:21) 0 are uniformly upper bounded in the L 2 sense.

Original: Lemma C.2.
Revised: Lemma C.2.

Original: If 0 < (cid:17) <
Revised: If 0 < (cid:17) <

Original: (cid:11)=L 2 , there exists a constant (cid:14) 1 ( d; (cid:28) 2 ; L; (cid:11); (cid:12) ) , which depends on the dimension d , the temperature parameter (cid:28) 2 , and the smoothness constant L and dissipative constants ( (cid:11); (cid:12) ) of U ( (cid:1) ) , such that
Revised: (cid:11)=L 2 , there exists a constant (cid:14) 1 ( d; (cid:28) 2 ; L; (cid:11); (cid:12) ) , which depends on the dimension d , the temperature parameter (cid:28) 2 , and the smoothness constant L and dissipative constants ( (cid:11); (cid:12) ) of U ( (cid:1) ) , such that

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: The proof idea is to show that the sequence f E
Revised: The proof idea is to show that the sequence f E

Original: [ ∥ Z (cid:17)k(cid:17) ∥ 2 ] g k (cid:21) 0 satisﬁes a contractive inequality based on the discretization scheme deﬁned in (3.15).
Revised: [ ∥ Z (cid:17)k(cid:17) ∥ 2 ] g k (cid:21) 0 satisﬁes a contractive inequality based on the discretization scheme deﬁned in (3.15).

Original: See x C.7 for a detailed proof.
Revised: See x C.7 for a detailed proof.

Original: By applying Lemma C.2 to (C.29), we have that there exists a constant (cid:14) 2 ( d; (cid:28) 2 ; L; (cid:11); (cid:12) ) such that
Revised: By applying Lemma C.2 to (C.29), we have that there exists a constant (cid:14) 2 ( d; (cid:28) 2 ; L; (cid:11); (cid:12) ) such that

Original: [Equation]
Revised: [Equation]

Original: Then based on (C.22), we obtain the following inequality
Revised: Then based on (C.22), we obtain the following inequality

Revised: [Equation]

Revised: C

Original: [Equation]
Revised: [Equation]

Original: which establishes an upper bound for the ﬁrst term on the right-hand side of (C.21).
Revised: which establishes an upper bound for the ﬁrst term on the right-hand side of (C.21).

Original: It remains to upper bound the second term on the right-hand side of (C.21).
Revised: It remains to upper bound the second term on the right-hand side of (C.21).

Original: According to Itˆo isometry, we have
Revised: According to Itˆo isometry, we have

Original: [Equation]
Revised: [Equation]

Original:  where (cid:6) s ( j ) and (cid:6) (cid:17)k(cid:17) ( j ) are the j -th diagonal entries of (cid:6) s and (cid:6) (cid:17)k(cid:17) , respectively.
Revised: ) where (cid:6) s ( j ) and (cid:6) (cid:17)k(cid:17) ( j ) are the j -th diagonal entries of (cid:6) s and (cid:6) (cid:17)k(cid:17) , respectively.

Original: Recall that (cid:6) s and (cid:6) (cid:17)s are diagonal matrices with all diagonal entires being p 2 (cid:28) 1 or p 2 (cid:28) 2 .
Revised: Recall that (cid:6) s and (cid:6) (cid:17)s are diagonal matrices with all diagonal entires being p 2 (cid:28) 1 or p 2 (cid:28) 2 .

Original: Then for all j , k , and possible realizations of (cid:6) s ( j ) and (cid:6) (cid:17)k(cid:17)
Revised: Then for all j , k , and possible realizations of (cid:6) s ( j ) and (cid:6) (cid:17)k(cid:17)

Original: ( j ) , we have
Revised: ( j ) , we have

Original: [Equation]
Revised: [Equation]

Original: +1) (cid:17)
Revised: +1) (cid:17)

Original: [Equation]
Revised: [Equation]

Original: To upper bound the probability on the right-hand side of (C.32), we take the expectation conditioning
Revised: To upper bound the probability on the right-hand side of (C.32), we take the expectation conditioning

Original: Z (cid:17)k , which yields
Revised: Z (cid:17)k , which yields

Original: [Equation]
Revised: [Equation]

Original: Recall that the rate of swapping is speciﬁed in (2.6) and (2.7).
Revised: Recall that the rate of swapping is speciﬁed in (2.6) and (2.7).

Original: The conditional probability on the right-hand side of (C.33) satisﬁes
Revised: The conditional probability on the right-hand side of (C.33) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: (C
Revised: C

Original: [Equation]
Revised: [Equation]

Original: 34) where Z (cid:17) (1) k(cid:17) ; Z (cid:17) (2) k(cid:17) are the ﬁrst and second components of Z (cid:17)k(cid:17) , s ( (cid:1) ; (cid:1) ) is deﬁned in (2.6), and o ( (cid:1) ) is the little-
Revised: 34) where Z (cid:17) (1) k(cid:17) ; Z (cid:17) (2) k(cid:17) are the ﬁrst and second components of Z (cid:17)k(cid:17) , s ( (cid:1) ; (cid:1) ) is deﬁned in (2.6), and o ( (cid:1) ) is the little-

Original: o notation, which denotes the higher-order term with respect to s (cid:0) k(cid:17) .
Revised: o notation, which denotes the higher-order term with respect to s (cid:0) k(cid:17) .

Original: Hence, by combining (C.32)-(C.34), we have that there exists a constant (cid:14) 3 ( (cid:28) 1 ; (cid:28) 2 ; a ) such that ∫ ( k +1) (cid:17) k(cid:17)
Revised: Hence, by combining (C.32)-(C.34), we have that there exists a constant (cid:14) 3 ( (cid:28) 1 ; (cid:28) 2 ; a ) such that ∫ ( k +1) (cid:17) k(cid:17)

Original: [Equation]
Revised: [Equation]

Original: Then based on (C.31), there exists a constant (cid:14) 4 ( d; (cid:28) 1 ; (cid:28) 2 ; a ) such that
Revised: Then based on (C.31), there exists a constant (cid:14) 4 ( d; (cid:28) 1 ; (cid:28) 2 ; a ) such that

Original: [Equation]
Revised: [Equation]

Original: Finally, by plugging (C.30) and (C.35) into (C.21), we have
Revised: Finally, by plugging (C.30) and (C.35) into (C.21), we have

Original: [Equation]
Revised: [Equation]

Original: Hence, by applying the Gr¨onwall’s inequality (Dragomir, 2003), we have that there exists a constant (cid:13) ( d; (cid:28) 1 ; (cid:28) 2 ; a; L; (cid:11); (cid:12); T ) such that for all 
Revised: Hence, by applying the Gr¨onwall’s inequality (Dragomir, 2003), we have that there exists a constant (cid:13) ( d; (cid:28) 1 ; (cid:28) 2 ; a; L; (cid:11); (cid:12); T ) such that for all t 2 [ [Equation]

Original: [Equation]
Revised: [Equation]

Original: In other words, the mean squared error of discretization grows linearly with respect to the stepsize (cid:17) , which concludes the proof of Theorem 3.6.
Revised: In other words, the mean squared error of discretization grows linearly with respect to the stepsize (cid:17) , which concludes the proof of Theorem 3.6.

Original: C. P ROOF OF L EMMA C. 
Revised: C. P ROOF OF L EMMA C.

Original: Proof.
Revised: Proof.

Original: Recall that for all integers k (cid:21) 0 , based on the discretization scheme in (3.15) and (3.16), for i 2 f 1 ; 2 g , we have
Revised: Recall that for all integers k (cid:21) 0 , based on the discretization scheme in (3.15) and (3.16), for i 2 f 1 ; 2 g , we have

Original: [Equation]
Revised: [Equation]

Original: where (cid:24) ( i ) ( k ) is a standard d -dimensional Gaussian random vector and the temperature (cid:28) ( i ) ( k ) takes value in f (cid:28) 1 ; (cid:28) 2 g .
Revised: where (cid:24) ( i ) ( k ) is a standard d -dimensional Gaussian random vector and the temperature (cid:28) ( i ) ( k ) takes value in f (cid:28) 1 ; (cid:28) 2 g .

Original: Also note that by the deﬁnition of the continuous-time interpolated process f Z (cid:17)t g t (cid:21) 0 in (3.17), we have Z ( k )
Revised: Also note that by the deﬁnition of the continuous-time interpolated process f Z (cid:17)t g t (cid:21) 0 in (3.17), we have Z ( k )

Original: =
Revised: =

Original: Z (cid:17)k(cid:17) .
Revised: Z (cid:17)k(cid:17) .

Original: Hence, we have
Revised: Hence, we have

Original: E [ 
Revised: E [ ∥ Z (cid:17) ( k +1) (cid:17) ∥]

Original: [Equation]
Revised: [Equation]

Original: Z
Revised: E

Original: [Equation]
Revised: [Equation]

Original: ∥ [Equation] (cid:28) (1) ( k ) (cid:1) ∥
Revised: (1) ( k ) (cid:1) ∥

Original: (cid:24) (1) ( k ) ∥ 2 ]
Revised: (cid:24) (1) ( k ) ∥ 2 ]

Original: [Equation] (cid:17) (cid:1) (cid:28) (2) ( k ) (cid:1) ∥
Revised: + E [ 2 (cid:17) (cid:1) (cid:28) (2) ( k ) (cid:1) ∥

Original: (cid:24) (2) ( k ) ∥ 2 ]
Revised: (cid:24) (2) ( k ) ∥ 2 ]

Original: +
Revised: +

Revised: [Equation]

Revised: (cid:1) E [

Original: [Equation]
Revised: [Equation]

Original:  √
Revised: ; √

Original: [Equation]
Revised: [Equation]

Original: (cid:6)
Revised: (cid:17)

Original: [Equation]
Revised: [Equation]

Original:  (cid:1) (cid:24) ( k [Equation] ⟩ ] : (C.36)
Revised: (cid:6) (cid:17)k(cid:17) (cid:1) (cid:24) ( k ) ⟩ ] : (C.36)

Original: Since (cid:24) ( k ) is independent of Z (cid:17)k(cid:17) and by (3.16) the distribution of (cid:28) ( k ) is only determined by Z (cid:17)k(cid:17) , the last term on the right-hand side of (C.36) is zero.
Revised: Since (cid:24) ( k ) is independent of Z (cid:17)k(cid:17) and by (3.16) the distribution of (cid:28) ( k ) is only determined by Z (cid:17)k(cid:17) , the last term on the right-hand side of (C.36) is zero.

Original: Moreover, note that each component of (cid:28) ( k ) only takes value in f (cid:28) 1 ; (cid:28) 2 g and (cid:28) 1 < (cid:28) 2 .
Revised: Moreover, note that each component of (cid:28) ( k ) only takes value in f (cid:28) 1 ; (cid:28) 2 g and (cid:28) 1 < (cid:28) 2 .

Original: Then by (C.36) we have
Revised: Then by (C.36) we have

Original: E [ ∥ Z (cid:17) ( k +1) (cid:17) ∥] (cid:20) E
Revised: E [ ∥ Z (cid:17) ( k +1) (cid:17) ∥] (cid:20) E

Original: [(cid:13)(cid:13) Z (cid:17)k(cid:17) (cid:0) (cid:17) (cid:1) ∇ U ( Z (cid:17)k(cid:17) )
Revised: [(cid:13)(cid:13) Z (cid:17)k(cid:17) (cid:0) (cid:17) (cid:1) ∇ U ( Z (cid:17)k(cid:17) )

Original: (cid:13)(cid:13)]
Revised: (cid:13)(cid:13)]

Original: + 4 d(cid:17)(cid:28) 2 : (C.37)
Revised: + 4 d(cid:17)(cid:28) 2 : (C.37)

Original: Recall that U ( (cid:1) ) is ( (cid:11); (cid:12) ) -dissipative
Revised: Recall that U ( (cid:1) ) is ( (cid:11); (cid:12) ) -dissipative

Original: and L -smooth by Assumption 3.1.
Revised: and L -smooth by Assumption 3.1.

Original: Then we have
Revised: Then we have

Original: E [(cid:13)(cid:13)
Revised: E [ [Equation]

Original: Z (cid:17)k(cid:17)
Revised: Z (cid:17)k(cid:17)

Original: [Equation]
Revised: [Equation]

Original: where x (cid:3) is a stationary point of U ( (cid:1) ) .
Revised: where x (cid:3) is a stationary point of U ( (cid:1) ) .

Original: Hence, based on (C.37)
Revised: Hence, based on (C.37)

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: Note that when (cid:17) 2 (0 ; (cid:11)=L 2 ) , we have 1 (cid:0) 2 (cid:11)(cid:17) + 2 (cid:17) 2 L 2 < 1 .
Revised: Note that when (cid:17) 2 (0 ; (cid:11)=L 2 ) , we have 1 (cid:0) 2 (cid:11)(cid:17) + 2 (cid:17) 2 L 2 < 1 .

Original: Hence, according to (C.38), there exists a constant (cid:14) 

Revised: Hence, according to (C.38), there exists a constant (cid:14)

Original: ( d; (cid:28) 2 ; L; (cid:11); (cid:12) ) such that
Revised: ( d; (cid:28) 2 ; L; (cid:11); (cid:12) ) such that

Original: [Equation]
Revised: [Equation]

Original: which concludes the proof of Lemma C.2.
Revised: which concludes the proof of Lemma C.2.