Original: I NTERACTING C ONTOUR S TOCHASTIC G RADIENT L ANGEVIN D YNAMICS
Revised: I NTERACTING C ONTOUR S TOCHASTIC G RADIENT L ANGEVIN D YNAMICS

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, an embarrassingly parallel multiple-chain contour stochastic gradient Langevin dynamics (CSGLD) sampler with efﬁcient interactions .
Revised: We propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, an embarrassingly parallel multiple-chain contour stochastic gradient Langevin dynamics (CSGLD) sampler with efﬁcient interactions .

Original: We show that ICSGLD can be theoretically more efﬁcient than a single-chain CSGLD with an equivalent computational budget.
Revised: We show that ICSGLD can be theoretically more efﬁcient than a single-chain CSGLD with an equivalent computational budget.

Original: We also present a novel random-ﬁeld function, which facilitates the estimation of self-adapting parameters in big data and obtains free mode explorations.
Revised: We also present a novel random-ﬁeld function, which facilitates the estimation of self-adapting parameters in big data and obtains free mode explorations.

Original: Empirically, we compare the proposed algorithm with popular benchmark methods for posterior sampling.
Revised: Empirically, we compare the proposed algorithm with popular benchmark methods for posterior sampling.

Original: The numerical results show a great potential of ICSGLD for large-scale uncertainty estimation tasks
Revised: The numerical results show a great potential of ICSGLD for large-scale uncertainty estimation tasks.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Stochastic gradient Langevin dynamics (SGLD)
Revised: Stochastic gradient Langevin dynamics (SGLD)

Original: (Welling & Teh, 2011) has achieved great successes in simulations of high-dimensional systems for big data problems.
Revised: (Welling & Teh, 2011) has achieved great successes in simulations of high-dimensional systems for big data problems.

Original: It, however, yields only a fast mixing rate when the energy landscape is simple, e.g., local energy wells are shallow and not well separated.
Revised: It, however, yields only a fast mixing rate when the energy landscape is simple, e.g., local energy wells are shallow and not well separated.

Original: To improve its convergence for the problems with complex energy landscapes, various strategies have been proposed, such as momentum augmentation (Chen et al., 2014; Ding et al., 2014),
Revised: To improve its convergence for the problems with complex energy landscapes, various strategies have been proposed, such as momentum augmentation (Chen et al., 2014; Ding et al., 2014),

Original: Hessian approximation (Ahn et al., 2012; Li et al., 2016), high-order numerical schemes (Chen et al., 2015; Li et al., 2019b), and cyclical learning rates (Izmailov et al., 2018; Maddox et al., 2019; Zhang et al., 2020b).
Revised: Hessian approximation (Ahn et al., 2012; Li et al., 2016), high-order numerical schemes (Chen et al., 2015; Li et al., 2019b) and cyclical learning rates (Izmailov et al., 2018; Maddox et al., 2019; Zhang et al., 2020b).

Original: In spite of their asymptotic properties in Bayesian inference (Vollmer et al., 2016) and non-convex optimization (Zhang et al., 2017), it is still difﬁcult to achieve compelling empirical results for pathologically complex deep neural networks (DNNs).
Revised: In spite of their asymptotic properties in Bayesian inference (Vollmer et al., 2016) and non-convex optimization (Zhang et al., 2017), it is still difﬁcult to achieve compelling empirical results for pathologically complex deep neural networks (DNNs).

Original: To simulate from distributions with complex energy landscapes, e.g., those with a multitude of modes well separated by high energy barriers, an emerging trend is to run multiple chains, where interactions between different chains can potentially accelerate the convergence of the simulation.
Revised: To simulate from distributions with complex energy landscapes, e.g., those with a multitude of modes well separated by high energy barriers, an emerging trend is to run multiple chains, where interactions between different chains can potentially accelerate the convergence of the simulation.

Original: For example, Song et al.
Revised: For example, Song et al.

Original: (2014) and Futami et al.
Revised: (2014) and Futami et al.

Original: (2020) showed theoretical advantages of appropriate interactions in ensemble/population simulations.
Revised: (2020) showed theoretical advantages of appropriate interactions in ensemble/population simulations.

Original: Other multiple chain methods include particle-based nonlinear Markov (Vlasov) processes (Liu & Wang, 2016; Zhang et al., 2020a) and replica exchange methods (also known as parallel tempering)
Revised: Other multiple chain methods include particle-based nonlinear Markov (Vlasov) processes (Liu & Wang, 2016; Zhang et al., 2020a) and replica exchange methods (also known as parallel tempering)

Original: (Deng et al., 2021).
Revised: (Deng et al., 2021a).

Original: However, the particle-based methods result in an expensive kernel matrix computation given a large number of particles (Liu & Wang, 2016); similarly, na¨ıvely extending replica exchange methods to population chains leads to a long waiting time to swap between non-neighboring chains (Syed et al., 2019).
Revised: However, the particle-based methods result in an expensive kernel matrix computation given a large number of particles (Liu & Wang, 2016); similarly, na¨ıvely extending replica exchange methods to population chains leads to a long waiting time to swap between non-neighboring chains (Syed et al., 2021).

Original: Therefore, how to conduct interactions between different chains, while maintaining the scalability of the algorithm, is the key to the success of the parallel stochastic gradient MCMC algorithms.
Revised: Therefore, how to conduct interactions between different chains, while maintaining the scalability of the algorithm, is the key to the success of the parallel stochastic gradient MCMC algorithms.

Original: In this paper, we propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, a pleasingly parallel extension of contour stochastic gradient Langevin dynamics (CSGLD)
Revised: In this paper, we propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, a pleasingly parallel extension of contour stochastic gradient Langevin dynamics (CSGLD)

Original: (Deng et al., 2020b) with efﬁcient interactions .
Revised: (Deng et al., 2020b) with efﬁcient interactions .

Original: The proposed algorithm requires minimal communication cost in that each chain shares with others the marginal energy likelihood estimate only.
Revised: The proposed algorithm requires minimal communication cost in that each chain shares with others the marginal energy likelihood estimate only.

Original: As a result, the interacting mechanism improves the convergence of the simulation, while the minimal communication mode between different chains enables the proposed algorithm to be naturally adapted to distributed computing with little overhead.
Revised: As a result, the interacting mechanism improves the convergence of the simulation, while the minimal communication mode between different chains enables the proposed algorithm to be naturally adapted to distributed computing with little overhead.

Original: For the single-chain CSGLD algorithm, despite its theoretical advantages as shown in Deng et al.
Revised: For the single-chain CSGLD algorithm, despite its theoretical advantages as shown in Deng et al.

Original: (2020b), estimation of the marginal energy likelihood remainsa challengingtask forthe big data problems with a wide energy range, jeopardizing the empirical performance of the class of importance sampling methods (Gordon et al., 1993;
Revised: (2020b), estimation of the marginal energy likelihood remains challenging for big data problems with a wide energy range, jeopardizing the empirical performance of the class of importance sampling methods (Gordon et al., 1993;

Original: Doucet et al., 2001;Wang & Landau, 2001;
Revised: Doucet et al., 2001;

Original:  Liang et al., 2007; Andrieu et al., 2010; Deng et al., 2020b) in big data applications.
Revised: Wang & Landau, 2001; Liang et al., 2007; Andrieu et al., 2010; Deng et al., 2020b) in big data applications.

Original: To resolve this issue, we resort to a novel  random-ﬁeld functionthat is constructed based on multiple chains  and proven easier to estimate.
Revised: To resolve this issue, we resort to a novel interacting random-ﬁeld function based on multiple chains for an ideal variance reduction and a more robust estimation.

Original: As such, we can greatly facilitate the estimation of the marginal energy likelihood so as to accelerate the simulations of notoriously complex distributions.
Revised: As such, we can greatly facilitate the estimation of the marginal energy likelihood so as to accelerate the simulations of notoriously complex distributions.

Original: To summarize, the algorithm has three main contributions:
Revised: To summarize, the algorithm has three main contributions:

Original:  We propose a scalable interacting importance sampling method for big data problems with the minimal communication cost.
Revised: • We propose a scalable interacting importance sampling method for big data problems with the minimal communication cost.

Original: A novel random-ﬁeld function is derived to tackle the incompatibility issue of the class of importance sampling methods in big data problems.
Revised: A novel random-ﬁeld function is derived to tackle the incompatibility issue of the class of importance sampling methods in big data problems.

Original:  Theoretically, we study the local stability of the non-linear mean-ﬁeld system and justify regularity properties of the solution of Poisson’s equation.
Revised: • Theoretically, we study the local stability of a non-linear mean-ﬁeld system and justify regularity properties of the solution of Poisson’s equation.

Original: In addition, we prove the asymptotic normality for the stochastic approximation process in mini-batch settings and show that ICSGLD is asymptotically more efﬁcient than the single-chain CSGLD with an equivalent computational budget.
Revised: We also prove the asymptotic normality for the stochastic approximation process in mini-batch settings and show that ICSGLD is asymptotically more efﬁcient than the single-chain CSGLD with an equivalent computational budget.

Original:  Our proposed algorithm achieves appealing mode explorations using a ﬁxed learning rate on the MNIST dataset and obtains remarkable performance in large-scale uncertainty estimation tasks.
Revised: • Our proposed algorithm achieves appealing mode explorations using a ﬁxed learning rate on the MNIST dataset and obtains remarkable performance in large-scale uncertainty estimation tasks.

Original:  P RELIMINARIES 2. S TOCHASTIC GRADIENT L ANGEVIN DYNAMICS
Revised:  P RELIMINARIES 2. S TOCHASTIC GRADIENT L ANGEVIN DYNAMICS

Original: A standard sampling algorithm for big data problems is SGLD (Welling & Teh, 2011), which is a numerical scheme of a stochastic differential equation in mini-batch settings:
Revised: A standard sampling algorithm for big data problems is SGLD (Welling & Teh, 2011), which is a numerical scheme of a stochastic differential equation in mini-batch settings:

Original: [Equation]
Revised: [Equation]

Original: (1) where x k ∈ X ∈ R d , (cid:15) k is the learning rate at iteration k , N denotes the number of total data points, τ is the temperature, and w k is a standard Gaussian vector of dimension d .
Revised: (1) where x k ∈ X ∈ R d , (cid:15) k is the learning rate at iteration k , N denotes the number of total data points, τ is the temperature, and w k is a standard Gaussian vector of dimension d .

Original: In particular, Nn ∇ x (cid:101) U ( x ) is an unbiased stochastic gradient estimator based on a mini-batch data B of size n and Nn (cid:101)
Revised: In particular, Nn ∇ x (cid:101) U ( x ) is an unbiased stochastic gradient estimator based on a mini-batch data B of size n and Nn (cid:101)

Original: U ( x ) is the unbiased energy estimator for the exact energy function U ( x ) .
Revised: U ( x ) is the unbiased energy estimator for the exact energy function U ( x ) .

Original: Under mild conditions on U , x k +1 is known to converge weakly to a unique invariant distribution π ( x ) ∝ e − U ( x ) τ as (cid:15) k
Revised: Under mild conditions on U , x k +1 is known to converge weakly to a unique invariant distribution π ( x ) ∝ e − U ( x ) τ as (cid:15) k

Original: → 0 .
Revised: → 0 .

Original: 2. C ONTOUR STOCHASTIC GRADIENT L ANGEVIN DYNAMICS
Revised: 2. C ONTOUR STOCHASTIC GRADIENT L ANGEVIN DYNAMICS

Original: Despite its theoretical guarantees, SGLD can converge exponentially slow when U ( x ) is non-convex and exhibits high energy barriers.
Revised: Despite its theoretical guarantees, SGLD can converge exponentially slow when U ( x ) is non-convex and exhibits high energy barriers.

Original: To remedy this issue, CSGLD (Deng et al., 2020b) exploits the ﬂat histogram idea and proposes to simulate from a ﬂattened density with much lower energy barriers
Revised: To remedy this issue, CSGLD (Deng et al., 2020b) exploits the ﬂat histogram idea and proposes to simulate from a ﬂattened density with much lower energy barriers

Original: [Equation]
Revised: [Equation]

Original: where ζ is a hyperparameter, Ψ 
Revised: where ζ is a hyperparameter, Ψ θ ( u ) =

Revised: (cid:80) mi

Original: [Equation]
Revised: [Equation]

Original: ( u ) =

Original: [Equation]
Revised: [Equation]

Original: In particular, { u i } mi =0 determines the partition {X i } mi =1 of X such that X i
Revised: In particular, { u i } mi =0 determines the partition {X i } mi =1 of X such that X i

Original: = { x : u i − 1 <
Revised: = { x : u i − 1 <

Original: U ( x ) ≤ u i } , where
Revised: U ( x ) ≤ u i } , where

Original: −∞ = u 0 < u 1 < · · · < u m − 1 < u m = ∞ .
Revised: −∞ = u 0 < u 1 < · · · < u m − 1 < u m = ∞ .

Original: For practical purposes, we assume u i +

Revised: For practical purposes, we assume u i +

Original: − u i =
Revised: − u i =

Original: ∆ u for i = 1 , · · · , m − 2 .
Revised: ∆ u for i = 1 , · · · , m − 2 .

Original: In addition, θ = ( θ (1) , θ (2) , . . .
Revised: In addition, θ = ( θ (1) , θ (2) , . . .

Original: , θ ( m )) is the self-adapting parameter in the space Θ = (cid:26) ( θ (1) , · · · , θ [Equation] m [Equation]
Revised: , θ ( m )) is the self-adapting parameter in the space Θ = (cid:26) ( θ (1) , · · · , θ ( m ))

Original: (cid:12)(cid:12) {+
+}
Revised: (cid:12)(cid:12) {+
+}

Original: [Equation]

Original: [Equation]

Original: [Equation] ) < [Equation] & (cid:80) 
Revised: < θ (1) , · · · , θ ( m ) < 1& (cid:80) mi

Original: [Equation] =1 θ ( i ) = [Equation] (cid:27) .
Revised: =1 θ ( i ) =(cid:27) .

Original: Ideally, setting ζ = 1 and θ ( i ) = θ ∞ ( i ) , where θ ∞ ( i ) =
Revised: Ideally, setting ζ = 1 and θ ( i ) = θ ∞ ( i ) , where θ ∞ ( i ) =

Original: (cid:82) X i π ( x ) d x for i ∈ { 1 , 2 , · · · , m } , enables
Revised: (cid:82) X i π ( x ) d x for i ∈ { 1 , 2 , · · · , m } , enables

Original: CSGLD to achieve a “random walk” in the space of energy and to penalize the over-visited partition (Wang & Landau, 2001;
Revised: CSGLD to achieve a “random walk” in the space of energy and to penalize the over-visited partition (Wang & Landau, 2001;

Original: Liang et al., 2007; Fort et al., 2011; 2015).
Revised: Liang et al., 2007; Fort et al., 2011; 2015).

Original: However, the optimal values of θ ∞ is unknown a priori .
Revised: However, the optimal values of θ ∞ is unknown a priori .

Original: To tackle this issue, CSGLD proposes the following procedure to adaptively estimate θ via stochastic approximation (SA) (Robbins & Monro, 1951; Benveniste et al., 1990):
Revised: To tackle this issue, CSGLD proposes the following procedure to adaptively estimate θ via stochastic approximation (SA) (Robbins & Monro, 1951; Benveniste et al., 1990):

Original: (1) Sample x k +

Revised: (1) Sample x k +

Original: = x k + (cid:15) k
Revised: = x k + (cid:15) k

Original: Nn ∇ x (cid:101) U Ψ
Revised: Nn ∇ x (cid:101) U Ψ

Original: θ k ( x k ) + √ 2 τ(cid:15) k w k , (2)
Revised: θ k ( x k ) + √ 2 τ(cid:15) k w k , (2)

Original: Optimize θ k +

Revised: Optimize θ k +

Original: =
Revised: =

Original: θ k + ω k +1 (cid:101)
Revised: θ k + ω k +1 (cid:101)

Original: H
Revised: H

Original: ( θ k , x k +1 ) ,
Revised: ( θ k , x k +1 ) ,

Original: Algorithm  stochastic  Langevin dynamics algorithm (ICSGLD).
Revised: Algorithm 1 Interacting contour stochastic gradient Langevin dynamics algorithm (ICSGLD).

Original: {X i } mi =1 is pre-deﬁned partition and ζ is a hyperparameter.
Revised: {X i } mi =1 is pre-deﬁned partition and ζ is a hyperparameter.

Original: The update rule in distributed-memory settings and discussions of hyperparameters is detailed in section B.1.1 in the supplementary material.
Revised: The update rule in distributed-memory settings and discussions of hyperparameters is detailed in section B.1.1 in the supplementary material.

Original: [1.] (Data subsampling)
Revised: [1.] (Data subsampling)

Original: Draw a mini-batch data B k from D , and compute stochastic gradients ∇ x (cid:101) U ( x ( p ) k ) and energies (cid:101) U
Revised: Draw a mini-batch data B k from D , and compute stochastic gradients ∇ x (cid:101) U ( x ( p ) k ) and energies (cid:101) U

Original: ( x ( p ) k )
Revised: ( x ( p ) k )

Original: for each x ( p ) , where p ∈ { 1 , 2 , · · · , P } , |B k | = n , and |D| = N .
Revised: for each x ( p ) , where p ∈ { 1 , 2 , · · · , P } , |B k | = n , and |D| = N .

Original: (Parallel simulation)
Revised: (Parallel simulation)

Original: Sample x (cid:78) P k
Revised: Sample x (cid:78) P k

Original: +1 :
Revised: +1 :

Original: = ( x (1) k
Revised: = ( x (1) k

Original: +1 , x (2) k +1 , · · · , x ( P ) k +1 )
Revised: +1 , x (2) k +1 , · · · , x ( P ) k +1 )

Original: (cid:62) based on SGLD and θ k
Revised: (cid:62) based on SGLD and θ k

Original: [Equation]
Revised: [Equation]

Original: where (cid:15) k is the learning rate, τ is the temperature, w (cid:78) P k denotes P independent standard
Revised: where (cid:15) k is the learning rate, τ is the temperature, w (cid:78) P k denotes P independent standard

Original: Gaussian vectors, ∇ x (cid:101) U
Revised: Gaussian vectors, ∇ x (cid:101) U

Original: Ψ θ
Revised: Ψ θ

Original: ( x (cid:78) P )
Revised: ( x (cid:78) P )

Original: = ( ∇ x (cid:101)
Revised: = ( ∇ x (cid:101)

Original: U Ψ θ
Revised: U Ψ θ

Original: ( x (1) ) , ∇ x (cid:101)
Revised: ( x (1) ) , ∇ x (cid:101)

Original: U Ψ θ
Revised: U Ψ θ

Original: ( x (2) ) , · · · , ∇ x (cid:101) U Ψ θ
Revised: ( x (2) ) , · · · , ∇ x (cid:101) U Ψ θ

Original: ( x ( P ) ))
Revised: ( x ( P ) ))

Original: (cid:62) , and ∇ x (cid:101) U
Revised: (cid:62) , and ∇ x (cid:101) U

Original: Ψ θ ( x ) =
Revised: Ψ θ ( x ) =

Original: (cid:104) 1 + ζτ ∆ u (cid:0) log θ ( J (cid:101) U ( x ))
Revised: (cid:104) 1 + ζτ ∆ u (cid:0) log θ ( J (cid:101) U ( x ))

Original: − log θ (( J (cid:101) U ( x ) − 1) ∨ 1) (cid:1)(cid:105) ∇ x (cid:101) U ( x ) for any x ∈ X .
Revised: − log θ (( J (cid:101) U ( x ) − 1) ∨ 1) (cid:1)(cid:105) ∇ x (cid:101) U ( x ) for any x ∈ X .

Original: (Stochastic approximation) Update the self-adapting parameter θ ( i ) for i ∈ { 1 , 2 , · · · , m }
Revised: (Stochastic approximation) Update the self-adapting parameter θ ( i ) for i ∈ { 1 , 2 , · · · , m }

Original: [Equation]
Revised: [Equation]

Original: ) where 1 A is an indicator function that takes value 1 if the event A appears and equals 0 otherwise.
Revised: ) where 1 A is an indicator function that takes value 1 if the event A appears and equals 0 otherwise.

Original: where ∇ x (cid:101) U Ψ θ
Revised: where ∇ x (cid:101) U Ψ θ

Original: ( · ) is a stochastic gradient function of (cid:36)
Revised: ( · ) is a stochastic gradient function of (cid:36)

Original: Ψ θ ( · ) to be detailed in Algorithm 1.
Revised: Ψ θ ( · ) to be detailed in Algorithm 1.

Original: (cid:101) H ( θ , x ) :
Revised: (cid:101) H ( θ , x ) :

Original: = (cid:16)(cid:101) H 1 ( θ , x ) , · · · , (cid:101)
Revised: = (cid:16)(cid:101) H 1 ( θ , x ) , · · · , (cid:101)

Original: H m ( θ , x ) (cid:17) is random-ﬁeld function where each entry follows
Revised: H m ( θ , x ) (cid:17) is random-ﬁeld function where each entry follows

Original: [Equation]
Revised: [Equation]

Original: θ ( i ) (cid:17) , where J (cid:101) U ( x ) =
Revised: i ) (cid:17) , where J (cid:101) U ( x ) =

Original: [Equation]
Revised: [Equation]

Original: Theoretically, CSGLD converges to a sampling-optimization equilibrium in the sense that θ k approaches to a ﬁxed point θ ∞ and the samples are drawn from the ﬂattened density (cid:36) Ψ θ ∞ ( x ) .
Revised: Theoretically, CSGLD converges to a sampling-optimization equilibrium in the sense that θ k approaches to a ﬁxed point θ ∞ and the samples are drawn from the ﬂattened density (cid:36) Ψ θ ∞ ( x ) .

Original: Notably, the mean-ﬁeld system is globally stable with a unique stable equilibrium point in a small neighborhood of θ ∞ .
Revised: Notably, the mean-ﬁeld system is globally stable with a unique stable equilibrium point in a small neighborhood of θ ∞ .

Original: Moreover, such an appealing property holds even when U ( x ) is non-convex.
Revised: Moreover, such an appealing property holds even when U ( x ) is non-convex.

Original:  I NTERACTING CONTOUR STOCHASTIC GRADIENT L ANGEVIN DYNAMICS
Revised:  I NTERACTING CONTOUR STOCHASTIC GRADIENT L ANGEVIN DYNAMICS

Original: The major goal of interacting CSGLD (ICSGLD) is to improve the efﬁciency of CSGLD.
Revised: The major goal of interacting CSGLD (ICSGLD) is to improve the efﬁciency of CSGLD.

Original: In particular, the self-adapting parameter θ is crucial for ensuring the sampler to escape from the local traps and traverse the whole energy landscape, and how to reduce the variability of θ k ’s is the key to the success of such a dynamic importance sampling algorithm.
Revised: In particular, the self-adapting parameter θ is crucial for ensuring the sampler to escape from the local traps and traverse the whole energy landscape, and how to reduce the variability of θ k ’s is the key to the success of such a dynamic importance sampling algorithm.

Original: To this end, we propose an efﬁcient variance reduction scheme via interacting parallel systems to improve the accuracy of θ k ’s.
Revised: To this end, we propose an efﬁcient variance reduction scheme via interacting parallel systems to improve the accuracy of θ k ’s.

Original: 3. I NTERACTIONS IN PARALLELISM
Revised: 3. I NTERACTIONS IN PARALLELISM

Original: Now we ﬁrst consider a na¨ıve parallel sampling scheme with P chains as follows
Revised: Now we ﬁrst consider a na¨ıve parallel sampling scheme with P chains as follows

Original: [Equation]
Revised: [Equation]

Original: where x (cid:78) P =
Revised: where x (cid:78) P =

Original: ( x (1) , x (2) , · · · , x ( P ) ) (cid:62) , w (cid:78) P k denotes P independent standard Gaussian vectors, and (cid:101)
Revised: ( x (1) , x (2) , · · · , x ( P ) ) (cid:62) , w (cid:78) P k denotes P independent standard Gaussian vectors, and (cid:101)

Original: U Ψ θ
Revised: U Ψ θ

Original: ( x (cid:78) P )
Revised: ( x (cid:78) P )

Original: =
Revised: =

Original: ( (cid:101)
Revised: ( (cid:101)

Original: U Ψ θ
Revised: U Ψ θ

Original: ( x (1) ) , (cid:101) U
Revised: ( x (1) ) , (cid:101) U

Original: Ψ θ
Revised: Ψ θ

Original: ( x (2) ) , · · · , (cid:101)
Revised: ( x (2) ) , · · · , (cid:101)

Original: U Ψ θ
Revised: U Ψ θ

Original: ( x ( P ) ))
Revised: ( x ( P ) ))

Original: (cid:62) .
Revised: (cid:62) .

Original: Stochastic approximation aims to ﬁnd the solution θ of the mean-ﬁeld system h ( θ ) such that
Revised: Stochastic approximation aims to ﬁnd the solution θ of the mean-ﬁeld system h ( θ ) such that

Original: [Equation]
Revised: [Equation]

Original: where (cid:36) θ is the invariant measure simulated via SGLD that approximates (cid:36) Ψ θ in (2) and (cid:101)
Revised: where (cid:36) θ is the invariant measure simulated via SGLD that approximates (cid:36) Ψ θ in (2) and (cid:101)

Original: H ( θ , x ) is the novel random-ﬁeld function to be deﬁned later in (8).
Revised: H ( θ , x ) is the novel random-ﬁeld function to be deﬁned later in (8).

Original: Since h ( θ ) is observable only up to large random perturbations (in the form of (cid:101) H ( θ , x ) ), the optimization of θ based on isolated random-ﬁeld functions may not be efﬁcient enough.
Revised: Since h ( θ ) is observable only up to large random perturbations (in the form of (cid:101) H ( θ , x ) ), the optimization of θ based on isolated random-ﬁeld functions may not be efﬁcient enough.

Original: However, due to the conditional independence of x (1) , x (2) , · · · , x ( P ) in parallel sampling, it is very natural to consider a Monte Carlo average
Revised: However, due to the conditional independence of x (1) , x (2) , · · · , x ( P ) in parallel sampling, it is very natural to consider a Monte Carlo average

Original: [Equation]
Revised: [Equation]

Original: Namely, we are considering the following stochastic approximation scheme
Revised: Namely, we are considering the following stochastic approximation scheme

Original: [Equation]
Revised: [Equation]

Original: where (cid:102) H ( θ k , x (cid:78) P k +1 ) is an interacting random-ﬁeld function (cid:102)
Revised: where (cid:102) H ( θ k , x (cid:78) P k +1 ) is an interacting random-ﬁeld function (cid:102)

Original: [Equation]
Revised: [Equation]

Original: Note that the Monte Carlo average is very effective to reduce the variance of the interacting randomﬁeld function (cid:102) H ( θ , x (cid:78) P ) based on the conditionally independent random ﬁeld functions.
Revised: Note that the Monte Carlo average is very effective to reduce the variance of the interacting randomﬁeld function (cid:102) H ( θ , x (cid:78) P ) based on the conditionally independent random ﬁeld functions.

Original: Moreover, each chain shares with others only a very short message during each iteration.
Revised: Moreover, each chain shares with others only a very short message during each iteration.

Original: Therefore, the interacting parallel system is well suited for distributed computing, where the implementations and communication costs are further detailed in section B.1.2 in the supplementary material.
Revised: Therefore, the interacting parallel system is well suited for distributed computing, where the implementations and communication costs are further detailed in section B.1.2 in the supplementary material.

Original: By contrast, each chain of the non-interacting parallel CSGLD algorithm deals with the parameter θ and a large-variance random-ﬁeld function (cid:101) H ( θ , x ) individually, leading to coarse estimates in the end.
Revised: By contrast, each chain of the non-interacting parallel CSGLD algorithm deals with the parameter θ and a large-variance random-ﬁeld function (cid:101) H ( θ , x ) individually, leading to coarse estimates in the end.

Original: Formally, for the population/ensemble interaction scheme (7), we deﬁne a novel random-ﬁeld function (cid:101) H ( θ , x ) =
Revised: Formally, for the population/ensemble interaction scheme (7), we deﬁne a novel random-ﬁeld function (cid:101) H ( θ , x ) =

Original: ( (cid:101) H 1 ( θ , x ) , (cid:101)
Revised: ( (cid:101) H 1 ( θ , x ) , (cid:101)

Original: H 2 ( θ , x ) , · · · , (cid:101)
Revised: H 2 ( θ , x ) , · · · , (cid:101)

Original: H m ( θ , x )) , where each component satisﬁes
Revised: H m ( θ , x )) , where each component satisﬁes

Original: [Equation]
Revised: [Equation]

Original: As shown in Lemma 1, the corresponding mean-ﬁeld function proposes to converge to a different ﬁxed point θ (cid:63) , s.t.
Revised: As shown in Lemma 1, the corresponding mean-ﬁeld function proposes to converge to a different ﬁxed point θ (cid:63) , s.t.

Original: [Equation]
Revised: [Equation]

Original: A large data set often renders the task of estimating θ ∞ numerically challenging.
Revised: A large data set often renders the task of estimating θ ∞ numerically challenging.

Original: By contrast, we resort to a different solution by estimating θ (cid:63) instead based on a large value of ζ .
Revised: By contrast, we resort to a different solution by estimating θ (cid:63) instead based on a large value of ζ .

Original: The proposed algorithm is summarized in Algorithm 1.
Revised: The proposed algorithm is summarized in Algorithm 1.

Original: For more study on the scalablity of the new scheme, we leave the discussion in section B.1.3.
Revised: For more study on the scalablity of the new scheme, we leave the discussion in section B.1.3.

Original: 3. R ELATED WORKS
Revised: 3. R ELATED WORKS

Original: Replica exchange SGLD (Deng et al., 2020a; 2021) has successfully extended the traditional replica exchange (Swendsen & Wang, 1986; Geyer, 1991; Earl & Deem, 2005) to big data problems.
Revised: Replica exchange SGLD (Deng et al., 2020a; 2021a) has successfully extended the traditional replica exchange (Swendsen & Wang, 1986; Geyer, 1991; Earl & Deem, 2005) to big data problems.

Original: However, it works with two chains only and has a low swapping rate.
Revised: However, it works with two chains only and has a low swapping rate.

Original: As shown in Figure 1(a), a na ¨ ıve extension of multi-chain replica exchange SGLD yields low communication efﬁciency.
Revised: As shown in Figure 1(a), a na ¨ ıve extension of multi-chain replica exchange SGLD yields low communication efﬁciency.

Original: Despite some recipe in the literature (Katzgraber et al., 2008; Bittner et al., 2008; Syed et al., 2019), how to conduct multi-chain replica exchange with low-frequency swaps is still an open question.
Revised: Despite some recipe in the literature (Katzgraber et al., 2008; Bittner et al., 2008; Syed et al., 2021), how to conduct multi-chain replica exchange with low-frequency swaps is still an open question.

Original: [Figure]
Revised: [Figure]

Original: Stein variational gradient descent (SVGD) (Liu & Wang, 2016) is a popular approximate inference method to drive a set of particles for posterior approximation.
Revised: Stein variational gradient descent (SVGD) (Liu & Wang, 2016) is a popular approximate inference method to drive a set of particles for posterior approximation.

Original: In particular, repulsive forces are proposed to prevent particles to collapse together into neighboring regions, which resembles our strategy of penalizing over-visited partition.
Revised: In particular, repulsive forces are proposed to prevent particles to collapse together into neighboring regions, which resembles our strategy of penalizing over-visited partition.

Original: However, SVGD tends to underestimate the uncertainty given a limited number of particles.
Revised: However, SVGD tends to underestimate the uncertainty given a limited number of particles.

Original: Moreover, the quadratic cost in kernel matrix computation further raises the scalability concerns as more particles are proposed.
Revised: Moreover, the quadratic cost in kernel matrix computation further raises the scalability concerns as more particles are proposed.

Original: Admittedly, ICSGLD is not the ﬁrst interacting importance sampling algorithm.
Revised: Admittedly, ICSGLD is not the ﬁrst interacting importance sampling algorithm.

Original: For example, a population stochastic approximation Monte Carlo (pop-SAMC) algorithm has been proposed in Song et al.
Revised: For example, a population stochastic approximation Monte Carlo (pop-SAMC) algorithm has been proposed in Song et al.

Original: (2014), and an interacting particle Markov chain Monte Carlo (IPMCMC) algorithm has been proposed in Rainforth et al.
Revised: (2014), and an interacting particle Markov chain Monte Carlo (IPMCMC) algorithm has been proposed in Rainforth et al.

Original: A key difference between our algorithm and others is that our algorithm is mainly devised for big data problems.
Revised: A key difference between our algorithm and others is that our algorithm is mainly devised for big data problems.

Original: The IPMCMC and pop-SAMC are gradient-free samplers, which are hard to be adapted to high-dimensional big data problems.
Revised: The IPMCMC and pop-SAMC are gradient-free samplers, which are hard to be adapted to high-dimensional big data problems.

Original: Other parallel SGLD methods (Ahn et al., 2014; Chen et al., 2016) aim to reduce the computational cost of gradient estimations in distributed computing, which, however, does not consider interactions for accelerating the convergence.
Revised: Other parallel SGLD methods (Ahn et al., 2014; Chen et al., 2016) aim to reduce the computational cost of gradient estimations in distributed computing, which, however, does not consider interactions for accelerating the convergence.

Original: Li et al. (2019a) proposed asynchronous protocols to reduce communication costs when the master aggregates model parameters from all workers.
Revised: Li et al. (2019a) proposed asynchronous protocols to reduce communication costs when the master aggregates model parameters from all workers.

Original: Instead, we don’t communicate the parameter x ∈ R d but only share θ ∈ R m and the indices, where m (cid:28) d .
Revised: Instead, we don’t communicate the parameter x ∈ R d but only share θ ∈ R m and the indices, where m (cid:28) d .

Original: Our work also highly resembles the well-known Federated Averaging (FedAvg) algorithm (Li et al., 2020), except that the stochastic gradient (cid:101) U ( x ) is replaced with the random ﬁeld function (cid:101)
Revised: Our work also highly resembles the well-known Federated Averaging (FedAvg) algorithm (Li et al., 2020; Deng et al., 2021b), except that the stochastic gradient (cid:101) U ( x ) is replaced with the random ﬁeld function (cid:101)

Original: H ( θ , x )
Revised: H ( θ , x )

Original: and we only share the low-dimensional latent vector θ .
Revised: and we only share the low-dimensional latent vector θ .

Original: Since privacy concerns and communication cost are not major bottlenecks of our problem, we leave the study of taking the Monte Carlo average in Eq.(6)
Revised: Since privacy concerns and communication cost are not major bottlenecks of our problem, we leave the study of taking the Monte Carlo average in Eq.(6)

Original: every K > 1 iterations for future works.
Revised: every K > 1 iterations for future works.

Original:  C ONVERGENCE PROPERTIES
Revised:  C ONVERGENCE PROPERTIES

Original: To study theoretical properties of ICSGLD, we ﬁrst show a local stability property that is well-suited to big data problems, and then we present the asymptotic normality for the stochastic approximation process in mini-batch settings , which eventually yields the desired result that ICSGLD is asymptotically more efﬁcient than a single-chain CSGLD with an equivalent computational cost.
Revised: To study theoretical properties of ICSGLD, we ﬁrst show a local stability property that is well-suited to big data problems, and then we present the asymptotic normality for the stochastic approximation process in mini-batch settings , which eventually yields the desired result that ICSGLD is asymptotically more efﬁcient than a single-chain CSGLD with an equivalent computational cost

Original:  4. L OCAL STABILITY FOR NON - LINEAR MEAN - FIELD SYSTEMS IN BIG DATA
Revised: . 4. L OCAL STABILITY FOR NON - LINEAR MEAN - FIELD SYSTEMS IN BIG DATA

Original: The ﬁrst obstacle for the theoretical study is to approximate the components of θ ∞ corresponding to the high energy region.
Revised: The ﬁrst obstacle for the theoretical study is to approximate the components of θ ∞ corresponding to the high energy region.

Original: To get around this issue, the random ﬁeld function (cid:101) H ( θ , x ) in (8) is adopted to estimate a different target θ (cid:63) ∝ θ 1 ζ ∞ .
Revised: To get around this issue, the random ﬁeld function (cid:101) H ( θ , x ) in (8) is adopted to estimate a different target θ (cid:63) ∝ θ 1 ζ ∞ .

Original: As detailed in Lemma 3 in the supplementary material, the mean-ﬁeld equation is now formulated as follows
Revised: As detailed in Lemma 3 in the supplementary material, the mean-ﬁeld equation is now formulated as follows

Original: [Equation]
Revised: [Equation]

Original: where C
Revised: where C

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: Z
Revised: Z

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: (cid:101)
Revised: (cid:19)

Original: Z

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: (cid:19) 1 ζ and (cid:101)
Revised: and (cid:101)

Original: Z ζ, θ = (cid:80) mk =
Revised: Z ζ, θ = (cid:80) mk =

Original: (cid:82)
Revised: (cid:82)

Original: X k π ( x ) d x θ ζ − 1 ( k ) .
Revised: X k π ( x ) d x θ ζ − 1 ( k ) .

Original: We see that (10) may not be linearly stable as in Deng et al.
Revised: We see that (10) may not be linearly stable as in Deng et al.

Original: (2020b).
Revised: (2020b).

Original: Although the solution of the mean-ﬁeld system h ( θ ) = 0 is still unique, there may exist unstable invariant subspaces, leading us to consider the local properties.
Revised: Although the solution of the mean-ﬁeld system h ( θ ) = 0 is still unique, there may exist unstable invariant subspaces, leading us to consider the local properties.

Original: For a proper initialization of θ , which can be achieved by pre-training the model long enough time through SGLD, the mean value theorem implies a linear property in a local region
Revised: For a proper initialization of θ , which can be achieved by pre-training the model long enough time through SGLD, the mean value theorem implies a linear property in a local region

Original: [Equation]
Revised: [Equation]

Original: Combining the perturbation theory (Vanden-Eijnden, 2001), we present the following stability result:
Revised: Combining the perturbation theory (Vanden-Eijnden, 2001), we present the following stability result:

Original: Lemma 1 (Local stability, informal version of Lemma 3) Assume Assumptions A1-A4 (given in the supplementary material) hold.
Revised: Lemma 1 (Local stability, informal version of Lemma 3) Assume Assumptions A1-A4 (given in the supplementary material) hold.

Original: For any properly initialized θ , we have (cid:104) h
Revised: For any properly initialized θ , we have (cid:104) h

Original: ( θ ) , θ − (cid:98) θ (cid:63) (cid:105) ≤ − φ (cid:107) θ − (cid:98) θ (cid:63) (cid:107) 2 , where (cid:98) θ (cid:63)
Revised: ( θ ) , θ − (cid:98) θ (cid:63) (cid:105) ≤ − φ (cid:107) θ − (cid:98) θ (cid:63) (cid:107) 2 , where (cid:98) θ (cid:63)

Original: = θ (cid:63) + O (cid:0) sup x Var( ξ n ( x ))
Revised: = θ (cid:63) + O (cid:0) sup x Var( ξ n ( x ))

Original: + (cid:15) + 1 m (cid:1) , θ (cid:63) ∝ θ 1 ζ ∞ , φ > 0 ,  and ξ n ( x ) denotes the noise in the stochastic energy estimator of batch size n and Var( · ) denotes the variance.
Revised: + (cid:15) + 1 m (cid:1) , θ (cid:63) ∝ θ 1 ζ ∞ , φ > 0 , (cid:15) denotes a learning rate, and ξ n ( x ) denotes the noise in the stochastic energy estimator of batch size n and Var( · ) denotes the variance.

Original: By justifying the drift conditions of the adaptive transition kernel and relevant smoothness properties, we can prove the existence and regularity properties of the solution of the Poisson’s equation in
Revised: By justifying the drift conditions of the adaptive transition kernel and relevant smoothness properties, we can prove the existence and regularity properties of the solution of the Poisson’s equation in

Original: Lemma 6 in the supplementary material.
Revised: Lemma 6 in the supplementary material.

Original: In what follows, we can control the ﬂuctuations in stochastic approximation and eventually yields the L 2 convergence.
Revised: In what follows, we can control the ﬂuctuations in stochastic approximation and eventually yields the L 2 convergence.

Original: Lemma 2 ( L 2 convergence rate, informal version of Lemma 7) Given standard Assumptions A1A5.
Revised: Lemma 2 ( L 2 convergence rate, informal version of Lemma 7) Given standard Assumptions A1A5.

Original: θ k converges to (cid:98) θ (cid:63) , where (cid:98) θ (cid:63)
Revised: θ k converges to (cid:98) θ (cid:63) , where (cid:98) θ (cid:63)

Original: = θ (cid:63) + O (cid:0) sup x Var [Equation] n ( x 
Revised: = θ (cid:63) + O (cid:0) sup x Var( ξ n ( x ))

Original: [Equation] + 1 m (cid:1) , such that
Revised: + (cid:15) + 1 m (cid:1) , such that

Original: [Equation]
Revised: [Equation]

Original: The result differs from Theorem 1 of Deng et al.
Revised: The result differs from Theorem 1 of Deng et al.

Original: (2020b) in that the biased ﬁxed point (cid:98) θ (cid:63) instead of θ (cid:63) is treated as the equilibrium of the continuous system, which provides us a user-friendly proof.
Revised: (2020b) in that the biased ﬁxed point (cid:98) θ (cid:63) instead of θ (cid:63) is treated as the equilibrium of the continuous system, which provides us a user-friendly proof.

Original: Similar techniques have been adopted by Durmus & ´Eric Moulines (2017); Xu et al.
Revised: Similar techniques have been adopted by Durmus & ´Eric Moulines (2017); Xu et al.

Original: Although the global stability (Deng et al., 2020b) may be sacriﬁced when ζ (cid:54) = 1 based on Eq.(8), θ (cid:63) ∝ θ 1 ζ ∞ is much easier to estimate numerically for any i that yields 0 < θ ∞ ( i ) (cid:28) 1 based on a large ζ > 1 .
Revised: Although the global stability (Deng et al., 2020b) may be sacriﬁced when ζ (cid:54) = 1 based on Eq.(8), θ (cid:63) ∝ θ 1 ζ ∞ is much easier to estimate numerically for any i that yields 0 < θ ∞ ( i ) (cid:28) 1 based on a large ζ > 1 .

Original: 4. A SYMPTOTIC NORMALITY
Revised: 4. A SYMPTOTIC NORMALITY

Original: To study the asymptotic behavior of ω − 12 k
Revised: To study the asymptotic behavior of ω − 12 k

Original: ( θ k − (cid:98) θ (cid:63) ) , where (cid:98)
Revised: ( θ k − (cid:98) θ (cid:63) ) , where (cid:98)

Original: θ (cid:63) is the equilibrium point s.t. (cid:98) θ (cid:63)
Revised: θ (cid:63) is the equilibrium point s.t. (cid:98) θ (cid:63)

Original: = θ (cid:63) + O (cid:0) Var( ξ n ( x ))
Revised: = θ (cid:63) + O (cid:0) Var( ξ n ( x ))

Original: + (cid:15) + 1 m (cid:1) , we consider a ﬁxed step size ω in the SA step for ease of explanation.
Revised: + (cid:15) + 1 m (cid:1) , we consider a ﬁxed step size ω in the SA step for ease of explanation.

Original: Let ¯ θ t denote the solution of the mean-ﬁeld system in continuous time ( ¯ θ 

Revised: Let ¯ θ t denote the solution of the mean-ﬁeld system in continuous time ( ¯ θ

Original: = θ 0 ), and rewrite the single-chain SA step (7) as follows
Revised: = θ 0 ), and rewrite the single-chain SA step (7) as follows

Original: [Equation]
Revised: [Equation]

Original: Further, we set (cid:101) θ kω :
Revised: Further, we set (cid:101) θ kω :

Original: = ω − 1 2 ( θ k − ¯ θ kω ) .
Revised: = ω − 1 2 ( θ k − ¯ θ kω ) .

Original: Then the stochastic approximation differs from the mean ﬁeld system in that
Revised: Then the stochastic approximation differs from the mean ﬁeld system in that

Original: [Equation]
Revised: [Equation]

Revised: ω

Revised: [Equation]

Original: where h θ ( θ ) :
Revised: where h θ ( θ ) :

Original: = dd θ h ( θ ) is a matrix, W ∈ R m is a standard Brownian motion, the last term follows from a certain central limit theorem (Benveniste et al., 1990) and R denotes the covariance matrix of the random-ﬁeld function s.t.
Revised: = dd θ h ( θ ) is a matrix, W ∈ R m is a standard Brownian motion, the last term follows from a certain central limit theorem (Benveniste et al., 1990) and R denotes the covariance matrix of the random-ﬁeld function s.t.

Original: R ( θ ) :
Revised: R ( θ ) :

Original: = (cid:80) ∞
Revised: = (cid:80) ∞

Original: k
Revised: k

Original: =
Revised: =

Original: −∞ Cov θ ( H ( θ , x k ) , H ( θ , x 0 )) .
Revised: −∞ Cov θ ( H ( θ , x k ) , H ( θ , x 0 )) .

Original: We expect the weak convergence of U k to the stationary distribution of a diffusion
Revised: We expect the weak convergence of U k to the stationary distribution of a diffusion

Original: [Equation]
Revised: [Equation]

Original: (11) where U t = ω − 1 / 2 t ( θ t − (cid:98) θ (cid:63) ) .
Revised: (11) where U t = ω − 1 / 2 t ( θ t − (cid:98) θ (cid:63) ) .

Original: Given that θ t converges to (cid:98) θ (cid:63) sufﬁciently fast and the local linearity of h θ , the diffusion (11) resembles the Ornstein–Uhlenbeck process and yields the following solution
Revised: Given that θ t converges to (cid:98) θ (cid:63) sufﬁciently fast and the local linearity of h θ , the diffusion (11) resembles the Ornstein–Uhlenbeck process and yields the following solution

Original: [Equation]
Revised: [Equation]

Original: Then we have the following theorem, whose formal proof is given in section C.3.
Revised: Then we have the following theorem, whose formal proof is given in section C.3.

Original: Theorem 1 (Asymptotic Normality) Assume Assumptions A1-A5 (given in the supplementary material) hold.
Revised: Theorem 1 (Asymptotic Normality) Assume Assumptions A1-A5 (given in the supplementary material) hold.

Original: We have the following weak convergence
Revised: We have the following weak convergence

Original: [Equation]
Revised: [Equation]

Original: N
Revised: N

Original: [Equation]
Revised: [Equation]

Original: Σ ) , where Σ = (cid:90)
Revised: where Σ = (cid:90)

Original: [Equation]
Revised: [Equation]

Original: 4. I NTERACTING PARALLEL CHAINS ARE MORE EFFICIENT
Revised: 4. I NTERACTING PARALLEL CHAINS ARE MORE EFFICIENT

Original: For clarity, we ﬁrst denote an estimate of θ based on ICSGLD with P interacting parallel chains by θ Pk and denote the estimate based on a single-long-chain CSGLD by θ kP .
Revised: For clarity, we ﬁrst denote an estimate of θ based on ICSGLD with P interacting parallel chains by θ Pk and denote the estimate based on a single-long-chain CSGLD by θ kP .

Original: Note that Theorem 1 holds for any step size ω k
Revised: Note that Theorem 1 holds for any step size ω k

Original: = O ( k − α ) , where α ∈ (0 . 5 , 1) .
Revised: = O ( k − α ) , where α ∈ (0 . 5 , 1] .

Original: If we simply run a single-chain CSGLD algorithm with P times of iterations, by Theorem 1,
Revised: If we simply run a single-chain CSGLD algorithm with P times of iterations, by Theorem 1,

Original: [Equation]
Revised: [Equation]

Original: As to ICSGLD, since the covariance Σ relies on R , which depends on the covariance of the martingale {M i } i ≥ 1 , the conditional independence of x (1) , x (2) , · · · , x ( P ) naturally results in an efﬁcient variance reduction such that
Revised: As to ICSGLD, since the covariance Σ relies on R , which depends on the covariance of the martingale {M i } i ≥ 1 , the conditional independence of x (1) , x (2) , · · · , x ( P ) naturally results in an efﬁcient variance reduction such that

Original: Corollary 1 (Asymptotic Normality for ICSGLD)
Revised: Corollary 1 (Asymptotic Normality for ICSGLD)

Original: Assume the same assumptions.
Revised: Assume the same assumptions.

Original: For ICSGLD with P interacting chains, we have the following weak convergence
Revised: For ICSGLD with P interacting chains, we have the following weak convergence

Original: [Equation]
Revised: [Equation]

Original: That is, under a similar computational budget, we have
Revised: That is, under a similar computational budget, we have

Original: [Equation]
Revised: [Equation]

Original: Corollary 2 (Efﬁciency)
Revised: Corollary 2 (Efﬁciency)

Original: Given a decreasing step size ω k
Revised: Given a decreasing step size ω k

Original: = O ( k − α ) , where 0 .
Revised: = O ( k − α ) , where 0 .

Original: 5 < α < 1 , ICSGLD is asymptotically more efﬁcient than the single-chain CSGLD with an equivalent training cost.
Revised: 5 < α < 1 , ICSGLD is asymptotically more efﬁcient than the single-chain CSGLD with an equivalent training cost.

Original: In practice, slowly decreasing step sizes are often preferred in stochastic algorithms for a better non-asymptotic performance (Benveniste et al., 1990).
Revised: In practice, slowly decreasing step sizes are often preferred in stochastic algorithms for a better non-asymptotic performance (Benveniste et al., 1990).

Original:  E XPERIMENTS 5. L ANDSCAPE EXPLORATION ON MNIST VIA THE SCALABLE RANDOM - FIELD FUNCTION
Revised:  E XPERIMENTS 5. L ANDSCAPE EXPLORATION ON MNIST VIA THE SCALABLE RANDOM - FIELD FUNCTION

Original: This section shows how the novel random-ﬁeld function (8) facilitates the exploration of multiple modes on the MNIST dataset § , while the standard methods, such as stochastic gradient descent (SGD) and SGLD, only get stuck in few local modes .
Revised: This section shows how the novel random-ﬁeld function (8) facilitates the exploration of multiple modes on the MNIST dataset § , while the standard methods, such as stochastic gradient descent (SGD) and SGLD, only get stuck in few local modes .

Original: To simplify the experiments, we choose a large batch size of 2500 and only pick the ﬁrst ﬁve classes, namely digits from 0 to 4.
Revised: To simplify the experiments, we choose a large batch size of 2500 and only pick the ﬁrst ﬁve classes, namely digits from 0 to 4.

Original: The learning rate is ﬁxed to 1e-6 and the temperature is set to 0 .
Revised: The learning rate is ﬁxed to 1e-6 and the temperature is set to 0 .

Original: We see from Fig.2(a) that both SGD and SGLD lead to fast decreasing losses.
Revised: We see from Figure 2(a) that both SGD and SGLD lead to fast decreasing losses.

Original: By contrast, ICSGLD yields ﬂuctuating losses that traverse freely between high energy and low energy regions.
Revised: By contrast, ICSGLD yields ﬂuctuating losses that traverse freely between high energy and low energy regions.

Original: As the particles stick in local regions, the penalty of re-visiting these zones keeps increasing until a negative learning rate is injected to encourage explorations.
Revised: As the particles stick in local regions, the penalty of re-visiting these zones keeps increasing until a negative learning rate is injected to encourage explorations.

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: We conducted a singular value decomposition (SVD) based on the ﬁrst two coordinates to visualize the trajectories:
Revised: We conducted a singular value decomposition (SVD) based on the ﬁrst two coordinates to visualize the trajectories:

Original: We ﬁrst choose a domain that includes all the coordinates, then we recover the parameter based on the grid point and truncated values in other dimensions, and ﬁnally we ﬁne-tune the parameters and present the approximate losses of the trajectories in Fig.2(b-d).
Revised: We ﬁrst choose a domain that includes all the coordinates, then we recover the parameter based on the grid point and truncated values in other dimensions, and ﬁnally we ﬁne-tune the parameters and present the approximate losses of the trajectories in Figure 2(b-d).

Original: We see SGD trajectories get stuck in a local region; SGLD exploits a larger region but is still quite limited in the exploration; ICSGLD, instead, ﬁrst converges to a local region and then escapes it once it overvisits this region .
Revised: We see SGD trajectories get stuck in a local region; SGLD exploits a larger region but is still quite limited in the exploration; ICSGLD, instead, ﬁrst converges to a local region and then escapes it once it overvisits this region .

Original: This shows the strength of ICSGLD in the simulations of complex multi-modal distributions.
Revised: This shows the strength of ICSGLD in the simulations of complex multi-modal distributions.

Original: More experimental details are presented in section D.1 of the supplementary material.
Revised: More experimental details are presented in section D.1 of the supplementary material.

Original: 5. S IMULATIONS OF MULTI - MODAL DISTRIBUTIONS
Revised: 5. S IMULATIONS OF MULTI - MODAL DISTRIBUTIONS

Original: This section shows the acceleration effect of ICSGLD via a group of simulation experiments for a multi-modal distribution.
Revised: This section shows the acceleration effect of ICSGLD via a group of simulation experiments for a multi-modal distribution.

Original: The baselines include popular Monte Carlo methods such as CSGLD,
Revised: The baselines include popular Monte Carlo methods such as CSGLD,

Original: SGLD, cyclical SGLD (cycSGLD), replica exchange SGLD (reSGLD), and the particle-based SVGD.
Revised: SGLD, cyclical SGLD (cycSGLD), replica exchange SGLD (reSGLD), and the particle-based SVGD.

Original: The target multi-modal density is presented in Figure 3(a).
Revised: The target multi-modal density is presented in Figure 3(a).

Original: Figure 3(b-g) displays the empirical performance of all the testing methods: the vanilla SGLD with 5 parallel chains ( × P5) undoubtedly performs the worst in this example and fails to quantify the weights of each mode correctly; the single-chain cycSGLD with 5 times of iterations ( × T5) improves the performance but is still not accurate enough; reSGLD ( × P5) and SVGD ( × P5) have good performances, while the latter is quite costly in computations; ICSGLD ( × P5) does not only traverse freely over the rugged energy landscape, but also yields the most accurate approximation to the ground truth distribution.
Revised: Figure 3(b-g) displays the empirical performance of all the testing methods: the vanilla SGLD with 5 parallel chains ( × P5) undoubtedly performs the worst in this example and fails to quantify the weights of each mode correctly; the single-chain cycSGLD with 5 times of iterations ( × T5) improves the performance but is still not accurate enough; reSGLD ( × P5) and SVGD ( × P5) have good performances, while the latter is quite costly in computations; ICSGLD ( × P5) does not only traverse freely over the rugged energy landscape, but also yields the most accurate approximation to the ground truth distribution.

Original: By contrast, CSGLD ( × T5) performs worse than ICSGLD and overestimates the weights on the left side.
Revised: By contrast, CSGLD ( × T5) performs worse than ICSGLD and overestimates the weights on the left side.

Original: For the detailed setups, the study of convergence speed, and runtime analysis, we refer interested readers to section D.2 in the supplementary material.
Revised: For the detailed setups, the study of convergence speed, and runtime analysis, we refer interested readers to section D.2 in the supplementary material.

Original: [Figure]
Revised: [Figure]

Original: 5. D EEP CONTEXTUAL BANDITS ON MUSHROOM TASKS
Revised: 5. D EEP CONTEXTUAL BANDITS ON MUSHROOM TASKS

Original: This section evaluates ICSGLD on the contextual bandit problem based on the UCI Mushroom data set as in Riquelme et al.
Revised: This section evaluates ICSGLD on the contextual bandit problem based on the UCI Mushroom data set as in Riquelme et al.

Original: The mushrooms are assumed to arrive sequentially and the agent needs to take an action at each time step based on past feedbacks.
Revised: The mushrooms are assumed to arrive sequentially and the agent needs to take an action at each time step based on past feedbacks.

Original: Our goal is to minimize the cumulative regret that measures the difference between the cumulative reward obtained by the proposed policy and optimal policy.
Revised: Our goal is to minimize the cumulative regret that measures the difference between the cumulative reward obtained by the proposed policy and optimal policy.

Original: We evaluate Thompson Sampling (TS) based on a variety of approximate inference methods for posterior sampling.
Revised: We evaluate Thompson Sampling (TS) based on a variety of approximate inference methods for posterior sampling.

Original: We choose one (cid:15) -greedy policy (EpsGreedy) based on the RMSProp optimizer with a decaying learning rate (Riquelme et al., 2018) as a baseline.
Revised: We choose one (cid:15) -greedy policy (EpsGreedy) based on the RMSProp optimizer with a decaying learning rate (Riquelme et al., 2018) as a baseline.

Original: Two variational methods, namely stochastic gradient descent with a constant learning rate (ConstSGD) (Mandt et al., 2017) and Monte Carlo Dropout (Dropout) (Gal & Ghahramani, 2016) are compared to approximate the posterior distribution.
Revised: Two variational methods, namely stochastic gradient descent with a constant learning rate (ConstSGD) (Mandt et al., 2017) and Monte Carlo Dropout (Dropout) (Gal & Ghahramani, 2016) are compared to approximate the posterior distribution.

Original: For the sampling algorithms, we include preconditioned
Revised: For the sampling algorithms, we include preconditioned

Original: SGLD (pSGLD) (Li et al., 2016), preconditioned CSGLD (pCSGLD)
Revised: SGLD (pSGLD) (Li et al., 2016), preconditioned CSGLD (pCSGLD)

Original: (Deng et al., 2020b), and preconditioned ICSGLD (pICSGLD).
Revised: (Deng et al., 2020b), and preconditioned ICSGLD (pICSGLD).

Original: Note that all the algorithms run 4 parallel chains with average outputs ( × P4) except that pCSGLD runs a single-chain with 4 times of computational budget ( × T4).
Revised: Note that all the algorithms run 4 parallel chains with average outputs ( × P4) except that pCSGLD runs a single-chain with 4 times of computational budget ( × T4).

Original: For more details, we refer readers to section D.3 in the supplementary material.
Revised: For more details, we refer readers to section D.3 in the supplementary material.

Original: Figure.4 shows that EpsGreedy × P4 tends to explore too much for a long horizon as expected;
Revised: Figure 4 shows that EpsGreedy × P4 tends to explore too much for a long horizon as expected;

Original: ConstSGD × P4 and Dropout × P4 perform poorly in the beginning but eventually outperform EpsGreedy × P4 due to the inclusion of uncertainty for exploration, whereas the uncertainty seems to be inadequate due to the nature of variational inference.
Revised: ConstSGD × P4 and Dropout × P4 perform poorly in the beginning but eventually outperform EpsGreedy × P4 due to the inclusion of uncertainty for exploration, whereas the uncertainty seems to be inadequate due to the nature of variational inference.

Original: By contrast, pSGLD × P4 signiﬁcantly
Revised: By contrast, pSGLD × P4 signiﬁcantly

Original: [Figure]
Revised: [Figure]

Original: outperforms the variational methods by considering preconditioners within an exact sampling framework (SGLD).
Revised: outperforms the variational methods by considering preconditioners within an exact sampling framework (SGLD).

Original: As a unique algorithm that runs in a single-chain manner, pCSGLD × T4 leads to the worst performance due to the inefﬁciency in learning the selfadapting parameters, fortunately, pCSGLD × T4 slightly outperform pSGLD × P4 in the later phase with the help of the well-estimated self-adapting parameters.
Revised: As a unique algorithm that runs in a single-chain manner, pCSGLD × T4 leads to the worst performance due to the inefﬁciency in learning the selfadapting parameters, fortunately, pCSGLD × T4 slightly outperform pSGLD × P4 in the later phase with the help of the well-estimated self-adapting parameters.

Original: Nevertheless, pICSGLD × P4 propose to optimize the shared self-adapting parameters at the same time, which in turn greatly contributes to the simulation of the posterior.
Revised: Nevertheless, pICSGLD × P4 propose to optimize the shared self-adapting parameters at the same time, which in turn greatly contributes to the simulation of the posterior.

Original: As a result, pICSGLD × P4 consistently shows the lowest regret excluding the very early period.
Revised: As a result, pICSGLD × P4 consistently shows the lowest regret excluding the very early period.

Original: This shows the great superiority of the interaction mechanism in learning the self-adapting parameters for accelerating the simulations.
Revised: This shows the great superiority of the interaction mechanism in learning the self-adapting parameters for accelerating the simulations.

Original: 5. U NCERTAINTY ESTIMATION
Revised: 5. U NCERTAINTY ESTIMATION

Original: This section evaluates the qualify of our algorithm in uncertainty quantiﬁcation.
Revised: This section evaluates the qualify of our algorithm in uncertainty quantiﬁcation.

Original: For model architectures, we use residual networks (ResNet) (He et al., 2016) and a wide ResNet (WRN) (Zagoruyko & Komodakis, 2016); we choose 20, 32, and 56-layer ResNets (denoted by ResNet20, et al.) and a WRN-16-8 network, a 16-layer WRN that is 8 times wider than ResNet16.
Revised: For model architectures, we use residual networks (ResNet) (He et al., 2016) and a wide ResNet (WRN) (Zagoruyko & Komodakis, 2016); we choose 20, 32, and 56-layer ResNets (denoted by ResNet20, et al.) and a WRN-16-8 network, a 16-layer WRN that is 8 times wider than ResNet16.

Original: We train the models on CIFAR100, and report the test accuracy (ACC) and test negative log-likelihood (NLL).
Revised: We train the models on CIFAR100, and report the test accuracy (ACC) and test negative log-likelihood (NLL) based ontrials with standard error.

Original: For the out-of-distribution prediction performance, we test the well-trained models in Brier scores (Brier) * on the Street View House Numbers dataset (SVHN).
Revised: For the out-of-distribution prediction performance, we test the well-trained models in Brier scores (Brier) * on the Street View House Numbers dataset (SVHN).

Original: Due to the wide adoption of momentum stochastic gradient descent (M-SGD), we use stochastic gradient Hamiltonian Monte Carlo (SGHMC)
Revised: Due to the wide adoption of momentum stochastic gradient descent (M-SGD), we use stochastic gradient Hamiltonian Monte Carlo (SGHMC)

Original: (Chen et al., 2014) as the baseline sampling algorithm and denote the interacting contour SGHMC by ICSHMC.
Revised: (Chen et al., 2014) as the baseline sampling algorithm and denote the interacting contour SGHMC by ICSHMC.

Original: In addition, we include several high performing baselines, such as SGHMC with cyclical learning rates (cycSGHMC) (Zhang et al., 2020b), SWAG based on cyclic learning rates of 10 cycles (cycSWAG) (Maddox et al., 2019) and variance-reduced replica exchange SGHMC (reSGHMC) (Deng et al., 2021).
Revised: In addition, we include several high performing baselines, such as SGHMC with cyclical learning rates (cycSGHMC) (Zhang et al., 2020b), SWAG based on cyclic learning rates of 10 cycles (cycSWAG) (Maddox et al., 2019) and variance-reduced replica exchange SGHMC (reSGHMC) (Deng et al., 2021a).

Original: For a fair comparison,
Revised: For a fair comparison,

Original: ICSGLD also conducts variance reduction on the energy function to alleviate the bias.
Revised: ICSGLD also conducts variance reduction on the energy function to alleviate the bias.

Original: Moreover, a large ζ = 3 × 10 6 is selected, which only induces mild gradient multipliers ranging from − 1 to 2 to penalize over-visited partitions.
Revised: Moreover, a large ζ = 3 × 10 6 is selected, which only induces mild gradient multipliers ranging from − 1 to 2 to penalize over-visited partitions.

Original: We don’t include SVGD (Liu & Wang, 2016) and SPOS (Zhang et al., 2020a) for scalability reasons.
Revised: We don’t include SVGD (Liu & Wang, 2016) and SPOS (Zhang et al., 2020a) for scalability reasons.

Original: A batch size of 256 is selected.
Revised: A batch size of 256 is selected.

Original: We run 4 parallel processes ( × P4) with 500 epochs for M-SGD, reSGHMC and ICSGHMC and run cycSGHMC and cycSWAGepochs ( × T4) based on a single process with 10 cycles.
Revised: We run 4 parallel processes ( × P4) with 500 epochs for M-SGD, reSGHMC and ICSGHMC and run cycSGHMC and cycSWAGepochs ( × T4) based on a single process with 10 cycles.

Original: Refer to section D.4 of the supplementary material for the detailed settings.
Revised: Refer to section D.4 of the supplementary material for the detailed settings.

Original: [Table]
Revised: [Table]

Original: Table 1 shows that the vanilla ensemble results via M-SGD × P4 surprisingly outperform cycSGHMC × T4 and cycSWAG × T4 on medium models, such as ResNet20 and ResNet32, and show very good performance on the out-of-distribution samples in Brier scores.
Revised: Table 1 shows that the vanilla ensemble results via M-SGD × P4 surprisingly outperform cycSGHMC × T4 and cycSWAG × T4 on medium models, such as ResNet20 and ResNet32, and show very good performance on the out-of-distribution samples in Brier scores.

Original: We suspect that the parallel implementation ( × P4) provides isolated initializations with less correlated samples; by contrast, cycSGHMC × T4 and cycSWAG × T4 explore the energy landscape contiguously, implying a risk to stay near the original region.
Revised: We suspect that the parallel implementation ( × P4) provides isolated initializations with less correlated samples; by contrast, cycSGHMC × T4 and cycSWAG × T4 explore the energy landscape contiguously, implying a risk to stay near the original region.

Original: reSGHMC × P4 shows a remarkable performance overall, but demonstrates a large variance occasionally; this indicates the insufﬁciency of the swaps when multiple processes are included.
Revised: reSGHMC × P4 shows a remarkable performance overall, but demonstrates a large variance occasionally; this indicates the insufﬁciency of the swaps when multiple processes are included.

Original: When it comes to testing WRN-16-8, cycSWAG × T4 shows a marvelous result and a large improvement compared to the other baselines.
Revised: When it comes to testing WRN-16-8, cycSWAG × T4 shows a marvelous result and a large improvement compared to the other baselines.

Original: We conjecture that cycSWAG is more independent of hyperparameter tuning, thus leading to better performance in larger models.
Revised: We conjecture that cycSWAG is more independent of hyperparameter tuning, thus leading to better performance in larger models.

Revised: We don’t report CSGHMC × P4 since it becomes quite unstable during the training of ResNet56 and

Original:  As to ICSGHMC × P4, it consistently performs the best in prediction accuracy (ACC) and the negative log likelihood (NLL) and perform comparable to M-SGD × P4 in Brier scores.
Revised: WRN-16-8 models and causes mediocre results. As to ICSGHMC × P4, it consistently performs remarkable in both ACC and NLL and performs comparable to M-SGD × P4 in Brier scores.

Revised: Code is available at github.com/WayneDW/Interacting-Contour-Stochastic-Gradient-Langevin-Dynamics .

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have proposed the ICSGLD as an efﬁcient algorithm for sampling from distributions with a complex energy landscape, and shown theoretically that ICSGLD is indeed more efﬁcient than the single-chain CSGLD for a slowly decreasing step size.
Revised: We have proposed the ICSGLD as an efﬁcient algorithm for sampling from distributions with a complex energy landscape, and shown theoretically that ICSGLD is indeed more efﬁcient than the single-chain CSGLD for a slowly decreasing step size.

Original: To our best knowledge, this is the ﬁrst interacting importance sampling algorithm that adapts to big data problems without scalability concerns.
Revised: To our best knowledge, this is the ﬁrst interacting importance sampling algorithm that adapts to big data problems without scalability concerns.

Original: ICSGLD has been compared with numerous state-of-the-art baselines for various tasks, whose remarkable results indicate its promising future in big data applications. 
Revised: ICSGLD has been compared with numerous state-of-the-art baselines for various tasks, whose remarkable results indicate its promising future in big data applications. A CKNOWLEDGMENT

Revised: Liang’s research was supported in part by the grants DMS-2015498, R01-GM117597 and R01GM126089.

Revised: Lin acknowledges the support from NSF (DMS-1555072, DMS-2053746, and DMS2134209), BNL Subcontract 382247, and DE-SC0021142.

Original: A P RELIMINARIES A. S TOCHASTIC APPROXIMATION
Revised: A P RELIMINARIES A. S TOCHASTIC APPROXIMATION

Original: Given a random-ﬁeld function (cid:101) H
Revised: Given a random-ﬁeld function (cid:101) H

Original: ( θ , x ) , the stochastic approximation algorithm (Benveniste et al., 1990) proposes to solve the mean-ﬁeld equation h ( θ )
Revised: ( θ , x ) , the stochastic approximation algorithm (Benveniste et al., 1990) proposes to solve the mean-ﬁeld equation h ( θ )

Original: = 

Revised: =

Original: in the analysis of adaptive algorithms
Revised: in the analysis of adaptive algorithms

Original: [Equation]
Revised: [Equation]

Original: where x ∈ X ⊂ R d , θ ∈ Θ ⊂ R m , (cid:36) θ ( x ) is a distribution that depends on the self-adapting parameter θ .
Revised: where x ∈ X ⊂ R d , θ ∈ Θ ⊂ R m , (cid:36) θ ( x ) is a distribution that depends on the self-adapting parameter θ .

Original: Given the transition kernel Π θ ( x , A ) for any Borel subset A ⊂ X , the algorithm can be written as follows
Revised: Given the transition kernel Π θ ( x , A ) for any Borel subset A ⊂ X , the algorithm can be written as follows

Original: (1) Simulate x k +1 ∼ Π θ k ( x k , · ) , which yields the invariant distribution (cid:36) θ k ( · ) , (2) Optimize θ k +

Revised: (1) Simulate x k +1 ∼ Π θ k ( x k , · ) , which yields the invariant distribution (cid:36) θ k ( · ) , (2) Optimize θ k +

Original: =
Revised: =

Original: θ k + ω k +1 (cid:101)
Revised: θ k + ω k +1 (cid:101)

Original: H
Revised: H

Original: ( θ k , x k +1 ) .
Revised: ( θ k , x k +1 ) .

Original: Compared with the standard Robbins–Monro algorithm (Robbins & Monro, 1951), the algorithm proposes to simulate x from a transition kernel Π θ ( · , · ) instead of the distribution (cid:36) θ ( · ) directly.
Revised: Compared with the standard Robbins–Monro algorithm (Robbins & Monro, 1951), the algorithm proposes to simulate x from a transition kernel Π θ ( · , · ) instead of the distribution (cid:36) θ ( · ) directly.

Original: In other words, , (cid:101) H ( θ k , x k +1 ) − h
Revised: In other words, , (cid:101) H ( θ k , x k +1 ) − h

Original: ( θ k ) is not a Martingale but rather a Markov state-dependent noise.
Revised: ( θ k ) is not a Martingale but rather a Markov state-dependent noise.

Original: A. P OISSON ’ S EQUATION
Revised: A. P OISSON ’ S EQUATION

Original: In the stochastic approximation algorithm, the sequence of { ( x k , θ k ) } ∞ k =1 on the product space
Revised: In the stochastic approximation algorithm, the sequence of { ( x k , θ k ) } ∞ k =1 on the product space

Original: X × Θ is generated, which is an inhomogeneous Markov chain and requires the tool of the Poisson’s equation to study the convergence
Revised: X × Θ is generated, which is an inhomogeneous Markov chain and requires the tool of the Poisson’s equation to study the convergence

Original: [Equation]
Revised: [Equation]

Original: where µ θ ( · ) is a function on X .
Revised: where µ θ ( · ) is a function on X .

Original: The solution µ θ ( x ) to the Poisson’s equation exists and is formulated in the form
Revised: The solution µ θ ( x ) to the Poisson’s equation exists and is formulated in the form

Original: [Equation]
Revised: [Equation]

Original: when the above series converges.
Revised: when the above series converges.

Original: To ensure such a convergence, Benveniste et al.
Revised: To ensure such a convergence, Benveniste et al.

Original: (1990) made the following regularity conditions on the solution µ θ ( · ) of the Poisson’s equation:
Revised: (1990) made the following regularity conditions on the solution µ θ ( · ) of the Poisson’s equation:

Original: There exist a Lyapunov function V : X → [1 , ∞ ) and a positive constant C > 0 such that ∀ θ , θ (
Revised: There exist a Lyapunov function V : X → [1 , ∞ ) and a positive constant C > 0 such that ∀ θ , θ (

Original: cid:48) ∈ Θ , we have
Revised: cid:48) ∈ Θ , we have

Original: [Equation]
Revised: [Equation]

Original: where a common choice for the Lyapunov function is to set V ( x ) = 

Revised: where a common choice for the Lyapunov function is to set V ( x ) =

Original: + (cid:107) x (cid:107) 

Revised: + (cid:107) x (cid:107)

Original: (Teh et al
Revised: (Teh et al

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Vollmer et al., 2016).
Revised: Vollmer et al., 2016).

Original: A. G AUSSIAN DIFFUSIONS
Revised: A. G AUSSIAN DIFFUSIONS

Original: Consider a stochastic linear differential equation
Revised: Consider a stochastic linear differential equation

Original: [Equation]
Revised: [Equation]

Original: 13) where
Revised: (13) where

Original: U is a m -dimensional random vector,
Revised: U is a m -dimensional random vector,

Original: [Equation]
Revised: [Equation]

Original: ∞ k =
Revised: ∞ k =

Original: −∞ Cov θ ( H ( θ , x k ) , H ( θ , x 0 )) is a positive deﬁnite matrix that depends on θ ( · ) , W ∈ R m is a standard Brownian motion.
Revised: −∞ Cov θ ( H ( θ , x k ) , H ( θ , x 0 )) is a positive deﬁnite matrix that depends on θ ( · ) , W ∈ R m is a standard Brownian motion.

Original: Given a large enough t such that θ t converges to a ﬁxed point (cid:98) θ (cid:63) sufﬁciently fast, we may write the diffusion associated with Eq.(13) as follow
Revised: Given a large enough t such that θ t converges to a ﬁxed point (cid:98) θ (cid:63) sufﬁciently fast, we may write the diffusion associated with Eq.(13) as follow

Original: [Equation]
Revised: [Equation]

Original: Suppose that the matrix h θ ( (cid:98) θ (cid:63) ) is negative deﬁnite, then U t converges in distribution to a Gaussian variable
Revised: Suppose that the matrix h θ ( (cid:98) θ (cid:63) ) is negative deﬁnite, then U t converges in distribution to a Gaussian variable

Original: [Equation]
Revised: [Equation]

Original: The main goal of this supplementary ﬁle is to study the Gaussian approximation of the process ω
Revised: The main goal of this supplementary ﬁle is to study the Gaussian approximation of the process ω

Original: − 1 / 2 k ( θ k − (cid:98) θ (cid:63) ) to the solution Eq.(14) for a proper step size ω k .
Revised: − 1 / 2 k ( θ k − (cid:98) θ (cid:63) ) to the solution Eq.(14) for a proper step size ω k .

Original: Thereafter, the advantage of interacting mechanisms can be naturally derived.
Revised: Thereafter, the advantage of interacting mechanisms can be naturally derived.

Original: B S TABILITY AND CONVERGENCE ANALYSIS
Revised: B S TABILITY AND CONVERGENCE ANALYSIS

Original: As required by the algorithm, we update P contour stochastic gradient Langevin dynamics (CSGLD) simultaneously.
Revised: As required by the algorithm, we update P contour stochastic gradient Langevin dynamics (CSGLD) simultaneously.

Original: For the notations, we denote the particle of the p-th chain at iteration k by x ( p ) k ∈ X ⊂
Revised: For the notations, we denote the particle of the p-th chain at iteration k by x ( p ) k ∈ X ⊂

Original: R d and the joint state of the P parallel particles at iteration k by x (cid:78) P k := (cid:16) x (1) k , x (2) k , · · · , x ( P ) k (cid:17) (cid:62) ∈
Revised: R d and the joint state of the P parallel particles at iteration k by x (cid:78) P k := (cid:16) x (1) k , x (2) k , · · · , x ( P ) k (cid:17) (cid:62) ∈

Original: X (cid:78) P ⊂ R dP .
Revised: X (cid:78) P ⊂ R dP .

Original: We also denote the learning rate and step size at iteration k by (cid:15) k and ω k , respectively.
Revised: We also denote the learning rate and step size at iteration k by (cid:15) k and ω k , respectively.

Original: We denote by N (0 , I dP ) a standard dP -dimensional Gaussian vector and denote by ζ a positive hyperparameter.
Revised: We denote by N (0 , I dP ) a standard dP -dimensional Gaussian vector and denote by ζ a positive hyperparameter.

Original: B. ICSGLD ALGORITHM
Revised: B. ICSGLD ALGORITHM

Original: First, we introduce the interacting contour stochastic gradient Langevin dynamics (ICSGLD) with P parallel chains:
Revised: First, we introduce the interacting contour stochastic gradient Langevin dynamics (ICSGLD) with P parallel chains:

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: where ∇ x (cid:101) L ( x (cid:78) P , θ )
Revised: where ∇ x (cid:101) L ( x (cid:78) P , θ )

Original: := (cid:16) ∇ x (cid:101)
Revised: := (cid:16) ∇ x (cid:101)

Original: L ( x (1) , θ ) , ∇ x (cid:101) L ( x (2) , θ ) , · · · , ∇ x (cid:101) L ( x ( P ) , θ ) (cid:17) (cid:62) ,
Revised: L ( x (1) , θ ) , ∇ x (cid:101) L ( x (2) , θ ) , · · · , ∇ x (cid:101) L ( x ( P ) , θ ) (cid:17) (cid:62) ,

Original: ∇ x (cid:101) L ( x , θ ) is the stochastic adaptive gradient given by
Revised: ∇ x (cid:101) L ( x , θ ) is the stochastic adaptive gradient given by

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: In particular, the interacting random-ﬁeld function is written as
Revised: In particular, the interacting random-ﬁeld function is written as

Original: [Equation]
Revised: [Equation]

Original: where each random-ﬁeld function (cid:101) H ( θ , x 
Revised: where each random-ﬁeld function (cid:101) H ( θ , x ) =

Original: [Equation] = ( 
Revised: ( (cid:101) H 1 ( θ , x ) , . .

Original: [Equation]

Original:  (cid:101) H m ( θ , x )) follows
Revised: , (cid:101) H m ( θ , x )) follows

Original: [Equation]
Revised: [Equation]

Original: Here J (cid:101) U ( x ) denotes the index i ∈ { 1 , 2 , 3 , · · · , m } such that u i − 1 < Nn (cid:101)
Revised: Here J (cid:101) U ( x ) denotes the index i ∈ { 1 , 2 , 3 , · · · , m } such that u i − 1 < Nn (cid:101)

Original: U
Revised: U

Original: ( x ) ≤ u i for a set of energy partitions { u i } mi =0 and (cid:101) U ( x )
Revised: ( x ) ≤ u i for a set of energy partitions { u i } mi =0 and (cid:101) U ( x )

Original: = (cid:80) i ∈
Revised: = (cid:80) i ∈

Original: B U i ( x ) where U i denotes the negative log of a posterior based on a single data point i and B denotes a mini-batch of data of size n .
Revised: B U i ( x ) where U i denotes the negative log of a posterior based on a single data point i and B denotes a mini-batch of data of size n .

Original: Note that the stochastic energy estimator (cid:101) U ( x ) results in a biased estimation for the partition index J (cid:101) U ( x ) due to a non-linear transformation.
Revised: Note that the stochastic energy estimator (cid:101) U ( x ) results in a biased estimation for the partition index J (cid:101) U ( x ) due to a non-linear transformation.

Original: To avoid such a bias asymptotically with respect to the learning rate (cid:15)  , we may consider a variance-reduced energy estimator (cid:101) U VR ( x ) following Deng et al. 
Revised: To avoid such a bias asymptotically with respect to the learning rate (cid:15) k , we may consider a variance-reduced energy estimator (cid:101) U VR ( x ) following Deng et al. (2021a)

Original: [Equation]
Revised: [Equation]

Original: where the control variate x q (cid:98) kq (cid:99) is updated every q iterations.
Revised: where the control variate x q (cid:98) kq (cid:99) is updated every q iterations.

Original: Compared with the na¨ıve parallelism of CSGLD, a key feature of the ICSGLD algorithm lies in the joint estimation of the interacting random-ﬁeld function (cid:102) H ( θ , x (cid:78) P ) in Eq.(16) for the same mean-ﬁeld function h ( θ ) .
Revised: Compared with the na¨ıve parallelism of CSGLD, a key feature of the ICSGLD algorithm lies in the joint estimation of the interacting random-ﬁeld function (cid:102) H ( θ , x (cid:78) P ) in Eq.(16) for the same mean-ﬁeld function h ( θ ) .

Original: B.1. D ISCUSSIONS ON THE HYPERPARAMETERS
Revised: B.1. D ISCUSSIONS ON THE HYPERPARAMETERS

Original: The most important hyperparameter is ζ .
Revised: The most important hyperparameter is ζ .

Original: A ﬁne-tuned ζ usually leads to a small or even slightly negative learning rate in low energy regions to avoid local-trap problems.
Revised: A ﬁne-tuned ζ usually leads to a small or even slightly negative learning rate in low energy regions to avoid local-trap problems.

Original: Theoretically, ζ affects the
Revised: Theoretically, ζ affects the

Original: L 2 convergence rate hidden in the big-O notation in Lemma 3.
Revised: L 2 convergence rate hidden in the big-O notation in Lemma 3.

Original: The other hyperparameters can be easily tuned.
Revised: The other hyperparameters can be easily tuned.

Original: For example, the ResNet models yields the full loss ranging from 10,000 to 60,000 after warm-ups, we thus partition the sample space according to the energy into 200 subregions equally without tuning; since the optimization of SA is nearly convex, tuning { ω k } is much easier than tuning { (cid:15) k } for non-convex learning.
Revised: For example, the ResNet models yields the full loss ranging from 10,000 to 60,000 after warm-ups, we thus partition the sample space according to the energy into 200 subregions equally without tuning; since the optimization of SA is nearly convex, tuning { ω k } is much easier than tuning { (cid:15) k } for non-convex learning.

Original: B.1. D ISCUSSIONS ON DISTRIBUTED COMPUTING AND COMMUNICATION COST
Revised: B.1. D ISCUSSIONS ON DISTRIBUTED COMPUTING AND COMMUNICATION COST

Original: In shared-memory settings, the implementation is trivial and the details are omitted.
Revised: In shared-memory settings, the implementation is trivial and the details are omitted.

Original: In distributed-memory settings: θ k +1 is updated by the central node as follows:
Revised: In distributed-memory settings: θ k +1 is updated by the central node as follows:

Original: • The p -th worker conducts the sampling step ( S 1 ) and sends the indices J (cid:101) U ( x ( p ) k +1 ) ’s to the central node; • The central node aggregates the indices from all worker and updates θ k based on ( S 2 ) ; • The central node sends θ k +1 back to each worker.
Revised: • The p -th worker conducts the sampling step ( S 1 ) and sends the indices J (cid:101) U ( x ( p ) k +1 ) ’s to the central node; • The central node aggregates the indices from all worker and updates θ k based on ( S 2 ) ; • The central node sends θ k +1 back to each worker.

Original: We emphasize that we don’t communicate the model parameters x ∈ R d , but rather share the selfadapting parameter θ ∈ R m , where m (cid:28) d .
Revised: We emphasize that we don’t communicate the model parameters x ∈ R d , but rather share the selfadapting parameter θ ∈ R m , where m (cid:28) d .

Original: For example, WRN-16-8 has 11 M parameters (40 MB), while θ can be set to dimension 200 of size 4 KB; hence, the communication cost is not a big issue.
Revised: For example, WRN-16-8 has 11 M parameters (40 MB), while θ can be set to dimension 200 of size 4 KB; hence, the communication cost is not a big issue.

Original: Moreover, the theoretical advantage still holds if the communication frequency is  reduced.
Revised: Moreover, the theoretical advantage still holds if the communication frequency is slightly reduced.

Original: B.1. S CALABILITY TO BIG DATA
Revised: B.1. S CALABILITY TO BIG DATA

Original: Recall that the adaptive sampler follows that
Revised: Recall that the adaptive sampler follows that

Original: [Equation]
Revised: [Equation]

Original: The key to the success of (I)CSGLD is to generate sufﬁciently strong bouncy moves ( negative gradient multiplier) to escape local traps.
Revised: The key to the success of (I)CSGLD is to generate sufﬁciently strong bouncy moves ( negative gradient multiplier) to escape local traps.

Original: To this end, ζ can be tuned to generate proper bouncy moves.
Revised: To this end, ζ can be tuned to generate proper bouncy moves.

Original: Take the CIFAR100 experiments for example:
Revised: Take the CIFAR100 experiments for example:

Original: • the self-adjusting mechanism fails if the gradient multiplier uniformly “equals” to 1 and a too small value of ζ = 1 could lead to this issue; • the self-adjusting mechanism works only if we choose a large enough ζ such as 3e6 to generate (desired) negative gradient multiplier in over-visited regions.
Revised: • the self-adjusting mechanism fails if the gradient multiplier uniformly “equals” to 1 and a too small value of ζ = 1 could lead to this issue; • the self-adjusting mechanism works only if we choose a large enough ζ such as 3e6 to generate (desired) negative gradient multiplier in over -visited regions.

Original: However, when we set ζ = 3e-6, the original stochastic approximation (SA) update proposed in (Deng et al., 2020b) follows that[Equation] k
Revised: However, when we set ζ = 3e6, the original stochastic approximation (SA) update proposed in (Deng et al., 2020b) follows that

Original: [Equation]
Revised: [Equation]

Original: i )

Original: [Equation] [Table] [Equation] Since θ ( i ) < 1 for any i ∈ { 1 , · · · , m } , θ ( i ) ζ is essentially 0 for such a large ζ , which means that the original SA fails to optimize when ζ is large .
Revised: Since θ ( i ) < 1 for any i ∈ { 1 , · · · , m } , θ ( i ) ζ is essentially 0 for such a large ζ , which means that the original SA fails to optimize when ζ is large .

Original: Therefore, the limited choices of ζ inevitably limits the scalability to big data problems.
Revised: Therefore, the limited choices of ζ inevitably limits the scalability to big data problems.

Original: Our newly proposed SA scheme
Revised: Our newly proposed SA scheme

Original: [Equation]
Revised: [Equation]

Original: is more independent of ζ and proposes to converge to  θ 1 /ζ ∞ instead of θ ∞ , where θ ∞ ( i ) =
Revised: is more independent of ζ and proposes to converge to a much smoother equilibrium θ 1 /ζ ∞ instead of θ ∞ , where θ ∞ ( i ) =

Original: (cid:82) χ
Revised: (cid:82) χ

Original: i π ( x ) dx ∝ (cid:82) χ i e − U ( x ) τ dx is the energy PDF.
Revised: i π ( x ) dx ∝ (cid:82) χ i e − U ( x ) τ dx is the energy PDF.

Original: As such, despite the linear stability is sacriﬁced, the resulting algorithm is more scalable.
Revised: As such, despite the linear stability is sacriﬁced, the resulting algorithm is more scalable.

Original: For example, estimating e − 10 , 000 × 1 ζ is numerically much easier than e − 10 , 000 for a large ζ such as 10 , 000 , where 10 , 000 can be induced by the high losses in training deep neural networks in big data.
Revised: For example, estimating e − 10 , 000 × 1 ζ is numerically much easier than e − 10 , 000 for a large ζ such as 10 , 000 , where 10 , 000 can be induced by the high losses in training deep neural networks in big data.

Original: B. A SSUMPTIONS
Revised: B. A SSUMPTIONS

Original: A long-standing problem for stochastic approximation is the difﬁculty in establishing the stability property and a practical remedy for this problem is to study Θ on a ﬁxed compact set.
Revised: A long-standing problem for stochastic approximation is the difﬁculty in establishing the stability property and a practical remedy for this problem is to study Θ on a ﬁxed compact set.

Original: Assumption A1 (Compactness)
Revised: Assumption A1 (Compactness)

Original: The space Θ is compact and for any i ∈ { 1 , 2 , . . .
Revised: The space Θ is compact and for any i ∈ { 1 , 2 , . . .

Original: , m } we have inf Θ θ ( i ) > 0 .
Revised: , m } we have inf Θ θ ( i ) > 0 .

Original: In addition, there exists a positive constant Q > 0 that satisﬁes ∀ θ ∈ Θ and x ∈ X ,
Revised: In addition, there exists a positive constant Q > 0 that satisﬁes ∀ θ ∈ Θ and x ∈ X ,

Original: [Equation]
Revised: [Equation]

Original: For weaker assumptions, we refer readers to Theorem 3.2 (Fort et al., 2015), where a recurrence property can be proved for the Metropolis-based Wang-Landau algorithm, which eventually established that the estimates return to a desired compact set often enough.
Revised: For weaker assumptions, we refer readers to Theorem 3.2 (Fort et al., 2015), where a recurrence property can be proved for the Metropolis-based Wang-Landau algorithm, which eventually established that the estimates return to a desired compact set often enough.

Original: Next, we lay out the smoothness assumption, which is standard in the convergence analysis of SGLD, see e.g. Mattingly et al.
Revised: Next, we lay out the smoothness assumption, which is standard in the convergence analysis of SGLD, see e.g. Mattingly et al.

Original: , Raginsky et al.
Revised: , Raginsky et al.

Original: (2017) and Xu et al.
Revised: (2017) and Xu et al.

Original: Assumption A2 (Smoothness) U ( x ) is M -smooth when there exists a positive constant M that satisﬁes ∀ x , x (cid:48) ∈ X ,
Revised: Assumption A2 (Smoothness) U ( x ) is M -smooth when there exists a positive constant M that satisﬁes ∀ x , x [Equation] ∈ [Equation]

Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: In addition, we assume the dissipativity condition to ensure that the geometric ergodicity of the dynamical system holds.
Revised: In addition, we assume the dissipativity condition to ensure that the geometric ergodicity of the dynamical system holds.

Original: This assumption is also crucial for verifying the solution properties of the solution of Poisson’s equation.
Revised: This assumption is also crucial for verifying the solution properties of the solution of Poisson’s equation.

Original: Similar assumptions have been made in Mattingly et al.
Revised: Similar assumptions have been made in Mattingly et al.

Original: Raginsky et al.
Revised: Raginsky et al.

Original: (2017) and Xu et al.
Revised: (2017) and Xu et al.

Original: Assumption A3 (Dissipativity)
Revised: Assumption A3 (Dissipativity)

Original: There exist constants ˜ m > 0 and ˜ b ≥ 0 that satisﬁes ∀ x ∈ X and any θ ∈ Θ ,
Revised: There exist constants ˜ m > 0 and ˜ b ≥ 0 that satisﬁes ∀ x ∈ X and any θ ∈ Θ ,

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: To further establish a bounded second moment on x ∈ X with respect to a proper Lyapunov function
Revised: To further establish a bounded second moment on x ∈ X with respect to a proper Lyapunov function

Original: V ( x ) , we impose the following conditions on the gradient noise:
Revised: V ( x ) , we impose the following conditions on the gradient noise:

Original: Assumption A4 (Gradient noise)
Revised: Assumption A4 (Gradient noise)

Original: The stochastic gradient based on mini-batch settings is an unbiased estimator such that
Revised: The stochastic gradient based on mini-batch settings is an unbiased estimator such that

Original: [Equation]
Revised: [Equation]

Original: furthermore, for some positive constants M and B , we have
Revised: furthermore, for some positive constants M and B , we have

Original: [Equation]
Revised: [Equation]

Original: where E [ · 
Revised: where E [ · ] acts on the distribution of the noise in the stochastic gradient ∇ x (cid:101) U ( x k ) .

Original: B. L OCAL STABILITY VIA THE SCALABLE RANDOM - FIELD FUNCTION
Revised: B. L OCAL STABILITY VIA THE SCALABLE RANDOM - FIELD FUNCTION

Original: Now, we are ready to present our ﬁrst result.
Revised: Now, we are ready to present our ﬁrst result.

Original: Lemma 3 establishes a local stability condition for the non-linear mean-ﬁeld system of ICSGLD, which implies a potential convergence of θ k to a unique ﬁxed point that adapts to a wide energy range under mild assumptions.
Revised: Lemma 3 establishes a local stability condition for the non-linear mean-ﬁeld system of ICSGLD, which implies a potential convergence of θ k to a unique ﬁxed point that adapts to a wide energy range under mild assumptions.

Original: Lemma 3 (Local stability, restatement of Lemma 1) Assume Assumptions A1-A4 hold.
Revised: Lemma 3 (Local stability, restatement of Lemma 1) Assume Assumptions A1-A4 hold.

Original: Given a small enough learning rate (cid:15) , a large enough m and batch size n , and any θ ∈ (cid:101) Θ , where (cid:101) Θ is a small neighborhood of θ (cid:63) that contains (cid:98) θ (cid:63) , we have (cid:104) h
Revised: Given any small enough learning rate (cid:15) , a large enough m and batch size n , and any θ ∈ (cid:101) Θ , where (cid:101) Θ is a small neighborhood of θ (cid:63) that contains (cid:98) θ (cid:63) , we have (cid:104) h

Original: ( θ ) , θ − (cid:98) θ (cid:63) (cid:105) ≤ − φ (cid:107) θ − (cid:98) θ (cid:63) (cid:107) 2 , where (cid:98) θ (cid:63)
Revised: ( θ ) , θ − (cid:98) θ (cid:63) (cid:105) ≤ − φ (cid:107) θ − (cid:98) θ (cid:63) (cid:107) 2 , where (cid:98) θ (cid:63)

Original: = θ (cid:63) + O ( ε ) , ε = O (cid:0) sup x Var( ξ n ( x ))
Revised: = θ (cid:63) + O ( ε ) , ε = O (cid:0) sup x Var( ξ n ( x ))

Original: + (cid:15) + 1 m (cid:1) and θ (cid:63) =
Revised: + (cid:15) + 1 m (cid:1) and θ (cid:63) =

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Revised: [Equation]

Revised: [Table]

Original: , φ = inf θ min i (cid:98) Z − 1 ζ,θ ( i ) (cid:0) 1 − O ( ε ) (cid:1) > 0 , (cid:98)
Revised: [Equation] [Table] [Equation] φ = inf θ min i (cid:98) Z − 1 ζ,θ ( i ) (cid:0) 1 − O ( ε ) (cid:1) > 0 , (cid:98)

Original: Z ζ,θ ( i ) is deﬁned below Eq.(29), and ξ n ( x ) denotes the noise in the energy estimator (cid:101)
Revised: Z ζ,θ ( i ) is deﬁned below Eq.(29), and ξ n ( x ) denotes the noise in the energy estimator (cid:101)

Original: U ( x ) of batch size n and Var( · ) denotes the variance.
Revised: U ( x ) of batch size n and Var( · ) denotes the variance.

Original: Proof
Revised: Proof

Original: The random-ﬁeld function (cid:101)
Revised: The random-ﬁeld function (cid:101)

Original: [Equation]

Original: [Equation]

Original: x ) −

Original: [Equation]
Revised: [Equation]

Original: i )
Revised: i )

Original: [Equation]
Revised: [Equation]

Original: based on the stochastic energy estimator (cid:101) U ( x ) yields a biased estimator of H i ( θ , x )
Revised: based on the stochastic energy estimator (cid:101) U ( x ) yields a biased estimator of H i ( θ , x )

Original: = θ
Revised: = θ

Original: ( J ( x )) (cid:0) 1 i =
Revised: ( J ( x )) (cid:0) 1 i =

Original: J ( x ) − θ ( i ) (cid:1) for any i ∈ { 1 , 2 , . .
Revised: J ( x ) − θ ( i ) (cid:1) for any i ∈ { 1 , 2 , . .

Original: . , m } based on the exact energy partition function J ( · ) .
Revised: . , m } based on the exact energy partition function J ( · ) .

Original: By Lemma.4, we know that the bias caused by the stochastic energy is of order O
Revised: By Lemma.4, we know that the bias caused by the stochastic energy is of order O

Original: (Var( ξ n ( x ))) .
Revised: (Var( ξ n ( x ))) .

Original: Now we compute the mean-ﬁeld function h ( θ ) based on the measure (cid:36)
Revised: Now we compute the mean-ﬁeld function h ( θ ) based on the measure (cid:36)

Original: θ ( x ) simulated from SGLD:
Revised: θ ( x ) simulated from SGLD:

Original: h i ( θ ) = (cid:90)
Revised: h i ( θ ) = (cid:90)

Original: [Equation]
Revised: [Equation]

Original: 22) where (cid:36) θ is the invariant measure simulated via SGLD that approximates (cid:36) Ψ θ ( x ) .
Revised: ) where (cid:36) θ is the invariant measure simulated via SGLD that approximates (cid:36) Ψ θ ( x ) .

Original: (cid:36)
Revised: (cid:36)

Original: Ψ θ ( x ) and (cid:36) (cid:101)
Revised: Ψ θ ( x ) and (cid:36) (cid:101)

Original: Ψ θ ( x ) are two invariant measures that follow (cid:36)
Revised: Ψ θ ( x ) are two invariant measures that follow (cid:36)

Original: Ψ θ ( x ) ∝ π ( x ) Ψ ζ θ ( U ( x )) and (cid:36) (cid:101)
Revised: Ψ θ ( x ) ∝ π ( x ) Ψ ζ θ ( U [Equation] x )) and (cid:36) (cid:101)

Original: Ψ θ ( x ) ∝ π ( x ) (cid:101)
Revised: Ψ θ ( x ) ∝ π ( x ) (cid:101)

Original: Ψ ζ θ ( U ( x )) ; Ψ θ ( u ) and (cid:101)
Revised: Ψ ζ θ ( U ( x )) ; Ψ θ ( u ) and (cid:101)

Original: Ψ θ ( u ) are piecewise continuous and constant functions, respectively
Revised: Ψ θ ( u ) are piecewise continuous and constant functions, respectively

Original: [Equation]
Revised: [Equation]

Original: (i)
Revised: (i)

Original: For the ﬁrst term I 1 , we have
Revised: For the ﬁrst term I 1 , we have

Original: [Equation]
Revised: [Equation]

Original: − θ ( i ) m (cid:88)
Revised: θ ( i ) m (cid:88)

Original: [Equation]
Revised: [Equation]

Original: = (cid:90)
Revised: = (cid:90)

Original: X k
Revised: X k

Original: [Equation]
Revised: [Equation]

Original: ( x 
Revised: ( x ) θ

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original:  (cid:35) =[Equation] (cid:101)[Equation]
Revised: x (cid:35) = (cid:101)

Original: +
Revised: Z

Original: [Equation]
Revised: [Equation]

Original:  , θ (cid:34)(cid:82)X
Revised: +1 , θ (cid:34)(cid:82)

Original: [Equation]

Original: ( x
Revised: X i

Original: [Equation]
Revised: [Equation]

Original:  θ ζ − 1 ( i ) − θ ( i ) (cid:101)
Revised: ( x ) d x θ ζ − 1 ( i ) − θ ( i ) (cid:101)

Original: Z ζ, θ (cid:35) , (24) where (cid:101)
Revised: Z ζ, θ (cid:35) , (24) where (cid:101)

Original: Z ζ +1 , θ = (cid:80) mk
Revised: Z ζ +1 , θ = (cid:80) mk

Original: = (cid:82)
Revised: = (cid:82)

Original: X k π ( x ) d x θ ζ ( k ) denotes the normalizing constant of (cid:36) (cid:101)
Revised: X k π ( x ) d x θ ζ ( k ) denotes the normalizing constant of (cid:36) (cid:101)

Original: Ψ θ ( x ) .
Revised: Ψ θ ( x ) .

Original: The solution θ (cid:63) that solves (cid:82) X k π ( x ) d x θ ζ − 1 ( k ) − θ ( k ) (cid:101)
Revised: The solution θ (cid:63) that solves (cid:82) X k π ( x ) d x θ [Equation] 1 ( k ) − θ ( k ) (cid:101)

Original: Z ζ, θ = 0 for any k ∈ { 1 , 2 , · · · , m } satisﬁes θ (cid:63) ( k ) =
Revised: Z ζ, θ = 0 for any k ∈ { 1 , 2 , · · · , m } satisﬁes θ (cid:63) ( k ) =

Original: (cid:18) (cid:82)
Revised: (cid:18) (cid:82)

Original: X k π ( x ) d x (cid:101)
Revised: X k π ( x ) d x (cid:101)

Original: Z ζ, θ (cid:63) (cid:19) 1 ζ .
Revised: Z ζ, θ (cid:63) (cid:19) 1 ζ .

Original: Combining the deﬁnition of (cid:101)
Revised: Combining the deﬁnition of (cid:101)

Original: Z ζ, θ (cid:63) =
Revised: Z ζ, θ (cid:63) =

Original: (cid:80) mk 
Revised: (cid:80) mk =

Original: = (cid:82)
Revised: [Equation]

Original: X k π ( x ) d x θ ζ − 1 (cid:63) ( k ) , we have(cid:101)
Revised: X k π ( x ) d x θ ζ [Equation] 1 (cid:63) ( k ) , we have

Original: Z ζ, θ (cid:63) = m (cid:88) k = (cid:82) X k π ( x ) d x θ ζ − 1 (cid:63) ( k ) = m (cid:88) k = (cid:82) X k π ( x ) d x (cid:18) (cid:82) X k π ( x ) d x (cid:101)
Revised: [Equation]

Original: Z ζ, θ (cid:63) (cid:19) ζ − 1 ζ =(cid:101)
Revised: =

Original: Z ζ − 1 ζ ζ, θ (cid:63) m (cid:88)  k
Revised: [Equation] (cid:88) [Equation] k

Original: = (cid:82) X k π ( x ) d x (cid:16)(cid:82) X k π ( x ) d x (cid:17) ζ − 1 ζ = (cid:101) Z ζ − 1 ζ ζ, θ (cid:63) m (cid:88) k
Revised: [Equation]

Original: = (cid:18)(cid:90)
Revised: (cid:18)(cid:90)

Original: X k π ( x ) d x (cid:19) 1 ζ , which leads to (cid:101) Z ζ, θ (cid:63) = (cid:18)(cid:80) mk =1 (cid:16)(cid:82)
Revised: [Equation] , which leads to (cid:101) Z ζ, θ (cid:63) = (cid:18)(cid:80) mk =1 (cid:16)(cid:82)

Original: X k π ( x ) d x (cid:17) 1 ζ (cid:19) ζ .
Revised: X k π ( x ) d x (cid:17) 1 ζ [Equation] ζ .

Original: In other words, the mean-ﬁeld system without perturbations yields a unique solution θ (cid:63)
Revised: In other words, the mean-ﬁeld system without perturbations yields a unique solution θ (cid:63)

Original: [Equation]
Revised: [Equation]

Original: for any
Revised: for any

Original: [Equation]
Revised: [Equation]

Original: (ii)
Revised: (ii)

Original: For the second term I 2 , we have
Revised: For the second term I 2 , we have

Original: [Equation]
Revised: [Equation]

Original: where the result follows from the boundedness of H ( θ , x ) in (A1) and Lemma B4 (Deng et al., 2020b).
Revised: where the result follows from the boundedness of H ( θ , x ) in (A1) and Lemma B4 (Deng et al., 2020b).

Original: (iii)
Revised: (iii)

Original: For the last term I 3 , following Theorem 6 of Sato & Nakagawa (2014), we have for any ﬁxed θ ,
Revised: For the last term I 3 , following Theorem 6 of Sato & Nakagawa (2014), we have for any ﬁxed θ ,

Original: [Equation]
Revised: [Equation]

Original: Plugging Eq.(24), Eq.(25) and Eq.(26) into Eq.(22), we have
Revised: Plugging Eq.(24), Eq.(25) and Eq.(26) into Eq.(22), we have

Original: [Equation]
Revised: [Equation]

Original: ) where ˜ β i ( θ ) is a bounded term such that (cid:101)
Revised: ) where ˜ β i ( θ ) is a bounded term such that (cid:101)

Original: Z − 1 ζ +1 , θ ε ˜ β
Revised: Z − 1 ζ +1 , θ ε ˜ β

Original: i ( θ ) =
Revised: i ( θ ) =

Original: O (cid:0) Var( ξ n ( x ))
Revised: O (cid:0) Var( ξ n ( x ))

Original: + (cid:15) + 1 m (cid:1) , ε =
Revised: + (cid:15) + 1 m (cid:1) , ε =

Original: O (cid:0) sup x Var( ξ n ( x ))
Revised: O (cid:0) sup x Var( ξ n ( x ))

Original: + (cid:15) + 1 m (cid:1) and C θ = (cid:16) (cid:101)
Revised: + (cid:15) + 1 m (cid:1) and C θ = (cid:16) (cid:101)

Original: Z ζ, θ (cid:101)
Revised: Z ζ, θ [Equation]

Original: Z ζ, θ
Revised: Z ζ [Equation] θ

Original: [Equation]
Revised: [Equation]

Original: (cid:17) 1 ζ . 
Revised: 1 ζ . By the deﬁnition of (cid:101) Z ζ, θ = (cid:80) mk =1 (cid:82) X k π ( x ) d x θ [Equation] 1 ( k

Original: By the deﬁnition of (cid:101) Z ζ, θ = (cid:80) mk =1 (cid:82) X k π ( x ) d x θ ζ − 1 ( k ) , when ζ = 1 , C θ ≡ 1 for any θ ∈ Θ , which suggests that the stability condition doesn’t rely on the initialization of θ ; however, when ζ (cid:54) = 1 , C θ (cid:54) = 1 when θ (cid:54) = θ (cid:63) , we see that h
Revised: [Equation] , when ζ = 1 , C θ ≡ 1 for any θ ∈ Θ , which suggests that the stability condition doesn’t rely on the initialization of θ ; however, when ζ (cid:54) = 1 , C θ (cid:54) = 1 when θ (cid:54) = θ (cid:63) , we see that h

Original: i ( θ ) ∝ θ (cid:63) ( i ) ζ − ( θ ( i ) C θ ) ζ + perturbations is a non-linear mean-ﬁeld system and requires a proper initialization of θ ∈ (cid:101) Θ .
Revised: i ( θ ) ∝ θ (cid:63) ( i ) ζ − ( θ ( i ) C θ ) ζ + perturbations is a non-linear mean-ﬁeld system and requires a proper initialization of θ ∈ (cid:101) Θ .

Original: For any θ ∈ (cid:101) Θ ⊂ Θ being close enough to θ (cid:63) , there exists a Lipschitz constant L (cid:101) θ = sup i ≤ m, θ ∈ (cid:101) Θ | C θ (cid:63) − C
Revised: For any θ ∈ (cid:101) Θ ⊂ Θ being close enough to θ (cid:63) , there exists a Lipschitz constant L (cid:101) θ = sup i ≤ m, θ ∈ (cid:101) Θ | C θ (cid:63) − C

Original: θ
Revised: θ

Original: | | θ (cid:63) ( i ) − θ ( i ) | < ∞ .
Revised: | | θ (cid:63) ( i ) − θ ( i ) | < ∞ .

Original: By C θ (cid:63) = 1 , θ ( i ) ≤ 1 , and mean value theorem for some (cid:101) θ ( i ) ∈
Revised: By C θ (cid:63) = 1 , θ ( i ) ≤ 1 , and mean value theorem for some (cid:101) θ ( i ) ∈

Original: [ θ ( i ) , θ (cid:63) ( i )] , we have
Revised: [ θ ( i ) , θ (cid:63) ( i )] , we have

Original: [Equation]
Revised: [Equation]

Original: Combining Eq.(27) and Eq.(28), we have
Revised: Combining Eq.(27) and Eq.(28), we have

Original: [Equation]
Revised: [Equation]

Original: where (cid:98) Z 
Revised: where (cid:98) Z − 1 ζ,θ ( i

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: Z − 1 ζ + [Equation] θ (cid:101)
Revised: Z − 1 ζ +1 , θ (cid:101)

Original: Z ζ
Revised: Z ζ

Original: [Equation]
Revised: [Equation]

Original: θ [Equation] ζ ( (cid:101) θ
Revised: θ (cid:63) ζ ( (cid:101) θ

Original: ( i ) C (cid:101) θ ) ζ − 1 (1+ L (cid:101) θ )
Revised: ( i ) C (cid:101) θ ) ζ − 1 (1+ L (cid:101) θ )

Original: θ ζ − 1 ( i ) ; β i ( θ ) is some bounded term such that β i ( θ ) ≤ ˜ β
Revised: θ ζ − 1 ( i ) ; β i ( θ ) is some bounded term such that β i ( θ ) ≤ ˜ β

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: Next, we apply the perturbation theory to solve the ODE system with small disturbances (Weinhart et al., 2010) and obtain the equilibrium (cid:98) θ (cid:63) , where ε β ( (cid:98) θ (cid:63) )
Revised: Next, we apply the perturbation theory to solve the ODE system with small disturbances (Weinhart et al., 2010) and obtain the equilibrium (cid:98) θ (cid:63) , where ε β ( (cid:98) θ (cid:63) )

Original: + θ
Revised: + θ

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original: θ
Revised: θ

Original: [Equation]
Revised: [Equation]

Original: = 0 , to the mean-ﬁeld equation
Revised: = 0 , to the mean-ﬁeld equation

Original: h
Revised: h

Original: i ( θ ) such that
Revised: i ( θ ) such that

Original: [Equation]
Revised: [Equation]

Original: where a smoothness condition clearly holds for the β ( · ) function.
Revised: where a smoothness condition clearly holds for the β ( · ) function.

Original: Given a positive deﬁnite Lyapunov function V ( θ ) = 12 (cid:107) (cid:98) θ
Revised: Given a positive deﬁnite Lyapunov function V ( θ ) = 12 (cid:107) (cid:98) θ

Original: [Equation]
Revised: [Equation]

Original: − θ (cid:107) 2 , the mean-ﬁeld system h ( θ ) = (cid:98) 
Revised: − θ (cid:107) 2 , the mean-ﬁeld system h ( θ ) = (cid:98) Z − 1 ζ,θ

Original: [Equation]
Revised: [Equation]

Original: − 1 ζ,θ
Revised: i )

Original: [Equation]
Revised: [Equation]

Original:  β
Revised: ε β

Original: [Equation]
Revised: [Equation]

Original: θ )
Revised: θ )

Original: + θ
Revised: + θ

Original: [Equation]
Revised: [Equation]

Original: − θ )
Revised: − θ )

Original: = (cid:98) Z − 1 ζ,θ ( i ) (1 −
Revised: = (cid:98) Z − 1 ζ,θ ( i ) (1 −

Original: O ( ε ))
Revised: O ( ε ))

Original: ( (cid:98) θ (cid:63) − θ ) for i ∈ { 1 , 2 , · · · , m } enjoys the following property
Revised: ( (cid:98) θ (cid:63) − θ ) for i ∈ { 1 , 2 , · · · , m } enjoys the following property

Original: [Equation]
Revised: [Equation]

Original: where φ = inf θ min i (cid:98) Z − 1 ζ,θ ( i ) (cid:0) 1 − O ( ε ) (cid:1) > 0 given the compactness assumption A1 and a small enough ε = O (cid:0) sup x Var( ξ n ( x ))
Revised: where φ = inf θ min i (cid:98) Z − 1 ζ,θ ( i ) (cid:0) 1 − O ( ε ) (cid:1) > 0 given the compactness assumption A1 and a small enough ε = O (cid:0) sup x Var( ξ n ( x ))

Original: + (cid:15) + 1 m (cid:1) .
Revised: + (cid:15) + 1 m (cid:1) .

Original: Remark 1 The newly proposed random-ﬁeld function Eq.(17) may sacriﬁce the global stability by including an approximately linear mean-ﬁeld system Eq.(29) instead of a linear stable system (see formula (15) in Deng et al.
Revised: Remark 1 The newly proposed random-ﬁeld function Eq.(17) may sacriﬁce the global stability by including an approximately linear mean-ﬁeld system Eq.(29) instead of a linear stable system (see formula (15) in Deng et al.

Original: (2020b)).
Revised: (2020b)).

Original: The advantage, however, is that such a mechanism facilitates the estimation of θ (cid:63) .
Revised: The advantage, however, is that such a mechanism facilitates the estimation of θ (cid:63) .

Original: We emphasize that the original energy probability in each partition (cid:8) (cid:82)
Revised: We emphasize that the original energy probability in each partition (cid:8) (cid:82)

Original: X k π ( x ) d x (cid:9) m k
Revised: X k π ( x ) d x (cid:9) m k

Original: =

Revised: =

Original: (Deng et al., 2020b) may be very difﬁcult to estimate for big data problems.
Revised: (Deng et al., 2020b) may be very difﬁcult to estimate for big data problems.

Original: By contrast, the estimation of (cid:8)(cid:0) (cid:82) X k π ( x ) d x (cid:1) 1 ζ (cid:9) m k
Revised: By contrast, the estimation of (cid:8)(cid:0) (cid:82) X k π ( x ) d x (cid:1) 1 ζ (cid:9) m k

Original: =1 becomes much easier given a proper ζ > 0 .
Revised: =1 becomes much easier given a proper ζ > 0 .

Original: Technical lemmas
Revised: Technical lemmas

Original: Lemma 

Revised: Lemma

Original: The stochastic energy estimator (cid:101) U ( x ) leads to a controllable bias in the random-ﬁeld function.
Revised: The stochastic energy estimator (cid:101) U ( x ) leads to a controllable bias in the random-ﬁeld function.

Original: [Equation]
Revised: [Equation]

Original: where the expectation E [ · ] is taken with respect to the random noise in the stochastic energy estimator of (cid:101) U ( · ) .
Revised: where the expectation E [ · ] is taken with respect to the random noise in the stochastic energy estimator of (cid:101) U ( · ) .

Original: Proof
Revised: Proof

Original: Denote the noise in the stochastic energy estimator by ξ ( x ) , such that (cid:101) U ( · ) =
Revised: Denote the noise in the stochastic energy estimator by ξ ( x ) , such that (cid:101) U ( · ) =

Original: U ( · ) + ξ ( · ) .
Revised: U ( · ) + ξ ( · ) .

Original: Recall that (cid:101)
Revised: Recall that (cid:101)

Original: H i ( θ , x )
Revised: H i ( θ , x )

Original: = θ ( J (cid:101)
Revised: = θ ( J (cid:101)

Original: U ( x ))
Revised: U ( x ))

Original: (cid:16) 1 i = J (cid:101)
Revised: (cid:16) 1 i = J (cid:101)

Original: U ( x ) − θ ( i ) (cid:17) and J (cid:101)
Revised: U ( x ) − θ ( i ) (cid:17) and J (cid:101)

Original: U ( x ) ∈ { 1 , 2 , · · · , m } satisﬁes u J (cid:101) U ( x ) − 1 < N n (cid:101) U ( x )
Revised: U ( x ) ∈ { 1 , 2 , · · · , m } satisﬁes u J (cid:101) U ( x ) − 1 < N n (cid:101) U ( x )

Original: ≤ u J (cid:101) U ( x ) for a set of energy partitions { u i } m i =0 .
Revised: ≤ u J (cid:101) U ( x ) for a set of energy partitions { u i } m i =0 .

Original: We can interpret (cid:101) H i ( θ , x ) as a non-linear transformation Φ that maps (cid:101) U ( x ) to (0 , 1) .
Revised: We can interpret (cid:101) H i ( θ , x ) as a non-linear transformation Φ that maps (cid:101) U ( x ) to (0 , 1) .

Original: Similarly, H i ( θ , x ) maps U ( x ) to (0 , 1) .
Revised: Similarly, H i ( θ , x ) maps U ( x ) to (0 , 1) .

Original: In what follows, the bias of random-ﬁeld function is upper bounded as follows
Revised: In what follows, the bias of random-ﬁeld function is upper bounded as follows

Original: [Equation]
Revised: [Equation]

Original: where the second equality follows from Taylor expansion for some energy u and the third equality follows because the stochastic energy estimator is unbiased; Φ (cid:48)
Revised: where the second equality follows from Taylor expansion for some energy u and the third equality follows because the stochastic energy estimator is unbiased; Φ (cid:48)

Original: ( U ( x ))
Revised: ( U ( x ))

Original: =
Revised: =

Original: O
Revised: O

Original: [Equation]
Revised: [Equation]

Original: x
Revised: x

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original: ) is clearly bounded due to the deﬁnition of θ ; a similar conclusion also applies to Φ (cid:48)(cid:48) ( · ) .
Revised: ) is clearly bounded due to the deﬁnition of θ ; a similar conclusion also applies to Φ (cid:48)(cid:48) ( · ) .

Original: B. C ONVERGENCE OF THE SELF - ADAPTING PARAMETERS
Revised: B. C ONVERGENCE OF THE SELF - ADAPTING PARAMETERS

Original: The following is a restatement of Lemma 3.2 of Raginsky et al.
Revised: The following is a restatement of Lemma 3.2 of Raginsky et al.

Original: (2017), which holds for any θ in the compact space Θ .
Revised: (2017), which holds for any θ in the compact space Θ .

Original: Lemma 5 (Uniform L 2 bounds) Assume Assumptions A1, A3 and A4 hold.
Revised: Lemma 5 (Uniform L 2 bounds) Assume Assumptions A1, A3 and A4 hold.

Original: We have a bounded second moment sup k ≥ 1 E [ (cid:107) x k (cid:107) 2 ] < ∞ given a small enough learning rate.
Revised: We have a bounded second moment sup k ≥ 1 E [ (cid:107) x k (cid:107) 2 ] < ∞ given a small enough learning rate.

Original: The following lemma justiﬁes the regularity properties of Poisson’s equation, which is crucial in controlling the perturbations through the stochastic approximation process.
Revised: The following lemma justiﬁes the regularity properties of Poisson’s equation, which is crucial in controlling the perturbations through the stochastic approximation process.

Original: The ﬁrst version was proposed in Lemma B2 of Deng et al.
Revised: The ﬁrst version was proposed in Lemma B2 of Deng et al.

Original: (2020b).
Revised: (2020b).

Original: Now we give a more detailed proof by utilizing a
Revised: Now we give a more detailed proof by utilizing a

Original: Lyapunov function V ( x )
Revised: Lyapunov function V ( x )

Original: = 1 + x 2 and Lemma 5.
Revised: = 1 + x 2 and Lemma 5.

Original: Lemma 6 (Solution of Poisson’s equation) Assume that Assumptions A1-A4 hold.
Revised: Lemma 6 (Solution of Poisson’s equation) Assume that Assumptions A1-A4 hold.

Original: There is a solution µ θ ( · ) on X to the Poisson’s equation
Revised: There is a solution µ θ ( · ) on X to the Poisson’s equation

Original: [Equation]
Revised: [Equation]

Original: Furthermore, there exists a constant C such that for all θ , θ (cid:48)
Revised: Furthermore, there exists a constant C such that for all θ , θ (cid:48)

Original: ∈ Θ
Revised: ∈ Θ

Original: [Equation]
Revised: [Equation]

Original: Proof
Revised: Proof

Original: The existence and the regularity property of Poisson’s equation can be used to control the perturbations.
Revised: The existence and the regularity property of Poisson’s equation can be used to control the perturbations.

Original: The key of the proof lies in verifying drift conditions proposed in Section 6 of Andrieu et al.
Revised: The key of the proof lies in verifying drift conditions proposed in Section 6 of Andrieu et al.

Original: (DRI)
Revised: (DRI)

Original: By the smoothness assumption A2, we have that U ( x ) is continuously differentiable almost everywhere.
Revised: By the smoothness assumption A2, we have that U ( x ) is continuously differentiable almost everywhere.

Original: By the dissipative assumption A3 and Theorem 2.1 (Roberts & Tweedie, 1996), we can show that the discrete dynamics system is irreducible and aperiodic.
Revised: By the dissipative assumption A3 and Theorem 2.1 (Roberts & Tweedie, 1996), we can show that the discrete dynamics system is irreducible and aperiodic.

Original: Now consider a Lyapunov function V = 1 + (cid:107) x (cid:107) 2 and any compact subset K ⊂ Θ , the drift conditions are veriﬁed as follows:(DRI
Revised: Now consider a Lyapunov function V = 1 + (cid:107) x (cid:107) 2 and any compact subset K ⊂ Θ , the drift conditions are veriﬁed as follows:

Original:  Given small enough learning rates { (cid:15) k } k ≥ 1 , the smoothness assumption A2, and the dissipative assumption A3, applyingCorollary 7.5 (Mattingly et al., 2002) yields the minorization condition for the CSGLD algorithm, i.e. there exists η > 0 , a measure ν , and a set C such that ν ( C )
Revised: (DRI1) Given small enough learning rates { (cid:15) k } k ≥ 1 , the smoothness assumption A2, and the dissipa tive assumption A applying

Revised: Corollary

Revised: Mattingly

Original: = 1 .
Revised: et al., yields the minorization condition for the CSGLD algorithm, i.e. there exists

Revised: η > 0 , a measure ν , and a set C such that ν ( C ) = 1 .

Original: Moreover, we have
Revised: Moreover, we have

Original: [Equation]
Revised: [Equation]

Original: where P θ ( x , y ) :
Revised: where P θ ( x , y ) :

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: | x (cid:3) denotes the transition kernel based on CSGLD with the parameter θ ∈ K and a learning rate (cid:15) , in addition, the expectation is taken over the adaptive gradient ∇ x (cid:101) L ( x , θ ) in Eq.(15).
Revised: | x (cid:3) denotes the transition kernel based on CSGLD with the parameter θ ∈ K and a learning rate (cid:15) , in addition, the expectation is taken over the adaptive gradient ∇ x (cid:101) L ( x , θ ) in Eq.(15).

Original: Using Assumption A1-A4, we can prove the uniform L upper bound by following Lemma 3.2 (Raginsky et al., 2017).
Revised: Using Assumption A1-A4, we can prove the uniform L upper bound by following Lemma 3.2 (Raginsky et al., 2017).

Original: Further, by Theorem 7.2 (Mattingly et al., 2002), there exist ˜ α ∈ (0 , 1) and ˜ β ≥ 0 such that[Equation] II
Revised: Further, by Theorem 7.2 (Mattingly et al., 2002), there exist ˜ α ∈ (0 , 1) and ˜ β ≥ 0 such that

Original: [Equation]
Revised: [Equation]

Original: Consider a Lyapunov function V = 1 + (cid:107) x (cid:107) 2 and a constant κ = ˜
Revised: Consider a Lyapunov function V = 1 + (cid:107) x (cid:107) 2 and a constant κ = ˜

Original: α + ˜ β , it yields that
Revised: α + ˜ β , it yields that

Original: [Equation]
Revised: [Equation]

Original: Now we have veriﬁed the ﬁrst condition (DRI1) by checking conditions (I),(II), and (III), (DRI2)
Revised: Now we have veriﬁed the ﬁrst condition (DRI1) by checking conditions (I),(II), and (III), (DRI2)

Original: In what follows, we check the boundedness and Lipshitz conditions on the random-ﬁeld function (cid:101)
Revised: In what follows, we check the boundedness and Lipshitz conditions on the random-ﬁeld function (cid:101)

Original: H ( θ , x ) , where each subcomponent is deﬁend as (cid:101)H i [Equation] =
Revised: H ( θ , x ) , where each subcomponent is deﬁend as (cid:101)

Original: [Equation]
Revised: [Equation]

Original: Recall that V = 

Revised: Recall that V =

Original: +
Revised: +

Original: (cid:107) x (cid:107) 2 , the compactness assumption A1 directly leads to
Revised: (cid:107) x (cid:107) 2 , the compactness assumption A1 directly leads to

Original: [Equation]
Revised: [Equation]

Original: For any θ 1 , θ 2 ∈ K and a ﬁxed x ∈ X , it sufﬁces for us to solely verify the i -th index, which is the index that maximizes | θ 1 ( i ) − θ 2 ( i ) | , then | 
Revised: For any θ 1 , θ 2 ∈ K and a ﬁxed x ∈ X , it sufﬁces for us to solely verify the i -th index, which is the index that maximizes | θ 1 ( i ) − θ 2 ( i ) | , then | [Equation]

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds since θ ( i ) ∈ (0 , 1] for any i ≤ m .
Revised: where the last inequality holds since θ ( i ) ∈ (0 , 1] for any i ≤ m .

Original: (DRI3) We proceed to verify the smoothness of the transitional kernel P θ ( x , y ) with respect to θ .
Revised: (DRI3) We proceed to verify the smoothness of the transitional kernel P θ ( x , y ) with respect to θ .

Original: For any θ 1 , θ 2 ∈ K and ﬁxed x and y , we have
Revised: For any θ 1 , θ 2 ∈ K and ﬁxed x and y , we have

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst inequality (up to a ﬁnite constant) follows by (cid:107) e x − e y (cid:107) (cid:46) (cid:107) x − y (cid:107) for any x , y in a compact space; the last inequality follows by the deﬁnition of the adaptive gradient in Eq.(15) and (cid:107) log( x ) − log( y ) (cid:107) (cid:46) (cid:107) x − y (cid:107) by the compactness assumption A1.
Revised: where the ﬁrst inequality (up to a ﬁnite constant) follows by (cid:107) e x − e y (cid:107) (cid:46) (cid:107) x − y (cid:107) for any x , y in a compact space; the last inequality follows by the deﬁnition of the adaptive gradient in Eq.(15) and (cid:107) log( x ) − log( y ) (cid:107) (cid:46) (cid:107) x − y (cid:107) by the compactness assumption A1.

Original: For f : X → R d , deﬁne the norm (cid:107) f (cid:107) V = sup x ∈X | f ( x ) | V ( x ) .
Revised: For f : X → R d , deﬁne the norm (cid:107) f (cid:107) V = sup x ∈X | f ( x ) | V ( x ) .

Original: Following the same technique proposed in Liang et al.
Revised: Following the same technique proposed in Liang et al.

Original: (2007) (page 319), we can verify the last drift condition
Revised: (2007) (page 319), we can verify the last drift condition

Original: [Equation]
Revised: [Equation]

Original: Having conditions (I), (II), · · · and (VI) veriﬁed, we are now able to prove the drift conditions proposed in Section 6 of Andrieu et al.
Revised: Having conditions (I), (II), · · · and (VI) veriﬁed, we are now able to prove the drift conditions proposed in Section 6 of Andrieu et al.

Original: Before we present the L 2 convergence of θ k , we make some extra assumptions on the step size.
Revised: Before we present the L 2 convergence of θ k , we make some extra assumptions on the step size.

Original: Assumption A5 (Learning rate and step size)
Revised: Assumption A5 (Learning rate and step size)

Original: The learning rate { (cid:15) k } k ∈ N is a positive nonincreasing sequence of real numbers satisfying the conditions
Revised: The learning rate { (cid:15) k } k ∈ N is a positive nonincreasing sequence of real numbers satisfying the conditions

Original: [Equation]
Revised: [Equation]

Original: The step size { ω k } k ∈ N is a positive non-increasing sequence of real numbers such that
Revised: The step size { ω k } k ∈ N is a positive non-increasing sequence of real numbers such that

Original: [Equation]
Revised: [Equation]

Original: A practical strategy is to set ω k :
Revised: A practical strategy is to set ω k :

Original: = O ( k − α )
Revised: = O ( k − α )

Original: to satisfy the above conditions for any α ∈ (0 . 5 , 1) .
Revised: to satisfy the above conditions for any α ∈ (0 . 5 , 1] .

Original: The following is an application of Theorem 24 (page 246) (Benveniste et al., 1990) given stability conditions (Lemma 3).
Revised: The following is an application of Theorem 24 (page 246) (Benveniste et al., 1990) given stability conditions (Lemma 3).

Original: Lemma 7 ( L 2 convergence rate, restatement of Lemma 2) Assume Assumptions
Revised: Lemma 7 ( L 2 convergence rate, restatement of Lemma 2) Assume Assumptions

Original: A 1 - A 5 hold.
Revised: A 1 - A 5 hold.

Original: For any θ 0 ∈ (cid:101) Θ ⊂ Θ , a large m , small learning rates { (cid:15) k } ∞ k =1 , and step sizes { ω k } ∞ k =1 , { θ k } ∞ k = converges to (cid:98) θ (cid:63) , where (cid:98) θ (cid:63) =
Revised: For any θ 0 ∈ (cid:101) Θ ⊂ Θ , a large m , small learning rates { (cid:15) k } ∞ k =1 , and step sizes { ω k } ∞ k =1 , { θ k } ∞ k = converges to (cid:98) θ (cid:63) , where (cid:98) θ (cid:63) =

Original: θ (cid:63) + O (cid:0) sup x Var( ξ n ( x ))
Revised: θ (cid:63) + O (cid:0) sup x Var( ξ n ( x ))

Original: + sup k ≥ k 0 (cid:15) k
Revised: + sup k ≥ k 0 (cid:15) k

Original: + 1 m (cid:1) for some k 0 ,
Revised: + 1 m (cid:1) for some k 0 ,

Original: such that
Revised: such that

Original: [Equation]
Revised: [Equation]

Original: The theoretical novelty is that we treat the biased (cid:98) θ (cid:63) as the equilibrium of the continuous system instead of analyzing how far we are away from θ (cid:63) in all aspects as in Theorem 1 (Deng et al., 2020b).
Revised: The theoretical novelty is that we treat the biased (cid:98) θ (cid:63) as the equilibrium of the continuous system instead of analyzing how far we are away from θ (cid:63) in all aspects as in Theorem 1 (Deng et al., 2020b).

Original: This enables us to directly apply Theorem 24 (page 246).
Revised: This enables us to directly apply Theorem 24 (page 246).

Original: Nevertheless, it can be interpreted as a special case of Theorem 1 (Deng et al., 2020b) except that there are no perturbation terms and the equilibrium is (cid:98) θ (cid:63) instead of θ (cid:63) .
Revised: Nevertheless, it can be interpreted as a special case of Theorem 1 (Deng et al., 2020b) except that there are no perturbation terms and the equilibrium is (cid:98) θ (cid:63) instead of θ (cid:63) .

Original: C G AUSSIAN APPROXIMATION C. P RELIMINARY : SUFFICIENT CONDITIONS FOR WEAK CONVERGENCE
Revised: C G AUSSIAN APPROXIMATION C. P RELIMINARY : SUFFICIENT CONDITIONS FOR WEAK CONVERGENCE

Original: To formally prove the asymptotic normality of the stochastic approximation process ω − 1 / 2 k ( θ k − (cid:98) θ (cid:63) ) , we ﬁrst lay out a preliminary result (Theorem 1 of Pelletier (1998)) that provides sufﬁcient conditions to guarantee the weak convergence.
Revised: To formally prove the asymptotic normality of the stochastic approximation process ω − 1 / 2 k ( θ k − (cid:98) θ (cid:63) ) , we ﬁrst lay out a preliminary result (Theorem 1 of Pelletier (1998)) that provides sufﬁcient conditions to guarantee the weak convergence.

Original: Lemma 8 (Sufﬁcient Conditions) Consider a stochastic algorithm as follows
Revised: Lemma 8 (Sufﬁcient Conditions) Consider a stochastic algorithm as follows

Original: [Equation]
Revised: [Equation]

Original: where (cid:101)
Revised: where (cid:101)

Original: ν k +1 denotes a perturbation and e k
Revised: ν k +1 denotes a perturbation and e k

Original: +1 is a random noise.
Revised: +1 is a random noise.

Original: Given three conditions ( C1 ), ( C2 ), and ( C3 ) deﬁned below, we have the desired weak convergence result
Revised: Given three conditions ( C1 ), ( C2 ), and ( C3 ) deﬁned below, we have the desired weak convergence result

Original: [Equation]
Revised: [Equation]

Original: where Σ =
Revised: where Σ =

Original: [Equation]
Revised: [Equation]

Original: {+∞ 
+} eth
Revised: {+∞ 
+} e

Original: [Equation]
Revised: [Equation]

Original: ◦ R ◦ e th
Revised: ◦ R ◦ e th

Original: [Equation]
Revised: [Equation]

Original: dt , R denotes the limiting covariance of the martingale lim k →∞ E [ e k +1 e (cid:62) k
Revised: dt , R denotes the limiting covariance of the martingale lim k →∞ E [ e k +1 e (cid:62) k

Original: +1 |F k ] and F k is the σ -algebra of the events up to iteration k , h θ (cid:63) 
Revised: +1 |F k ] and F k is the σ -algebra of the events up to iteration k , h θ (cid:63) =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (C1)
Revised: (C1)

Original: There exists an equilibrium point (cid:98) θ (cid:63) and a stable matrix h θ
Revised: There exists an equilibrium point (cid:98) θ (cid:63) and a stable matrix h θ

Original: (cid:63) :
Revised: (cid:63) :=

Original: =
Revised: h

Original: h θ ( (cid:98) θ (cid:63) 
Revised: θ ( (cid:98) θ (cid:63) )

Original: ∈
Revised: ∈

Original: R m × m such that forany θ ∈ {
Revised: R m × m such that for

Original:  θ : [Equation] θ − [Equation]
Revised: any θ ∈ { θ : (cid:107) θ − (cid:98) θ (cid:63) (cid:107) ≤ (cid:102)

Original: θ
Revised: M

Original: [Equation] (cid:107)
Revised: } for some (cid:102) M >

Original: ≤
Revised: , the

Original: (cid:102) M } for some (cid:102) M > 0 , the mean-ﬁeld function h : R m → R m satisﬁes
Revised: mean-ﬁeld function h : R m → R m satisﬁes

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (C2)
Revised: (C2)

Original: The step size ω k decays with an order α ∈ (0 , 1] such that ω k
Revised: The step size ω k decays with an order α ∈ (0 , 1] such that ω k

Original: = O 
Revised: = O (

Original: [Equation]

Original: k − [Equation]
Revised: k − α )

Original:  (C3) Assumptions on the disturbances .
Revised: . (C3) Assumptions on the disturbances .

Original: There exists constants (cid:102) M > 0 and (cid:101) α > 2 such that
Revised: There exists constants (cid:102) M > 0 and (cid:101) α > 2 such that

Original: [Equation]
Revised: [Equation]

Original: Remark 

Revised: Remark

Original: By the deﬁnition of the mean-ﬁeld function h ( θ ) in Eq.(27), it is easy to verify the condition C1.
Revised: By the deﬁnition of the mean-ﬁeld function h ( θ ) in Eq.(27), it is easy to verify the condition C1.

Original: Moreover, Assumption A5 also fulﬁlls the condition C2.
Revised: Moreover, Assumption A5 also fulﬁlls the condition C2.

Original: Then, the proof hinges on the veriﬁcation of the condition C3.
Revised: Then, the proof hinges on the veriﬁcation of the condition C3.

Original: C. 
Revised: C. P RELIMINARY : CONVERGENCE OF THE COVARIANCE ESTIMATORS

Original: P RELIMINARY : CONVERGENCE OF THE COVARIANCE ESTIMATORS

Original: In particular, to verify the condition E (cid:2) e k +1 e (cid:62) k +1 |F k (cid:3) 1 {(cid:107) θ − (cid:98) θ (cid:63) (cid:107)≤ (cid:102) M } → R , , we study the convergence of the empirical sample mean E
Revised: In particular, to verify the condition E (cid:2) e k +1 e (cid:62) k +1 |F k (cid:3) 1 {(cid:107) θ − (cid:98) θ (cid:63) (cid:107)≤ (cid:102) M } → R , , we study the convergence of the empirical sample mean E

Original: [ f ( x k )] for a test function f to the posterior expectation ¯ f = (cid:82)
Revised: [ f ( x k )] for a test function f to the posterior expectation ¯ f = (cid:82)

Original: X f ( x ) (cid:36) (cid:98) θ (cid:63) ( x )( d x ) .
Revised: X f ( x ) (cid:36) (cid:98) θ (cid:63) ( x )( d x ) .

Original: Poisson’s equation is often used to characterize the ﬂuctuation between f ( x ) and ¯ f :
Revised: Poisson’s equation is often used to characterize the ﬂuctuation between f ( x ) and ¯ f :

Original: [Equation]
Revised: [Equation]

Original: 35) where L refers to an inﬁnitesimal generator and g ( x ) denotes the solution of the Poisson’s equation.
Revised: 35) where L refers to an inﬁnitesimal generator and g ( x ) denotes the solution of the Poisson’s equation.

Original: Similar to the proof of Lemma 6, the existence of the solution of the Poisson’s equation has been established in (Mattingly et al., 2002; Vollmer et al., 2016).
Revised: Similar to the proof of Lemma 6, the existence of the solution of the Poisson’s equation has been established in (Mattingly et al., 2002; Vollmer et al., 2016).

Original: Moreover, the perturbations of E [ f ( x k )] − ¯ f are properly bounded given regularity properties for g ( x ) , where the 0-th, 1st, and 2nd order of the regularity properties has been established in Erdogdu et al.
Revised: Moreover, the perturbations of E [ f ( x k )] − ¯ f are properly bounded given regularity properties for g ( x ) , where the 0-th, 1st, and 2nd order of the regularity properties has been established in Erdogdu et al.

Original: The following result helps us to identify the convergence of the covariance estimators, which is adapted from Theorem 5 (Chen et al., 2015) with decreasing learning rates { (cid:15) k } k ≥ 1 .
Revised: The following result helps us to identify the convergence of the covariance estimators, which is adapted from Theorem 5 (Chen et al., 2015) with decreasing learning rates { (cid:15) k } k ≥ 1 .

Original: The gradient biases from Theorem 2 (Chen et al., 2015) are also included to handle the adaptive biases.
Revised: The gradient biases from Theorem 2 (Chen et al., 2015) are also included to handle the adaptive biases.

Original: Lemma 9 (Convergence of the Covariance Estimators) Suppose Assumptions A1-A5 hold.
Revised: Lemma 9 (Convergence of the Covariance Estimators) Suppose Assumptions A1-A5 hold.

Original: For any θ 0 ∈ (cid:101) Θ ⊂ Θ , a large m , small learning rates { (cid:15) k } ∞ k =1 , step sizes { ω k } ∞ k =1 and any bounded function f , we have
Revised: For any θ 0 ∈ (cid:101) Θ ⊂ Θ , a large m , small learning rates { (cid:15) k } ∞ k =1 , step sizes { ω k } ∞ k =1 and any bounded function f , we have

Original: [Equation]
Revised: [Equation]

Original: where (cid:36) (cid:98) θ (cid:63) ( x ) is the invariant measure simulated via SGLD that approximates (cid:36) (cid:101)
Revised: where (cid:36) (cid:98) θ (cid:63) ( x ) is the invariant measure simulated via SGLD that approximates (cid:36) (cid:101)

Original: Ψ θ (cid:63) ( x ) ∝ π ( x ) θ ζ(cid:63)
Revised: Ψ θ (cid:63) ( x ) ∝ π ( x ) θ ζ(cid:63)

Original: ( J ( x )) .
Revised: ( J ( x )) .

Original: Proof
Revised: Proof

Original: We study the single-chain CSGLD and reformulate the adaptive algorithm as follows:
Revised: We study the single-chain CSGLD and reformulate the adaptive algorithm as follows:

Original: [Equation]
Revised: [Equation]

Original: where ∇ x (cid:101) L ( x , θ ) = Nn (cid:104) 1 + ζτ ∆ u (log θ ( J ( x )) − log θ ( [Equation] J ( x ) − 1) ∨ 1))
Revised: where ∇ x (cid:101) L ( x , θ ) = Nn (cid:104) 1 + ζτ ∆ u (log θ ( J ( x )) − log θ (( J ( x ) − 1) ∨ 1))

Original: (cid:105) ∇ x (cid:101) U ( x ) ‡ , ∇ x (cid:101) L ( x , θ )
Revised: (cid:105) ∇ x (cid:101) U ( x ) ‡ , ∇ x (cid:101) L ( x , θ )

Original: is deﬁned in Section B.1 and the bias term is given by Υ( x k , θ k ) =
Revised: is deﬁned in Section B.1 and the bias term is given by Υ( x k , θ k ) =

Original: ∇ x (cid:101) L ( x k , θ k ) − ∇ x (cid:101) L ( x k , (cid:98) θ (cid:63) ) .
Revised: ∇ x (cid:101) L ( x k , θ k ) − ∇ x (cid:101) L ( x k , (cid:98) θ (cid:63) ) .

Original: Then, by Jensen’s inequality and Lemma 7, we have
Revised: Then, by Jensen’s inequality and Lemma 7, we have

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: Combining Eq.(36) and Theorem 5 (Chen et al., 2015)
Revised: Combining Eq.(36) and Theorem 5 (Chen et al., 2015)

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: f

Original: [Equation]

Original: [Equation]

Original: where the last argument directly follows from the conditions on learning rates and step sizes in
Revised: where the last argument directly follows from the conditions on learning rates and step sizes in

Original: Assumption A
Revised: Assumption A

Original: [Equation]
Revised: [Equation]

Original: C. P ROOF OF T HEOREM
Revised: C. P ROOF OF T HEOREM

Original: Recall that the stochastic approximation based on a single process follows from
Revised: Recall that the stochastic approximation based on a single process follows from

Original: [Equation]
Revised: [Equation]

Original: where the second equality holds from the solution of Poisson’s equation in Eq.(31).
Revised: where the second equality holds from the solution of Poisson’s equation in Eq.(31) [Equation]

Original: We denote ¨ 
Revised: We denote ¨ θ k

Revised: = θ k + ω k

Revised: +

Revised: [Equation]

Revised: Π θ

Original: [Equation]
Revised: [Equation]

Original: θ k ( x k ) .
Revised: k ( x k ) .

Original: Adding ω
Revised: Adding ω

Original: k +2 Π
Revised: k +2 Π

Original: θ k +1 µ[-θ k +
-]
Revised: θ k +1 µ[-θ k +
-]

Original:  ( x k +1 ) on both sides of Eq.(37), we have[Equation]
Revised: [Equation] k +1 ( x k +1 ) on both sides of Eq.(37), we have

Original: [Equation]
Revised: [Equation]

Original: 38) where ˜ ν
Revised: 38) where ˜ ν

Original: k +1 =
Revised: k +1 =

Revised: [Equation]

Original: ν k +

Revised: k +

Original: + h ( θ k ) − h
Revised: + h ( θ k ) − h

Original: (¨ θ k ) .
Revised: (¨ θ k ) .

Original: Next, we proceed to verify the conditions in C3 .
Revised: Next, we proceed to verify the conditions in C3 .

Original: (I)
Revised: (I)

Original: By the martingale difference property of { e k } and the compactness assumption A1, we know that for any (cid:101) α >
Revised: By the martingale difference property of { e k } and the compactness assumption A1, we know that for any (cid:101) α >

Original: [Equation]
Revised: [Equation]

Original: (II)
Revised: (II)

Original: By the deﬁnition of h ( θ k ) in Eq.(27), we can easily check that h ( θ k ) is Lipschitz continuous in a neighborhood of (cid:98) θ (cid:63) .
Revised: By the deﬁnition of h ( θ k ) in Eq.(27), we can easily check that h ( θ k ) is Lipschitz continuous in a neighborhood of (cid:98) θ (cid:63) .

Original: Combining Eq.(32), we have (cid:107) h
Revised: Combining Eq.(32), we have (cid:107) h

Original: ( θ k ) − h
Revised: ( θ k ) − h

Original: (¨ θ k ) (cid:107) =
Revised: (¨ θ k ) (cid:107) =

Original: O ( (cid:107) θ k − ¨ θ k (cid:107) ) =
Revised: O ( (cid:107) θ k − ¨ θ k (cid:107) ) =

Original: O ( (cid:107) ω
Revised: O ( (cid:107) ω

Original: k +1 Π θ k µ θ k
Revised: k +1 Π θ k µ θ k

Original: ( x k ) (cid:107) )
Revised: ( x k ) (cid:107) )

Original: =
Revised: =

Original: O ( ω k +1 ) .
Revised: O ( ω k +1 ) .

Original: Then E [ (cid:107) ν k +1 (cid:107) ] ≤ C (cid:107) θ k − ¨ θ k (cid:107) + O ( ω k +2 )
Revised: Then E [ (cid:107) ν k +1 (cid:107) ] ≤ C (cid:107) θ k − ¨ θ k (cid:107) + O ( ω k +2 )

Original: =
Revised: =

Original: O ( ω k +1 ) by the step size condition Eq.(33).
Revised: O ( ω k +1 ) by the step size condition Eq.(33).

Original: In what follows, we can verify
Revised: In what follows, we can verify

Original: [Equation]
Revised: [Equation]

Original: ) (III) For the martingale difference noise e k
Revised: ) (III) For the martingale difference noise e k

Original: +

Revised: +

Original: =
Revised: =

Original: µ θ k ( x k +1 ) − Π θ k µ θ k
Revised: µ θ k ( x k +1 ) − Π θ k µ θ k

Original: ( x k ) with mean 0, we have
Revised: ( x k ) with mean 0, we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: We denote E [ e k +1 e (cid:62)
Revised: We denote E [ e k +1 e (cid:62)

Original: k +1 |F k ] by a function f ( x k ) .
Revised: k +1 |F k ] by a function f ( x k ) .

Original: Applying Lemma 9, we have
Revised: Applying Lemma 9, we have

Original: [Equation]
Revised: [Equation]

Original: where R :=
Revised: where R :=

Original: R ( (cid:98) θ (cid:63) ) and R ( θ ) is also equivalent to (cid:80) ∞
Revised: R ( (cid:98) θ (cid:63) ) and R ( θ ) is also equivalent to (cid:80) ∞

Original: k =
Revised: k =

Original: −∞ Cov θ ( H ( θ , x k ) , H ( θ , x 0 )) .
Revised: −∞ Cov θ ( H ( θ , x k ) , H ( θ , x 0 )) .

Original: Having the conditions C1, C2 and C3 veriﬁed, we apply Lemma 8 and have the following weak convergence for ¨ θ k
Revised: Having the conditions C1, C2 and C3 veriﬁed, we apply Lemma 8 and have the following weak convergence for ¨ θ k

Original: [Equation]
Revised: [Equation]

Original: 0 , Σ ) , where Σ = (cid:82) ∞
Revised: , Σ ) , where Σ = (cid:82) ∞

Original: [Equation]
Revised: [Equation]

Original: e 
Revised: e th [Equation] ◦ R ◦ e [Equation]

Revised: and

Original: [Equation]
Revised: [Equation]

Original: Considering the deﬁnition that ¨ θ k
Revised: Considering the deﬁnition that ¨ θ k

Original: = θ k + ω k +1 Π θ k µ θ k
Revised: = θ k + ω k +1 Π θ k µ θ k

Original: ( x k ) and E [ (cid:107) Π θ k µ θ k ( x k ) (cid:107) ] is uniformly bounded by Eq.(32), we have
Revised: ( x k ) and E [ (cid:107) Π θ k µ θ k ( x k ) (cid:107) ] is uniformly bounded by Eq.(32), we have

Original: [Equation]
Revised: [Equation]

Original: in probability.
Revised: in probability.

Original: By Slutsky’s theorem, we eventually have the desired result
Revised: By Slutsky’s theorem, we eventually have the desired result

Original: [Equation]
Revised: [Equation]

Original: where the step size ω k decays with an order α ∈ (0 . 5 , 1] such that ω k
Revised: where the step size ω k decays with an order α ∈ (0 . 5 , 1] such that ω k

Original: = O ( k − α ) .
Revised: = O ( k − α ) .

Original: D M ORE ON EXPERIMENTS D. M ODE EXPLORATION ON MNIST VIA THE SCALABLE RANDOM - FIELD FUNCTION
Revised: D M ORE ON EXPERIMENTS D. M ODE EXPLORATION ON MNIST VIA THE SCALABLE RANDOM - FIELD FUNCTION

Original: For the network structure, we follow Jarrett et al.
Revised: For the network structure, we follow Jarrett et al.

Original: and choose a standard convolutional neural network (CNN).
Revised: and choose a standard convolutional neural network (CNN).

Original: Such a CNN has two convolutional (conv) layers and two fully-connected (FC) layers.
Revised: Such a CNN has two convolutional (conv) layers and two fully-connected (FC) layers.

Original: The two conv layers has 32 and 64 feature maps, respectively.
Revised: The two conv layers has 32 and 64 feature maps, respectively.

Original: The FC layers both have 50 hidden nodes and the network has 5 outputs.
Revised: The FC layers both have 50 hidden nodes and the network has 5 outputs.

Original: A large batch size of 2500 is selected to reduce the gradient noise and reduce the stochastic approximation bias.
Revised: A large batch size of 2500 is selected to reduce the gradient noise and reduce the stochastic approximation bias.

Original: We ﬁx ζ = 3 e 4 and weight decay 25.
Revised: We ﬁx ζ = 3 e 4 and weight decay 25.

Original: For simplicity, we choose 10000 partitions and ∆ u = 10 .
Revised: For simplicity, we choose 100,000 partitions and ∆ u = 10 .

Original: The step size follows ω k = min { 0 . 01 ,1 k
Revised: The step size follows ω k = min { 0 . 01 ,

Original: [Figure]
Revised: [Equation]

Original: 6 +100 } .
Revised: [Equation]

Original: D. S IMULATIONS OF MULTI - MODAL DISTRIBUTIONS
Revised: D. S IMULATIONS OF MULTI - MODAL DISTRIBUTIONS

Original: The target density function is given by π ( x ) ∝ exp( − U ( x )) , where x = ( x 1 , x 2 ) and U ( x ) follows U ( x )
Revised: The target density function is given by π ( x ) ∝ [Equation] ( − U ( x )) , where x = ( x 1 , x 2 ) and U ( x ) follows U ( x )

Original: [Equation]
Revised: [Equation]

Original: We also include a regularization term L ( x )
Revised: We also include a regularization term L ( x )

Original: =
Revised: =

Original: I ( x 21 + x 22 ) > 20 × (( x 21 + x 22 ) − 20) .
Revised: I ( x 21 + x 22 ) > 20 × (( x 21 + x 22 ) − 20) .

Original: This design leads to a highly multi-modal distribution with 25 isolated modes.
Revised: This design leads to a highly multi-modal distribution with 25 isolated modes.

Original: Figure 5 shows the contour and the 3-D plot of the target density.
Revised: Figure 5 shows the contour and the 3-D plot of the target density.

Original: The ICSGLD and baseline algorithms are applied to this example.
Revised: The ICSGLD and baseline algorithms are applied to this example.

Original: For ICSGLD, we set (cid:15) k
Revised: For ICSGLD, we set (cid:15) k

Original: = 3 e − 3 , τ = 1 , ζ = 0 .
Revised: = 3 e − 3 , τ = 1 , ζ = 0 .

Original: 75 and total number of iterations = 8 e 4 .
Revised: 75 and total number of iterations = 8 e 4 .

Original: Besides, we partition the sample space into 100 subregions with bandwidth ∆ u = 0 .
Revised: Besides, we partition the sample space into 100 subregions with bandwidth ∆ u = 0 .

Original: 125 and set ω k = min(

Revised: 125 and set ω k = min(

Original: e − 3 , 1 k 0 .
Revised: e − 3 , 1 k 0 .

Original: 6 +100 ) .
Revised: 6 +100 ) .

Original: For comparison, we run the baseline algorithms under similar settings.
Revised: For comparison, we run the baseline algorithms under similar settings.

Original: For CSGLD, we run a single process 5 times of the time budget and all the settings are the same as those used by ICSGLD.
Revised: For CSGLD, we run a single process 5 times of the time budget and all the settings are the same as those used by ICSGLD.

Original: For reSGLD, we run ﬁve parallel chains with learning rates 0 .
Revised: For reSGLD, we run ﬁve parallel chains with learning rates 0 .

Original: 005 and temperatures 1 , 2 , · · · , 5 , respectively.
Revised: 005 and temperatures 1 , 2 , · · · , 5 , respectively.

Original: We estimate the correction every 100 iterations.
Revised: We estimate the correction every 100 iterations.

Original: We ﬁx the initial correction 30 and choose the same step size for the stochastic approximation as in ICSGLD.
Revised: We ﬁx the initial correction 30 and choose the same step size for the stochastic approximation as in ICSGLD.

Original: For SGLD, we run ﬁve chains in parallel with the learning rate 3 e − 3 and a temperature of 1 .
Revised: For SGLD, we run ﬁve chains in parallel with the learning rate 3 e − 3 and a temperature of 1 .

Original: For cycSGLD, we run a single-chain withtimes of the time budget.
Revised: For cycSGLD, we run a single-chain withtimes of the time budget.

Original: We set the initial learning rate as 1 e − 2 and choose 10 cycles.
Revised: We set the initial learning rate as 1 e − 2 and choose 10 cycles.

Original: For the particle-based SVGD, we run ﬁve chains in parallel.
Revised: For the particle-based SVGD, we run ﬁve chains in parallel.

Original: For each chain, we initialize 100 particles as being drawn from a uniform distribution over a rectangle.
Revised: For each chain, we initialize 100 particles as being drawn from a uniform distribution over a rectangle.

Original: The learning rate is set to 3 e − 3 .
Revised: The learning rate is set to 3 e − 3 .

Original: [Figure] To compare the convergence rates in terms of running steps and time between ICSGLD and other algorithms, we repeat each algorithm 20 times and calculate the mean and standard error over trials.
Revised: To compare the convergence rates in terms of running steps and time between ICSGLD and other algorithms, we repeat each algorithm 20 times and calculate the mean and standard error overtrials.

Original: Note that we run all the algorithms based on 5 parallel chains ( × P5) except that cycSGLD and CSGLD are run in a single-chain with 5 times of time budget ( × T5) and the steps and running time are also scaled accordingly.
Revised: Note that we run all the algorithms based on 5 parallel chains ( × P5) except that cycSGLD and CSGLD are run in a single-chain with 5 times of time budget ( × T5) and the steps and running time are also scaled accordingly.

Original: Figure 6 shows that the vanilla SGLD × P5 converges the slowest among the ﬁve algorithms due to the lack of mechanism to escape local traps; cycSGLD × T5 slightly alleviates that problem by adopting cyclical learning rates; reSGLD × P5 greatly accelerates the computations by utilizing high-temperature chains for exploration and low-temperature chains for exploitation, but the large correction term inevitably slows down the convergence; ICSGLD × P converges faster than all the others and the noisy energy estimators only induce a bias for the latent variables and don’t affect the convergence rate signiﬁcantly.
Revised: Figure 6 shows that the vanilla SGLD × P5 converges the slowest among the ﬁve algorithms due to the lack of mechanism to escape local traps; cycSGLD × T5 slightly alleviates that problem by adopting cyclical learning rates; reSGLD × P5 greatly accelerates the computations by utilizing high-temperature chains for exploration and low-temperature chains for exploitation, but the large correction term inevitably slows down the convergence; ICSGLD × P converges faster than all the others and the noisy energy estimators only induce a bias for the latent variables and don’t affect the convergence rate signiﬁcantly.

Original: For the particle-based SVGD method, since more particles require expensive computations while fewer particles lead to a crude approximation.
Revised: For the particle-based SVGD method, since more particles require expensive computations while fewer particles lead to a crude approximation.

Original: Therefore, we don’t show the convergence of SVGD and only compare the Monte Carlo methods.
Revised: Therefore, we don’t show the convergence of SVGD and only compare the Monte Carlo methods.

Original: D. D EEP CONTEXTUAL BANDITS ON MUSHROOM TASKS
Revised: D. D EEP CONTEXTUAL BANDITS ON MUSHROOM TASKS

Original: For the UCI Mushroom data set, each mushroom is either edible or poisonous.
Revised: For the UCI Mushroom data set, each mushroom is either edible or poisonous.

Original: Eating an edible mushroom yields a reward of 5, but eating a poisonous mushroom has a 50% chance to result in a reward of -35 and a reward of 5 otherwise.
Revised: Eating an edible mushroom yields a reward of 5, but eating a poisonous mushroom has a 50% chance to result in a reward of -35 and a reward of 5 otherwise.

Original: Eating nothing results in 0 reward.
Revised: Eating nothing results in 0 reward.

Original: All the agents use the same architecture.
Revised: All the agents use the same architecture.

Original: In particular, we ﬁt a two-layer neural network with 100 neurons each and ReLU activation functions.
Revised: In particular, we ﬁt a two-layer neural network with 100 neurons each and ReLU activation functions.

Original: The input of the network is a feature vector with dimension 22 (context) and there are 2 outputs, representing the predicted reward for eating or not eating a mushroom.
Revised: The input of the network is a feature vector with dimension 22 (context) and there are 2 outputs, representing the predicted reward for eating or not eating a mushroom.

Original: The mean squared loss is adopted for training the models.
Revised: The mean squared loss is adopted for training the models.

Original: We initialize 1024 data points and keep a data buffer of size 4096 as the training proceeds.
Revised: We initialize 1024 data points and keep a data buffer of size 4096 as the training proceeds.

Original: The size of the mini-batch data is set to 512.
Revised: The size of the mini-batch data is set to 512.

Original: To adapt to online scenarios, we train models after every 20 new observations.
Revised: To adapt to online scenarios, we train models after every 20 new observations.

Original: We choose one (cid:15) -greedy policy (EpsGreedy) based on the RMSProp optimizer with a decaying learning rate (Riquelme et al., 2018) as a baseline.
Revised: We choose one (cid:15) -greedy policy (EpsGreedy) based on the RMSProp optimizer with a decaying learning rate (Riquelme et al., 2018) as a baseline.

Original: Two variational methods, namely stochastic gradient descent with a constant learning rate (ConstSGD) (Mandt et al., 2017) and Monte Carlo Dropout (Dropout) (Gal & Ghahramani, 2016) are compared to approximate the posterior distribution.
Revised: Two variational methods, namely stochastic gradient descent with a constant learning rate (ConstSGD) (Mandt et al., 2017) and Monte Carlo Dropout (Dropout) (Gal & Ghahramani, 2016) are compared to approximate the posterior distribution.

Original: For the sampling algorithms, we include preconditioned SGLD (pSGLD)
Revised: For the sampling algorithms, we include preconditioned SGLD (pSGLD)

Original: (Li et al., 2016), preconditioned
Revised: (Li et al., 2016), preconditioned

Original: CSGLD (pCSGLD) (Deng et al., 2020b), and preconditioned ICSGLD (pICSGLD).
Revised: CSGLD (pCSGLD) (Deng et al., 2020b), and preconditioned ICSGLD (pICSGLD).

Original: Note that all the algorithms run 4 parallel chains with average outputs ( × P4) except that pCSGLD runs a single-chain with 4 times of computational budget ( × T4).
Revised: Note that all the algorithms run 4 parallel chains with average outputs ( × P4) except that pCSGLD runs a single-chain with 4 times of computational budget ( × T4).

Original: In particular for the two contour algorithms, we set ζ = 20 and choose a constant step size for the stochastic approximation to ﬁt for the time-varying posterior distributions.
Revised: In particular for the two contour algorithms, we set ζ = 20 and choose a constant step size for the stochastic approximation to ﬁt for the time-varying posterior distributions.

Original: For more details on the experimental setups, we refer readers to section D in the supplementary material.
Revised: For more details on the experimental setups, we refer readers to section D in the supplementary material.

Original: We report the experimental setups for each algorithm.
Revised: We report the experimental setups for each algorithm.

Original: Similar to Table 2 of Riquelme et al.
Revised: Similar to Table 2 of Riquelme et al.

Original: (2018), the inclusion of advanced techniques may change the optimal settings of the hyperparameters.
Revised: (2018), the inclusion of advanced techniques may change the optimal settings of the hyperparameters.

Original: Nevertheless, we try to report the best setups for each individual algorithm.
Revised: Nevertheless, we try to report the best setups for each individual algorithm.

Original: We train each algorithm 2000 steps.
Revised: We train each algorithm 2000 steps.

Original: We initialize 1024 mushrooms and keep a data buffer of size 4096 as the training proceeds.
Revised: We initialize 1024 mushrooms and keep a data buffer of size 4096 as the training proceeds.

Original: For each step, we are given 20 random mushrooms and train the model 16 iterations every step for the parallel algorithms ( × P4); we train pCSGLD × T4 64 iterations every step.
Revised: For each step, we are given 20 random mushrooms and train the model 16 iterations every step for the parallel algorithms ( × P4); we train pCSGLD × T4 64 iterations every step.

Original: EpsGreedy decays the learning rate by a factor of 0.999 every step; by contrast, all the others choose a ﬁxed learning rate.
Revised: EpsGreedy decays the learning rate by a factor of 0.999 every step; by contrast, all the others choose a ﬁxed learning rate.

Original: RMSprop adopts a regularizer of 0 .
Revised: RMSprop adopts a regularizer of 0 .

Original: 001 and a learning rate of 0 .
Revised: 001 and a learning rate of 0 .

Original: 01 to learn the preconditioners.
Revised: 01 to learn the preconditioners.

Original: Dropout proposes a 50% dropout rate and each subprocess simulates 5 models for predictions.
Revised: Dropout proposes a 50% dropout rate and each subprocess simulates 5 models for predictions.

Original: For the two importance sampling (IS) algorithms, we partition the energy space into m = 100 subregions and set the energy depth ∆ u as 10.
Revised: For the two importance sampling (IS) algorithms, we partition the energy space into m = 100 subregions and set the energy depth ∆ u as 10.

Original: We ﬁx the hyperrameter ζ = 20 .
Revised: We ﬁx the hyperrameter ζ = 20 .

Original: The step sizes for pICSGLD × P4 and pCSGLD × T4 are chosen as 0.03 and 0.006, respectively.
Revised: The step sizes for pICSGLD × P4 and pCSGLD × T4 are chosen as 0.03 and 0.006, respectively.

Original: A proper regularizer is adopted for the low importance weights.
Revised: A proper regularizer is adopted for the low importance weights.

Original: See Table 2 for details.
Revised: See Table 2 for details.

Revised: T ABLE 2: D ETAILS OF THE EXPERIMENTAL SETUPS .

Original: D. U NCERTAINTY ESTIMATION
Revised: A LGORITHM L EARNING RATE

Revised: Temperature

Revised: RMSprop

Revised: IS

Revised: Train

Revised: Dropout (cid:15) -Greedy

Revised: E PS G REEDY ×

Revised: P 5e-7 (0.999)

Revised: Y ES

Revised: N O

Revised: N O 0.3% C ONST

Revised: SGD × P 1eN O

Revised: N O

Revised: N O

Revised: N O D ROPOUT × P 1eN O

Revised: N O

Revised: Y ES (50%)

Revised: N O

Revised: P CSGLD × T 5e0.

Revised: Y ES

Revised: Y ES

Revised: N O

Revised: N O

Revised: P SGLD × P 3e0.

Revised: Y ES

Original: All the algorithms, excluding M-SGD × P4, choose a temperature of 0.0003 † .
Revised: N O N O N O P ICSGLD × P 3e0.

Original: We run the parallel algorithms 500 epochs ( × P4) and run the single-chain algorithms 2000 epochs ( × T4).
Revised: Y ES Y ES N O N O

Original: The initial † We use various data augmentation techniques, such as random ﬂipping, cropping, and random erasing (Zhong et al., 2017).
Revised: D. U NCERTAINTY ESTIMATION

Original: This leads to a much more concentrated posterior and requires a very low temperature.
Revised: All the algorithms, excluding M-SGD × P4, choose a temperature of 0.0003 † .

Original: [Table] [Table] [Table]
Revised: We run the parallel algorithms 500 epochs ( × P4) and run the single-chain algorithms 2000 epochs ( × T4).

Original:  learning rate is 2e-6 (Bayesian settings), which corresponds to the standard 0.1 for averaged data likelihood.
Revised: The initial learning rate is 2e-6 (Bayesian settings), which corresponds to the standard 0.1 for averaged data likelihood.

Original: We train cycSGHMC × T4 and MultiSWAG × T4 based on the cosine learning rates with 10 cycles.
Revised: We train cycSGHMC × T4 and MultiSWAG × T4 based on the cosine learning rates with 10 cycles.

Original: The learning rate in the last 15% of each cycle is ﬁxed at a constant value.
Revised: The learning rate in the last 15% of each cycle is ﬁxed at a constant value.

Original: MultiSWAG simulatesrandom models at the end of each cycle.
Revised: MultiSWAG simulatesrandom models at the end of each cycle.

Original: M-SGD × P4 follows the same cosine learning rate strategy with one cycle.
Revised: M-SGD × P4 follows the same cosine learning rate strategy with one cycle.

Original: reSGHMC × P4 proposes swaps between neighboring chains and requires a ﬁxed correction offor ResNet20, 32, and 56 and a correction of 1000 for WRN-16-8.
Revised: reSGHMC × P4 proposes swaps between neighboring chains and requires a ﬁxed correction offor ResNet20, 32, and 56 and a correction of 1000 for WRN-16-8.

Original: The learning rate is annealed at 250 and 375 epochs with a factor of 0.2.
Revised: The learning rate is annealed at 250 and 375 epochs with a factor of 0.2.

Original: ICSGHMC × P4 also applies the same learning rate.
Revised: ICSGHMC × P4 also applies the same learning rate.

Original: We choose m = 200 and ∆ u = 200 for ResNet20, 32, and 56 and ∆ u = 60 for WRN-16-8.
Revised: We choose m = 200 and ∆ u = 200 for ResNet20, 32, and 56 and ∆ u = 60 for WRN-16-8.

Original: A proper regularization is applied to the importance weights.
Revised: Proper regularizations may be applied to the importance weights and gradient multipliers for training deep neural networks.

Original: Variance reduction (Deng et al., 2021) only applies to reSGHMC × P4 and ICSGHMC × P4 because they are the only two algorithms that require accurate estimations of the energy.
Revised: Variance reduction (Deng et al., 2021a) only applies to reSGHMC × P4 and ICSGHMC × P4 because they are the only two algorithms that require accurate estimations of the energy.

Original: We only update control variates every 2 epochs in the last 100 epochs, which maintain a reasonable training time and a higher reduction of variance due to a small learning rate.
Revised: We only update control variates every 2 epochs in the last 100 epochs, which maintain a reasonable training time and a higher reduction of variance due to a small learning rate.

Original: Other algorithms yield a worse performance when variance reduction is applied to the gradients.
Revised: Other algorithms yield a worse performance when variance reduction is applied to the gradients.

Revised: D. E MPIRICAL V ALIDATION OF R EDUCED V ARIANCE

Revised: To compare the θ ’s learned from ICSGLD and CSGLD, we try to simulate from a Gaussian mixture distribution 0 .

Revised: 4 N ( − 6 , 1) + 0 .

Revised: 6 N (4 , 1) , where N ( u, v ) denotes a Gaussian distribution with mean u and standard deviation v .

Revised: We ﬁx ζ = 0 .

Revised: and ∆ u = 1 .

Revised: We run ICSGLD with 1,000,000 iterations 0.

Revised: 10.0 Energy

Revised: [Figure]

Revised: q

Revised: [Figure]

Revised: based on 10 interacting parallel chains and run CSGLD with 10,000,000 iterations using a single chain.

Revised: We refer to them as ICSGLD × P10 and CSGLD × T10, respectively.

Revised: The rest of the settings follows from the experimental setup in section 4.1 (Deng et al., 2020a).

Revised: To measure the variance of the estimates, we repeated the experiments 10 times and present the mean and two standard deviations for both CSGLD × T10 and ICSGLD × P in Figure 7.

Revised: The results indicate that both estimates of θ ζ (by CSGLD and ICSGLD) converge to the equilibrium that approximates the ground truth of the density of states.

Revised: Notably, ICSGLD × P10 yields a signiﬁcantly smaller variance than CSGLD × T10, but with the same computational budget.

Revised: This shows the clear advantage of ICSGLD (many interacting short runs) over CSGLD (a single long run) in tackling the large variance issue for importance sampling.