Title: Langevin Quasi-Monte Carlo
Venue: NeurIPS 2023
Accepted: yes
Keywords: Completely uniformly distributed; log-concave sampling; low-discrepancy; MCMC;
Abstract: Langevin Monte Carlo (LMC) and its stochastic gradient versions are powerful algorithms for sampling from complex high-dimensional distributions. To sample from a distribution with density $\pi(\theta)\propto \exp(-U(\theta)) $, LMC iteratively generates the next sample by taking a step in the gradient direction $\nabla U$ with added Gaussian perturbations. Expectations w.r.t. the target distribution $\pi$ are estimated by averaging over LMC samples. In ordinary Monte Carlo, it is well known that the estimation error can be substantially reduced by replacing independent random samples by quasi-random samples like low-discrepancy sequences. In this work, we show that the estimation error of LMC can also be reduced by using quasi-random samples. Specifically, we propose to use completely uniformly distributed (CUD) sequences with certain low-discrepancy property to generate the Gaussian perturbations. Under smoothness and convexity conditions, we prove that LMC with a low-discrepancy CUD sequence achieves smaller error than standard LMC. The theoretical analysis is supported by compelling numerical experiments, which demonstrate the effectiveness of our approach.

Langevin Quasi-Monte Carlo

Abstract

Langevin Monte Carlo (LMC) and its stochastic gradient versions are powerful algorithms for sampling from complex high-dimensional distributions. To sample from a distribution with density π ( θ ) ∝ exp( -U ( θ )) , LMC iteratively generates the next sample by taking a step in the gradient direction ∇ U with added Gaussian perturbations. Expectations w.r.t. the target distribution π are estimated by averaging over LMC samples. In ordinary Monte Carlo, it is well known that the estimation error can be substantially reduced by replacing independent random samples by quasi-random samples like low-discrepancy sequences. In this work, we show that the estimation error of LMC can also be reduced by using quasirandom samples. Specifically, we propose to use completely uniformly distributed (CUD) sequences with certain low-discrepancy property to generate the Gaussian perturbations. Under smoothness and convexity conditions, we prove that LMC with a low-discrepancy CUD sequence achieves smaller error than standard LMC. The theoretical analysis is supported by compelling numerical experiments, which demonstrate the effectiveness of our approach.

1 Introduction

Sampling from probability distributions is a crucial task in both statistics and machine learning. However, when the target distribution does not permit exact sampling, researchers often rely on Markov chain Monte Carlo (MCMC) methods. These techniques simulate a Markov chain that converges to the target distribution as its stationary distribution. Recently, MCMC samplers based on discretizing the continuous-time Langevin diffusion have become popular, due to its ease of implementation and ability to handle stochastic gradients (Welling and Teh, 2011).

The primary focus of this work is on the quality of samples generated by Langevin Monte Carlo (LMC) algorithms in terms of estimating the expectation E θ ∼ π [ f ( θ )] for some integrand f by sample averages. In the context of Bayesian inference, the target distribution π is typically the posterior distribution, and computing the posterior expectation, posterior variance, or confidence intervals are of great interest. In the context of post-selection inference, the target distribution π is the probability distribution conditioned on the selection event, and computing the selection-adjusted p-value is the main task. LMC has been widely used in this problem as well (Markovic and Taylor, 2016; Shi et al., 2022). In all these situations, the accuracy of the sample average estimator is critical and affects the downstream data analysis.

In traditional Monte Carlo sampling, it is well known that using quasi-Monte Carlo (QMC) samples, instead of independent and identically distributed (i.i.d.) random samples, can lead to significant error reduction. So it is natural to ask whether we can apply QMC techniques to improve Langevin Monte Carlo sampling as well. In this work, we introduce the Langevin quasi-Monte Carlo (LQMC) algorithm, which replaces the i.i.d. random inputs in the LMC algorithm with quasi-random numbers.

0

1

0

1

251 points from MT19937

0

1

251 points from LCG

These quasi-random numbers are carefully designed to sample from the target distribution more evenly and more balanced, leading to improved estimation accuracy.

Not all quasi-Monte Carlo point sets are suitable for simulating Markov chains. Suppose the Markov chain is driven by a sequence of uniform random vectors in the unit cube. A sufficient condition for the sequence is known as completely uniformly distributed (CUD). In our implementation of the driving sequence, we use an entire period of a pseudo-random number generator (PRNG). While modern computer simulations often use PRNGs with a large period, such as Mersenne Twister with a period of 2 19937 -1 , our approach runs through the entire period of a PRNG with a relatively small period in the LMC algorithm. The advantage of using an entire period of a PRNG is that the points are more evenly distributed, which is more desirable for numerical integration. We illustrate the balancing property of an entire PRNG in Figure 1.

The main contributions of this paper are threefold. First, we propose a novel technique of using quasi-random numbers in Langevin-type algorithms, which can be applied to a wide range of such algorithms by substituting i.i.d. random numbers with a sequence of quasi-random numbers. The quasi-random numbers are constructed similarly as usual PRNGs, therefore no extra computational complexity is required. Second, we evaluate the performance of the proposed LQMC algorithm in a variety of numerical experiments, demonstrating that it can significantly reduce the mean squared error (MSE) of traditional LMC by a factor ranging from 2 to 500, depending on the problem. Finally, we provide theoretical analysis showing that LQMC can reduce the Monte Carlo part of the error from O ( n -1 / 2 ) to O ( n -1+ δ ) for any δ > 0 in situations where the Markov chain is strongly contracting and the integrand function f is sufficiently regular. This error reduction is consistent with the usual improvement achieved by using quasi-Monte Carlo in place of plain Monte Carlo.

The rest of the paper is organized as follows. In Section 2, we provide some background on LMC and QMC, followed by a review of related work. Section 3 describes the LQMC algorithm and its implementation details. In Section 4, we present theoretical guarantees for the proposed method. Finally, in Section 5, we provide empirical results to evaluate the performance of LQMC and compare it with the standard LMC algorithm.

2 Backgrounds

This section provides some background on Langevin Monte Carlo and quasi-Monte Carlo.

2.1 Langevin Monte Carlo

Suppose we want to sample from the target distribution π ( θ ) ∝ exp( -U ( θ )) where θ ∈ R d and U is known as the potential function. LMC algorithms are based on Euler-Maruyama discretization of the Langevin diffusion θ ( t ) , which satisfies the stochastic differential equation

where { W t } t ≥ 0 is a d -dimensional standard Brownian motion. Under mild technical conditions, the Langevin diffusion θ ( t ) has π as its unique invariant distribution (Roberts and Tweedie, 1996). With a discretization step size h , LMC updates the sample θ k by

where ξ k iid ∼ N (0 , I d ) .

In many applications, we are interested in computing the expectation µ := E θ ∼ π [ f ( θ )] over π for some π -integrable function f . The LMC estimator of µ is the sample average

where n is the number of iterations.

Teh et al. (2016) provide an asymptotic bias-variance decomposition of the MSE of the weighted average ∑ n k =1 h k f ( θ k ) ∑ n k =1 h k and show that the optimal step size scales as h k glyph[equivasymptotic] k -1 / 3 , leading to an MSE of order O ( n -2 / 3 ) . Here h k is the step size used at the k -th iteration. V ollmer et al. (2016) generalize this result to the non-asymptotic setting with a constant step size h . They show that the MSE is of order O ( h 2 + 1 nh ) , where h 2 corresponds to the squared bias and 1 nh corresponds to the variance.

2.2 Quasi-Monte Carlo

QMCis an alternative to Monte Carlo for numerical integration and is well-known for having much higher accuracy than Monte Carlo. QMC is primarily designed to numerically evaluate the integral µ = ∫ [0 , 1] d f ( u )d u . It estimates µ by taking points u i ∈ [0 , 1] d and let the estimator be

Unlike Monte Carlo which takes u i to be identically independently distributed (i.i.d.), QMC constructs the point set { u i } n i =1 that aims to minimize the star discrepancy

The star discrepancy measures the uniformity of the point sets by comparing the fraction of points inside [ 0 , a ) and the volume ∏ d j =1 a j , taking supreme over all the rectangles inside [0 , 1] d anchored at 0 . QMCcan generate points with D ∗ n = O ( n -1 (log n ) d -1 ) , thus QMC is also known as low-discrepancy sequence. Commonly used QMC points include Sobol' sequence (Sobol', 1967), Niederreiter's sequence (Niederreiter, 1987), Halton's sequence, and lattice rules. For a comprehensive survey, we refer to the monograph Dick and Pillichshammer (2010). If the integrand f has bounded variation in the sense of Hardy and Krause ‖ f ‖ HK, then the Koksma-Hlawka inequality (see e.g. Dick and Pillichshammer (2010)) bounds the integration error by

While the Koksma-Hlawka inequality shows that QMC is asymptotically better than usual Monte Carlo, it doesn't provide a practical way to estimate the error. Moreover, integrands might have infinite Hardy-Krause variation.

One can apply randomization techniques to QMC to address both problems. Common randomization techniques include random shifts (Cranley and Patterson, 1976) and scrambling (Owen, 1995). For RQMC samples u 1 , . . . , u n , each u i ∼ Unif([0 , 1] d ) individually but they have the low-discrepancy property collectively with probability 1. One can estimate the error by multiple independent random replicates. For sufficiently smooth f , the scrambled Sobol' sequence has variance O ( n -3 (log n ) d -1 ) (Owen, 1997a,b).

2.3 Related work

The first attempt to apply quasi-random numbers to simulate stochastic differential equations was made by Hofmann and Mathé (1997). They showed that if a numerical scheme is weakly convergent with i.i.d. samples, then using completely uniformly distributed (CUD) sequences also leads to consistent estimation. They also demonstrated that certain low-discrepancy sequences are not suitable for simulating SDEs. There have also been some efforts to apply QMC to MCMC. Owen and Tribble (2005) proposed to apply CUD sequences to a Metropolis algorithm and showed that the method is consistent in problems with finite state spaces. Chen et al. (2011) generalized the consistency result to continuous state spaces under the assumption that the Markov chain is a contraction. More recently, Dick et al. (2016); Dick and Rudolf (2014) proved that there exists constructions of the driving sequence { u k } k ≥ 1 such that the discrepancy between the empirical distribution of MCMC samples and the target distribution is bounded by O ( n -1 / 2 (log n ) 1 / 2 ) , the same rate achieved by random inputs. Another line of applying QMC to Markov chains is known as array-RQMC proposed by L'Ecuyer et al. (2008). Array-RQMC runs in parallel multiple Markov chains, and each iteration involves a complicated reordering of the states so that the low-discrepancy among the chains is maintained. Empirically, it achieves significantly smaller estimation error than usual MCMC, but theoretical guarantees remain a challenging open problem.

There has been a growing interest in using QMC techniques in various machine learning tasks, such as variational inference (Buchholz et al., 2018; Liu and Owen, 2021), policy learning and evaluation (Arnold et al., 2022), reinforcement learning with evolution strategies (Choromanski et al., 2019; Rowland et al., 2018), compression of large datasets (Dick and Feischl, 2021), example selection in stochastic gradient descent (SGD) (Lu et al., 2021), and deep learning for solving partial differential equations (Longo et al., 2021).

Numerous efforts have been devoted to improving LMC and stochastic gradient Langevin dynamics (SGLD). To overcome the instability of Euler-Maruyama discretization, various numerical schemes have been proposed, including higher-order integrators (Chen et al., 2015), underdamped LMC (Cheng et al., 2018), and stochastic Runge-Kutta diffusion (Li et al., 2019). For SGLD, variance reduction techniques such as SAGA and SVGR (Dubey et al., 2016) and control variates (Baker et al., 2019) have been proposed. LMC also provides a useful perspective for optimization, as demonstrated by the analyses in Chen et al. (2016); Dalalyan (2017); Raginsky et al. (2017); Xu et al. (2018); Erdogdu et al. (2018). Our contribution is orthogonal to all the aforementioned work, as our algorithm only modifies the random numbers used in the algorithm. Therefore, our method can be combined with other algorithms without interference.

3 QMCfor LMC

In the LMC algorithm, we can think of the Markov chain as being driven by a sequence of uniform variables u k in the unit cube [0 , 1] d . For instance, the Gaussian perturbation can be represented as ξ k = Φ -1 ( u k ) , where Φ -1 denotes the inverse Gaussian CDF applied element-wise to u k . If a stochastic gradient is employed, the randomness associated with the stochastic gradient can also be expressed as uniform variables. Therefore, we can write the transition of the Markov chain as θ k +1 = ψ ( θ k , u k +1 ) . In typical computer experiments, u k are not really i.i.d. but are deterministic pseudo-random numbers. In this section, we will describe an alternative method of generating the pseudo-random numbers u k , which are carefully constructed and can lead to more accurate sample averages.

The idea here is to use point sets that are more evenly distributed such as QMC points, which can lead to significant improvement in the usual Monte Carlo estimation. However, caution is required when using QMC points to simulate an SDE like (1). This is because the correlation between successive QMCsamples may introduce undesired behavior in the Markov chain, as demonstrated in (Tribble, 2007, Section 3.2). To avoid the dependence among successive values, we require that the blocks of points ( v i , v i +1 , . . . , v i + d -1 ) for any lag d are uniformly distributed. This notion of uniformity is formally known as completely uniformly distributed (CUD, Korobov (1948)), which we define next.

Wesay an infinite sequence { u i } ∞ i =1 ⊆ [0 , 1] d is uniformly distributed on [0 , 1] d if the star discrepancy D ∗ ( { u i } n i =1 ) goes to 0 as n →∞ , where the star discrepancy is defined in Equation 3.

Definition 3.1 (Completely uniformly distributed sequence (CUD)) . An infinite sequence { v i } ∞ i =0 ⊂ [0 , 1] is called completely uniformly distributed, if for all positive integer d , the sequence { ( v k , . . . , v k + d -1 ) } ∞ k =0 ⊆ R d is uniformly distributed on [0 , 1] d . A triangular array v n = ( v n, 1 , . . . , v n,N n ) is called array-CUD, if for all positive integer d , D ∗ (( v n, 1 , . . . , v n,d ) , ( v n, 2 , . . . , v n,d +1 ) , . . . , ( v n,N n -d +1 , . . . , v n,N n )) → 0 as n →∞ , N n →∞ .

In other words, the subsequent d -tuples in a CUD sequence are uniformly distributed in the d -dimensional unit cube for any positive dimension d . Now we are ready to present the main algorithm.

3.1 LQMCalgorithm

Let { v i } ∞ i =0 be a CUD sequence. Let u k = ( v kd , . . . , v ( k +1) d -1 ) ∈ R d be the k -th non-overlapping d -tuple from the sequence ( k ≥ 0 ). A CUD sequence is often constructed deterministically. They can further be randomized using the Cranley-Patterson (i.e. random shift) rotation (Cranley and Patterson, 1976)

where ∆ ∼ Unif([0 , 1] d ) . The Cranley-Patterson rotation randomly shifts each dimension of u k by a uniform random number separately. Then each u k is uniformly distributed on [0 , 1] d . If we apply the inverse Gaussian CDF to each coordinate of u k , then Φ -1 ( u k ) ∼ N (0 , I d ) . In the Langevin-type algorithms, we will let ξ k = Φ -1 ( u k ) and use ξ k as the Gaussian perturbation in the k -th iteration. Specifically, each iteration takes the form

Thus the transition map is ψ ( θ, u ) = θ -h ∇ U ( θ ) + √ 2 h Φ -1 ( u ) . In practice, we can only run finite many iterations. In the following, we will describe how to construct a finite CUD sequence and feed it into the LMC algorithm.

3.2 Construction of CUD sequences

A finite CUD (array-CUD) sequence is often implemented by using an entire period of a pseudo random number generator with a small period (Tribble, 2007). There exist other constructions of CUD sequences. For further details, interested readers can refer to Levin (1999). We propose to use the linear-feedback shift register (LFSR) provided in Chen (2011), because it has demonstrated good performance and the computational effort required is comparable to other commonly used PRNGs.

The binary Galois LFSR (Tausworthe generator, Tausworthe (1965)) of order m updates the states b i ∈ { 0 , 1 } recursively by

with initial states b 0 , b 1 , . . . , b m -1 pre-specified. The m -tuple ( b i , b i +1 , . . . , b i + m -1 ) ∈ GF (2) m can only take 2 m different values. If there is an m -tuple that is all zero, then all b i 's in this sequence must be zero. So the period of the sequence { b i } i ≥ 0 is at most n = 2 m -1 . Moreover, the period is exactly equal to 2 m -1 if and only if the characteristic polynomial

is a primitive polynomial over GP (2) (Niederreiter, 1992, Lemma 9.1). Given the states { b i } i ≥ 0 and an offset s > 0 such that gcd( s, 2 m -1) = 1 , v i is computed with

That is, for each i , we take the m -tuple ( b si + j ) 0 ≤ j<m and interpret it as the binary expansion of v i . For the next step, we jump s bits ahead in the sequence { b i } i ≥ 0 and use the m -tuple starting from b s ( i +1) . Chen (2011) provided a table of the LFSR generators for 10 ≤ m ≤ 32 . They searched the offsets so that the LFSR has good equi-distributed properties. Our experiments use the LFSR generators listed there.

Given the sequence { v i } n -1 i =0 of length n , we repeat it d times and arrange v i 's in the following n × d matrix

We run the LMC algorithm n = 2 m -1 iterations. The k -th uniform vector u k is the k -th row of the above matrix. The procedure is summarized in Algorithm 1.

Algorithm 1 Langevin quasi-Monte Carlo (LQMC)

Input: Number of iterations n = 2 m -1 such that gcd(2 m -1 , d ) = 1 , step size h , initial value θ 0 Generate an LFSR sequence { v i } i ≥ 0 of period 2 m -1 .

Let u k = ( v ( k -1) d , . . . , v kd -1 ) ∈ [0 , 1] d , for 1 ≤ k ≤ n .

end for

Output: θ 1 , . . . , θ n

If gcd( n, d ) = 1 , then each column of the matrix (5) contains no repeated values. This means that among the n = 2 m -1 iterations of the LQMC algorithm, each dimension uses one value in each sub-interval ( k 2 m , k +1 2 m ] at most once ( 0 ≤ k ≤ 2 m -1 ). This perfect one-dimensional stratification is one of the reasons why CUD may achieve smaller estimation error than pseudo-random numbers. If gcd( n, d ) > 1 , then we take d ′ to be the smallest integer greater than d and co-prime with n . We then create the matrix in (5) similarly but with d ′ columns. In the LQMC algorithm, we take u k to be the k -th row of the matrix but only use the first d coordinates.

Algorithm 1 may seem to be restricted by having a fixed number of iterations, n = 2 m -1 . However, in practice, the LQMC algorithm can be started with an initial value of m . If the chain does not converge after 2 m -1 iterations, one can continue the chain with another freshly generated LFSR, possibly with a larger period. This allows for flexibility in adjusting the number of iterations based on the convergence of the chain. Additionally, if a burn-in period is required, one can first run the algorithm with an LFSR of a small period to serve as the burn-in stage and then continue with a larger LFSR. Furthermore, running multiple chains with independent random shifts is embarrassingly parallel. We present the algorithm in the form of the basic LMC algorithm with accurate gradient and constant learning rate. However, as we noted previously, other Langevin-type algorithms can also utilize the CUD sequence directly by substituting the pseudo-random numbers with the LFSR sequence.

4 Theoretical guarantee

Here we study the estimation error | n -1 ∑ n k =1 f ( θ k ) -π ( f ) | of LQMC for some test function f that is 1-Lipschitz and bounded. As the first attempt to prove the convergence rate of using QMC in LMC, we impose the relatively strong conditions of smoothness and convexity.

Assumption 1. The potential function U is L -smooth

and M -strongly convex

We will also assume a constant step size h . While LMC with vanishing step sizes converges weakly to the target distribution, in practice a constant step size is often used (V ollmer et al., 2016; Brosse et al., 2018). With a constant step size, we can derive a non-asymptotic error bound for LQMC.

Assumption 1 implies that if the step size h ≤ 2 L + M , then the transition map ψ is a strong contraction with parameter ρ = 1 -hM , i.e.

See e.g. Lemma 2 of Dalalyan and Karagulyan (2019). The strong contraction implies that if we start two chains from θ and θ ′ , and use the same random numbers at every step, then the two chains will merge exponentially fast. In other words, the state θ k largely depends on the most recent iterations and quickly forgets about the past history. Formally, let w ( glyph[lscript] ) k = ( u k , . . . , u k -glyph[lscript] +1 ) denote the random numbers used in the most recent glyph[lscript] steps. Define the glyph[lscript] -step transition as

and let ¯ f glyph[lscript] ( w ( glyph[lscript] ) k ) denote the value of f ( θ k ) marginalized over θ k -glyph[lscript] ∼ π , i.e.

Thus ¯ f glyph[lscript] ( w ( glyph[lscript] ) k ) only depends on the most recent glyph[lscript] iterations. Due to the strong contraction, | ¯ f glyph[lscript] ( w ( glyph[lscript] ) k ) -f ( θ k ) | decays exponentially fast with glyph[lscript] . So for large glyph[lscript] , the estimation error of n -1 ∑ n k =1 f ( θ k ) is close to the error of 1 n -glyph[lscript] ∑ n k = glyph[lscript] +1 ¯ f glyph[lscript] ( w ( glyph[lscript] ) k ) . The latter can be viewed as a dglyph[lscript] -dimensional numerical integration scheme based on the point set { w ( glyph[lscript] ) k } n k = glyph[lscript] +1 . By leveraging the discrepancy bound of the LFSR sequence and assuming that ¯ f glyph[lscript] has bounded variation in the sense of Hardy and Krause, we can derive an error bound using the Koksma-Hlawka inequality (4). Now we state the main error bound and leave the detailed proof in the Appendix A.

Theorem 4.1 (Error bound of LQMC) . Let Assumption 1 hold. Define the step size h ≤ 2 L + M , ρ = 1 -hM , glyph[lscript] = glyph[ceilingleft] (1 / 2) log ρ h glyph[ceilingright] . Let θ 1 , . . . , θ n be the output of Algorithm 1 which runs n iterations with step size h ≤ 2 L + M . Assume the LFSR sequence { v i } i ≥ 0 in use has period n = 2 m -1 , offset s , and gcd ( m,n ) = gcd( dglyph[lscript],n ) = 1 . If ¯ f glyph[lscript] has bounded variation in the sense of Hardy and Krause, then as n →∞ we have

Here δ hides poly-logarithmic factors (log n ) d , C 1 depends on d, glyph[lscript] and ‖ ¯ f glyph[lscript] ‖ HK, and C 2 = 3 √ 2 2 L M d + max 0 ≤ k ≤ n ‖ θ k ‖ + E π [ ‖ θ ‖ ] .

The upper bound consists of two terms. The first term represents the numerical integration error, which arises from the discrepancy of the point set used in the integration scheme. By utilizing low-discrepancy CUD sequences, we can reduce this numerical integration error (the first term) from the standard rate of O ( n -1 / 2 ) to a faster rate of O ( n -1+ δ ) for any δ > 0 . However, it is important to note that when using a constant step size h in LMC, the bias term (second term) does not vanish. This bias term includes not only the discretization error of the Langevin diffusion, but also the difference between f ( θ k ) and its truncated version ¯ f glyph[lscript] ( w ( glyph[lscript] ) k ) . Consequently, the bias term in our analysis is larger than the bias term in V ollmer et al. (2016), which employs different techniques and assumptions based on the Poisson equation.

The theorem's assumption of finite Hardy-Krause variation is a common requirement in error bounds for QMC methods, and it can be challenging to verify in practice. Basu and Owen (2016) provide sufficient conditions in order for f ◦ ψ glyph[lscript] to have finite HK variation, requiring the glyph[lscript] -step transition ψ glyph[lscript] to be sufficiently smooth. In the next section, we aim to assess the practical performance of the proposed LQMC algorithm through numerical experiments.

5 Numerical experiments

To comprehensively evaluate the performance of the algorithm, we will consider both convex and non-convex potentials, both low-dimensional and high-dimensional state spaces, both accurate and stochastic gradients, both smooth and discontinuous integrands, as well as different learning rate schedules. Additional numerical results can be found in the Appendix B.

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

9

2

6

2

3

MSE

f

(

x

) =

xj

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

6

2

4

2

2

2

0

f

(

x

) =

x

2

j

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

11

2

9

2

7

2

5

f

(

x

) =

1

xj

>0

MC

CUD

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

11

2

8

2

5

MSE

f

(

x

) =

xj

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

8

2

5

2

2

f

(

x

) =

x

2

j

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

12

2

10

2

8

2

6

f

(

x

) =

1

xj

>0

MC

CUD

5.1 Bayesian logistic regression

We first consider the Bayesian logistic model

We take N = 20 , d = 10 . The features x i are generated from N (0 , Σ) with Σ ij = 2 -| i -j | . The coefficients β and the data y i 's are generated from the same model. We consider the test functions f ( x ) = x j , x 2 j , 1 { x j > 0 } for j = 1 , . . . , d . The step size h is fixed to 0 . 001 .

We compute the MSE of the estimator based on usual LMC and the proposed LQMC with CUD sequences and report the MSE averaged over all coordinates and 20 random replicates. We do not have a closed form for the expectations E [ f ] , so the ground truth is estimated using a high-accuracy estimator proposed in He et al. (2023) using scrambled Sobol' sequence with a very large sample size.

In Figure 2 (top panel), we present a log-log plot of the MSE against the number of iterations. Across all three test functions, we observe that LQMC reduces the MSE by a factor ranging from 4 to 8. As the number of iterations increases, the curve corresponding to LQMC reaches a plateau. This behavior can be attributed to the discretization error inherent in the unadjusted LMC, which cannot be further reduced by increasing the number of iterations.

In the bottom panel of Figure 2, we increase the number of observations to N = 100 and incorporate stochastic gradient estimation in the Langevin algorithm. Specifically, at each iteration, we estimate the gradient using a random subset of 10 observations. The results demonstrate that LQMC still provides a big improvement when n is smaller than 2 14 . However, as n surpasses 2 14 , we observe that the LQMC curve flattens again. It is worth noting that the improvement achieved by LQMC in this scenario is less pronounced compared to the previous example, primarily due to the presence of noise in the gradient estimates.

5.2 Bayesian linear regression

Now we try a higher-dimensional example with Bayesian linear regression. The model is defined as

We take d = 100 and N = 20 . We generate x i ∈ R d similarly as in the logistic regression example. The test functions and step size are also unchanged. The posterior distribution of β has the closed form N ( ( X ᵀ X σ 2 + I ) -1 X ᵀ Y σ 2 , ( X ᵀ X σ 2 + I ) -1 ) . The results are shown in Figure 3. We see that even at 100 dimension, LQMC still brings a substantial improvement over LMC in terms of MSE. In

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

11

2

8

2

5

2

2

MSE

f

(

x

) =

xj

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

4

2

3

2

2

2

1

f

(

x

) =

x

2

j

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

9

2

7

2

5

f

(

x

) =

1

xj

>0

MC

CUD

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

2

5

2

2

2

1

MSE

h

=

0.0001

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

h

=

0.01

2 10

2 11

2 12

2 13

2 14

2 15

2 16

n

decreasing

h

MC

CUD

particular, for the integrand f ( x ) = x j , LQMC achieves a reduction in MSE of approximately 500-fold compared to LMC.

5.3 A hierarchical Bayesian model

We consider a hierarchical Bayesian model known as the crossed random effect model

The goal is to sample from the posterior distribution of ( µ, a , b , log( σ 2 a ) , log( σ 2 b )) , which has dimension d = I + J +3 . We take I = 3 , J = 5 . We will consider the test functions f ( x ) = x j ( 1 ≤ j ≤ d ). The ground truth of E [ f ( x )] is estimated by Langevin dynamics with Metropolis adjustments (MALA) using a large sample size.

We will compare the performance of the LQMC algorithm using three different step sizes: a constant step size of 10 -4 , a constant step size of 10 -2 , and decreasing step sizes with h k = c 0 ( c 1 + k ) -1 / 3 . The choice of c 0 and c 1 ensures that the step size decreases from 10 -2 to 10 -4 throughout the entire algorithm. The use of the exponent -1 / 3 in the decreasing step sizes is recommended in Teh et al. (2016). The results of these comparisons are presented in Figure 4.

In the small step size case (left panel), we observe that the errors of LMC and LQMC are initially comparable for small values of n . This is because the algorithm converges slowly, and thus the error is dominated by the bias. However, as n increases, the improvement of LQMC becomes evident. In the large step size case (middle panel), the MSE of LQMC is consistently smaller than that of LMC even for small values of n . This is because the algorithm converges faster to the target distribution with a larger step size h . Therefore, the improvement of LQMC is more pronounced. Interestingly, in this particular example, using decreasing step sizes yields similar accuracy to using a constant step size of 10 -4 . It is worth noting that the MSE of LMC does not decrease at a rate of n -2 / 3 as in Teh et al. (2016). This is because the line in the plot does not represent the accuracy against the iteration k within a single training process. Instead, it reflects the accuracy achieved after completing all n iterations of the algorithm, considering different values of n .

2 15

2 16

2 17

2 18

2 19

2 20

2 21

2 22

2 23

n

2

16

2

12

2

8

2

4

MSE

f

(

x

) =

x

2 15

2 16

2 17

2 18

2 19

2 20

2 21

2 22

2 23

n

2

12

2

9

2

6

2

3

f

(

x

) =

x

2

2 15

2 16

2 17

2 18

2 19

2 20

2 21

2 22

2 23

n

2

18

2

15

2

12

2

9

f

(

x

) =

1

x

>0

LMC

5.4 Nonconvex potential

Finally we investigate a double-well potential function U ( x ) = 1 4 x 2 -1 2 log(1 + x 2 ) from Pagès and Panloup (2018). We know E [ x ] = 0 and E [ 1 { x> 0 } ] = 0 . 5 . The second moment E [ x 2 ] is computed by Gaussian quadrature. See the results in Figure 5. Since the potential has two separate local minimums, it takes longer for the Langevin algorithm to explore the space sufficiently and converge to the target distribution. Once converged, the improvement of LQMC over LMC is still significant.

Acknowledgments and Disclosure of Funding

The author thanks Prof. Art Owen for helpful conversations. This work was partially funded by the NSF grant DMS-2152780 and the Stanford Data Science Scholars program.

References

Arnold, S. M., L'Ecuyer, P., Chen, L., Chen, Y.-F., and Sha, F. (2022). Policy learning and evaluation with randomized quasi-Monte Carlo. In International Conference on Artificial Intelligence and Statistics , pages 1041-1061. PMLR.

Baker, J., Fearnhead, P., Fox, E. B., and Nemeth, C. (2019). Control variates for stochastic gradient MCMC. Statistics and Computing , 29:599-615.

Basu, K. and Owen, A. B. (2016). Transformations and Hardy-Krause variation. SIAM Journal on Numerical Analysis , 54(3):1946-1966.

Brosse, N., Durmus, A., and Moulines, E. (2018). The promises and pitfalls of stochastic gradient Langevin dynamics. Advances in Neural Information Processing Systems , 31.

Buchholz, A., Wenzel, F., and Mandt, S. (2018). Quasi-Monte Carlo variational inference. In International Conference on Machine Learning , pages 668-677. PMLR.

Chen, C., Carlson, D., Gan, Z., Li, C., and Carin, L. (2016). Bridging the gap between stochastic gradient MCMC and stochastic optimization. In Artificial Intelligence and Statistics , pages 1051-1060. PMLR.

Chen, C., Ding, N., and Carin, L. (2015). On the convergence of stochastic gradient MCMC algorithms with high-order integrators. Advances in neural information processing systems , 28.

Chen, S. (2011). Consistency and convergence rate of Markov chain quasi Monte Carlo with examples . PhD thesis, Stanford University.

Chen, S., Dick, J., and Owen, A. (2011). Consistency of Markov chain quasi-Monte Carlo on continuous state spaces. The Annals of Statistics , 39(2):673-701.

Cheng, X., Chatterji, N. S., Bartlett, P. L., and Jordan, M. I. (2018). Underdamped Langevin MCMC: A non-asymptotic analysis. In Conference on learning theory , pages 300-323. PMLR.

Choromanski, K., Pacchiano, A., Parker-Holder, J., and Tang, Y. (2019). Structured Monte Carlo sampling for nonisotropic distributions via determinantal point processes. arXiv preprint arXiv:1905.12667 .

Cranley, R. and Patterson, T. N. (1976). Randomization of number theoretic methods for multiple integration. SIAM Journal on Numerical Analysis , 13(6):904-914.

Dalalyan, A. (2017). Further and stronger analogy between sampling and optimization: Langevin Monte Carlo and gradient descent. In Conference on Learning Theory , pages 678-689. PMLR.

Dalalyan, A. S. and Karagulyan, A. (2019). User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient. Stochastic Processes and their Applications , 129(12):5278-5311.

Dick, J. and Feischl, M. (2021). A quasi-monte carlo data compression algorithm for machine learning. Journal of Complexity , 67:101587.

Dick, J. and Pillichshammer, F. (2010). Digital Sequences, Discrepancy and Quasi-Monte Carlo Integration . Cambridge University Press, Cambridge.

Dick, J. and Rudolf, D. (2014). Discrepancy estimates for variance bounding Markov chain quasiMonte Carlo. Electron. J. Probab , 19(105):1-24.

Dick, J., Rudolf, D., and Zhu, H. (2016). Discrepancy bounds for uniformly ergodic Markov chain quasi-Monte Carlo. Annals of Applied Probability , 26(5):3178-3205.

Dubey, K. A., J Reddi, S., Williamson, S. A., Poczos, B., Smola, A. J., and Xing, E. P. (2016). Variance reduction in stochastic gradient Langevin dynamics. Advances in neural information processing systems , 29.

Erdogdu, M. A., Mackey, L., and Shamir, O. (2018). Global non-convex optimization with discretized diffusions. Advances in Neural Information Processing Systems , 31.

He, Z., Zheng, Z., and Wang, X. (2023). On the error rate of importance sampling with randomized quasi-Monte Carlo. SIAM Journal on Numerical Analysis , 61(2):515-538.

Hofmann, N. and Mathé, P. (1997). On quasi-Monte Carlo simulation of stochastic differential equations. Mathematics of computation , 66(218):573-589.

Korobov, N. (1948). On functions with uniformly distributed fractional parts. In Dokl. Akad. Nauk SSSR , volume 62, pages 21-22.

L'Ecuyer, P., Lécot, C., and Tuffin, B. (2008). A randomized quasi-Monte Carlo simulation method for Markov chains. Operations Research , 56(4):958-975.

Levin, M. B. (1999). Discrepancy estimates of completely uniformly distributed and pseudorandom number sequences. International Mathematics Research Notices , 1999(22):1231-1251.

Li, X., Wu, Y., Mackey, L., and Erdogdu, M. A. (2019). Stochastic Runge-Kutta accelerates Langevin Monte Carlo and beyond. Advances in neural information processing systems , 32.

Liu, S. and Owen, A. B. (2021). Quasi-Monte Carlo quasi-Mewton in variational Bayes. The Journal of Machine Learning Research , 22(1):11043-11065.

Longo, M., Mishra, S., Rusch, T. K., and Schwab, C. (2021). Higher-order quasi-monte carlo training of deep neural networks. SIAM Journal on Scientific Computing , 43(6):A3938-A3966.

Lu, Y., Meng, S. Y., and De Sa, C. (2021). A general analysis of example-selection for stochastic gradient descent. In International Conference on Learning Representations .

Markovic, J. and Taylor, J. (2016). Bootstrap inference after using multiple queries for model selection. arXiv preprint arXiv:1612.07811 .

Niederreiter, H. (1987). Point sets and sequences with small discrepancy. Monatshefte für Mathematik , 104:273-337.

Niederreiter, H. (1992). Random Number Generation and Quasi-Monte Carlo Methods . SIAM.

Owen, A. B. (1995). Randomly permuted ( t, m, s ) -nets and ( t, s ) -sequences. In Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing , pages 299-317, New York. Springer-Verlag.

Owen, A. B. (1997a). Monte Carlo variance of scrambled net quadrature. SIAM Journal of Numerical Analysis , 34(5):1884-1910.

Owen, A. B. (1997b). Scrambled net variance for integrals of smooth functions. Annals of Statistics , 25(4):1541-1562.

Owen, A. B. and Tribble, S. D. (2005). A quasi-Monte Carlo Metropolis algorithm. Proceedings of the National Academy of Sciences , 102(25):8844-8849.

Pagès, G. and Panloup, F. (2018). Weighted multilevel langevin simulation of invariant measures. Annals of Applied Probability , 28(6):3358-3417.

Raginsky, M., Rakhlin, A., and Telgarsky, M. (2017). Non-convex learning via stochastic gradient Langevin dynamics: a nonasymptotic analysis. In Conference on Learning Theory , pages 16741703. PMLR.

Roberts, G. O. and Tweedie, R. L. (1996). Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli , pages 341-363.

Rowland, M., Choromanski, K. M., Chalus, F., Pacchiano, A., Sarlos, T., Turner, R. E., and Weller, A. (2018). Geometrically coupled monte carlo sampling. Advances in Neural Information Processing Systems , 31.

Shi, J., Liu, C., and Mackey, L. (2022). Sampling with mirrored stein operators. In International Conference on Learning Representations .

Sobol', I. M. (1967). On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki , 7(4):784-802.

Tausworthe, R. C. (1965). Random numbers generated by linear recurrence modulo two. Mathematics of Computation , 19(90):201-209.

Teh, Y. W., Thiery, A. H., and Vollmer, S. J. (2016). Consistency and fluctuations for stochastic gradient Langevin dynamics. Journal of Machine Learning Research , 17.

Tribble, S. D. (2007). Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences . PhD thesis, Citeseer.

Vollmer, S. J., Zygalakis, K. C., and Teh, Y . W. (2016). Exploration of the (non-) asymptotic bias and variance of stochastic gradient Langevin dynamics. The Journal of Machine Learning Research , 17(1):5504-5548.

Welling, M. and Teh, Y. W. (2011). Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11) , pages 681-688.

Xu, P., Chen, J., Zou, D., and Gu, Q. (2018). Global convergence of Langevin dynamics based algorithms for nonconvex optimization. Advances in Neural Information Processing Systems , 31.

A Proofs

A.1 Proof of Theorem 4.1

We start by decomposing the error | 1 n ∑ n k =1 f ( θ k ) -π ( f ) | into three parts

We first upper bound ( I ) .

Lemma 1 (Upper bound of ( I ) ; adapted from Lemma 6.1.4 of Chen (2011)) . If the transition map ψ is a contraction with parameter ρ and if f is 1-Lipschitz, then

Proof of Lemma 1. Note that

To bound ( II ) , note that 1 n -glyph[lscript] ∑ n k = glyph[lscript] +1 ¯ f glyph[lscript] ( w ( glyph[lscript] ) k ) is estimating

Here, πP glyph[lscript] denote the distribution of the glyph[lscript] -step state θ glyph[lscript] starting from θ 0 ∼ π . So we have the further decomposition

The first term ( II ) ′ is due to the discretization in time. The second term ( II ) ′′ is the numerical integration error.

To bound ( II ) ′ , we use the following result.

Lemma 2 (Upper bound on discretization error ( II ) ′ ) . Under Assumption 1, we have for f 1Lipschitz,

Proof of Lemma 2. We let θ ( t ) be the continuous-time Langevin diffusion with θ (0) = θ 0 ∼ π , W t k +1 -W t k = √ hξ k +1 , where ξ k +1 iid ∼ N (0 , I d ) , t k = kh . So we have

and

Combing the previous two equations gives

Let ∆ k = θ ( t k ) -θ k . The last display reads

By the contracting property (6) in the main paper,

Taking expectation and use L -smoothness of U , we have

By Lemma 3 of Dalalyan and Karagulyan (2019), E [ ‖∇ U ( θ ) ‖ 2 2 ] ≤ Ld . So we have E [ ‖∇ U ( θ ) ‖ ] ≤ √ d E [ ‖∇ U ( θ ) ‖ 2 2 ] ≤ √ Ld . Because θ ( t ) is a stationary process,

Note that

Thus,

Denote r = 3 √ 2 2 Lh 3 / 2 d . So

Therefore, for any k ≥ 1 ,

If we use a noisy gradient ˆ g ( θ k ) = ∇ U ( θ k ) + e k where e k is the noise with mean zero and bounded variance such that E ( || e k || 2 2 ) ≤ σ 2 , then an extra term 2 hσ will appear in Lemma 2. As σ 2 is usually expected to be proportional to the dimension , this additional term is of the same order as the other term.

Theorem A.1 (Theorem 9.8 of Niederreiter (1992)) . Let v 0 , v 1 , . . . be an LFSR with offset s and period n = 2 m -1 which satisfy gcd ( m,n ) = 1 . Then the sequence { u i } n -1 i =0 ⊂ [0 , 1] s with u i = ( v i , v i +1 , . . . , v i + s -1 ) has, on average, star-discrepancy

with an implied constant depending only on d and the average is taken over all primitive polynomials over GF(2) of degree m .

Proof of Theorem 4.1. The error on the left-hand-side is bounded by

Lemma1shows that ( I ) ≤ (max 0 ≤ i<n ‖ θ i ‖ + E π [ ‖ θ ‖ ]) ρ glyph[lscript] ≤ (max 0 ≤ i ≤ n ‖ θ i ‖ + E π [ ‖ θ ‖ ]) h 1 / 2 since glyph[lscript] = glyph[ceilingleft] (1 / 2) log ρ h glyph[ceilingright] . Lemma 2 shows that ( II ) ′ ≤ 3 √ 2 2 L M dh 1 / 2 . Denote C 2 = max 0 ≤ i ≤ n ‖ θ i ‖ + E π [ ‖ θ ‖ ] + 3 √ 2 2 L M d . So ( I ) + ( II ) ′ ≤ C 2 h 1 / 2 .

By Theorem A.1 and the condition that gcd( dglyph[lscript],n ) = 1 , the star-discrepancy D ∗ ( { ¯ w ( glyph[lscript] ) k } k ≥ 1 ) is upper bounded by O ( n -1 (log n ) dglyph[lscript] +1 log log n ) . Finally, by Koksma-Hlawka inequality, we have ( II ) ′′ ≤ ‖ ¯ f glyph[lscript] ‖ HK · D ∗ ( { ¯ w ( glyph[lscript] ) k } k ≥ 1 ) . Thus, ( II ) ′′ + 4 glyph[lscript] n ‖ f ‖ ∞ ≤ C 1 n -1+ δ , where δ hides the polylogarithmic terms in log n and C 1 depends on d, glyph[lscript], ‖ ¯ f glyph[lscript] ‖ HK.

Therefore, the upper bound becomes

B Additional numerical results

The primary contribution of this work is to improve LMC as a Monte Carlo sampling algorithm, not as an optimization algorithm. Therefore, our main focus is on providing a better estimation of π ( f ) for some function of interest. Downstream tasks relying on such expectations can also benefit from LQMC. For posterior prediction, it is essential to recognize that the prediction error is not solely determined by the sampling method. Even with infinite perfect samples from the posterior, the prediction error can still arise due to model misspecification, noisy data, biased sampling, etc. So the improvement achieved by LQMC might be less pronounced when assessing the prediction error.

To investigate the performance of LQMC in a posterior prediction setting, we conducted experiments similar to those presented in Dubey et al. (2016) using three UCI datasets. Each dataset was split into a training set (70%), a validation set (10%), and a test set (20%). We performed a tuning process for the constant step size on a grid using the validation set and evaluated the prediction error on the test set. Each iteration computes the stochastic gradient using 32 data points sampled at random. Details of the datasets are in Table 1.

The results are presented in Figure 6. The x -axes represent the total number of iterations of Langevin algorithm and the y -axes represent the test error. The error bars represent the variation across 10 random replicates. It is evident that LQMC reduced the test error, although the improvement is not substantial. This aligns with our initial expectation, as the proposed method primarily enhances the accuracy of estimating the posterior mean. However, the test error often consists of other sources of error, thus the improvement achieved by the proposed method in reducing the test error might be limited.

2 10

2 11

2 12

2 13

2 14

0.095

0.100

0.105

0.110

0.115

0.120

0.125

0.130

0.135

Test error

Parkinsons

MC

CUD

2 10

2 11

2 12

2 13

2 14

n

0.00000

0.00005

0.00010

0.00015

0.00020

0.00025

0.00030

Test error

Bike

MC

CUD

2 12

2 13

2 14

2 15

2 16

n

0.7205

0.7210

0.7215

0.7220

0.7225

Test error

Protein

MC

CUD

n

Peer reviews:
comments: SUMMARY:
This paper analyzes the effect of using quasi-random numbers in place of the usual IID Gaussians for the driving noise of a Langevin algorithm. Assuming that the loss is strong convex and a the quasi-random numbers are completely uniformly distributed, a bound on the Monte Carlo estimation error is derived. This bound is substantially better than what is currently achieved for IID Gaussian driving noise.

STRENGTHS:
The paper makes a nice contribution to the theory of sampling with Langevin dynamics, by bringing in the use of quasi-Monte Carlo methods. The proof is quite clean, and it is very clear where the power of the proposed method becomes useful. 

The experiments are promising.

WEAKNESSES:
My only real concern is that this paper focuses on a relatively easy case, albeit an important one. It would be interesting to see how easy or difficult it is to extend the results to more complex scenarios, such as with non-convex losses or external random variables.

QUESTIONS:
* Can the result be extended to common measures between distributions? For example, the result looks almost like a bound on the 1-Wasserstein distance between the distribution of the iterates and the stationary distribution. The main difference is that it assumed that $f$ is both bounded and Lipschitz, as opposed to just Lipschitz.
* Are there general classes of strongly convex functions for which the you can check that the Hardy-Krause variation of $\bar f_{\ell}$ is finite?

LIMITATIONS:
These are adequately addressed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nFor suitably smooth functions defined on a bounded support, it is well known that quasi Monte Carlo (QMC) can achieve faster rates of convergence than standard Monte Carlo when the goal is to integrate the function. This paper studies whether techniques from QMC can be beneficial for Langevin Monte Carlo (LMC), a sampling procedure based on the discretization of stochastic differential equations. The paper shows both theoretically and empirically that when the function $f$ is suitably smooth, one can augment the standard Gaussian perturbations in LMC with a specialized sequence such that the error rates converge at a faster rate than standard LMC.\n\nSTRENGTHS:\nThe paper is very easy to follow and understand. The authors do a great job of laying out the appropriate context, defining the necessary notions, and laying out their new procedure. \n\nThe idea, while relatively simple to implement, seems quite powerful. It’s simplicity seems like a feature rather than bug, and should make it relatively easy to incorporate into open source implementations.\n\nThe main result and empirical work seem sound. And while QMC is not new, incorporating it into LMC would be a nice extension of work that has previously been confined to relatively small use cases.\n\nWEAKNESSES:\nThe main critiques of this paper are associated with its overall applicability. On the theoretical side, the authors make some assumptions to prove their result. How strong are these assumptions? Can they apply generally as the number of samples increases?\n\nIn terms of the empirical work, all the examples are done on toy models where the final distributions are known or can be estimated for long chains. Can the authors show LQMC is useful for problems with real data? That would bolster the case for using this as a drop in replacement for vanilla LMC.\n\nQUESTIONS:\n[Q1] The main result requires $\\bar{f}_l$ to have bounded variation. Is this possible to show this for any of your examples, i.e. if f is Lipschitz and the target distribution is Gaussian? It would be great to know whether this assumption is weak or strong. \n\n[Q2] Given the main results assumptions on d, m and n, can this result always be applicable for values of n going to infinity?\n\n[Q3] The empirical examples only demonstrate the utility of LQMC on toy models. Can the authors demonstrate its usefulness in a Bayesian context on real data, say for improving the posterior predictions associated with some application?\n\nLIMITATIONS:\nIt would be great if the authors mentioned the importance of doing proper inference in ML.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors proposed the Langevin quasi Monte Carlo algorithm (LQMC), that is, to use a completely uniformly distributed series (CUD quasi random number) in the Langevin Monte Carlo instead of iid pseudo random numbers. The quasi-random number is generated by the LFSR method. For a smooth and strongly convex potential, the estimation error scales as $O(n^{-1 + \\delta})$, where $\\delta$ encodes the dependence on $\\log n$ and $d$. The numerical experiment is performed for various $d$, using gradient or stochastic gradient. In all the cases, LQMC has better performance than the Langevin MC.\n\nSTRENGTHS:\n* Originality: It appears to me that the combination of CUD and Langevin MC is new.\n* Quality: This paper contains rigorous derivation of error bounds and numerical experiments covering a wide range of cases. Both derivation and experiment seem convincing for me.\n* Clarity: This is a well written paper in general. This paper demonstrates the backbone idea well by using examples and pseudo codes. This paper also provides a good context for the discussion. The setup for the experiments are stated clearly, and the numerical results are interpreted. I also appreciate attaching the code together with the paper.\n* Significance: This paper shows that, without making any change to the Langevin MC, changing the random number in the algorithm can lead to vast performance gain. If the performance gain shown in the paper migrates to realistic problems, the precision of a lot of computational work could be improved without a lot of software development.\n\nWEAKNESSES:\n* The numerical experiments are performed with synthesized data instead of real ones.\n* Unlike classic Monte Carlo, the scaling of error of quasi-Monte Carlo depends on the dimension of the system $d$. As a result, when $n$ is small and $d$ is large, quasi-Monte Carlo may perform worse than classic Monte Carlo in theory. In this paper, the error still depends on $d$ if I am not mistaken. The $d$ dependence is encoded in $\\delta$, and not shown explicitly in the main text. Although, in the numerical experiments, it is shown that the LQMC performs very well for $d=100$, I still think it would be nice to make the reader know that, in theory, the error of LQMC depends on $d$.\n* The error bound is given in a rather constraint setting.\n\nThe weaknesses and questions are addressed by the authors in their rebuttal.\n\nQUESTIONS:\nOverall, I think this is already a very solid paper. However, I think it would be nice if the authors:\n* State what this algorithm could do in reality. New experiments using realistic data are welcomed. Discussions on the time scale of the realistic tasks are also interesting (for example, is there any task that was impossible and made possible by LQMC).\n* Write the error bound in the format of equation (4), that it, express the $d$ dependence explicitly.\n\nLIMITATIONS:\nMy only concern regarding the limitation is that the dependence on $d$ of estimation error is not well studied. After all, $d$ is the criterion of when we should use lattice rules, quasi MC, or MC. However, I understand that the dimension of quasi MC is itself a tricky problem, and, in my humble opinion some ambiguity in this regard could be forgiven.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper analyses a variant of Langevin Monte Carlo which, instead of using standard Gaussian random variables (using standard random number generation), instead uses correlated random variables following the quasi-MCMC literature. The paper demonstrates that this provably improves the efficiency of certain statistical tests, such as e.g. mean estimation.\n\nSTRENGTHS:\nThe usage of non-random Gaussians (using an LSFR sequence instead of standard Mersenne twister) is a novelty for Langevin Monte Carlo. Consequently, the Algorithm is entirely novel, and is provably better in the sense shown in Theorem 4.1. This improvement is a significant technical innovation and is of use to both theoreticians and practitioners.\n\nThe experiments are thorough, in that many settings are considered, both in terms of convexity, dimensionality and type of gradient. The improvement for LQMC is clear and significant, which therefore lends credence to the theoretical claims.\n\nThe paper is very well written and the proofs are cleanly presented. I could not find any issues with the results.\n\nWEAKNESSES:\nThe improvement of this method is weak, in the sense that the expectation of certain test functions converges . In contrast, standard Langevin MCMC results will hold in “stronger” measures of convergence, such as KL divergence, total variation, etc. This is expected since quasi-MCMC methods typically can only outperform in these specific circumstances, but is nonetheless a disadvantage of the work.\n\nThe analysis seems to be rather trivial and combines the standard inequalities in previous works on quasi-MCMC, with the standard contractivity results of LMC-type algorithms under strong convexity. \n\nOverall, while the analytic simplicity of this paper makes the result seem obvious in hindsight, I would still argue that this result is novel and impactful enough to merit publication.\n\nQUESTIONS:\nAre the assumptions of smoothness, strong convexity and bounded variation (of f) necessary? I would recommend a deeper exploration of these conditions in order to enrich the paper.\n\nAlthough stochastic gradient LMC appears in the experiments, it is not analysed. Could similar results be established in this setting?\n\nCould this approach be combined with e.g. Metropolis/HMC algorithms, or other MCMC methods based on similar walks? It seems there may be some significant theoretical barriers to this end.\n\nTypos: \nSome citations are not properly formatted, e.g. in L. 22, L. 126, L. 151 (using citet instead of citep or vice versa).\n\nLIMITATIONS:\nNone beyond those raised in my earlier comments.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-07"}