SlowGuess/NeurIPS_2023_OCR huggingface dataset:
See e.g. Lemma 2 of Dalalyan and Karagulyan (2019). The strong contraction implies that if we start two chains from  $\theta$  and  $\theta'$ , and use the same random numbers at every step, then the two chains will merge exponentially fast. In other words, the state  $\theta_k$  largely depends on the most recent iterations and quickly forgets about the past history. Formally, let  $\mathbf{w}_k^{(\ell)} = (\mathbf{u}_k, \dots, \mathbf{u}_{k - \ell + 1})$  denote the random numbers used in the most recent  $\ell$  steps. Define the  $\ell$ -step transition as
$$
\boldsymbol {\theta} _ {k} = \psi_ {\ell} (\boldsymbol {\theta} _ {k - \ell}, \mathbf {w} _ {k} ^ {(\ell)})
$$
and let  $\bar{f}_{\ell}(\mathbf{w}_k^{(\ell)})$  denote the value of  $f(\theta_k)$  marginalized over  $\theta_{k - \ell}\sim \pi$ , i.e.
$$
\bar {f} _ {\ell} (\mathbf {w} _ {k} ^ {(\ell)}) = \int f \circ \psi_ {\ell} (x, \mathbf {w} _ {k} ^ {(\ell)}) \pi (\mathrm {d} x).
$$
Thus  $\bar{f}_{\ell}(\mathbf{w}_k^{(\ell)})$  only depends on the most recent  $\ell$  iterations. Due to the strong contraction,  $|\bar{f}_{\ell}(\mathbf{w}_k^{(\ell)}) - f(\theta_k)|$  decays exponentially fast with  $\ell$ . So for large  $\ell$ , the estimation error of  $n^{-1}\sum_{k=1}^{n}f(\theta_k)$  is close to the error of  $\frac{1}{n-\ell}\sum_{k=\ell+1}^{n}\bar{f}_{\ell}(\mathbf{w}_k^{(\ell)})$ . The latter can be viewed as a  $d\ell$ -dimensional numerical integration scheme based on the point set  $\{\mathbf{w}_k^{(\ell)}\}_{k=\ell+1}^n$ . By leveraging the discrepancy bound of the LFSR sequence and assuming that  $\bar{f}_{\ell}$  has bounded variation in the sense of Hardy and Krause, we can derive an error bound using the Koksma-Hlawka inequality (4). Now we state the main error bound and leave the detailed proof in the Appendix A.
Theorem 4.1 (Error bound of LQMC). Let Assumption 1 hold. Define the step size  $h \leq \frac{2}{L + M}$ ,  $\rho = 1 - hM$ ,  $\ell = \lceil (1/2) \log_{\rho} h \rceil$ . Let  $\theta_1, \ldots, \theta_n$  be the output of Algorithm 1 which runs  $n$  iterations with step size  $h \leq \frac{2}{L + M}$ . Assume the LFSR sequence  $\{v_i\}_{i \geq 0}$  in use has period  $n = 2^m - 1$ , offset  $s$ , and  $\gcd(m, n) = \gcd(d\ell, n) = 1$ . If  $\bar{f}_{\ell}$  has bounded variation in the sense of Hardy and Krause, then as  $n \to \infty$  we have
$$
\left| \frac {1}{n} \sum_ {k = 1} ^ {n} f (\theta_ {k}) - \pi (f) \right| \leq C _ {1} n ^ {- 1 + \delta} + C _ {2} h ^ {1 / 2}, \quad \forall \delta > 0.
$$
Here  $\delta$  hides poly-logarithmic factors  $(\log n)^{d}$ ,  $C_1$  depends on  $d, \ell$  and  $\| \bar{f}_{\ell} \|_{HK}$ , and  $C_2 = \frac{3\sqrt{2}}{2} \frac{L}{M} d + \max_{0 \leq k \leq n} \| \theta_k \| + \mathbb{E}_{\pi} [ \| \theta \| ]$ .
The upper bound consists of two terms. The first term represents the numerical integration error, which arises from the discrepancy of the point set used in the integration scheme. By utilizing low-discrepancy CUD sequences, we can reduce this numerical integration error (the first term) from the standard rate of  $O(n^{-1/2})$  to a faster rate of  $O(n^{-1+\delta})$  for any  $\delta > 0$ . However, it is important to note that when using a constant step size  $h$  in LMC, the bias term (second term) does not vanish. This bias term includes not only the discretization error of the Langevin diffusion, but also the difference between  $f(\theta_k)$  and its truncated version  $\bar{f}_{\ell}(\mathbf{w}_k^{(\ell)})$ . Consequently, the bias term in our analysis is larger than the bias term in Vollmer et al. (2016), which employs different techniques and assumptions based on the Poisson equation.
The theorem's assumption of finite Hardy-Krause variation is a common requirement in error bounds for QMC methods, and it can be challenging to verify in practice. Basu and Owen (2016) provide sufficient conditions in order for  $f \circ \psi_{\ell}$  to have finite HK variation, requiring the  $\ell$ -step transition  $\psi_{\ell}$  to be sufficiently smooth. In the next section, we aim to assess the practical performance of the proposed LQMC algorithm through numerical experiments.
To comprehensively evaluate the performance of the algorithm, we will consider both convex and non-convex potentials, both low-dimensional and high-dimensional state spaces, both accurate and stochastic gradients, both smooth and discontinuous integrands, as well as different learning rate schedules. Additional numerical results can be found in the Appendix B.
Figure 2: Bayesian logistic regression with accurate gradients (top) and stochastic gradients (bottom).
5.1 Bayesian logistic regression
We first consider the Bayesian logistic model
$$
y _ {i} \mid x _ {i} \sim \operatorname {B e r n o u l l i} \left(\left(1 + \exp \left(- x _ {i} ^ {\mathbf {T}} \beta\right)\right) ^ {- 1}\right), \quad 1 \leq i \leq N,
$$
$$
\beta \sim \mathcal {N} (0, I _ {d}).
$$
We take  $N = 20$ ,  $d = 10$ . The features  $x_{i}$  are generated from  $\mathcal{N}(0,\Sigma)$  with  $\Sigma_{ij} = 2^{-|i - j|}$ . The coefficients  $\beta$  and the data  $y_{i}$ 's are generated from the same model. We consider the test functions  $f(x) = x_{j}, x_{j}^{2}, \mathbf{1}_{\{x_{j} > 0\}}$  for  $j = 1, \ldots, d$ . The step size  $h$  is fixed to 0.001.
We compute the MSE of the estimator based on usual LMC and the proposed LQMC with CUD sequences and report the MSE averaged over all coordinates and 20 random replicates. We do not have a closed form for the expectations  $\mathbb{E}[f]$ , so the ground truth is estimated using a high-accuracy estimator proposed in He et al. (2023) using scrambled Sobol' sequence with a very large sample size.
In Figure 2 (top panel), we present a log-log plot of the MSE against the number of iterations. Across all three test functions, we observe that LQMC reduces the MSE by a factor ranging from 4 to 8. As the number of iterations increases, the curve corresponding to LQMC reaches a plateau. This behavior can be attributed to the discretization error inherent in the unadjusted LMC, which cannot be further reduced by increasing the number of iterations.
In the bottom panel of Figure 2, we increase the number of observations to  $N = 100$  and incorporate stochastic gradient estimation in the Langevin algorithm. Specifically, at each iteration, we estimate the gradient using a random subset of 10 observations. The results demonstrate that LQMC still provides a big improvement when  $n$  is smaller than  $2^{14}$ . However, as  $n$  surpasses  $2^{14}$ , we observe that the LQMC curve flattens again. It is worth noting that the improvement achieved by LQMC in this scenario is less pronounced compared to the previous example, primarily due to the presence of noise in the gradient estimates.
5.2 Bayesian linear regression
Now we try a higher-dimensional example with Bayesian linear regression. The model is defined as
$$
y _ {i} \sim \mathcal {N} \left(x _ {i} ^ {\intercal} \beta , \sigma^ {2} = 4 ^ {- 1}\right), \quad 1 \leq i \leq N,
$$
$$
\beta \sim \mathcal {N} (0, I).
$$
We take  $d = 100$  and  $N = 20$ . We generate  $x_{i}\in \mathbb{R}^{d}$  similarly as in the logistic regression example. The test functions and step size are also unchanged. The posterior distribution of  $\beta$  has the closed form  $\mathcal{N}\left((\frac{X^{\intercal}X}{\sigma^2} +I)^{-1}\frac{X^{\intercal}Y}{\sigma^2},(\frac{X^{\intercal}X}{\sigma^2} +I)^{-1}\right)$ . The results are shown in Figure 3. We see that even at 100 dimension, LQMC still brings a substantial improvement over LMC in terms of MSE. In
Figure 3: Bayesian linear regression in 100 dimensions.
Figure 4: Crossed random effect.
particular, for the integrand  $f(x) = x_{j}$ , LQMC achieves a reduction in MSE of approximately 500-fold compared to LMC.
5.3 A hierarchical Bayesian model
We consider a hierarchical Bayesian model known as the crossed random effect model
$$
Y _ {i j} \sim \mathcal {N} (\mu + a _ {i} + b _ {j}, 1), \quad 1 \leq i \leq I, 1 \leq j \leq J,
$$
$$
\mu \sim \mathcal {N} (0, 1), a _ {i} \stackrel {i i d} {\sim} \mathcal {N} (0, \sigma_ {a} ^ {2}), b _ {j} \stackrel {i i d} {\sim} \mathcal {N} (0, \sigma_ {b} ^ {2}),
$$
$$
\log (\sigma_ {a} ^ {2}), \log (\sigma_ {b} ^ {2}) \stackrel {i i d} {\sim} \mathcal {N} (0, 1).
$$
The goal is to sample from the posterior distribution of  $(\mu, \mathbf{a}, \mathbf{b}, \log(\sigma_a^2), \log(\sigma_b^2))$ , which has dimension  $d = I + J + 3$ . We take  $I = 3$ ,  $J = 5$ . We will consider the test functions  $f(x) = x_j$  ( $1 \leq j \leq d$ ). The ground truth of  $\mathbb{E}[f(x)]$  is estimated by Langevin dynamics with Metropolis adjustments (MALA) using a large sample size.
We will compare the performance of the LQMC algorithm using three different step sizes: a constant step size of  $10^{-4}$ , a constant step size of  $10^{-2}$ , and decreasing step sizes with  $h_k = c_0(c_1 + k)^{-1/3}$ . The choice of  $c_0$  and  $c_1$  ensures that the step size decreases from  $10^{-2}$  to  $10^{-4}$  throughout the entire algorithm. The use of the exponent  $-1/3$  in the decreasing step sizes is recommended in Teh et al. (2016). The results of these comparisons are presented in Figure 4.
In the small step size case (left panel), we observe that the errors of LMC and LQMC are initially comparable for small values of  $n$ . This is because the algorithm converges slowly, and thus the error is dominated by the bias. However, as  $n$  increases, the improvement of LQMC becomes evident. In the large step size case (middle panel), the MSE of LQMC is consistently smaller than that of LMC even for small values of  $n$ . This is because the algorithm converges faster to the target distribution with a larger step size  $h$ . Therefore, the improvement of LQMC is more pronounced. Interestingly, in this particular example, using decreasing step sizes yields similar accuracy to using a constant step size of  $10^{-4}$ . It is worth noting that the MSE of LMC does not decrease at a rate of  $n^{-2/3}$  as in Teh et al. (2016). This is because the line in the plot does not represent the accuracy against the iteration  $k$  within a single training process. Instead, it reflects the accuracy achieved after completing all  $n$  iterations of the algorithm, considering different values of  $n$ .
Figure 5: Double well potential.
5.4 Nonconvex potential
Finally we investigate a double-well potential function  $U(x) = \frac{1}{4} x^2 - \frac{1}{2}\log (1 + x^2)$  from Pages and Panloup (2018). We know  $\mathbb{E}[x] = 0$  and  $\mathbb{E}\left[\mathbf{1}_{\{x > 0\}}\right] = 0.5$ . The second moment  $\mathbb{E}\left[x^{2}\right]$  is computed by Gaussian quadrature. See the results in Figure 5. Since the potential has two separate local minimums, it takes longer for the Langevin algorithm to explore the space sufficiently and converge to the target distribution. Once converged, the improvement of LQMC over LMC is still significant.
The author thanks Prof. Art Owen for helpful conversations. This work was partially funded by the NSF grant DMS-2152780 and the Stanford Data Science Scholars program.
Arnold, S. M., L'Ecuyer, P., Chen, L., Chen, Y.-F., and Sha, F. (2022). Policy learning and evaluation with randomized quasi-Monte Carlo. In International Conference on Artificial Intelligence and Statistics, pages 1041-1061. PMLR.
Baker, J., Fearnhead, P., Fox, E. B., and Nemeth, C. (2019). Control variates for stochastic gradient MCMC. Statistics and Computing, 29:599-615.
Basu, K. and Owen, A. B. (2016). Transformations and Hardy-Krause variation. SIAM Journal on Numerical Analysis, 54(3):1946-1966.
Brosse, N., Durmus, A., and Moulines, E. (2018). The promises and pitfalls of stochastic gradient Langevin dynamics. Advances in Neural Information Processing Systems, 31.
Buchholz, A., Wenzel, F., and Mandt, S. (2018). Quasi-Monte Carlo variational inference. In International Conference on Machine Learning, pages 668-677. PMLR.
Chen, C., Carlson, D., Gan, Z., Li, C., and Carin, L. (2016). Bridging the gap between stochastic gradient MCMC and stochastic optimization. In Artificial Intelligence and Statistics, pages 1051-1060. PMLR.
Chen, C., Ding, N., and Carin, L. (2015). On the convergence of stochastic gradient MCMC algorithms with high-order integrators. Advances in neural information processing systems, 28.
Chen, S. (2011). Consistency and convergence rate of Markov chain quasi Monte Carlo with examples. PhD thesis, Stanford University.
Chen, S., Dick, J., and Owen, A. (2011). Consistency of Markov chain quasi-Monte Carlo on continuous state spaces. The Annals of Statistics, 39(2):673-701.
Cheng, X., Chatterji, N. S., Bartlett, P. L., and Jordan, M. I. (2018). Underdamped Langevin MCMC: A non-asymptotic analysis. In Conference on learning theory, pages 300-323. PMLR.
Niederreiter, H. (1992). Random Number Generation and Quasi-Monte Carlo Methods. SIAM.
Owen, A. B. (1995). Randomly permuted  $(t,m,s)$ -nets and  $(t,s)$ -sequences. In *Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing*, pages 299-317, New York. Springer-Verlag.
Owen, A. B. (1997a). Monte Carlo variance of scrambled net quadrature. SIAM Journal of Numerical Analysis, 34(5):1884-1910.
Owen, A. B. (1997b). Scrambled net variance for integrals of smooth functions. Annals of Statistics, 25(4):1541-1562.
Owen, A. B. and Tribble, S. D. (2005). A quasi-Monte Carlo Metropolis algorithm. Proceedings of the National Academy of Sciences, 102(25):8844-8849.
Pages, G. and Panloup, F. (2018). Weighted multilevel Langevin simulation of invariant measures. Annals of Applied Probability, 28(6):3358-3417.
Raginsky, M., Rakhlin, A., and Telgarsky, M. (2017). Non-convex learning via stochastic gradient Langevin dynamics: a nonasymptotic analysis. In Conference on Learning Theory, pages 1674–1703. PMLR.
Roberts, G. O. and Tweedie, R. L. (1996). Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli, pages 341-363.
Rowland, M., Choromanski, K. M., Chalus, F., Pacchiano, A., Sarlos, T., Turner, R. E., and Weller, A. (2018). Geometrically coupled monte carlo sampling. Advances in Neural Information Processing Systems, 31.
Shi, J., Liu, C., and Mackey, L. (2022). Sampling with mirroredstein operators. In International Conference on Learning Representations.
Sobol', I. M. (1967). On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal Vychislitel'noi Matematiki i Matematicheskoi Fiziki, 7(4):784-802.
Tausworth, R. C. (1965). Random numbers generated by linear recurrence modulo two. Mathematics of Computation, 19(90):201-209.
Teh, Y. W., Thiery, A. H., and Vollmer, S. J. (2016). Consistency and fluctuations for stochastic gradient Langevin dynamics. Journal of Machine Learning Research, 17.
Tribble, S. D. (2007). Markov chain Monte Carlo algorithms using completely uniformly distributed driving sequences. PhD thesis, CiteSeer.
Vollmer, S. J., Zygalakis, K. C., and Teh, Y. W. (2016). Exploration of the (non-) asymptotic bias and variance of stochastic gradient Langevin dynamics. The Journal of Machine Learning Research, 17(1):5504-5548.
Welling, M. and Teh, Y. W. (2011). Bayesian learning via stochastic gradient Langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 681-688.
Xu, P., Chen, J., Zou, D., and Gu, Q. (2018). Global convergence of Langevin dynamics based algorithms for nonconvex optimization. Advances in Neural Information Processing Systems, 31.
We start by decomposing the error  $\left| \frac{1}{n} \sum_{k=1}^{n} f(\theta_k) - \pi(f) \right|$  into three parts
$$
\begin{array}{l} \left| \frac {1}{n} \sum_ {k = 1} ^ {n} f (\theta_ {k}) - \pi (f) \right| \leq \left| \frac {1}{n} \sum_ {k = \ell + 1} ^ {n} f (\theta_ {k}) - \frac {1}{n} \sum_ {k = \ell + 1} ^ {n} \bar {f} _ {\ell} (\mathbf {w} _ {k} ^ {(\ell)}) \right| + \left| \frac {1}{n} \sum_ {k = \ell + 1} ^ {n} \bar {f} _ {\ell} (\mathbf {w} _ {k} ^ {(\ell)}) - \pi (f) \right| + \frac {\ell}{n} 2 \| f \| _ {\infty} \\ = (I) + (I I) + \frac {2 \ell}{n} \| f \| _ {\infty}. \\ \end{array}
$$
We first upper bound  $(I)$ .
Lemma 1 (Upper bound of  $(I)$ ; adapted from Lemma 6.1.4 of Chen (2011)). If the transition map  $\psi$  is a contraction with parameter  $\rho$  and if  $f$  is 1-Lipschitz, then
$$
| \bar {f} _ {\ell} (\mathbf {w} _ {k} ^ {(\ell)}) - f (\theta_ {k}) | \leq \left(\max  _ {0 \leq i \leq n} \| \theta_ {i} \| + \mathbb {E} _ {\pi} [ \| \theta \| ]\right) \rho^ {\ell}.
$$
Proof of Lemma 1. Note that
$$
\begin{array}{l} \left| \bar {f} _ {\ell} \left(\mathbf {w} _ {k} ^ {(\ell)}\right) - f \left(\theta_ {k}\right) \right| \leq \int | f \left(\psi_ {\ell} \left(\theta , \mathbf {w} _ {k} ^ {(\ell)}\right)\right) - f \left(\psi_ {\ell} \left(\theta_ {k - \ell}, \mathbf {w} _ {k} ^ {(\ell)}\right)\right) | \pi (\mathrm {d} \theta) \\ \leq \int \| (\psi_ {\ell} (\theta , \mathbf {w} _ {k} ^ {(\ell)})) - (\psi_ {\ell} (\theta_ {k - \ell}, \mathbf {w} _ {k} ^ {(\ell)})) \| \pi (\mathrm {d} \theta) \\ \leq \rho \int \| (\psi_ {\ell - 1} (\theta , \mathbf {w} _ {k - 1} ^ {(\ell - 1)})) - (\psi_ {\ell - 1} (\theta_ {k - \ell}, \mathbf {w} _ {k - 1} ^ {(\ell - 1)})) \| \pi (d \theta) \\ \leq \rho^ {\ell} \int \| \theta - \theta_ {k - \ell} \| \pi (\mathrm {d} \theta) \\ \leq \rho^ {\ell} (\max  _ {0 \leq i \leq n} \| \theta_ {i} \| + \mathbb {E} _ {\pi} [ \| \theta \| ]). \\ \end{array}
$$
To bound (II), note that  $\frac{1}{n - \ell}\sum_{k = \ell +1}^{n}\bar{f}_{\ell}(\mathbf{w}_k^{(\ell)})$  is estimating
$$
\mathbb {E} \left[ \bar {f} _ {\ell} \left(\mathbf {w} ^ {(\ell)}\right) \right] = \int \psi_ {\ell} (\theta , \mathbf {w} ^ {(\ell)}) \pi (\mathrm {d} \theta) \mathrm {d} \mathbf {w} ^ {(\ell)} =: \pi P _ {\ell} (f).
$$
Here,  $\pi P_{\ell}$  denote the distribution of the  $\ell$ -step state  $\theta_{\ell}$  starting from  $\theta_0\sim \pi$ . So we have the further decomposition
$$
\begin{array}{l} (I I) \leq \left| \frac {1}{n - \ell} \sum_ {k = \ell + 1} ^ {n} \bar {f} _ {\ell} \left(\mathbf {w} _ {k} ^ {(\ell)}\right) - \pi (f) \right| + \frac {\ell}{n - \ell} \| f \| _ {\infty} \\ \leq | \pi (f) - \pi P _ {\ell} (f) | + \left| \frac {1}{n - \ell} \sum_ {k = \ell + 1} ^ {n} \bar {f} _ {\ell} \left(\mathbf {w} _ {k} ^ {(\ell)}\right) - \pi P _ {\ell} (f) \right| + \frac {\ell}{n - \ell} \| f \| _ {\infty} \\ \leq (I I) ^ {\prime} + (I I) ^ {\prime \prime} + \frac {\ell}{n - \ell} \| f \| _ {\infty}. \\ \end{array}
$$
The first term  $(II)'$  is due to the discretization in time. The second term  $(II)''$  is the numerical integration error.
To bound  $(II)'$ , we use the following result.
Lemma 2 (Upper bound on discretization error  $(II)'$ ). Under Assumption 1, we have for  $f$  1-Lipschitz,
$$
| \pi (f) - \pi P _ {\ell} (f) | \leq \frac {3 \sqrt {2}}{2} \frac {L}{M} h ^ {1 / 2} d.
$$
Proof of Lemma 2. We let  $\theta(t)$  be the continuous-time Langevin diffusion with  $\theta(0) = \theta_0 \sim \pi$ ,  $W_{t_{k+1}} - W_{t_k} = \sqrt{h}\xi_{k+1}$ , where  $\xi_{k+1} \stackrel{iid}{\sim} \mathcal{N}(0, I_d)$ ,  $t_k = kh$ . So we have
$$
\theta (t _ {k + 1}) = \theta (t _ {k}) - \int_ {t _ {k}} ^ {t _ {k + 1}} \nabla U (\theta (s)) \mathrm {d} s + \sqrt {2 h} \xi_ {k + 1}
$$
$$
\theta_ {k + 1} = \theta_ {k} - h \nabla U (\theta_ {k}) + \sqrt {2 h} \xi_ {k + 1}.
$$
Combing the previous two equations gives
$$
\theta (t _ {k + 1}) - \theta_ {k + 1} = \theta (t _ {k}) - \theta_ {k} - h [ \nabla U (\theta (t _ {k})) - \nabla U (\theta_ {k}) ] - \int_ {t _ {k}} ^ {t _ {k + 1}} \nabla U (\theta (s)) - \nabla U (\theta (t _ {k})) \mathrm {d} s.
$$
Let  $\Delta_k = \theta(t_k) - \theta_k$ . The last display reads
$$
\Delta_ {k + 1} = \Delta_ {k} - h [ \nabla U (\theta_ {k} + \Delta_ {k}) - \nabla U (\theta_ {k}) ] - \int_ {t _ {k}} ^ {t _ {k + 1}} \nabla U (\theta (s)) - \nabla U (\theta (t _ {k})) \mathrm {d} s.
$$
By the contracting property (6) in the main paper,
$$
\left\| \Delta_ {k} - h \left[ \nabla U \left(\theta_ {k} + \Delta_ {k}\right) - \nabla U \left(\theta_ {k}\right) \right] \right\| \leq \rho \| \Delta_ {k} \|.
$$
Taking expectation and use  $L$ -smoothness of  $U$ , we have
$$
\mathbb {E} \left[ \| \Delta_ {k + 1} \| \right] \leq \rho \mathbb {E} \left[ \| \Delta_ {k} \| \right] + L \int_ {t _ {k}} ^ {t _ {k + 1}} \mathbb {E} \left[ \| \theta (s) - \theta (t _ {k}) \| \right] d s.
$$
By Lemma 3 of Dalalyan and Karagulyan (2019),  $\mathbb{E}\left[\|\nabla U(\theta)\|_2^2\right] \leq Ld$ . So we have  $\mathbb{E}\left[\|\nabla U(\theta)\|\right] \leq \sqrt{d\mathbb{E}\left[\|\nabla U(\theta)\|_2^2\right]} \leq \sqrt{L}d$ . Because  $\theta(t)$  is a stationary process,
$$
\begin{array}{l} \int_ {t _ {k}} ^ {t _ {k + 1}} \mathbb {E} \left[ \| \theta (s) - \theta \left(t _ {k}\right) \| \right] d s = \int_ {0} ^ {h} \mathbb {E} \left[ \| \theta (t) - \theta (0) \| \right] d t \\ = \int_ {0} ^ {h} \mathbb {E} \left[ \| - \int_ {0} ^ {t} \nabla U (\theta (s)) \mathrm {d} s + \sqrt {2} W _ {t} \| \right] \mathrm {d} t \\ \leq \int_ {0} ^ {h} \int_ {0} ^ {t} \mathbb {E} \left[ \| \nabla U (\theta (s)) \| \right] d s d t + \int_ {0} ^ {h} \sqrt {2} \mathbb {E} \left[ \| W _ {t} \| \right] d t \\ = \frac {h ^ {2}}{2} \sqrt {L} d + \int_ {0} ^ {h} \sqrt {2 t} \mathbb {E} [ \| \xi_ {1} \| ] \mathrm {d} t. \\ \end{array}
$$
$$
\mathbb {E} \left[ \| \xi_ {1} \| \right] = \sqrt {2} \frac {\Gamma (d / 2 + 1 / 2)}{\Gamma (d / 2)} \leq \sqrt {2} \left(\frac {d + 1}{2}\right) ^ {1 / 2} = \sqrt {d + 1}.
$$
$$
\begin{array}{l} \int_ {t _ {k}} ^ {t _ {k + 1}} \mathbb {E} \left[ \left\| \theta (s) - \theta \left(t _ {k}\right) \right\| \right] d s \leq \frac {1}{2} L ^ {1 / 2} h ^ {2} d + \frac {3 \sqrt {2}}{2} h ^ {3 / 2} d ^ {1 / 2} \\ \leq \frac {\sqrt {2}}{2} h ^ {3 / 2} d + \frac {3 \sqrt {2}}{2} h ^ {3 / 2} d ^ {1 / 2} \\ \leq \frac {3 \sqrt {2}}{2} h ^ {3 / 2} d. \\ \end{array}
$$
Denote  $r = \frac{3\sqrt{2}}{2} Lh^{3 / 2}d$  So
$$
\begin{array}{l} \mathbb {E} \left[ \| \Delta_ {k + 1} \| \right] \leq \rho \mathbb {E} \left[ \| \Delta_ {k} \| \right] + r \leq \rho^ {k + 1} \mathbb {E} \left[ \| \Delta_ {0} \| \right] + \sum_ {i = 0} ^ {k} \rho^ {i} r \\ \leq \frac {r}{1 - \rho} = \frac {3 \sqrt {2}}{2} \frac {L}{M} h ^ {1 / 2} d \\ \end{array}
$$
Therefore, for any  $k \geq 1$ ,
$$
\begin{array}{l} | \pi (f) - \pi P _ {k} (f) | = | \mathbb {E} [ f (\theta (t _ {k})) - \mathbb {E} [ f (\theta_ {k}) ] | \leq \mathbb {E} [ | f (\theta (t _ {k})) - f (\theta_ {k}) | ] \\ \leq \mathbb {E} \left[ \| \Delta_ {k} \| \right] \leq \frac {3 \sqrt {2}}{2} \frac {L}{M} h ^ {1 / 2} d. \\ \end{array}
$$
If we use a noisy gradient  $\hat{g} (\theta_k) = \nabla U(\theta_k) + e_k$  where  $e_k$  is the noise with mean zero and bounded variance such that  $\mathbb{E}(||e_k||_2^2)\leq \sigma^2$ , then an extra term  $2h\sigma$  will appear in Lemma 2. As  $\sigma^2$  is usually expected to be proportional to the dimension, this additional term is of the same order as the other term.
Theorem A.1 (Theorem 9.8 of Niederreiter (1992)). Let  $v_0, v_1, \ldots$  be an LFSR with offset  $s$  and period  $n = 2^m - 1$  which satisfy  $\gcd(m, n) = 1$ . Then the sequence  $\{\mathbf{u}_i\}_{i=0}^{n-1} \subset [0,1]^s$  with  $\mathbf{u}_i = (v_i, v_{i+1}, \ldots, v_{i+s-1})$  has, on average, star-discrepancy
$$
O (n ^ {- 1} (\log n) ^ {d + 1} \log \log n)
$$
with an implied constant depending only on  $d$  and the average is taken over all primitive polynomials over  $GF(2)$  of degree  $m$ .
Proof of Theorem 4.1. The error on the left-hand-side is bounded by
$$
(I) + (I I) ^ {\prime} + (I I) ^ {\prime \prime} + \frac {4 \ell}{n} \| f \| _ {\infty}.
$$
Lemma 1 shows that  $(I) \leq (\max_{0 \leq i < n} \| \theta_i \| + \mathbb{E}_{\pi} [ \| \theta \| ]) \rho^\ell \leq (\max_{0 \leq i \leq n} \| \theta_i \| + \mathbb{E}_{\pi} [ \| \theta \| ]) h^{1/2}$  since  $\ell = \lceil (1/2) \log_{\rho} h \rceil$ . Lemma 2 shows that  $(II)' \leq \frac{3\sqrt{2}}{2} \frac{L}{M} dh^{1/2}$ . Denote  $C_2 = \max_{0 \leq i \leq n} \| \theta_i \| + \mathbb{E}_{\pi} [ \| \theta \| ] + \frac{3\sqrt{2}}{2} \frac{L}{M} d$ . So  $(I) + (II)' \leq C_2 h^{1/2}$ .