SlowGuess/NeurIPS_2023_OCR huggingface dataset:
By Theorem A.1 and the condition that  $\gcd(d\ell, n) = 1$ , the star-discrepancy  $D^{*}(\{\bar{w}_{k}^{(\ell)}\}_{k \geq 1})$  is upper bounded by  $O(n^{-1}(\log n)^{d\ell + 1}\log \log n)$ . Finally, by Koksma-Hlawka inequality, we have  $(II)'' \leq \| \bar{f}_{\ell}\|_{\mathrm{HK}} \cdot D^{*}(\{\bar{w}_{k}^{(\ell)}\}_{k \geq 1})$ . Thus,  $(II)'' + \frac{4\ell}{n}\| f\|_{\infty} \leq C_{1}n^{-1 + \delta}$ , where  $\delta$  hides the poly-logarithmic terms in  $\log n$  and  $C_1$  depends on  $d, \ell, \| \bar{f}_{\ell}\|_{\mathrm{HK}}$ .
Therefore, the upper bound becomes
$$
(I) + (I I) ^ {\prime} + (I I) ^ {\prime \prime} + \frac {4 \ell}{n} \| f \| _ {\infty} \leq C _ {1} n ^ {- 1 + \delta} + C _ {2} h ^ {1 / 2}.
$$
The primary contribution of this work is to improve LMC as a Monte Carlo sampling algorithm, not as an optimization algorithm. Therefore, our main focus is on providing a better estimation of  $\pi(f)$  for some function of interest. Downstream tasks relying on such expectations can also benefit from LQMC. For posterior prediction, it is essential to recognize that the prediction error is not solely determined by the sampling method. Even with infinite perfect samples from the posterior, the prediction error can still arise due to model misspecification, noisy data, biased sampling, etc. So the improvement achieved by LQMC might be less pronounced when assessing the prediction error.
To investigate the performance of LQMC in a posterior prediction setting, we conducted experiments similar to those presented in Dubey et al. (2016) using three UCI datasets. Each dataset was split into a training set  $(70\%)$ , a validation set  $(10\%)$ , and a test set  $(20\%)$ . We performed a tuning process for the constant step size on a grid using the validation set and evaluated the prediction error on the test set. Each iteration computes the stochastic gradient using 32 data points sampled at random. Details of the datasets are in Table 1.
Datasets	Parkinsons	Bike	Protein
N (number of instances)	5875	17379	45730
p (number of features)	21	12	9
Table 1: Summary of datasets used for Bayesian posterior prediction.
The results are presented in Figure 6. The  $x$ -axes represent the total number of iterations of Langevin algorithm and the  $y$ -axes represent the test error. The error bars represent the variation across 10 random replicates. It is evident that LQMC reduced the test error, although the improvement is not substantial. This aligns with our initial expectation, as the proposed method primarily enhances the accuracy of estimating the posterior mean. However, the test error often consists of other sources of error, thus the improvement achieved by the proposed method in reducing the test error might be limited.
Figure 6: Test error versus number of iterations for the three UCI datasets.