We propose a fast stochastic Hamilton Monte Carlo (HMC) method, for sampling from a smooth and strongly log-concave distribution. At the core of our proposed method is a variance reduction technique inspired by the recent advance in stochastic optimization. We show that, to achieve accuracy in 2-Wasserstein distance, our algorithm achieves O n + κ 2 d 1/2 / + κ 4/3 d 1/3 n 2/3 / 2/3 gradient complexity (i.e., number of component gradient evaluations), which outperforms the state-of-the-art HMC and stochastic gradient HMC methods in a wide regime. We also extend our algorithm for sampling from smooth and general log-concave distributions, and prove the corresponding gradient complexity as well. Experiments on both synthetic and real data demonstrate the superior performance of our algorithm.
Very recently, REF used a stochastic variance-reduced HMC for sampling from smooth and strongly log-concave distributions which requires f is smooth and strongly convex.
Stochastic Variance-Reduced Hamilton Monte Carlo Methods
ArXiv
ArXiv
Mathematics Computer Science