We propose an accelerated-gradient-based MCMC method. It relies on a modification of the Nesterov's accelerated gradient method for strongly convex functions (NAG-SC): We first reformulate NAG-SC as a Hessian-Free High-Resolution ODE, then release the high-resolution coefficient as a free hyperparameter, and finally inject appropriate noise and discretize the  diffusion process. Accelerated sampling enabled by this new hyperparameter is not only experimentally demonstrated on several learning tasks, but also theoretically quantified, both at the continuous level and after discretization. For (not-necessarily-strongly-) convex and L-smooth potentials, exponential convergence in \chi^2 divergence  is proved, with a rate analogous to  state-of-the-art results of underdamped Langevin dynamics, plus an additional acceleration. At the same time, the method also works for nonconvex potentials, for which we also establish exponential convergence as long as the potential satisfies a Poincar\'e inequality\tilde{\mathcal{O}}}{\epsilon}) iteration complexity in 2-Wasserstein distance, same as underdamped Langevin dynamics, but with a reduced constant. Empirical experiments are conducted to numerically verify our theoretical results and demonstrate accelerated sampling in practice} .
We propose an accelerated-gradient-based MCMC method. It relies on a modification of the Nesterov's accelerated gradient method for strongly convex functions (NAG-SC): We reformulate NAG-SC as a Hessian-Free High-Resolution ODE, lift its high-resolution coefficient to be a hyperparameter, and then inject appropriate noise and discretize the resulting diffusion process. Accelerated sampling enabled by the new hyperparameter is theoretically quantified. At continuous-time level, for log-concave/log-strongly-concave target measures, exponential convergence in \chi^2 divergence /2-Wasserstein distance is proved, with a rate analogous to the state-of-the-art results of underdamped Langevin dynamics, plus an additional acceleration. At discrete algorithm level, a dedicated discretization algorithm is proposed to simulate the Hessian-Free High-Resolution SDE in a cost-efficient manner. For log-strong-concave target measures, the proposed algorithm achieves\tilde{\mathcal{O}}(\frac{\sqrt{d}{\epsilon}) iteration complexity in 2-Wasserstein distance, same as underdamped Langevin dynamics, but with a reduced constant. Empirical experiments are conducted to numerically verify our theoretical results and demonstrate accelerated sampling in practice} .
D
first
R
then release the
lift its
R
as a free
to be a
R
finally
then
A
resulting
R
this
the
R
not only experimentally demonstrated on several learning tasks, but also theoretically quantified, both at the continuous level and after discretization. For (not-necessarily-strongly-) convex and L-smooth potentials,
theoretically quantified. At continuous-time level, for log-concave/log-strongly-concave target measures,
A
/2-Wasserstein distance
A
the
R
additional acceleration. At the same time, the method also works for nonconvex potentials, for which we also establish exponential convergence as long as the potential satisfies a Poincar\'e inequality
additional acceleration
A
. At discrete algorithm level, a dedicated discretization algorithm is proposed to simulate the Hessian-Free High-Resolution SDE in a cost-efficient manner. For log-strong-concave target measures, the proposed algorithm achieves
A
(\frac{\sqrt{d