  Stochastic gradient Markov Chain Monte Carlo algorithms are popular samplers
for approximate inference, but they are generally biased. We show that many
recent versions of these methods (e.g. Chen et al. (2014)) cannot be corrected
using Metropolis-Hastings rejection sampling, because their acceptance
probability is always zero. We can fix this by employing a sampler with
realizable backwards trajectories, such as Gradient-Guided Monte Carlo
(Horowitz, 1991), which generalizes stochastic gradient Langevin dynamics
(Welling and Teh, 2011) and Hamiltonian Monte Carlo. We show that this sampler
can be used with stochastic gradients, yielding nonzero acceptance
probabilities, which can be computed even across multiple steps.
