Paper:
Stochastic Gradient Geodesic MCMC Methods
We propose two stochastic gradient MCMC methods for sampling from Bayesian posterior distributions defined on Riemann manifolds with a known geodesic flow, e.g. hyperspheres. Our methods are the first scalable sampling methods on these manifolds, with the aid of stochastic gradients. Novel dynamics are conceived and 2nd-order integrators are developed. By adopting embedding techniques and the geodesic integrator, the methods do not require a global coordinate system of the manifold and do not involve inner iterations. Synthetic experiments show the validity of the method, and its application to the challenging inference for spherical topic models indicate practical usability and efficiency.

Review:
This paper exploits the geodesic flow to perform stochastic MCMC. The authors leverage the symmetric splitting integrator for the numerical simulation. The two proposed methods are evaluated by synthetic and real data.  
1. The extension of SGGMC from previous work (SGRHMC)[1] are in two folds. First, the proposed method use Geodesic flow rather than Riemmannian manifold. Second, the proposed method leverage a symmetric splitting integrator (ABOBA) scheme. However, unfortunately none of extensions have a clear and convincing novelty as far as I can see. The drop-in replacement of D and Q are not surprising. The main difference comparing to SGRHMC is to replace the fisher information $G(q)$ with geodesic metric $M(q)^T C M(q)$, thus eliminating the need for global coordinates, which may induce artificial boundaries in some manifolds [4]. This is well-studied in [4], thus the contribution seems marginal. The SSI scheme is originally proposed in [2], and what they did seems to be a direct application of the standard SSI. In fact as pointed out by [3], it does not lead to correct sampler if SSI is not applied, as the motion is not reversible and symplectic. The gSGNHT is not very appealing either because its an immediate extension from [1]. I am expecting to see some cases where the SGRHMC fails (e.g. due to a lack of global coordinates), and their methods survives. However, in their experiments, they didn't perform any comparison between their proposed method with [1]. Thereby I am not convinced that they did provide something more useful than [1].  2. In GMC[4] the splitting integrator is guaranteed to be reversible and symplectic. However this may not directly applied to cases where stochastic gradient is used. I haven't found any clear theoretical evidences in the paper supporting that their splitting integrator is still reversible and symplectic.  3. The toy and synthetic experiments does not provide insight on why SGGMC is more desirable than GMC. The results of SGGMC seems suboptimal, particularly in Fig. 3(b) and Fig. 4(a). A comparison on running time would help. There is no comparison with SGRHMC.  4. In real data experiment, the compared sampling methods does not seem to reach a convergence. I would provide a better intuition on why the stochastic version gives a very large performance gain on log-perplexity (from 3000 to 1500 in Fig. 5(a)). I would also provide more information about the parameters (e.g. stepsize) and initialization setting, otherwise it is hard to conclude where the benefit comes from.   The paper is well written and pedagogical, though more introductions about Geodesic flow would be helpful.  [1] Yi-An Ma, Tianqi Chen, and Emily Fox. A complete recipe for stochastic gradient mcmc. [2] Changyou Chen, Nan Ding, and Lawrence Carin. On the convergence of stochastic gradient mcmc algorithms with high-order integrators. [3] Mark Girolami and Ben Calderhead. Riemann manifold langevin and hamiltonian monte carlo methods. [4] Simon Byrne and Mark Girolami. Geodesic monte carlo on embedded manifolds.