In this paper, the authors present an analysis of SGD within an SDE framework.  The ideas and the presented results are interesting and are clearly of interest to the deep learning community.  The paper is well-written overall. However, the paper has important problems.  1) The analysis is widely based on the recent paper by Mandt et al.  While being an interesting work on its own, the assumptions made in that paper are very strict and not very realistic.  For instance, the assumption that the stochastic gradient noise being Gaussian is very restrictive and trying to justify it just by the usual CLT is not convincing especially when the parameter space is extremely large,  the setting that is considered in the paper. 2) There is a mistake in the proof Theorem 1.  Even with the assumption that the gradient of sigma is bounded, eq 20 cannot be justified and the equality can only be "approximately equal to".  The result will only hold if sigma does not depend on theta.  However, letting sigma depend on theta is the only difference from Mandt et al.  On the other hand, with constant sigma the result is very trivial and can be found in any text book on SDEs (showing the Gibbs distribution).  Therefore, presenting it as a new result is misleading.   3) Even if the sigma is taken constant and theorem 1 is corrected, I don't think theorem 2 is conclusive.  Theorem 2 basically assumes that the distribution is locally a proper Gaussian (it is stated as locally convex, however it is taken as quadratic)  and the result just boils down to computing some probability under a Gaussian distribution,  which is still quite trivial.  Apart from this assumption not being very realistic,  the result does not justify the claims on "the probability of ending in a certain minimum"  -- which is on the other hand a vague statement.  First of all "ending in" a certain area depends on many different factors, such as the structure of the distribution, the initial point, the distance between the modes etc.  Also it is not very surprising that the inverse image of a wider Gaussian density is larger than of a pointy one.  This again does not justify the claims.  For instance consider a GMM with two components, where the means of the individual components are close to each other, but one component having a very large variance and a smaller weight, and the other one having a lower variance and higher weight.  With authors' claim, the algorithm should spend more time on the wider one,  however it is evident that this will not be the case.  4) There is a conceptual mistake that the authors assume that SGD will attain the exact stationary distribution even when the SDE is simulated by the fixed step-size Euler integrator.  As soon as one uses eta>0 the algorithm will never attain the stationary distribution of the continuous-time process, but will attain a stationary distribution that is close to the ideal one (of course with several smoothness, growth assumptions).  The error between the ideal distribution and the empirical distribution will be usually O(eta) depending on the assumption  and therefore changing eta will result in a different distribution than the ideal one.  With this in mind the stationary distributions for (eta/S) and (2eta/2S) will be clearly different.  The experiments are very interesting and I do not underestimate their value.  However, the current analysis unfortunately does not properly explain the rather strong claims of the authors, which is supposed to be the main contribution of this paper.