The authors perform a set of experiments in which they inspect the Hessian matrix of the loss of a neural network, and observe that most of the eigenvalues are very close to zero.  This is a potentially important observation,  and the experiments were well worth performing,  but I don't find them fully convincing  (partly because I was confused by the presentation). They perform four sets of experiments: 1) In section 3.1, they show on simulated data that for data drawn from k clusters, there are roughly k significant eigenvalues in the Hessian of the solution. 2) In section 3.2, they show on MNIST that the solution contains few large eigenvalues, and also that there are negative eigenvalues. 3) In section 3.3, they show (again on MNIST) that at their respective solutions, large batch and small batch methods find solutions with similar numbers of large eigenvalues, but that for the large batch method the magnitudes are larger. 4) In section 4.1, they train (on CIFAR10) using a large batch method, and then transition to a small batch method, and argue that the second solution appears to be better than the first, but that they are a part of the same basin  (since linearly while interpolating between them they don't run into any barriers). I'm not fully convinced by the second and third experiments,  partly because I didn't fully understand the plots (more on this below),  but also because it isn't clear to me what we should expect from the spectrum of a Hessian,  so I don't know whether the observed specra have fewer large eigenvalues, or more large eigenvalues, then would be "natural".  In other words, there isn't a *baseline*. For the fourth experiment, it's unsurprising that the small batch method winds up in a different location in the same basin as the large batch method,  since it was initialized to the large batch method's solution  (and it doesn't appear to me, in figure 9, that the small batch solution is significantly different). Section 2.1 is said to contain an argument that the second term of equation 5 can be ignored, but only says that if \ell' and \nabla^2 of f are uncorrelated, then it can be ignored.  I don't see any reason that these two quantities should be correlated,  but this is not an argument that they are uncorrelated.  Also, it isn't clear to me where this approximation was used--everywhere?  In section 3.2, it sounds as if the exact Hessian is used,  and at the end of this section the authors say that figure 6 demonstrates that the effect of this second term is small,  but I don't see why this is,  and it isn't explained. My main complaint is that I had a great deal of difficulty interpreting the plots:  it often wasn't clear to me what exactly was being plotted,  and most of the language describing them was frustratingly vague.  For example, figure 6 is captioned "left edge of the spectrum, eigenvalues are scaled by their ratio".  The text explains that "left edge of the spectrum" means "small but negative eigenvalues"  (this would be better in the caption),  but what are the ratios?  Ratio of what to what?  I think it would greatly enhance clarity if every plot caption described exactly, and unambiguously, what quantities were plotted on the horizontal and vertical axes. Some minor notes:There are a number of places where "it's" is used, where it should be "its". In the introduction, the definition of \mathcal{L}' is slightly confusing,  since it's an expectation,  but the use of "'" makes one expect a derivative.  Perhaps use \hat{\mathcal{L}} for the empirical loss, and \mathcal{L} for the expected one? On the bottom of page 4, "if \ell' and \nabla f are not correlated": I think the \nabla should be \nabla^2. It's "principal components", not "principle components".