Venue: ICLR 2017 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: I think that the paper is quite interesting and useful. 
It might benefit from additional investigations, e.g., by adding some rescaled Gaussian noise to gradients during the LB regime one can get advantages of the SB regime.
review_title: Good paper

final_score: rating: 10: Top 5% of accepted papers, seminal paper
review_content: The paper is an empirical study to justify that: 1. SGD with smaller batch sizes converges to flatter minima, 2. flatter minima have better generalization ability. 

Pros and Cons:
Although there is little novelty in the paper, I think the work is of great value in shedding light into some interesting questions around generalization of deep networks. 

Significance:
I think such results may have impact on both theory and practice, respectively by suggesting what assumptions are legitimate for real scenarios for building new theories, or be used heuristically to develop new algorithms with generalization by smart manipulation of mini-batch sizes.

Comments:
Earlier I had some concern about the correctness of a claim made by the authors, which is resolved now. They had claimed their proposed sharpness criterion is scale invariance. They took care of it by removing this claim in the revised version.




review_title: Little novelty but valuable empirical evidence

final_score: rating: null
review_content: If I understand correctly, the paragraphs above Eq (3) claim the sharpness criterion is scale invariant. Does that happen when the denominator in Eq (3) is "1+f(x)". f(x) could have any scale/unit but "1" is a unitless constant. How the proposed criterion is scale invariant with "1+f(x)"? Specifically, if I replace the objective function f(x) with a scaled version c*f(x), does your criterion produces the same value?
review_title: Contant "1" in equation (3)?

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Interesting paper, definitely provides value to the community by discussing why large batch gradient descent does not work too well
review_title: Analysis of large batch training

final_score: rating: null
review_content: true "local minima" or just "local minima" of one-dimensional slices of a unimodal (within the search boundary) nonconvex problem?
review_title: local minima

Meta-review: All reviews (including the public one) were extremely positive, and this sheds light on a universal engineering issue that arises in fitting non-convex models. I think the community will benefit a lot from the insights here.
Decision: Accept (Oral)