Venue: ICLR 2022 Track Blog

Reviews:

final_score: rating: 7: Good paper, accept
review_content: **Is there a significant added value in comparison to the cited papers?**
Depends. I didn't read the selected paper, but enjoyed the initial background section; which seems to go beyond what is presented in the original work and distilling various work on the topic; which I believe has a clear added value on its own.

**Is this added value supported by accurate, convincing, and clear arguments?**
I found the blog post clear and very informative overall. Authors respond to the questions asked in the intro by discussing related work and their arguments are clear.

# Major
- Maybe I'm missing the difference but the plots presented here seems almost same as Brock et al. (2021a) Figure 2. Overall it would be nice to make it clear what is new compared to the existing work. It would be also nice to support performance differences, by bringing related numbers/plots from relevant work. How bad/or close Normalizer-free nets?

- Authors have a nice background section, which I really appreciate. However it is not clear why Dense-net's Highway-nets are introduced. It looks like the discussion only focuses on ResNets. It would be nice to have a concise background.

- I think it would be nice to reiterate the questions posed in the intro again in the last section. Maybe matching the paragraphs or subsection titles? 

# Minor
- It would be nice to define S, B, C in Figure 2 in caption.
- Figure 3 can be improved I think to make it standalone. I had to read the main text to understand the differences. I believe the differences can be made more clear. I would start from the resnet diagram (which is very common and well known) and add deltas to the digram, that indicates the differences mentioned in the main text.
- " to some extent. On one side by using...' Something is missing here I believe.
- ".. there do seem to be alternative .. "->there are solutions to this problem 
- "Old ideas" section repeats part of the Moment Control section. For example; Zhang's method. It might help thinking of making the background section as concise as possible without removing important relevant information.
review_title: Review

Decision: Accept