Venue: ICLR 2022 Track Blog

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: **Summary**

This blog post presents a summary of the ViT method as well as a timeline on attention models in deep learning. The authors present a brief but polished background study and key design aspects of the model. Several aspects of these models are explained through comparative studies of previous works. The blog post is concise and easy to follow, however, the followings are my concerns:

* Considering this is a blog post, it would be better to give a briefing on self-attention and image patching.
* The results and conclusions on the effect of scaling could be more thorough, using some comparative results on Transformer and ConvNet models from previous works [1].
* About the results on how ViT processes image data, the comparative results from [2] could be very insightful considering the whole flow of this blog post.

I highly recommend the authors extend their analysis reports and conclusions based on recent works in this domain.

**References**

1. Zhai, Xiaohua, et al. "Scaling Vision Transformers." arXiv preprint arXiv:2106.04560 (2021)
2. Raghu, Maithra, et al. "Do vision transformers see like convolutional neural networks?." Advances in Neural Information Processing Systems 34 (2021).
review_title: Blog post is concise and easy to follow but reported analysis and conclusions should be more thorough

Decision: Reject