A 1.58-bit Vision Transformer for Efficient Image Classification
Authors: Zhengqing Yuan, Rong Zhou, Hongyi Wang, Lifang He, Yanfang Ye, Lichao Sun


Reviewer paper summaries: This paper explores the migration of 1.58 BitNet to ViT, demonstrating the generalization of this low-bit solution in the vision domain. It provides a feasible approach for deploying vision models on edge devices. Experiments were conducted on CIFAR-10 and ImageNet-1K, showing that its performance is close to 8-bit ViT but weaker than 16-bit ViT.
Summary of strengths: - The research problem focused on this paper has significant practical value.
Summary of weaknesses: - The paper does not compare against other necessary baselines, such as edge-deployed ViT models like ElasticViT [1].
- The number of downstream tasks tested is limited, which does not adequately reflect the method's generalization capability.

[1] EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention, CVPR 2023.
[2] Levit: a vision transformer in convnet's clothing for faster inference, ICCV 2023.
Comments, suggestions and typos: None
Overall assessment scores: 3