Next-Token is Not All You Need: Unleashing Subsequent Tokens Parallelly for Zero-Shot Text Classification
Authors: Junlang Qian, Zixiao Zhu, Hanzhang Zhou, Zijian Feng, Zepeng Zhai, Kezhi Mao


Reviewer paper summaries: This paper proposes an extension to zero-shot LMs verbalizers based on prompting. The idea is quite simple, instead of predicting a single token following the prompt, several tokens are predicted, with the possibility of skipping non-discriminative tokens. The experiments show that this extension can significantly improve text classification.
Summary of strengths: On the plus side:
1- The paper is clearly written.
2- The idea of using several tokens is well motivated and it seems natural.
3- Experiments show that the proposed extension improves performance.
Summary of weaknesses: On the minus side:
4- The contribution is rather incremental, maybe it would have been more appropriate for a short paper. Although it is true that the paper manages to fill the content with an interesting analysis of results.
Comments, suggestions and typos: My understanding is that all the datasets used in the experiments are in English, it migth be interesting to test the approach on some non-English datasets.
Overall assessment scores: 3