SlowGuess/NAACL_2022_OCR huggingface dataset:
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019a. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 4171-4186.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019b. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Volume 1 (Long and Short Papers), pages 4171-4186.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1126-1135. PMLR.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Association for Computational Linguistics (ACL).
Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recognizing textual entailment challenge. In the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing.
Belize Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2020. Supervised contrastive learning for pre-trained language model fine-tuning. In International Conference on Learning Representations.
Song Han, Jeff Pool, John Tran, and William J Dally. 2015. Learning both weights and connections for efficient neural network. In NIPS.
Junxian He, Jiatao Gu, Jiajun Shen, and Marc'Aurelio Ranzato. 2019. Revisiting self-training for neural sequence generation.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. Deberta: decoding-enhanced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790-2799. PMLR.
Yann LeCun, John S Denker, and Sara A Solla. 1990. Optimal brain damage. In Advances in neural information processing systems, pages 598-605.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021a. The power of scale for parameter-efficient prompt tuning. CoRR, abs/2104.08691.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021b. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. CoRR, abs/2101.00190.
Xinzhe Li, Qianru Sun, Yaoyao Liu, Qin Zhou, Shibao Zheng, Tat-Seng Chua, and Bernt Schiele. 2019. Learning to self-train for semi-supervised few-shot classification. Advances in Neural Information Processing Systems, 32:10276-10286.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019a. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101.
Subhabrata Mukherjee and Ahmed Awadallah. 2020. Uncertainty-aware self-training for few-shot text classification. Advances in Neural Information Processing Systems, 33.
Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts.
Ethan Perez, Douwe Kiela, and Kyunghyun Cho. 2021. True few-shot learning with language models. arXiv preprint arXiv:2105.11447.
Jonas Pfeiffer, Andreas Rückle, Clifton Poth, Aishwarya Kamath, Ivan Vulić, Sebastian Ruder, Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterhub: A framework for adapting transformers. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020): Systems Demonstrations, pages 46-54, Online. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In International Conference on Machine Learning, pages 4334-4343. PMLR.
Timo Schick and Hinrich Schütze. 2021a. Exploiting cloze-questions for few-shot text classification and natural language inference. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269.
Timo Schick and Hinrich Schütze. 2021b. It's not just size that matters: Small language models are also few-shot learners. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2339-2352, Online. Association for Computational Linguistics.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank.
Tu Vu, Minh-Thang Luong, Quoc V Le, Grady Simon, and Mohit Iyyer. 2021. Strata: Self-training with task augmentation for better few-shot learning. arXiv preprint arXiv:2109.06270.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding.
Yaqing Wang, Haoda Chu, Chao Zhang, and Jing Gao. 2021a. Learning from language description: Low-shot named entity recognition via decomposed framework. In *Findings of the Association for Computational Linguistics: EMNLP* 2021, pages 1618-1630.
Yaqing Wang, Fenglong Ma, Haoyu Wang, Kishlay Jha, and Jing Gao. 2021b. Multimodal emergent fake news detection via meta neural process networks. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 3708-3716.
Yaqing Wang, Subhabrata Mukherjee, Haoda Chu, Yuancheng Tu, Ming Wu, Jing Gao, and Ahmed Hassan Awadallah. 2021c. Meta self-training for few-shot neural sequence labeling. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 1737-1747.
Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language resources and evaluation, 39(2):165-210.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018a. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1
(Long Papers), pages 1112-1122. Association for Computational Linguistics.
Adina Williams, Nikita Nangia, and Samuel Bowman. 2018b. A broad-coverage challenge corpus for sentence understanding through inference.
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le. 2019. Unsupervised data augmentation for consistency training. arXiv preprint arXiv:1904.12848.
Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q Weinberger, and Yoav Artzi. 2021. Revisiting few-sample BERT fine-tuning.
Wanjun Zhong, Duyu Tang, Jiahai Wang, Jian Yin, and Nan Duan. 2021. Useradapter: Few-shot user learning in sentiment analysis. In ACL/IJCNLP (Findings).
A.1 Dataset information
Table 6 summarize dataset statistics and task descriptions. All the datasets are in English Language. The licence information is as follows.
MNLI: The majority of the corpus is released under the OANC's license, which allows all content to be freely used, modified, and shared under permissive terms. The data in the FICTION section falls under several permissive licenses; Seven Swords is available under a Creative Commons Share-Alike 3.0 Unported License, and with the explicit permission of the author, Living History and Password Incorrect are available under Creative Commons Attribution 3.0 Unported Licenses; the remaining works of fiction are in the public domain in the United States (but may be licensed differently elsewhere).
RTE: The dataset is public release but the corresponding licence information is not found in the source website  $^{5}$ .
QQP: We did not find the responding license. The source website is not accessible.
SST-2 dataset: CC0: Public Domain
Subj: The dataset is public release but the Licence information is not presented in the source website<sup>7</sup>.
MPQA: The dataset<sup>8</sup> is public release. Made available under the terms of GNU General Public License. They are distributed without any warranty.
We follow the licence of datasets for research use. We manually check no offensive content in our few-shot training dataset.
Table 7 summarizes manually-designed prompts and label words for each dataset in our experiments. These prompts and label words were adopted from (Gao et al., 2021).
B Algorithm Flow
Algorithm 1 summarizes overall flow of LiST. We adopt a light self-training mechanism which keeps
the shared student and teacher encoder parameters frozen and only updates the adapter parameters along with the corresponding language model heads. Beside the lightweight tuning design, another key step in our self-training framework is to utilize the few-shot labeled data to fine-tune the student model  $\psi_{\mathrm{stu}}^{(T)}$  ) in every self-training session. Such a step is different with conventional self-training framework, which either leverages labeled data for initial teacher fine-tuning or combine labeled data with unlabeled data for joint training of student model. The iterative usage of unlabeled data and labeled data helps in better teacher initialization before next round of adapter prompt-tuning on  $\widetilde{\mathcal{D}}_K^{Ttrain}$  which further helps in improving model tuning and the quality of pseudo labels.
Following the true few-shot learning spirit, we do not have any additional development set for hyperparameter tuning. Instead we keep all the hyperparameter same for different tasks, different model families and sizes as well as different shots  $K$ . We retain most of the default hyper-parameter configurations from related work. For each task, we run the model five times with different data splits and different random seeds in  $\{1,2,3,4,5\}$ . Our experiments are conducted in few-shot supervision setting and few-shot semi-supervised setting. In
Category	Dataset	#Labels	#Full Train	#Test	Type	Labels
sentence-pair	MNLI	3	392,702	9,815	NLI	entailment, neutral, contradiction
RTE	2	2,490	277	NLI	entailment, not_entailment
QQP	2	363,846	40,431	paraphrase	equivalent, not_equivalent
single-sentence	SST-2	2	6,920	872	sentiment	positive, negative
Subj	2	8,000	2,000	subjectivity	subjective, objective
MPQA	2	8,606	2,000	opinion polarity	positive, negative
Table 6: Dataset summary and task descriptions. For each task, we sample  $\mathcal{K} \in \{10, 20, 30\}$  labeled examples to form five different splits with different random seeds from the original training set, and add the remaining to the unlabeled set while ignoring their labels.
Task	Prompt	Label words
SST-2	<S1> It was [MASK] .	positive: great, negative: terrible
MR	<S1> It was [MASK] .	positive: great, negative: terrible
Subj	<S1> This is [MASK] .	subjective: subjective, objective: objective
MNLI	<S1> ? [MASK] , <S2>	entailment: Yes, netural: Maybe, contradiction: No
RTE	<S1> ? [MASK] , <S2>	entailment: Yes, not_ entailment: No
QQP	<S1> [MASK] , <S2>	equivalent: Yes, not_equivalent: No
the following, we introduce the hyper-parameters for each setting respectively.
Few-shot supervision setting. We set learning rate as 5e-6, training epochs as 400 and batch size as 4. The bottleneck dimension  $d$  of Adapter is set to 128. The optimizer is AdamW (Loshchilov and Hutter, 2017) with default settings besides learning rate. We use variance for adapter as 0.002 and observe that the performance is not sensitive to variance values when the scale of variance values are equal or less than 0.002. Since experiments are run with different number of labeled examples, the GPU hours range from 5 minutes to 1 hour per task.
Few-shot semi-supervised setting. For initial teacher fine-tuning, we adopt the same hyperparameter configuration as in few-shot supervision setting. To facilitate training on a large amounts of unlabeled data, the learning rate in self-training is set to 1e-4 following fully supervised adapter work (Pfeiffer et al., 2020). The batch size of unlabeled data for student adapter training is 16 and the size of minibatch  $\widetilde{\mathcal{D}}\in \widetilde{\mathcal{D}}_K^{T_{train}}$  for meta reweighting in Eq. 6 is 4. For each self-training session, we train student adapter for  $T = 1000$  steps and further fine-tune 50 epochs on given labeled data. The student KD warmup ratio is set to  $60\%$  i.e.,  $T_{warm} = 600$  steps, without extra hyper-parameter tuning. We repeat all the steps in self-training training  $M = 6$  times. Since experiments are run with different number of labeled
examples and datasets, the GPU hours of all approaches are different, ranging from 1 hour to 10 hours per task.
Table 7: Task prompt and label words summary.  $< S_1 >$  and  $< S_2 >$  indicate input sentences.
Models	#Params	Avg Acc
BERT-base	110M	67.4
BERT-large	336M	68.0
RoBERTa-base	125M	73.7
RoBERTa-large	355M	77.6
T5-small	60M	66.5
T5-base	220M	71.9
T5-large	770M	77.3
Table 8: Average accuracy of prompt FN with different encoders using  $|\mathcal{K}| = 30$  labels on six tasks.
C.2 Few-shot Supervision with Varying Model Sizes and Labels
To better understand the role of different model families in few-shot prompt-based FN, we evaluate the performance of representative state-of-the-art PLMs like BERT (Devlin et al., 2019b), RoBERTa (Liu et al., 2019b) and T5 (Raffel et al., 2020) of different sizes (parameters) using varying amounts of labeled data. We report macro-averaged results over six tasks where each has five different splits for easy comparison.
Effect of model choices. Table 8 shows the performance comparison of three representative PLMs with different parameters using prompt-based FN on 30 labeled samples. We observe that average performance increases with increase in model size within each model family. Overall, we observe
RoBERTa models to perform much better than BERT, and marginally outperform T5 models of much bigger size. Accordingly, we use RoBERTa-large as the base encoder for both LiST and other baseline methods.
Effect of varying the number of labels  $|\mathcal{K}|$ . From Figure 5, we observe prompt-based FN to consistently outperform classic-tuning under all labeled data settings when using the same encoder. With increase in the amount of labeled examples, prompt-based FN and classic-tuning both improve in performance, although with reduced performance gap. This demonstrates prompt-based FN to be the most impactful for low-resource settings with few training labels. LiST improves over both classic and prompt-based FN in all settings with massive reduction in number of tunable parameters.
C.3 Experimental result details
Fine-tuning strategies with varying number of shots. Table 9 shows the performance comparison of RoBERTa-large with two fine-tuning strategies and varying number of labeled samples including zero-shot supervision, few-shot supervision from 10 to 30 and full supervision. Prompt fine-tuning shows competitive performance in zero-shot learning, outperforming classic fine-tuning strategy with 30 labeled examples on several tasks like MNLI and SST-2. As the size of labeled examples increases, the average performance of classic and prompt fine-tuning strategy improves significantly and prompt fine-tuning strategy consistently improves classic fine-tuning with a big gap in the few-shot setting. With full supervision, Prompt fine-tuning strategy and classic fine-tuning strategy achieve similar performance, demonstrating that Prompt fine-tuning is most impactful for low-resource settings with few training labels.
Task performance of varying number of shots and models. We show performance changes regarding varying number of shots and varying model choices in Figure 5 and include more detailed results including average accuracy over 5 runs and corresponding standard deviation on MNLI and RTE in Table 10.
Task performance of different modules with varying number of shots. We show the average accuracy on tuning different modules of RoBERTa-large with  $|K| = 30$  on six tasks in Table 3. In Table 11, we show average accuracy with standard deviation of RoBERTa-large on each task using
varying shots of labeled data. We can observe that Feedforward-output performs best in average while Attention module achieves best performance on some tasks. The conclusion is consistent across different shots of labeled data. Such observations motivate us to insert Adapter into Feedforward Output and Attention modules to handle diverse tasks.
Task performance of lightweight model tuning strategies. We show the average accuracy of serveral lightweight strategies with  $|K| = 30$  labeled examples on six tasks in Table 4. In Table 12, we show average accuracy with standard deviation of lightweight tuning strategies on each task with  $|K| = 30$  labeled examples. We can observe that LiST Adapter outperforms all the lightweight tuning strategies for all six tasks, demonstrating the effective design in adapter placement and parameter initialization.
Comparisons over different PLMs. Table 13, 14 and 15 show the performance comparison of two representative PLMs with different parameters using prompt-based FN on 10, 20 and 30 labeled samples. We observe that average performance increases with increase in model size within each model family. Overall, we observe RoBERTa models to perform much better than BERT. This observation is consistent with the observation in Table 8.
More ablation Analysis. Tables 16, 17 and 18 show the performance of LiST (14 MM parameters) by removing different components as well as LiST without (w/o) adapter (355 MM parameters). It can be observed that the trend is consistent over different shots. "w/o re-init" leads to performance drop consistently in various shots and different data sets. Adapter with  $4\%$  tunable parameters obtains similar performance to full model tuning for shots of 10, 20 and 30 as shown in Table 8.
Adapters w/ different number of training labels. We compare the performance of LiST Adapter (14 MM parameters) against full model tuning (355 MM parameters) where we obtain  $96\%$  tunable parameter reduction with almost matching performance across 10, 20 and 30 shots in Table 20.
Labels	Models	Avg	MNLI (m/mm) (acc)	RTE (acc)	QQP (acc)	SST-2 (acc)	Subj (acc)	MPQA (acc)
|K|=0	Classic Prompt	-58.4	-51.7/52.4	-51.3	-38.6	-83.6	-51.4	-67.6
|K|=10	Classic Prompt	50.069.3	34.9 (0.3) / 35.2 (0.7)54.8 (3.7) / 55.6 (4.6)	50.3 (2.1)60.0 (4.4)	61.1 (3.5)58.7 (4.6)	51.8 (2.9)89.5 (1.7)	71.2 (17.5)84.5 (8.6)	52.4 (3.2)67.8 (6.9)
|K|=20	Classic Prompt	55.275.4	35.8 (1.0) / 36.8 (1.5)60.3 (2.0) / 61.6 (2.7)	51.0 (4.8)64.3 (2.4)	61.3 (9.0)67.8 (4.2)	57.2 (7.7)90.6 (1.8)	84.8 (9.0)88.3 (2.2)	55.9 (4.1)80.6 (7.5)
|K|=30	Classic Prompt	59.777.6	38.0 (1.7) / 39.0 (3.1)62.8 (2.6) / 64.1 (3.3)	51.4 (3.7)66.1 (2.2)	64.3 (8.1)71.1 (1.5)	65.0 (11.5)91.5 (1.0)	90.2 (2.2)91.0 (0.5)	56.1 (5.3)82.7 (3.8)
Full supervision	Classic Prompt	90.791.8	89.6 / 89.589.3 / 88.8	83.088.4	91.892.1	95.295.9	97.297.1	88.889.3
Table 9: Average performance and standard deviation of RoBERTa-large with Classic and Prompt-tuning strategies with varying training labels  $\left| \mathcal{K}\right|$  .
Labels	Models	MNLI (m/mm) (acc)	RTE (acc)
|K|=10	BERT-base-Classic	32.1 (1.2) / 32.4 (1.2)	49.3 (2.6)
RoBERTa-base-Classic	35.2 (1.1) / 35.3 (1.1)	50.6 (3.3)
RoBERTa-large-Classic	34.9 (0.3) / 35.2 (0.7)	50.3 (2.1)
BERT-base-Prompt	43.0 (2.1) / 44.2 (2.1)	50.6 (3.2)
RoBERTa-base-Prompt	49.5 (2.9) / 50.5 (3.1)	56.5 (2.3)
RoBERTa-large-Prompt	54.8 (3.7) / 55.6 (4.6)	59.1 (3.8)
LiST	62.6 (5.7) / 63.1 (6.7)	62.1 (4.1)
|K|=20	BERT-base-Classic	33.1 (1.9) / 33.4 (2.0)	49.5 (5.4)
RoBERTa-base-Classic	36.1 (1.4) / 36.5 (1.4)	51.9 (4.5)
RoBERTa-large-Classic	35.8 (1.0) / 36.8 (1.5)	51.0 (4.8)
BERT-base-Prompt	42.8 (2.1) / 44.5 (2.8)	50.5 (3.1)
RoBERTa-base-Prompt	51.9 (2.9) / 52.8 (3.1)	57.5 (3.4)
RoBERTa-large-Prompt	60.3 (2.0) / 61.6 (2.7)	63.0 (2.4)
LiST	70.3 (4.0) / 71.9 (4.4)	68.2 (3.6)
|K|=30	BERT-base-Classic	34.3 (2.0) / 34.5 (1.9)	51.6 (3.8)
RoBERTa-base-Classic	38.2 (1.9) / 38.6 (2.2)	53.1 (2.4)
RoBERTa-large-Classic	38.0 (1.7) / 39.0 (3.1)	51.4 (3.7)
BERT-base-Prompt	44.7 (2.4) / 45.7 (2.4)	52.6 (4.0)
RoBERTa-base-Prompt	53.6 (2.4) / 55.0 (3.0)	61.0 (4.7)
RoBERTa-large-Prompt	62.8 (2.6) / 64.1 (3.3)	66.1 (2.2)
LiST	73.5 (2.8) / 75.0 (3.7)	71.0 (2.4)
|K|=100	BERT-base-Classic	41.6 (3.5) / 42.8 (3.3)	54.0 (3.4)
RoBERTa-base-Classic	45.3 (0.9) / 46.8 (0.8)	55.6 (5.0)
RoBERTa-large-Classic	49.1 (6.6) / 51.5 (6.7)	56.8 (4.9)
BERT-base-Prompt	47.7 (1.9) / 49.8 (1.7)	52.0 (3.3)
RoBERTa-base-Prompt	59.7 (1.3) / 61.3 (1.4)	64.3 (2.2)
RoBERTa-large-Prompt	69.5 (1.7) / 70.9 (2.0)	72.3 (2.9)
LiST	78.6 (2.4) / 79.9 (1.6)	74.3 (2.2)
|K|=500	BERT-base-Classic	52.4 (3.7) / 53.9 (3.6)	59.2 (2.3)
RoBERTa-base-Classic	61.3 (2.1) / 63.4 (1.8)	62.7 (7.5)
RoBERTa-large-Classic	73.9 (1.8) / 75.6 (1.5)	66.8 (4.9)
BERT-base-Prompt	54.9 (0.8) / 57.6 (1.1)	57.0 (1.6)
RoBERTa-base-Prompt	69.3 (0.6) / 70.3 (0.5)	69.5 (2.1)
RoBERTa-large-Prompt	78.8 (0.8) / 80.0 (0.6)	78.2 (0.5)
LiST	81.9 (0.6) / 82.8 (0.6)	81.9 (1.1)
|K|=1000	BERT-base-Classic	57.4 (2.6) / 59.3 (2.2)	60.4 (3.2)
RoBERTa-base-Classic	68.9 (0.9) / 70.2 (0.8)	66.8 (2.9)
RoBERTa-large-Classic	79.0 (0.9) / 80.2 (0.8)	77.0 (1.7)
BERT-base-Prompt	58.9 (1.0) / 61.2 (1.0)	60.5 (1.7)
RoBERTa-base-Prompt	73.5 (0.9) / 74.4 (1.1)	73.9 (1.1)
RoBERTa-large-Prompt	81.6 (1.0) / 82.6 (0.5)	78.5 (1.8)
LiST	83.9 (0.8) / 84.6 (0.5)	82.9 (1.5)
Table 10: Average performance and standard deviation of different encoders with Classic and Prompt-tuning strategies with various training labels  $|\mathcal{K}|$