SlowGuess/NAACL_2022_OCR huggingface dataset:
$$
\mathcal {L} = \mathcal {L} _ {S} + \alpha \cdot \underbrace {\mathbb {E} _ {\boldsymbol {x} \sim \mathcal {D} _ {p o o l}} \mathbb {D} \left(p _ {\theta} (\cdot | \boldsymbol {x}) , p _ {\theta} (\cdot | \boldsymbol {x} ^ {\prime})\right)} _ {\mathcal {L} _ {U}}, \tag {5}
$$
where  $\mathcal{L}_S$  is the cross-entropy supervised learning loss over labeled samples,  $\mathcal{L}_U$  is the consistency regularization term, and  $\alpha$  is a coefficient (Tarvainen and Valpola, 2017; Berthelot et al., 2019b).
For prompt-based few-shot learning, we run experiments on 1.5B-parameters GPT-2 (Radford et al., 2019), a Transformer (Vaswani et al., 2017) based language model. It largely follows the details of the OpenAI GPT model (Radford et al., 2018). We take the TF-IDF as the default augmentation method and provide a rich analysis of other augmentation methods in Section 5. More detailed experimental settings are included in Appendix A.
We evaluate the performance of our acquisition and learning framework in this section. We bold the best results within Random, Entropy, BADGE, CAL, and the proposed ALLSH (Ours) in tables. Then, we bold the best result within each column block. All experimental results are obtained with five independent runs to determine the variance. See Appendix A for the full results with error bars.
4.1 In-Domain Classification Task Results
In Table 2, we evaluate the impact of our acquisition function under three different annotation budgets (1%, 5%, and 10%). With a constrained annotation budget, we see substantial gains on test accuracy with our proposed acquisition: ALLSH and selecting worst-case augmentation. With this encouraging initial results, we further explore our acquisition with curriculum learning. Across all settings, ALLSH is consistently the top performing method especially in SST-2, IMDB, and AG News. With a tight budget, our proposed acquisition can successfully integrate the local sensitivity and learning difficulty to generate annotated data.
For BADGE, despite combining both uncertainty and diversity sampling, it only achieves the compa
rable results on QNLI, showing that gradient computing may not directly benefit data acquisitions. In addition, requiring clustering for high dimensional data, BADGE is computationally heavy as its complexity grows exponentially with the acquisition size (Yuan et al., 2020). We provide rich analysis of the sampling efficiency and running time for each method in Appendix A and include the results in Table 13. Also, ALLSH outperforms the common uncertainty sampling in most cases. Given the current model state, uncertainty sampling chooses the samples that are not representative to the whole unlabeled data, leading to ineffective sampling. CAL has an effective contrastive acquiring on QNLI. We hypothesize that due to the presence of lexical and syntactic ambiguity between a pair of sentence, the contrastive examples can be used to push away the inputs in the feature space.
Acquired dataset size:	1%	5%	10%
SST-2	Random	84.11	86.53	88.05
Entropy	84.53	87.82	89.45
BADGE	84.32	87.11	88.72
CAL	84.95	87.34	89.16
Ours	85.97	88.61	90.05
+ WCA	86.12	88.56	90.14
+ CL	86.37	88.79	90.18
IMDB	Random	65.90	84.22	86.25
Entropy	68.32	84.51	87.29
BADGE	67.80	84.46	87.17
CAL	73.55	84.72	87.27
Ours:	75.23	85.82	87.91
+ WCA	75.17	85.79	87.83
+ CL	77.57	86.02	88.43
AG News	Random	85.43	90.05	91.93
Entropy	86.48	92.21	92.65
BADGE	86.81	90.72	92.41
CAL	87.12	92.13	92.82
Ours	88.42	92.86	93.13
+ WCA	88.50	92.84	93.22
+ CL	88.57	92.94	93.20
QNLI	Random	76.33	83.61	84.63
Entropy	77.95	83.83	84.75
BADGE	77.74	84.90	84.32
CAL	78.53	85.14	84.99
Ours	78.44	84.93	84.87
+ WCA	78.47	85.12	84.91
+ CL	78.92	85.06	84.96
QQP	Random	77.32	81.73	84.22
Entropy	78.47	81.92	86.03
BADGE	78.02	81.63	84.06
CAL	78.23	82.52	84.25
Ours	78.97	82.43	84.77
+ WCA	78.90	82.55	84.83
+ CL	79.32	82.91	84.95
Table 2: Results of the in-domain test accuracies for different acquired dataset size. + WCA refers to Ours + select worst-case augmentation. + CL refers to Ours + curriculum learning. We provide error bars in Table 11 in the Appendix.
4.2 Out-of-Domain Classification Task Results
We compare our proposed method with the baselines for their performance in an out-of-domain (OD) setting and summarize the results in Table 3. We test domain generalization on three datasets with two tasks, including sentiment analysis and paraphrase detection. We set the annotation budget as  $15\%$  of  $\mathcal{D}_{pool}$  for all OD experiments. For OD in SST-2 and IMDB, ALLSH yields better results than all baselines with a clear margin (1.7% and 2.0%, respectively). With curriculum learning, the results are continually improved. The performance gains on out-of-domain are often greater than the gains on in-domain, implying that ALLSH can significantly help the model to generalize across domains. On QQP, ALLSH achieves comparable results as CAL without curriculum learning while the performance can be further improved by adding curriculum learning.
ID
OD	SST-2
IMDB	IMDB
SST-2	QQP
TwitterPPDB
Random	76.31	82.01	85.57
Entropy	75.88	85.32	85.18
BADGE	75.23	85.11	85.39
CAL	78.88	84.92	86.14
Ours	80.54	86.97	86.03
+ WCA	80.72	86.99	86.07
+ CL	80.91	87.07	86.18
Table 3: Results of out-of-domain (OD) generalization. We report the out-of-domain accuracy on the target domain. ID refers to in-domain dataset. OD refers to out-of-domain dataset.
4.3 Prompt-Based Few-Shot Learning Results
We present the prompt-based few-shot learning results with GPT-2 in Table 4, in which we follow the setting (4-shot, 8-shot, and 12-shot) in Zhao et al. (2021). Few-shot learners suffer from the quality of labeled data (Sohn et al., 2020), and previous acquisition functions usually fail to boost the performance from labeling random sampled data. In Table 4, we observe that uncertain prompts performs similar to random selected prompts. A potential reason is that an under-trained model treats all examples as uncertainty examples and hard to distinguish the informativeness. However, our proposed acquisition demonstrates the strong capability in modeling the local sensitivity and learning from easy to hard. It comes to the best performance in most of the settings. These findings show the potential of using our acquisition to improve prompt-based few-shot learning and make a good in-context examples for GPT-2 model.
4-shot	8-shot	12-shot
SST-2	Random	64.9	54.5	56.3
Calibrated	73.8	64.6	73.0
Uncertainty	59.7	64.5	66.8
Ours	75.3	77.8	79.7
TREC	Random	23.1	32.7	37.5
Calibrated	44.2	44.1	44.4
Uncertainty	34.8	52.2	54.1
Ours	46.4	58.7	59.8
RTE	Random	53.2	54.9	56.0
Calibrated	57.5	57.7	58.2
Uncertainty	57.0	57.3	57.8
Ours	57.9	58.4	59.7
Table 4: Results across different strategies of acquiring training examples (the prompt format is fixed). The language model here is GPT-2 (1.5B).
Can we use our proposed acquisition in the imbalance setting? Extreme label imbalance is an important challenge in many non-pairwise NLP tasks (Sun et al., 2009; Zhang et al., 2017; Mussmann et al., 2020b). We set up the imbalance setting by sampling a subset with class-imbalance sample rate. For binary classification, we set the positive-class data sample rate as 1.0 and negative-class data sample rate as 0.1. As our acquisition focuses on local sensitivity and informativeness, it tends to select examples close to the decision boundary. Once too many positive examples and few negative examples are labeled, the local perturbation around negative samples are easy to be positive, and thus ALLSH selects examples that are close to the negative examples. We conduct the experiments on SST-2, IMDB, and AG News with annotation budget as  $1\%$ . In Table 5, Ours<sup>3</sup> indicates strong improvements. This further proves that our selection method can generalize better.
SST-2	IMDB	AG News
Random	79.45	62.33	82.95
Entropy	81.71	65.69	82.79
CAL	83.23	72.75	83.27
Ours	85.48	74.48	84.11
Table 5: Main results of different active learning strategies on the imbalanced SST-2, IMDB, and AG News.
Would different augmentations make meaningful difference? We test if our results are sensitive to the choice of augmentation: TF-IDF and backtranslation. For TF-IDF, we compare the random sample augmentation and worst-case augmentation (WCA). TF-IDF and Backtranslation generate diverse paraphrases while preserving the semantics of the original sentences. Select-worst case aug-ments the inputs by incorporating the approximate adversarial perturbations. Table 6 indicates our method is insensitive to different augmentations.
<sup>3</sup>Ours in the Section 5 refers to ours + curriculum learning.
SST-2	IMDB	AG News
Backtranslation	86.01	75.12	88.39
TF-IDF	85.97	75.23	88.42
+WCA	86.37	75.17	88.50
Table 6: Acquisition performance for different augmentations. We report results of our acquisition with different augmentations to get the local copies of the samples.
We also observe that WCA achieves the highest gains on two datasets. This confirms our discussion in Section 2.3 that select-worst case is capable of imposing local sensitivity.
What is the influence of the choice of divergence? We select different divergences in the statistical distance family and study their abilities in encoding different information. Corresponding to Section 2.3, we present the results in Table 7. We experiment on the KL divergence, JSD, and  $\alpha$ -divergence (Minka et al., 2005) with the  $\alpha$  value set as  $-0.5$  or  $0.5$ . We notice that for our case the difference between different divergences is small. A possible reason is that the number of class categories is small and therefore the choice of divergence does not have a large influence.
Can we use the proposed acquisition with extremely few labeled data? We have presented the results under very limited annotation budgets in Table 2. We set the annotation budget as  $0.8\%$  and  $0.4\%$ . The key observation is that the degradation of performance in the other acquisition functions are dramatic. For example, in IMDB, the uncertainty sampling (Entropy) shows the obvious performance drop. It suffers from the sampling bias problem because of the frequent variation of the decision boundary in the early phase of training with very few labeled data available, which results in ineffective sampling. Even under this extreme case, our acquisition still aims to select the most informative examples for the model. This further verifies our empirical results in Section 4.3 on prompt-based few-shot learning where only a very few in-context prompts are provided.
Active Learning Active Learning has been widely used in many applications in NLP (Lowell et al., 2018; Dor et al., 2020; Ru et al., 2020). The uncertainty-based methods (Fletcher et al., 2008) have become the most common strategy. Instead of only considering uncertainty, diversity sampling has also become an alternative direction. Recent works (Geifman and El-Yaniv, 2017; Sener
SST-2	IMDB	AG News
KL	86.37	77.57	88.57
JSD	86.25	77.38	88.41
α = -0.5	86.31	77.42	88.43
α = 0.5	86.39	77.53	88.61
Table 7: Ablation study on different choices of divergences. We report KL, JSD, and  $\alpha$ -divergence, and set  $\alpha = \pm 0.5$  respectively.
SST-2	IMDB
Dataset size	0.4%	0.8%	0.4%	0.8%
Random	64.64	61.08	60.84	73.86
Entropy	67.88	63.94	58.96	71.32
CAL	73.81	65.72	61.65	74.15
Ours	76.45	69.46	64.54	75.88
Table 8: Results on the SST-2 and IMDB datasets under limited annotation budget (0.4%, 0.8%).
and Savarese, 2017; Ash et al., 2020; Yuan et al., 2020) focus on different parts of diversity. Most recent works (e.g. Zhang and Plank, 2021; Margatina et al., 2021) have been more on exploiting the model behavior and each individual instance. Our work focuses more on the local sensitivity and informativeness of data, leading to better performance under various limited annotation settings.
Annotation Budgeting Annotation budgeting with learning has long been studied (Turney, 2002). Sheng et al. (2008) study the tradeoff between collecting multiple labels per example versus annotating more examples. On the other hand, different labeling strategies such as providing fine-grained rationales (Dua et al., 2020), active learning (Kirsch et al., 2019), and the training dynamics approach (Swayamdipta et al., 2020) are studied. Except standard classification, class-imbalance (Mussmann et al., 2020a) or noisy label cases (Fan et al., 2021; Chen et al., 2021) have also been explored. We utilize active learning to explore the labeling strategies and aim to select the most informative data for annotations.
Our work demonstrates the benefits of introducing local sensitivity and learning from easy to hard into the acquisition strategy. The proposed acquisition function shows noticeable gains in performance across classification tasks and prompt-based few-shot learning. In this work, we conduct the detailed study with the proposed acquisition strategy in different settings, including imbalanced and extremely limited labels. We also verify the impact of different choice of designs such as the choice of divergence and augmentations. To summarize, the proposed ALLSH is effective and general, with the
potential to be incorporated into existing models for various NLP tasks.
S. Zhang and M. Zhou acknowledge the support of NSF IIS-1812699 and Texas Advanced Computing Center.
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. 2020. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450.
Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41-48.
David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. 2019a. Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. arXiv preprint arXiv:1911.09785.
David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. 2019b. Mixmatch: A holistic approach to semi-supervised learning. arXiv preprint arXiv:1905.02249.
Avrim Blum and Shuchi Chawla. 2001. Learning from labeled and unlabeled data using graph mincuts.
Zalán Bodó, Zsolt Minier, and Lehel Csató. 2011. Active learning with clustering. In Active Learning and Experimental Design workshop In conjunction with AISTATS 2010, pages 127-139. JMLR Workshop and Conference Proceedings.
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), abs/1508.05326.
Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien. 2009. Semi-supervised learning (chapelle, o. et al., eds.; 2006)[book reviews]. IEEE Transactions on Neural Networks, 20(3):542-542.
Olivier Chapelle, Jason Weston, and Bernhard Schölkopf. 2002. Cluster kernels for semi-supervised learning. In Advances in neural information processing systems. Citeseer.
Derek Chen, Zhou Yu, and Samuel R Bowman. 2021. Learning with noisy labels by targeted relabeling. arXiv preprint arXiv:2110.08355.
Aron Culotta and Andrew McCallum. 2005. Reducing labeling effort for structured prediction tasks. In AAAI, volume 5, pages 746-751.
Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The pascal recognising textual entailment challenge. In *Machine Learning Challenges Workshop*, pages 177-190. Springer.
Shrey Desai and Greg Durrett. 2020. Calibration of pre-trained transformers. Conference on Empirical Methods in Natural Language Processing, abs/2003.07892.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: pre-training of deep bidirectional transformers for language understanding. arxiv. arXiv preprint arXiv:1810.04805.
Liat Ein Dor, Alon Halfon, Ariel Gera, Eyal Shnarch, Lena Dankin, Leshem Choshen, Marina Danilevsky, Ranit Aharonov, Yoav Katz, and Noam Slonim. 2020. Active learning for bert: An empirical study. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7949-7962.
Dheeru Dua, Sameer Singh, and Matt Gardner. 2020. Benefits of intermediate annotations in reading comprehension. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL).
Melanie Ducoffe and Frederic Precioso. 2018. Adversarial active learning for deep networks: a margin based approach. arXiv preprint arXiv:1802.09841.
Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. arXiv preprint arXiv:1808.09381.
Dominik Maria Endres and Johannes E Schindelin. 2003. A new metric for probability distributions. IEEE Transactions on Information theory, 49(7):1858-1860.
Xinjie Fan, Shujian Zhang, Bo Chen, and Mingyuan Zhou. 2020. Bayesian attention modules. arXiv preprint arXiv:2010.10604.
Xinjie Fan, Shujian Zhang, Korawat Tanwisuth, Xiaoning Qian, and Mingyuan Zhou. 2021. Contextual dropout: An efficient sample-dependent dropout module. arXiv preprint arXiv:2103.04181.
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George J Pappas. 2019. Efficient and accurate estimation of lipschitz constants for deep neural networks. arXiv preprint arXiv:1906.04893.
Elisa Ferracane, Greg Durrett, Junyi Jessy Li, and Katrina Erk. 2021. Did they answer? subjective acts and intents in conversational discourse. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1626-1644, Online. Association for Computational Linguistics.
Alyson K Fletcher, Sundeep Rangan, and Vivek K Goyal. 2008. Resolution limits of sparse coding in high dimensions. In NIPS, pages 449-456.
Mingfei Gao, Zizhao Zhang, Guo Yu, Sercan Ö Ark, Larry S Davis, and Tomas Pfister. 2020. Consistency-based semi-supervised active learning: Towards minimizing labeling cost. In European Conference on Computer Vision, pages 510-526. Springer.
Yonatan Geifman and Ran El-Yaniv. 2017. Deep active learning over the long tail. arXiv preprint arXiv:1711.00941.
Daniel Gissin and Shai Shalev-Shwartz. 2019. Discriminative active learning. arXiv preprint arXiv:1907.06347.
Chengyue Gong, Tongzheng Ren, Mao Ye, and Qiang Liu. 2020. Maxup: A simple way to improve generalization of neural network training. arXiv preprint arXiv:2002.09024.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Andreas Kirsch, Joost Van Amersfoort, and Yarin Gal. 2019. Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning. Advances in neural information processing systems, 32:7026-7037.
Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics, 22(1):79-86.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942.
Qi Lei, Lingfei Wu, Pin-Yu Chen, Alexandros G Dimakis, Inderjit S Dhillon, and Michael Witbrock. 2018. Discrete adversarial attacks and submodular optimization with applications to text classification. arXiv preprint arXiv:1812.00151.
David D Lewis and William A Gale. 1994. A sequential algorithm for training text classifiers. In SIGIR'94, pages 3-12. Springer.
David Lowell, Zachary C Lipton, and Byron C Wallace. 2018. Practical obstacles to deploying active learning. arXiv preprint arXiv:1807.04801.
Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142-150.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.
Katerina Margatina, Giorgos Vernikos, Loic Barrault, and Nikolaos Aletras. 2021. Active learning by acquiring contrastive examples. arXiv preprint arXiv:2109.03764.
Tom Minka et al. 2005. Divergence measures and message passing. Technical report, Citeseer.
Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii. 2018. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE transactions on pattern analysis and machine intelligence, 41(8):1979-1993.
Stephen Mussmann, Robin Jia, and P. Liang. 2020a. On the importance of adaptive data collection for extremely imbalanced pairwise tasks. In Conference on Empirical Methods in Natural Language Processing.
Stephen Mussmann, Robin Jia, and Percy Liang. 2020b. On the importance of adaptive data collection for extremely imbalanced pairwise tasks. arXiv preprint arXiv:2010.05103.
Nathan Ng, Kyra Yee, Alexei Baevski, Myle Ott, Michael Auli, and Sergey Edunov. 2019. Facebook fair's wmt19 news translation task submission. arXiv preprint arXiv:1907.06616.
Yixin Nie, Xiang Zhou, and Mohit Bansal. 2020. What can we learn from collective human opinions on natural language inference data? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9131-9143.
Ellie Pavlick and Tom Kwiatkowski. 2019. Inherent disagreements in human textual inferences. Transactions of the Association for Computational Linguistics, 7:677-694.
Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances in Neural Information Processing Systems, 34.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.