SlowGuess/NAACL_2022_OCR huggingface dataset:
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).
Dongyu Ru, Jiangtao Feng, Lin Qiu, Hao Zhou, Mingxuan Wang, Weinan Zhang, Yong Yu, and Lei Li. 2020. Active sentence learning by adversarial uncertainty sampling in discrete space. arXiv preprint arXiv:2004.08046.
Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen. 2016. Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Advances in neural information processing systems, 29:1163-1171.
Kevin Scaman and Aladin Virmaux. 2018. Lipschitz regularity of deep neural networks: analysis and efficient estimation. arXiv preprint arXiv:1805.10965.
Matthias Seeger. 2000. Learning with labeled and unlabeled data.
Ozan Sener and Silvio Savarese. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489.
Rico Sennrich, B. Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. ArXiv, abs/1511.06709.
Burr Settles. 2009. Active learning literature survey.
V. Sheng, F. Provost, and Panagiotis G. Ipeirotis. 2008. Get another label? improving data quality and data mining using multiple, noisy labelers.
Jasdeep Singh, Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2019. Xlda: Cross-lingual data augmentation for natural language inference and question answering. arXiv preprint arXiv:1905.11471.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631-1642.
Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in Neural Information Processing Systems, 33.
Aixin Sun, Ee-Peng Lim, and Ying Liu. 2009. On strategies for imbalanced text classification usingsvm: A comparative study. Decision Support Systems, 48(1):191-201.
Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A. Smith, and Yejin Choi. 2020. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Conference on Empirical Methods in Natural Language Processing.
Antti Tarvainen and Harri Valpola. 2017. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 1195–1204.
Peter D. Turney. 2002. Types of cost in inductive concept learning. ArXiv, cs.LG/0212034.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Ellen M Voorhees and Dawn M Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 200-207.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018a. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461.
Haotao Wang, Chaowei Xiao, Jean Kossaifi, Zhiding Yu, Anima Anandkumar, and Zhangyang Wang. 2021. Augmax: Adversarial composition of random augmentations for robust training. In Thirty-Fifth Conference on Neural Information Processing Systems.
Huan Wang, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018b. Identifying generalization properties in neural networks. arXiv preprint arXiv:1809.07402.
Thomas Wolf, Julien Chaumont, Lysandre Debut, Victor Sanh, Clement Delangue, Anthony Moi, Pierric Cistac, Morgan Funtowicz, Joe Davison, Sam Shleifer, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45.
Qizhe Xie, Zihang Dai, E. Hovy, Minh-Thang Luong, and Quoc V. Le. 2020. Unsupervised data augmentation for consistency training. arXiv: Learning.
Zhao Xu, Kai Yu, Volker Tresp, Xiaowei Xu, and Jizhi Wang. 2003. Representative sampling for text classification using support vector machines. In European conference on information retrieval, pages 393-407. Springer.
Zhaozhuo Xu, Beidi Chen, Chaojian Li, Weiyang Liu, Le Song, Yingyan Lin, and Anshumali Shrivastava. 2021. Locality sensitive teaching. Advances in Neural Information Processing Systems, 34.
Adams Wei Yu, David Dohan, Minh-Thang Luong, R. Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. 2018a. Qanet: Combining local convolution with global self-attention for reading comprehension. volume abs/1804.09541.
Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. 2018b. Qanet: Combining local convolution with global self-attention for reading comprehension. arXiv preprint arXiv:1804.09541.
Michelle Yuan, Hsuan-Tien Lin, and Jordan Boyd-Graber. 2020. Cold-start active learning through self-supervised language modeling. arXiv preprint arXiv:2010.09535.
Mike Zhang and Barbara Plank. 2021. Cartography active learning. arXiv preprint arXiv:2109.04282.
Shujian Zhang, Xinjie Fan, Bo Chen, and Mingyuan Zhou. 2021a. Bayesian attention belief networks. In International Conference on Machine Learning, pages 12413-12426. PMLR.
Shujian Zhang, Chengyue Gong, and Eunsol Choi. 2021b. Knowing more about questions can help: Improving calibration in question answering. arXiv preprint arXiv:2106.01494.
Shujian Zhang, Chengyue Gong, and Eunsol Choi. 2021c. Learning with different amounts of annotation: From zero to many labels. arXiv preprint arXiv:2109.04408.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28:649-657.
Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35-45.
Tony Z Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. arXiv preprint arXiv:2102.09690.
Dengyong Zhou, Olivier Bousquet, Thomas N Lal, Jason Weston, and Bernhard Schölkopf. 2004. Learning with local and global consistency. In Advances in neural information processing systems, pages 321-328.
Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. 2003. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the
20th International conference on Machine learning (ICML-03), pages 912-919.
A.1 Full Results and Examples
We report the full results of out-of-domain and in-domain tasks in Tables 9 and 11, respectively. The full results of prompt-based few-shot learning are shown in Table 10 and Table 12 show prompt examples of each task.
ID
OD	SST-2
IMDB	IMDB
SST-2	QQP
TwitterPPDB
Random	76.31±0.66	82.01±3.45	85.57±0.42
Entropy	75.88±1.82	85.32±2.36	85.18±1.79
BADGE	75.23±0.87	85.11±2.92	85.39±3.44
CAL	78.88±1.27	84.92±2.30	86.14±0.31
Ours	80.24±0.91	86.07±2.45	86.03±0.40
+ WCA	80.42±0.85	86.19±2.37	86.07±0.36
+ CL	80.51±0.67	86.24±1.98	86.18±0.29
Table 9: Results of out-of-domain (OD) generalization. We report the out-of-domain accuracy on the target domain. ID refers to in-domain dataset. OD refers to out-of-domain dataset.
4-shot	8-shot	12-shot
SST-2	Random	64.9±8.4	54.5±4.6	56.3±2.3
Calibrated	73.8±10.9	64.6±8.8	73.0 ±5.3
Uncertainty	59.7±7.3	64.5±5.9	66.8±4.8
Ours	75.3±7.8	77.8±4.7	79.7±3.2
TREC	Random	23.1±5.9	32.7±7.5	37.5±7.8
Calibrated	44.2±2.2	44.1±3.6	44.4±4.0
Uncertainty	34.8±3.4	52.2±4.1	54.1±5.2
Ours	46.4±2.8	58.7±3.6	59.8±4.3
RTE	Random	53.2±6.0	54.9±3.0	56.0±2.2
Calibrated	57.5±1.8	57.7±1.3	58.2±1.1
Uncertainty	57.0±1.5	57.3±1.4	57.8±1.1
Ours	57.9±2.3	58.4±1.6	59.7±1.2
A.2 Classification Task Hyperparameters and Experimental Settings
Our implementation is based on the BERT-base (Devlin et al., 2018) from HuggingFace Transformers (Wolf et al., 2020). We optimize the KL divergence as the objective with the Adam optimizer (Kingma and Ba, 2014) and batch size is set to 16 for all experiments. The curriculum learning is trained for 200 iterations. The learning rate is  $2 \times 10^{-5}$ . The  $\alpha$  in Eqn (5) is set as 0.01 for all experiments. With longer input texts such as IMDB, we use 256 as the maximum sequence length. For others, we use 128. Following Ash et al. (2020) and Margatina et al. (2021), for the initial training set  $\mathcal{D}_{\text {label }}$ , we begin the active learning loop by
Table 10: Full results across different choices of the training examples (the prompt format is fixed). The language model at here is GPT-2XL (1.5B).
Acquired dataset size:	1%	5%	10%
SST-2	Random	84.11±0.45	86.53±0.61	88.05±0.73
Entropy	84.53±0.81	87.82±0.73	89.45±0.92
BADGE	84.32±0.64	87.11±0.82	88.72±0.44
CAL	84.95±0.56	87.34±0.61	89.16±0.67
Ours	85.97±0.53	88.61±0.48	90.05±0.61
+ WCA	86.12±0.47	88.56±0.55	90.14±0.57
+ CL	86.37±0.43	88.79±0.46	90.18±0.48
IMDB	Random	65.96±0.66	84.22±0.52	86.25±0.54
Entropy	68.32±0.53	84.51±0.48	87.29±0.51
BADGE	67.80±0.44	84.46±0.50	87.17±0.41
CAL	73.55±0.56	84.72±0.48	87.27±0.50
Ours:	75.23±0.43	85.82±0.35	87.91±0.53
+ WCA	75.17±0.58	85.79±0.67	87.83±0.71
+ CL	77.57±0.64	86.02±0.62	88.43±0.57
AG News	Random	85.43±0.53	90.05±0.51	91.93±0.60
Entropy	86.48±0.46	92.21±0.41	92.65±0.39
BADGE	86.81±0.48	90.72±0.51	92.41±0.53
CAL	87.12±0.31	92.13±0.38	92.82±0.35
Ours	88.42±0.37	92.86±0.40	93.13±0.39
+ WCA	88.50±0.35	92.84±0.37	93.22±0.42
+ CL	88.57±0.30	92.94±0.32	93.20±0.35
QNLI	Random	76.33±0.54	83.61±0.57	84.63±0.62
Entropy	77.95±0.50	83.83±0.61	84.75±0.55
BADGE	77.74±0.53	84.90±0.48	84.32±0.46
CAL	78.53±0.49	85.14±0.45	84.99±0.53
Ours	78.44±0.41	84.93±0.32	84.87±0.39
+ WCA	78.47±0.43	85.12±0.37	84.91±0.38
+ CL	78.92±s0.40	85.06±0.36	84.96±0.33
QQP	Random	77.32±0.66	81.73±0.72	84.22±0.75
Entropy	78.47±0.57	81.92±0.64	86.03±0.49
BADGE	78.02±0.49	81.63±0.55	84.06±0.60
CAL	78.23±0.52	82.52±0.57	84.25±0.48
Ours	78.97±0.46	82.43±0.44	84.77±0.52
+ WCA	78.90±0.50	82.55±0.48	84.83±0.48
+ CL	79.32±0.53	82.91±0.51	84.95±0.58
Table 11: Full results of the in-domain test accuracies for different acquired dataset size. + WCA refers to Ours + select worst-case augmentation. + CL refers to Ours + curriculum learning.
uniformly random sampling from  $\mathcal{D}_{pool}$ . For all experiments in the Section 5, we set the annotation budget as  $1\%$  and use Ours (ours + curriculum learning) as the default methods.
TF-IDF based data augmentation (Xie et al., 2020) aims to generate both diverse and valid examples. It is designed to retain keywords and replace uninformative words with other uninformative words. BERT is used as the word tokenizer. We set  $\mathrm{IDF}(w)$  is the IDF score for word  $w$  computed on the whole corpus, and  $\mathrm{TF}(w)$  is the TF score for word  $w$  in a sentence. Then, we compute the TF-IDF score as  $\mathrm{TFIDF}(w) = \mathrm{TF}(w)\mathrm{IDF}(w)$ . Suppose the maximum TF-IDF score in a sentence  $x$  is  $\mathbf{C} = \max_{i}\mathrm{TFIDF}(x_{i})$ . We set the probability to  $\min(p(\mathbf{C} - \mathrm{TFIDF}(x_{i})) / \mathbf{Z}, 1)$ , where  $p$  is a hyperparameter that controls the magnitude of the augmentation and we set  $p = 0.3$ . Z is the average score over the inputs sentence. For backtranslation, we use a pre-trained EN-DE $^4$  and DE-EN $^5$  translation models (Ng et al., 2019) to perform backtranslation
4https://dl.fbaipublicfiles.com/  
fairseq/models/wmt19.en-dejoined-dict.  
single_model.tar.gz
5https://dl.fbaipublicfiles.com/ fairseq/models/wmt19.de-enjoined-dict. single_model.tar.gz
Task	Prompt	Label Names
SST-2	Review: At times, the movie looks genuinely pretty.Sentiment: Positive	Positive, Negative
Review: The movie is amateurish, but it's a minor treat.Sentiment:
TREC	Question: Where can I find information on becoming a journalist?Answer Type: Location	Number, Location, Person, Description, Entity, Abbreviation
Question: What is the temperature today?Answer Type:
RTE	The motor industry accounts for as much as 40 percent of the 450,000 installed industrial robots worldwide but their use is changing and applications are expanding.Question: The most common use for robots is the manufacture of automobiles. True or False?Answer: True	True, False
Arroyo was the favorite of investors because of her experience as a trained economist and government manager.Question: Arroyo has experience as an economist and as a government manager. True or False?Answer:
on each sentence.
A.3 Prompt-based Few-Shot Learning Hyperparameters and Experimental Settings
We use the 1.5B parameters GPT-2 (Radford et al., 2019), with a Transformer (Vaswani et al., 2017) based architecture. The model largely follows the details of the OpenAI GPT model (Radford et al., 2018) with a few modifications. Layer normalization (Ba et al., 2016; Fan et al., 2020; Zhang et al., 2021a) is moved to the input of each sub-block and an additional layer normalization is added after the final self-attention block. Following the settings in Zhao et al. (2021), the maximum input length is 2048 tokens or 1500 words. In Table 12, we show the default prompt format for SST-2, TREC, and RTE. For datasets, Stanford Sentiment Treebank (SST-2) (Socher et al., 2013) is one of benchmarks in General Language Understanding Evaluation (GLUE) (Wang et al., 2018a). With fully labeled parse tress, This corpus allows a complete analysis of the compositional effects of sentiment in language. TREC (Voorhees and Tice, 2000) is a 6-way question classification. The target is to classify the questions based on whether their answer type is a Number, Location, Person, Description, Entity, or Abbreviation. Similarly, RTE (Recognizing Textual Entailment) (Dagan et al., 2005) is also a benchmark dataset from GLUE. It is a binary classification task to determine if a given premise entails a given hypothesis.
Table 12: The different prompts we use for SST-2, TREC, and RTE. One training example per task is presented. The language model is used to predicted the label probability as shown in the right column.
SST-2	IMDB	AG News	AVG.
Random	0	0	0	0
Entropy	173	107	402	227
BADGE	25640	3816	1961	10303
CAL	708	273	1284	755
Ours	513	228	881	541
+ WCA	611	275	1023	636
Table 13: Running time (seconds) per sampling iteration (inference and selection) during AL acquisition for each datasets. AVG. refers the average acquisition time for all three datasets. For each acquisition, we report the running time under three adversarial attacks respectively
A.4 Sampling Efficiency and Running Time
We mask  $m$  as the number of labeled data in  $\mathcal{D}_{\text{label}}$ ,  $n$  the number of unlabeled data in  $\mathcal{D}_{\text{pool}}$ ,  $C$  the number of classes in the downstream classification task,  $d$  the dimension of embeddings,  $l$  the maximum sequence length, and  $s_{\text{acq}}$  the acquisition size. We set these values following Yuan et al. (2020) and Margatina et al. (2021). In Table 13, running time in seconds are summarized per sampling iteration (inference and selection) during AL acquisition for each dataset. Experiments in this part are performed on a Tesla V100 GPU. We keep  $s_{\text{acq}} = 100$ ,  $d = 768$ ,  $t = 10$ , and  $l = 128$ . For IMDB, we change the maximum sequence length to 256. As demonstrated in Table 13, BADGE requires a significantly amount of running time, since it has to cluster high-dimensional vectors and is a very computationally-heavy method. CAL also requires relative long running time as it needs to find the contrastive examples by finding nearest neighbors and computing contrastive score for unlabeled candidates. Our method achieves the second best
efficiency. Even with the select worst-case augmentation, our acquisition function is still computationally productive as the augmentation and ranking candidates can be well deployed in the current computational machines. Entropy is overall the most efficient method as it only requires to rank the list of uncertainty scores, while it tends to have weaker performance.