SlowGuess/NAACL_2021_OCR huggingface dataset:
The architecture of the clustering model  $\mathcal{C}$  (Section 3.3) is similar to that of the Siamese encoder. Additionally, it has a linear layer with a softmax activation function that maps the concatenation of terminal hidden states of the forward and backward LSTMs to  $K$  units, where  $K$  is the number of clusters. To assign an input example to a cluster, we first pass it through the encoder in  $\mathcal{M}$  and feed the resulting encodings to the clustering model  $\mathcal{C}$ . The example is assigned to the cluster with the highest softmax output. This network is also retrained after every 10 iterations to retain model-awareness.
The initial data splits used for training the model  $\mathcal{M}$  were set at  $2\%$  of randomly sampled data for Sequence Tagging (20% for NMT). These are in
accordance with the splitting techniques used in the existing literature on AL (Siddhant and Lipton, 2018; Liu et al., 2018). The model is then used to provide input to train the Siamese/Clustering network using the SICK/Quora Pairs. At each iteration, we gradually add another  $2\%$  of data for sequence tagging ( $5\%$  for NMT) by retrieving low confidence examples using an AL strategy, followed by clustering to extract the most representative examples. We average the results over five independent runs with randomly chosen initial splits. [Hyperparameters details in Appendix A].
We claim that  $\mathrm{A}^2\mathrm{L}$  mitigates the redundancies in the existing AL strategies by working in conjunction with them. We validate our claims by comparing our approaches with three baselines that highlight the importance of various components.
Cosine: Clustering is done based on cosine similarity between last output encodings (corresponding to sentence length) from encoder in  $\mathcal{M}$ . Although this similarity computation is model-aware, it is simplistic and shows the benefit of using a more expressive similarity measure.
None: In this baseline, we use the AL strategy without applying Active $^2$  learning to remove redundant examples. This validates our claim about redundancy in examples chosen by AL strategies.
Random: No active learning is used, and random examples are selected at each time.
We perform ablation studies to demonstrate the utility of model-awareness using these baselines:
Inferent: Clustering is done based on cosine similarity between sentence embeddings (Chen et al., 2015) obtained from a pre-trained InferSent model (Conneau et al., 2017). This similarity computation is not model-aware and shows the utility of model-aware similarity computation.
Iso Siamese: To show that the Siamese network alone is not sufficient and model-awareness is needed, in this baseline, we train the Siamese network by directly using GloVe embeddings of the words as input rather than using output from the
<sup>5</sup>We process the dataset to use only those sentences which are present in at least 5 other pairs. We retrieve 16000 sets, each with a source sentence and 5 other samples (comprising both positive and negative labels). An additional 1000 sets were generated for evaluation.
Figure 2: [Best viewed in color] Comparison of our approach  $(\mathrm{A}^2\mathrm{L})$  with baseline approaches on different tasks using different active learning strategies.  $1^{st}$  row: POS (error bound at convergence:  $\pm 0.05$ ),  $2^{nd}$  row: NER ( $\pm 0.08$ ),  $3^{rd}$  row: SEMTR ( $\pm 0.09$ ),  $4^{th}$  row: NMT ( $\pm 0.04$ ). In the first three rows, from left to right, the three columns represent BALD, Entropy, and Margin AL strategies.  $4^{th}$  row represents AL strategies for NMT, from left to right (LC: Least Confidence, CS: Coverage Sampling, ADS: Attention Distraction Sampling): Legend Description {100% data: full data performance,  $\mathrm{A}^2\mathrm{L}$  (MA Siamese): Model Aware Siamese,  $\mathrm{A}^2\mathrm{L}$  (Int Model): Integrated Clustering Model, Cosine: Cosine similarity, None: Active learning strategy without clustering step, Random: Random split (no active learning applied)}. See Section 4.4 for more details about the baselines. All the results were obtained by averaging over 5 random splits. These plots have been magnified to highlight the regions of interest. For original plots, refer to Fig 8 in the Appendix.
Figure 3: [Best viewed in color] Ablations studies on POS task using different active learning strategies. From left to right, the three columns represent BALD, Entropy and Margin based AL strategies. Legend Description {100% data : full data performance,  $\mathbf{A}^2\mathbf{L}$  (MA Siamese): Model Aware Siamese,  $\mathbf{A}^2\mathbf{L}$  (Int Model): Integrated Clustering Model, Iso Siamese: Model isolated Siamese, InferSent: Cosine similarity based on InferSent encodings}. See Figure 7 in Appendix for experiments on other tasks. All the results were obtained by averaging over 5 splits.
model  $\mathcal{M}$  's encoder. This similarity, which is not model-aware, is then used for clustering.
Figure 4: Comparison between various components of our approach in terms of the time required by them per training epoch for NMT. LSTM (Base) encoder-decoder translation model ( $\approx$  10 hours), Model Aware (MA) Siamese ( $\approx$  3 minutes) and Integrated Clustering (Int) Model ( $\approx$  10 seconds). It can be observed that  $\mathrm{A}^2\mathrm{L}$  adds a negligible overhead to the overall training time ( $\approx$  0.45% of the time taken by the base model).
Figure 2 compares the performance of our methods with baselines. It shows the test-set metric on the  $y$ -axis against the percentage of training data used on the  $x$ -axis for all tasks. See Figures 7 and 8 in the Appendix for additional results.
1. As shown in Figure 2, our approach consistently outperforms all baselines on all tasks. Note that one should observe how fast the performance increases with the addition of training data (and
not just the final performance) as we are trying to evaluate the effect of adding new examples. Our ablation studies in Figure 3 show the utility of using model-aware similarity.
2. In sequence tagging, we match the performance obtained by training on the full dataset using only a smaller fraction of the data (3 - 25% less data as compared to state-of-art AL strategies) (Table 2). On a large dataset in NMT task (Europarl),  $\mathrm{A}^2\mathrm{L}$  takes ≈ 4300 sentences fewer than the Least Confidence AL strategy to reach a Bleu score of 12.
3. While comparing different AL strategies is not our motive, Figure 2 also demonstrates that one can achieve performance comparable to a complex AL strategy like BALD, using simple AL strategies like margin and entropy, by using the proposed  $\mathrm{A}^2\mathrm{L}$  framework.
4. Additionally, from Figure 1, it can be observed that for one step of data selection: (i) The proposed MA Siamese model adds minimal overhead to the overall AL pipeline since it takes less than 5 additional seconds ( $\approx \frac{1}{12}$  of the time taken for ALS); (ii) By approximating the clustering step, Integrated Clustering (Int) Model further reduces the overhead down to 2 seconds. However, owing to this approximation, MA Siamese is observed to perform slightly better than the Int Model (Fig 3). A comparison of training time for various stages of the  $\mathrm{A}^2\mathrm{L}$  pipeline is provided in Figure 4.
We wish to state that our approach should be evaluated not in terms of the gain in the F1 score but
Figure 5: [Best viewed in color] Qualitative case study to convey the notion of redundancy and the model aware similarity. We compare the examples grouped in the same cluster using (i) MA similarity scores [left] and (ii) Cosine similarity scores on the InferSent embedding (Infersent baseline described in Section 4.4) [Right]. As expected, when cosine similarity is used, sentences that have roughly similar content have been assigned to the same cluster. However, when model aware similarity is used, in addition to having similar content, the sentences also have a similar tagging structure. It is sensible to eliminate sentences having similar tagging structures, as they are redundant. [Dataset: CoNLL 2003 dataset, AL strategy: BALD, Data usage:  $10\%$  of the total data]
No. of examples selected by AL Strategy	Spectral (examples processed per second)	Integrated (examples processed per second)
5000	491.64	2702.70
10000	248.08	2557.54
15000	163.10	2508.36
20000	122.89	2493.76
Table 3: Number of samples processed per second by the Spectral Clustering (MA Siamese) compared to the Integrated Clustering (Int Model) methods.
in terms of the reduction in data required to achieve the same (3-25 % on multiple datasets). More importantly, this improvement comes at a negligible computation overhead cost. The reported improvements are not relative with respect to any baseline but represent an absolute value and are very significant in the context of similar performance improvements reported in the literature. In Figure 5, we provide a qualitative case study that demonstrates the problem of redundancy.
This paper shows that one can further reduce the data requirements of Active Learning strategies
by proposing a new method,  $\mathrm{A}^2\mathrm{L}$ , which uses a model-aware-similarity computation. We empirically demonstrated that our proposed approaches consistently perform well across many tasks and AL strategies. We compared the performance of our approach with strong baselines to ensure that the role of each component is properly understood.
This work was funded by British Telecom India Research Center project on Advanced Chatbot.
Pranjal Awasthi, Maria Florina Balcan, and Philip M. Long. 2017. The power of localization for efficiently learning linear separators with noise. J. ACM, 63(6):50:1-50:27.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. In proceedings of ICLR 2015 as oral presentation.
Michael Bloodgood and Chris Callison-Burch. 2010. Bucking the trend: Large-scale cost-focused active learning for statistical machine translation. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL '10, page 854-864, USA. Association for Computational Linguistics.
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. 2015. Weight uncertainty in neural networks. In Proceedings of the 32Nd International Conference on International Conference on Machine Learning - Volume 37, ICML'15, pages 1613-1622. JMLR.org.
Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. 1994. Signature verification using a "siamese" time delay neural network. In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Advances in Neural Information Processing Systems, pages 737-744. Morgan-Kaufmann.
Robert Burchfield. 1985. Frequency analysis of english usage: Lexicon and grammar. by w. nelson francis and henry kucera with the assistance of andrew w. mackie. boston: Houghton mifflin. 1982. x + 561. Journal of English Linguistics, 18(1):64-70.
Yukun Chen, Thomas A. Lasko, Qiaozhu Mei, Joshua C. Denny, and Hua Xu. 2015. A study of active learning methods for named entity recognition in clinical text. J. of Biomedical Informatics, 58(C):11-18.
Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bah-danau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. In Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pages 103-111, Doha, Qatar. Association for Computational Linguistics.
Vincent Claveau and Ewa Kijak. 2017. Strategies to select examples for active learning with conditional random fields. In CICling 2017 - 18th International Conference on Computational Linguistics and Intelligent Text Processing, pages 1-14.
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670-680.
Sanjoy Dasgupta, Adam Tauman Kalai, and Claire Monteleoni. 2009. Analysis of perceptron-based active learning. J. Mach. Learn. Res., 10:281-299.
Rosa Figueroa, Qing Zeng-Treitler, Long Ngo, Sergey Goryachev, and Eduardo Wiechmann. 2012. Active learning for clinical text classification: Is it better than random sampling? Journal of the American Medical Informatics Association : JAMIA, 19:809-16.
Yoav Freund, H. Sebastian Seung, Eli Shamir, and Naftali Tishby. 1997. Selective sampling using the query by committee algorithm. *Mach. Learn.*, 28(2-3):133-168.
Yarin Gal and Zoubin Ghahramani. 2016a. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the
33rd International Conference on International Conference on Machine Learning - Volume 48, ICML'16, pages 1050-1059. JMLR.org.
Yarin Gal and Zoubin Ghahramani. 2016b. A theoretically grounded application of dropout in recurrent neural networks. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 1019-1027. Curran Associates, Inc.
Gholamreza Haffari, Maxim Roy, and Anoop Sarkar. 2009. Active learning for statistical phrase-based machine translation. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 415-423.
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 9(8):1735-1780.
Neil Houlsby, Ferenc Huszar, Zoubin Ghahramani, and Mate Lengyel. 2011. Bayesian active learning for classification and preference learning. CoRR, abs/1112.5745.
W. John Hutchins. 2004. The georgetown-ibm experiment demonstrated in january 1954. In Machine Translation: From Real Users to Research, pages 102-114, Berlin, Heidelberg. Springer Berlin Heidelberg.
Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Conference Proceedings: the tenth Machine Translation Summit, pages 79-86, Phuket, Thailand. AAMT, AAMT.
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 260-270.
Shari Landes, Claudia Leacock, and Christiane Fellbaum. 1998. Building semantic concordances. WordNet: An Electronic Lexical Database, pages 199-216.
David D. Lewis and Jason Catlett. 1994. Heterogeneous uncertainty sampling for supervised learning. In Proceedings of the Eleventh International Conference on Machine Learning, pages 148-156, San Francisco (CA). Morgan Kaufmann.
Ming Liu, Wray Buntine, and Gholamreza Haffari. 2018. Learning to actively learn neural machine translation. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 334-344, Brussels, Belgium. Association for Computational Linguistics.
Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Comput. Linguist., 19(2):313-330.
Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella bernardi, and Roberto Zamparelli. 2014. A sick cure for the evaluation of compositional distributional semantic models. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014).
Héctor Martínez Alonso and Barbara Plank. 2017. When is multitask learning effective? semantic sequence prediction under varying data conditions. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 44-53. Association for Computational Linguistics.
Andrew McCallum and Kamal Nigam. 1998. Employing em and pool-based active learning for text classification. In Proceedings of the Fifteenth International Conference on Machine Learning, ICML '98, pages 350-358.
Jonas Mueller and Aditya Thyagarajan. 2016. Siamese recurrent architectures for learning sentence similarity. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI'16, pages 2786-2792.
David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Linguisticae Investigationes, 30(1):3-26.
Laurent Nepveu, Guy Lapalme, Philippe Langlais, and George Foster. 2004. Adaptive language and translation models for interactive machine translation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 190-197, Barcelona, Spain. Association for Computational Linguistics.
Andrew Y. Ng, Michael I. Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 849-856. Curran Associates, Inc.
Daniel Ortiz-Martínez. 2016. Online learning for statistical machine translation. Computational Linguistics, 42(1):121-161.
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532-1543.
Álvaro Peris and Francisco Casacuberta. 2018. Active learning for interactive neural machine translation of data streams. In Proceedings of the 22nd Conference on Computational Natural Language Learning, pages 151-160, Brussels, Belgium. Association for Computational Linguistics.
Matthew S. Ryan and Graham R. Nudd. 1993. The viterbi algorithm. Technical report, University of Warwick.
Erik F. Tjong Kim Sang and Fien De Meulder 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition Proceeding of the Computational Natural Language Learning (CoNLL).
Fei Sha and Lawrence K. Saul. 2007. Large margin hidden markov models for automatic speech recognition. In B. Scholkopf, J. C. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 1249-1256. Curran Associates, Inc.
Yanyao Shen, Hyokun Yun, Zachary C. Lipton, Yakov Kronrod, and Animashree Anandkumar 2018. Deep active learning for named entity recognition. In 6th International Conference on Learning Representations.
Aditya Siddhant and Zachary C Lipton. 2018. Deep bayesian active learning for natural language processing: Results of a large-scale empirical study. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2904-2909.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014 Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1):1929-1958.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014 Sequence to sequence learning with neural networks In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS'14, page 3104-3112, Cambridge, MA, USA. MIT Press.
Cynthia A. Thompson, Mary Elaine Califf, and Raymond J. Mooney. 1999. Active learning for natural language parsing and information extraction. In Proceedings of the Sixteenth International Conference on Machine Learning, ICML '99, page 406-414, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
Erik F. Tjong Kim Sang and Sabine Buchholz. 2000 Introduction to the CoNLL-2000 shared task chunking. In Fourth Conference on Computational Natural Language Learning and the Second Learning Language in Logic Workshop.
Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 76-85, Berlin, Germany. Association for Computational Linguistics.
Active $^2$  Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation: Appendix
Figure 6: Modeling similarity using the Siamese encoder (enclosed by dotted lines). A pair of sentences from AUX dataset is fed to the pretrained ST/NMT model. The output of the encoder is then passed to the Siamese encoder. Last hidden state of the Siamese encoder, corresponding to the sequence length of the sentence, is used for assigning a similarity score.
We use two different sequence tagging architectures: CNN-BiLSTM-CRF model (CNN for character-level encoding and BiLSTM for word-level encoding) and a BiLSTM-BiLSTM-CRF model (Lample et al., 2016) (BiLSTM for both character-level and word-level encoding). The CNN-BiLSTM-CRF architecture is a light-weight variant of the model proposed in (Siddhant and Lipton, 2018), having one layer in CNN encoder with two filters of sizes 2 and 3, followed by a max pool, as opposed to three layers in the original setup. This modification was found to improve the results. We use glove embeddings (Pennington et al., 2014) for all datasets. We apply normal dropout in the character encoder instead of the use of recurrent dropout (Gal and Ghahramani, 2016b) in the word encoder of the model presented in (Siddhant and Lipton, 2018) owing to an improvement in performance. For numerical stability, we use log probabilities and, thus, the value for margin-based AL strategy's threshold is outside the interval [0, 1]. We use the spectral clustering (Ng et al., 2002) algorithm to cluster the sentences chosen by the AL
strategy. We chose two representative examples from each cluster.
Hyperparameters are given for Sequence Tagging (CoNLL 2003 NER) and NMT.
Active Learning strategy
threshold (Margin)	15
threshold (Entropy)	40
threshold (BALD)	0.2
dropout (BALD)	0.5
number of forward passes (BALD)	51
Sequence tagging model
CNN filter sizes	[2,3]
training batch size	12
number of train epochs	16
dimension of character embedding	100
learning rate (Adam)	0.005
learning rate decay	0.9
Siamese encoder
training batch size	48
number of train epochs	41
train/dev split	0.8
learning rate (Adam)	1e-5
period (of retrain)	10
Clustering
Number of clusters	20
Training
Batch size	12
NMT model
training batch size	128
number of train epochs	20
dimension of (sub)word embedding	256
learning rate (Adam)	1e-3
Siamese encoder
training batch size	1150
number of train epochs	25
dimension of (sub)word embedding	300
learning rate (Adam)	1e-3
period (of retrain)	3
Clustering
Number of clusters	50
Training
Batch size	128
Figure 7: [Best viewed in color] Ablations studies on different tasks using different active learning strategies.  $1^{st}$  row: NER,  $2^{nd}$  row: SEMTR,  $3^{rd}$  row: CHUNK,  $4^{th}$  row: NMT. In first three rows, from left to right, the three columns represent BALD, Entropy and Margin AL strategies.  $4^{th}$  row represents AL strategies for NMT, from left to right (LC: Least Confidence, CS: Coverage Sampling, ADS: Attention Distraction Sampling). Legend Description {100% data : full data performance,  $\mathbf{A}^2\mathbf{L}$  (MA Siamese) : Model Aware Siamese,  $\mathbf{A}^2\mathbf{L}$  (Int Model) : Integrated Clustering Model, Iso Siamese : Model isolated Siamese, InferSent : Cosine similarity based on InferSent encodings}. See Section 4.5 for more details. All results were obtained by averaging over 5 random splits.
Figure 8: [Best viewed in color] Comparison of our approach (A²L) with baseline approaches on different tasks using different active learning strategies.  $1^{st}$  row: POS,  $2^{nd}$  row: NER,  $3^{rd}$  row: SEMTR,  $4^{th}$  row: CHUNK. In each row, from left to right, the three columns represent BALD, Entropy and Margin based AL strategies. Legend Description  $\{100\%$  data : full data performance, A²L (MA Siamese) : Model Aware Siamese, A²L (Int Model) : Integrated Clustering Model, Cosine : Cosine similarity, None : Active learning strategy without clustering step, Random : Random split (no active learning applied)}\}. See Section 4.4 for more details. All the results were obtained by averaging over 5 random splits.