Uncertainty-Based Active Learning for Reading Comprehension
Authors: Jing Wang
Jie Shen
Xiaofei Ma
Andrew Arnold
Venue: Accepted by TMLR ()

Abstract: Recent years have witnessed a surge of successful applications of machine reading comprehension. Of central importance to these tasks is the availability of massive amount of labeled data, which facilitates training of large-scale neural networks. However, in many real-world problems, annotated data are expensive to gather not only because of time cost and budget, but also  of certain domain-specific restrictions such as privacy for healthcare data. In this regard, we propose an uncertainty-based active learning algorithm for reading comprehension, which interleaves data annotation and model updating to mitigate the demand of labeling. Our key techniques are two-fold: 1) an unsupervised uncertainty-based sampling scheme that queries the labels of the most informative instances with respect to the currently learned model; and 2) an adaptive loss minimization paradigm that simultaneously fits the data and controls the degree of model updating. We demonstrate on  benchmark datasets that 25% less labeled samples suffice to guarantee similar, or even improved performance. Our results show strong evidence that for label-demanding scenarios, the proposed approach offers a practical guide on data collection and model training. 

Published in Transactions on Machine Learning Research (12/2022)
Uncertainty-Based Active Learning for Reading Comprehen- sion
Jing Wang" Amazon
fing.julia. woggia....gmmmmaa.. com
Jie Shen Stevens Institute of Technology
jic.shen
Xiaofei Ma Amazon Web Services
eiaofeimm@omazon.cco
Andrew O. Arnold
wndrewa@delphia..oom
Delphia
Reviewed on OpenReview:
https: // openreview..ee.... net/ forum? id- QaDevCemeg
Abstract
Recent years have witnessed surge of successful applications of machine reading compre- hension. Of central importance to these tasks is the availability of massive amount of labeled data, which facilitates training of large-scale neural networks. However, in many real-world problems, annotated data are expensive to gather not only because of time cost and bud- get, but also of certain domain-specific restrictions such as privacy for healthcare data. In this regard, we propose an uncertainty-bast
active learning algorithm for reading compre- hension, which interleaves data amnotation and model updating to mitigate the demand of labeling. Our key techniques are two-fold: 1) unsupervised uncertainty- based sampling scheme that queries the labels of the most informative instances with respect to the currently learned model; and 2) an adaptive loss minimization paradigm that simultaneously fits the data and controls the degree of model updating. We demonstrate on benchmark datasets that 25% less labeled samples suffice to guarantee comparable, or even improved perfor- mance. Our results show strong evidence that for label-demanding scenarios, the proposed approach offers practical guide data collection and model training.
1 Introduction
The goal of machine reading comprehension (MRC) to train a model to understand natural language text (e.g. passage) and answer questions related to it (Hirschman et al. 1999); Figure for an illustration. MRC has been one of the most important problems in natural language processing thanks to its various successful applications, such as smooth-talking AT speaker assistants a technology that was highlighted as among 10 breakthrough technologies by MIT Technology Review (Karen 2019
Of central importance the success of MRC is the availability of benchmarking question-ant -anttiionnnnwering datasets, where larger dataset often enables training of more powerful neural networks. In this regard, there have been number of benchmark datascts proposed in recent years, with the effort of pushing forward the development of MRC. A partial list includes the SQuAD (Rajpurkar et al. 2016). NewsQA (Trischler et al.) 2017). MSMARCO (Nguyen et al. 2016). and Natural Questions (Kwiatkowski et al.) 2019). While the emergence of these high-quality datasets have stimulated a surge of research and have witnessed a large volume of deployments of MRC, it is often challenging to go beyond the scale the current architectures of neural
*The work was done when Jing Wang and Andrew Arnold were AWS Labs.

---

Published in Transactions on Machine Learning Research (12/2022)
Qucstion: What causes precipitation to fall?
In meteorology, precipitation is any product of the condensation of atmospheric water vapor that falls under gravity The main forms ... intense periods of rain in scattered locations are called "shower".
Passage:
Answer:
gravity
Figure 1: An illustrative example in the SQuAD datasel (Rajpurkar al. 2016).
networks, in that it is extremely expensive to obtain massive amount of labeled data. The barrier of data collection can be seen from SQuAD: the research group at Standford University spent 1,547 working hours for the annotation of SQuAD dataset, with the cost over $14,000. This issue was set out and partially addressed by the industry as well. However, even equipped with machine learning assisted labeling tools (e.g. Amazon SageMaker Ground Truth), it is still expensive to hire and educate expert workers for annotation. What makes the issue more serious is that there is rise of security and privacy concerns in various problems, which prevents researchers from scaling their projects to diverse domains efficiently. For example, all annotators are advised to get a scries of training about privacy rules, such as Health Insurance Portability & Accountability Act, before they can work on medical records.
In this work, we tackle the challenge by proposing a computationally efficient learning algorithm that is amenable for label-demanding problems. Unlike prior MRC methods that separate data annotation and model training, our algorithm interleaves these two phases. Our algorithm, in spirit, falls into the active learning framework (Balcan et al., 2007], where the promise of active learning is that can always con- centrate on fitting only the most informative instances without suffering a degraded performance. While there have been a considerable number of works showing that active learning often guarantees exponential savings of labels, the analysis holds typically for linear classification models (Awasthi et al.) 2017 Zhang let al. 2020). In stark contrast, less is explored for the more practical neural network based models since it is nontrivial to extend important concepts such as large margin of linear classifiers to neural networks.
As remedy, we consider an unsupervised sampling scheme based on the uncertainty of instances (Settles 2009). Our sampling scheme is active in the sense that it chooses instances that the currently learned model is most uncertain on. To this end, we recall that the purpose of MRC is to take as input passage and a question, and find the most accurate answer from the passage. Roughly speaking, this can be thought of as a weight assignment problem, where we need to calculate how likely each word span in the passage could be the correct answer. Idcally, we would hope that the algorithm assigns 1 to the correct answer, and assigns to the remaining, leading lo a large separation bet ween the correct and those incorrect (Sun et al. 2018). Alternatively, if the algorithm assigns, say 0.5 two different answers and assigns 0 to others, then is very uncertain about its response this is a strong criterion that need to qucry the correct answer from expert, ie. performing active labeling. Our uncertainty- based sampling scheme is essentially motivated by this observation: the uncertainty of an instance (i.e. pair of passage and question) is defined as the gap between the weight of the best candidate answer and the second best. We will present a more formal description in Section 2
After identifying these most uncertain, and hence most informative instances, we query their labels and use them to update the model. In this phase, in addition to minimizing the widely used entropy-based loss function, we consider time-varying regularizer which has two important properties. First, it enforces that the new model will not deviate far from the current model, since 1) with reasonable initialization we would expect that the initial model should perform not too poorly; and 2) we do not want to overfit the data even if they are recognized as informative. Second, the regularizer has a coefficient that is increasing with iterations. Namely, as the algorithm proceeds the stability of model updating outweighs loss minimization. In Scction [ we claborate on the concrete form of our objective function. It is also worth mentioning that since in each iteration, the algorithm only fits the uncertain instances, the model updating is faster than traditional methods.
2

---

Published in Transactions on Machine Learning Research (12/2022)
Paseeg
Question
Passage
Question
Passagi
Passage
Question
Passage
Answer
MRC model
MRC model
Passage
Passage
Question
Quastion
Passage
Answer
Passage
Quastton
Passago
Question
Figure 2: Illustration of our learning algorithm in each iteration. Given pool of unlabeled pairs of passage-questions, the algorithm first identifies the instances that is most uncertain on, e.g. those marked in red. Then it queries their answers (i.e. labels), and restricts to fit the newly labeled instances.
The pipeline is illustrated in Figure 2 Given abundant unlabeled instances, our algorithm first evaluates their uncertainty and detects the most informative ones, marked as pink. Then we query an expert on the correct answers, marked as orange. With the newly added labeled samples, it is possible to perform incremental updating of the MRC model.
Roadmap. We summarize our main technical contributions below, and discuss more related works
Section 5 In Section 2 we present. detailed description of the components of algorithm, and in
Section g provide end-to-end learning paradigm for MRC with implementation details. In Section 4 we demonstrate the efficacy of our algorithm in terms of exact match, F-1 score, and the savings of labels. Finally we conclude this paper in Section 0
1.1 Summary of contributions
We consider the problem of learning MRC models in the label-demanding context, and propose active learning algorithm that interleaves data annotation and model updating, In particular, there are two core components for this end: 1) an unsupervised uncertainty- sampling scheme that only queries labels of the most informative instances with respect to the currently learned model, which is inspired by the probability selection strategy introduced by Abe & Long (1999) for action selection in reinforcement learning, and 2) timo-varying loss minimization paradigm that simultancously fits the data and controls the degree of model updating, which is inspired by margin-based active learning of linear separators (Balcan et al., 2007). We present a comprehensive empirical study to demonstrate the efficacy of both components in reducing the labeling cost and in boosting the prediction accuracy. Lastly, though mainly focus the MRC problem in this work, we believe that our approach is fairly general and can be applied to other real-world problems.
2 Main Algorithms
In this section, formally introduce the problem setup and our main algorithm, ALBUS (Active Learning By Uneertainty-basee Sampling in Algorithm I We use == (p, q) to represent pair of passage p and question q, which will also be referred to as an instance. Tf there are multiple questions, say Qı (2, to a same passage p, we will use two instances x1 := (p. qı) and Ø2 == (p,42). Given an instance æ, our goal is to predict an answer. We use a zero-one vector labeled instance. The prediction made by the learner is denoted by à. We will always assume that all the coordinates of non-negative, and their sum equals one, which can be casily satisfied if the last layer of the neural network is softmax.
q2). indicate the correct answer, and (æ, a) is called a
2.1 Unsupervised Uncertainty-Bant Based Random Sampling
Since data annotation is expensive, we treat the problem as such that all the instances are unlabeled before running the algorithm, and as the algorithm proceeds, it may detect. the most informative instances and
3

---

Published in Transactions on Machine Learning Research (12/2022)
Algorithm ALBUS: Active Learning By Uncertainty-based Sampling
Require:
a of unlabeled instances U = {a1,.... Fn initial MRC model wn, maximum iteration number T, thresholds (TI...., Tr) the number of instances to be labeled nu-
Ensure: A new MRC model wr.
U.
2: for t= T do
3: Compute Am, (Φ) for all x E U--
4: B fxr E U₁ A (z) S TT).
5: Compute the sampling probability Pr(œ) for all a Bz.
S, + randomly choose no instances from Be by the distribution (Pr(z))æEB, and query their labels.
Update the model wt arg min St):
8: U141 S..
end for
have experts crowdworkers annotate. Thus, the central questions to learning are: 1) how to measure the informativeness of the unlabeled instances in a computationally efficient manner; and 2) how to select a manageable number of instances for annolation (since the algorithm might identify a large number of useful instances). We address both questions in the following.
2.1.1 Metric of Informativeness
Intuition. We first address the finst question, i.e. design metric to evaluate the informativeness. To ease the discussion, suppose that for a given instance æ, there are only two answers to choose from, i.e. a is a two-dimensional vector, and that the algorithm has been initialized, e.g. via pre-training. If the current model takes an input æ, and predicts à = (1, 0), then we think of this instance as less informative, in that the algorithm has an extremely high confidence on its prediction | On the other end of spectrum, if the prediction à (0.5, 0.5), then indicates that the current model is not able to distinguish the two answers. Thus, sending the correct answer together with the instance to the algorithm will lead to significant progress.
We observe that underlying the intuition is notion of separation between the answer with highest confidence and the second highest that was introduced in the literature before (Settles) 2009). The scparation is denoted by Aw (x), where w the current model. In fact, let our algorithm be a function fw x + â. Denote by a and (() the highest and second highest value à. Then
Aw = (1)) - a(2)
(1)
Given the unlabeled training set (œ) ........ Fn and the currently learned model, we can evaluate the degree of scparation {A1, A An} where we write A, := (z;)) to reduce notation clutter since most of the time, the model w is clear from the context. This answers the first question proposed at the beginning of the section, ie. how to measure the informativeness of the instances.
2.1.2 Bncertainty-Based Sampling
It remains to design a mechanism so that we can gather a manageable number of instances to be labeled. A natural approach will be specifying the maximum number no, so that each iteration the algorithm chooses at most Ag instances with lowest degree separation. Yet, we observe that for some marginal cases, many instances have very close A; e.g. A = 0.101 and Ag 0.102. Using the above strategy may annotate æ while throwing away 22. From the practical perspective, however, hope both instances will have a chance to be selected to increase diversity. Henceforth, consider "soft" approach based random sampling.
The algorithm may of course make mistake, but this will be treated by future model updating. Here we just giving intuitive explanation following the idealized scenario.
4

---

Published in Transactions on Machine Learning Research (12/2022)
Fixing an iteration of the algorithm, first define threshold (0, 1]. Based the current model 14-1, we calculate A A1...., An. Then we obtain a sampling region
B. = (a, S T.).
(2)
which contains informative instances (recall that lower degree of separation implies more informative model). Inspired by the probability selection scheme (Abe & Long 1999), we define the sampling probability as
Va E B 12
Pr(æ)) |B (B:1t((an- Ex'da" (B((tn(A)-AA.)) when x = x².
In the above expression, x² is the instance in B, with a lowest degree of separation, i.c. the most uncertain instance; 2 0 is a tunable hyper-parameter. Observe that when = 0, it becomes uniform sampling, In addition, in view of the sampling probability in equation 3 the instance x + x² will be sampled with probability less than B/) and xt will be sampled with probability higher than 1/ 1/1,, as
Pr(x") 2
Σ 2
[ B₁
1
(4)
B
[Br
Therefore, the sampling scheme always guarantees that x" will be selected with the highest probability, and if needed, it is possible to make this probability close to 1 by increasing In our algorithm, we set O ((VlB,) which works well in practice.
2.2 Time-Varying Loss Minimization
Another crucial component in ALBUS is the loss minimization. Here our novelty is the introduction of a time-varying regularizer that balances the progress of model updating and per-iteration data fitting.
Let St be the set of labeled instances determined by our random sampling method at the t-th iteration. For any (x. a) E St, since a an indicator vector, the problem can be thought of as multiclass classification. Therefore, a typical choice of sample-wise loss function the logistic loss, denoted by E(wr; æz a), which can be easily implemented by using softmax layer in the neural network. On top of the logistic loss, we also consider tarcingrying fo-norm regularizer, which gives the following objoctive function:
L(w; St) = |Sel
Σ (2.0)ES.
e(w; x. a) + A w - w.-1l1
(5)
2
Different from the broadly utilized fz-norm regularizer || we appeal localized form, in the sense that the objective function will ensure that the updated model be not far from the current. model w,-1 under the Euclidean distance. Practically speaking, this is because in many cases, pre-training often exhibits decent performance.
Regarding the coefficient Ae, we increase it by constant factor greater than one in each iteration. Therefore, as the algorithm procoeds, the localization property plays a more important role than the logistic loss. Our treatment is inspired by the literature active learning (Balcan et al. 2007 Zhang et al., 2020), where a similar localized fz-norm constraint is imposed as shown Figure 3 The objective function in equation 5
is the Lagrangian duality function of the following nonlinear programming problem:
min 1
Σ (Z,a)ES
f(w; a), s.t. ||w ------ll
wi-ill S at
(6)
S
where or is the parameter related to dr. As shown Figure 3 the updated model in the 1-th iteration w. is far away from the previous iteration w1-1, we project the w, w/ formally
wi wi-1
w w. + at Iw: wi
(7)

---

Published in Transactions on Machine Learning Research (12/2022)
Wt
at
Wt
Wt-1
Figure 3: Localization regularizer of the optimization.
This can be viewed stability property of our algorithm, and discover that it works very well benchnark datasets.
From a practical perspective, we remark that the regularized formulation is easier to implement in deep neural networks than the constrained one. This is because the back propagation algorithm updates the weights of layers one by one, making the projection onto an f--norm ball computationally expensive.
3 Implementation Details
Uncertainty-based sampling. We introduce how to select the batch S. in each iteration with current MRC model wn-1- For a given pair of (p. q), an answer is of the form of word span from the i-th position to the j-th position of the passage. Given the span (i,j) and the passage p, we BERT our embedding method (Devlin et al. 2019), which produces feature description denoted by Ep(i,j). We then construct probability matrix M whose (i,j)-th entry Mi, is given by the following:
exp(w,-1 Ep((,))
M.j En.g exp w: Fp((".)))
(8)
Observe that the matrix M forms a distribution over all possible word spans, i.e. all possible answers. It is then straightforward to convert M into the vector à, for example, by concalenating all the columns. Based on the obtained answer à, we are able to perform uncertainty based sampling as discussed in Section 2
Adaptive loss minimization. We already derived the probability matrix M in equation g During loss minimization, i.e. supervised fine-tuning, we aim to update w:-1 by minimizing L(w; S.) Since we have clarified the regularizer, it suffices to give the detailed form of the loss f(w; æ, a) where recall that æ (p.0) q). Note that using the groundtruth answer we know the correct span (aa,ja) for question q.
Thus, the likelihood that we observe S, is
exp(w Ep(ia: II (p.q.a)Es,
Pr(Si)
(9)
Exs exp(w (' (')
The loss function f(w; St) simply the negative log- lieeliod.
4 Experiments
Datascts. We focus on the span-based datasots, namely Stanford Question Answering Datasct (SQuAD)
(Rajpurkar et al. 2016) and NewsQA (Trischler al.) 2017 SQuAD consists of over 100,000 questions posed by crowdworkers set of 536 Wikipedia articles. We use the original split: 87,599 questions for

---

Published in Transactions on Machine Learning Research (12/2022)
Table 1: EM and Fl score on the SQuAD dataset.
EM
F1 scorc
# Labels queried
Badge
Conf
Entropy
Margin
Rand
ALBUS
Badge
Conf
Entropy
Margin
Rand
ALBUS
5000
60.94
59.62
60.52
62.71
62.58
64.03
72.20
72.29
72.78
74.28
73.97
75.30
15000
71.75
72.05
71.89
72.69
71.54
74.13
81.62
82.18
82.31
82.59
81.41
83.50
21000
73.88
74.38
74.67
74.31
73.75
75.48
83.54
83.90
84.23
83.77
83.08
84.53
410000
77.55
77.37
77.80
78.16
75.86
79.02
86.02
85.69
85.95
86.19
84.51
87.09
61000
77.90
77.98
77.75
78.06
77.98
80.44
86.15
85.86
85.43
86.32
86. 10
88.13
81000
76.08
76.08
75.66
76.34
78.63
81.14
84.75
84.08
83.74
84.64
86.98
88.53
training and 10,570 questions for testing. NewsQA is a machine comprehension dataset of over 100,000 human-generated question-answer pairs from over 10,000 news articles from CNN. The dataset is composed of 74,160 questions for training and 4,212 questions for validation F
Evaluation Metrics. We use two standard metrics: Exact Match (EM) and Fl score. EM measures the percentage of predictions that matches any one the annotated answers exactly. EM gives credit for predictions that exactly match (one of) the golden answers. F1 score measures the average overlap between the prediction and the annotated answer.
Baselines. We compare our approach ALBUS to the following baseline algorithms:
2020 : it learns the gradient embodding of samples and Badge (batchod based sampling) (Ash et al.
selects set of samples by K-MEANS++ (Athhr & Vassilvitskii 2007).
Conf (confidence sampling) WWang& & Shang 2014): an uncertainty- algorithm that selects samples for which the model produces the most uncertain classification results.
Entropy (Wang & Shang 2014): it selects samples based on the entropy of the predicted probability distribution.
Marg (margin- based sampling) RRoth & Small 2006): it checks the degree of separation as our algorithm, but selects no samples with most uncertainty rather than "soft" " approach to encourage diversity as we did.
Rand (Random sampling): the naive baseline of uniformly selecting samples from an unlabeled data pool.
Other Settings. To ensure a comprehensive comparison among state-of-the-art approaches, we simulate the annotation process with human experts in the loop by selecting fixed number of examples no to query their labels from training set in each iteration (we set no = 2, 000 for SQuAD and no = 5, 000 for NewsQA). The labeled data is used to update the MRC model. We report the exact match and F1 score with the number of iterations. BERT-base is used as the pretrained model and fine-tuned for 2 epochs with a learning rate of 35 and a batch size of 12, the default setting of Huggingface 3 The MRC model is initialized with 1,000 labeled samples for SQuAD and 10,000 for NewsQA. The parameter To is chosen from the range of [0.01, 0.1] based on the training set and decreases at the rate of 1.1.
4.1 Efficacy Study
Figure and Figure present EM and F1 score with the increase of the number of labeled samples selected by various active learning algorithms. We show the results with all labeled data (Figure [aa and Figure 5 ())) and 20,000 labeled data (Figure (b) and Figure b)). Our algorithm outperforms state-of-the-ae active learning algorithms in almost all the cases.
Table I lists detailed results with specific numbers of labeled samples. Our algorithm reaches the best performance in all cases and the advantage is significant specially when only a small subset of labeled
7

---

Published in Transactions on Machine Learning Research (12/2022)
samples available. For example, with 5,000 labeled examples available, algorithm reaches EM of 64.07% while the best of compared algorithms is 62.71% %. Figure 1 and Figure S plot the trends of EM and Fl score with the rise of labeled examples on SQuAD dataset. We observe that active learning algorithms reach the best performance before accessing all labeled data compared with Rand. It demonstrates the active learning effectively reduces the number of required labeled data for the learning process. Specifically, our algorithm achieves EM of 80.44% and F1 score of 88.53% with 61 61,000 queries which close the best result but with 25% less labeled samples. Our algorithm enjoys significant advantages over compared algorithms the NewsQA dataset as shown in Figure 6
SQuAD Dataset
SQUAD Dataset
80
70
70
Methods
60
Methods
60
Badge
Badge
EM
Conf
Conf
50
50
Entropy
Entropy
Margin
Margin
40
Rand
Rand
Ours
Ours
30
0
20000
40000
60000
80000
0
5000
10000
15000
20000
#Labels queried
#Labels queried
Figure 4: Compared results of FM on SQuAD with over 80,000 (left) and 20,000 (right) labeled data.
SQuAD Dataset
SQuAD Dataset
90
Methods
Methods
70
Badge
Badge
Conf
Conf
60
Entropy
60
Entropy
Margin
Margin
50
Rand
50
Rand
Ours
Ours
0
20000
40000
60000
80000
5000
10000
15000
20000
#Labels queried
# Laabels queried
Figure 5: Compared results of Fl score SQuAD with over 80,000 (left) and 20,000 (right) labeled data.
4.2 Ablation Study
In Table provide ablation experiments of uncertainty-based sampling and localization regularization the SQuAD dataset. 1) To examine the effect of uncertainty-bast based sampling, only apply it to select each batch of samples for model training without localization regularization. The method is termed Uncertainty- based Sampling (US) as shown in Table 2 2) To examine the ellect of localization regularization, we replace the uncertainty-basee sampling with the default data loading in the reading comprehension framework. The framework minimizes the softmax loss with localization regularization as in equation 5 The method is denoted localization regularization (LR). We also list the results of random sampling (denoted Rand) and those of combining uncertainty-based sampling and localization regularization (denoted ALBUS). As
8

---

Published in Transactions on Machine Learning Research (12/2022)
NewsQA Dataset
NewsQA Dataset
67.5
50
65.0
Methods
62.5
Methods
Badge
Badge
60.0
Conf
Conf
Entropy
57.5
Entropy
40
Margin
Margin
Rand
55.0
Rand
52.5
Ours
20000 40000 60000 80000 100000120000140000
20000 40000 60000 80000
#Labels queried
#Labels queried
Figure 6: Compared results of EM and F'l score on NewsQA with increase of labeled data.
Table 2: Ablation study of Uncertainty Sampling (US) and Localized Regularization (LR) the SQuAD
dataset in terms of EM and F1 score.
EM
F1 score
#Labels queried
Rand
CS
LR
ALBUS
Rand
US
LR
ALBUS
5000
62.58
62.82
62.26
64.03
73.97
73.16
73.07
75.30
15000
71.54
72.33
72.85
74.13
81.41
81.72
82.45
83.50
21000
73.75
73.80
74.62
75.48
83.08
82.67
83.64
84.53
41000
75.86
76.10
77.00
79.02
84.51
84.45
85.26
87.09
61000
77.98
78.21
77.71
80.44
86.10
86.25
86.95
88. 13
81000
78.63
79.13
78.21
81.14
86.98
86.88
86.25
88.53
the results shown in the table demonstrate, both techniques improve the performance of random sampling in terms of FM and F1 score with different number of labeled samples in most cases. The full pipeline provides significant improvement. The main reason is that the uncertainty- which help improve the model. The default optimization does not fit well with the new batches of data. Here the localization regularization guarantees that the model improves in each step. Hence, the combination of the two techniques leads to pipeline with impressive empirical improvement.
sampling selects samples
5 Related Works
Machine Reading Comprchension. MRC is the ability to read text and answer questions about it. It is challenging task it requires the abilities of understanding both the questions and the context. A data- driven approach to reading comprehension goes back to (Hirschman et al. 1999). In recent years, number of large-scale question answering datasets are constructed to provide opportunity for the development of machine reading comprehension methods (Rajpurkar et al.) 2016 Kwiatkowski et al. 2019). For example, Stanford Question Answering Dataset (SQuAD) dataset consists of 100,000 questions on a set of Wikipedia articles (Rajpurkar et al., 2016). Natural Questions dataset consists of queries issued to the Google scarch engine and answers annotated from Wikipedia pages (Kwiatkowski et. al.) 2019). Recently, some language models generate answers given question and the database instead of retrieving the result Lazaridou et al. (2022). The difference is like feature selection and dimension reduction Wang et al. (2018); Candès et al. (2011). Feature selection choose a subset of the original feature space which is explainable. In this work, our framework retrieves the answer from the database.
There are many approaches proposed learn a rich language representation for questions and context. For example, ELMo learned forward and backward language models: the forward one reads the text from left right, and the other encodes the text. from right to left. (Peters et. al. 2018). GPT used left-to-right
9

---

Published in Transactions on Machine Learning Research (12/2022)
Transformer to predict a text sequence word-by-word (Radford et al. 2018 Devlin et al (0119) designed BERT to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers. Our work is based on pretrained BERT model. We fine-tuned BERT with one additional softmax layer to calculate the correlation between the question and answer candidates.
Active Learning. Active learning is a machine learning paradigm that mainly aims at reducing label requirements through interacting with the oracle experts/annotatonn (Angluin & Laird 1988). Active learning has been well studied in both theory and applications. For example, under the probably approx- imately correct model (Valiant [984), margin-
active learning was utilized in (Balcan et al.) 2007; Balcan & Long) 2013; Awasthi et al.) 2017; Shen & Zhang] 2021 Shen 2021), showing that as long as the instance distribution satisfies certain conditions, it is possible to save exponential number of labels compared to passive learning when learning a halfspace. There is also evidence showing that without any restriction on the underlying problem, active learning may not be able to provide savings in labeling cost (Dasgupta) 2005). For more general learning problems, the disagrecment- active learning framework a more natural fit. (Hanneke) 2011).
Another rich set of works mainly develop new sampling schemes for data annotation. For example, repre- sentative sampling selects samples that are representative of the whole unlabeled dataset. It can be achieved by performing an optimization minimizing the difference between the selected subset and the global dataset GGissin& & Shalev-Shwartz, 2019). The setertainty-based sampling selects samples that maximally reduce the uncertainty the algorithm has on a target learning model, such as samples lying closest to the current decision boundary (Tür et al.) 2005). Citovsky et al. (2021) defined the "cluster passter-margnn" uncertainty sam- pling variant based the difference between the largest two predicted class probabilities. The round-robin sampling scheme is employed, that is the algorithm iterates through the clusters to select samples until the desired number of samples are chosen. The work in this paper belongs to uncertainty-basee sampling but equipped with a localization regularization scheme tailored to MRC, Our time-varying regularizer may appear similar to Kirkpatrick et al. (2017) which also uses a localized regularizer to overcome catastrophic forgetting in neural works However, their goal is to ensure that on two lasks A and B, the model learned for B needs to be close to A, while in our work, the regularization is introduced to refine the model iteratively (on one task). Second, their regularization fixes the learned model of task A and uses it as an anchor to guide the training
task B, and when a third task Comes in, will fix the models of A and B and tune that of C. In contrast, the anchor in our work is dynamically updated as the previous iteration, which follows from the active learning literature. We also note that our algorithm naturally updates the model in an online fashion, which thus has a potential to be adapted to continual learning (Chen et al., 2022). For example, our regularization can be used in the loss function of continual learning to discourage changes in already learned parameters in the face of new tasks. Replaying is widely believed to reduce forgetting (McClelland et al., 1995). The uncertainty-based sampling strategy can be used to select previously visited data to enhance the stability Rolnick et al. 2019). We leave it our future work.
On the application side, active learning has shown outstanding performance in real-world applications, such as computer vision and natural language processing (Joshi et al. 2009 Culotta & McCallum 2005 Reichart let al.) 2008). Recent studies combining deep neural networks and active learning approaches have been proposed (Zhang et al.] 2017 Shen et al. 20188 Geifman & EI- Yaniv 2019). However, these approaches do not consider the correlation between adaptively learned models of sclected samples. While Ash & Adams (2020) called in question on using warm-starting in neural networks, we note that in this work, we update the model with only fresh batches of labeled data, which is different from the setting thereof.
6 Conclusion and Future Works
In this work, we have proposed an active learning algorithm and apply it for the machine reading comprehen- sion task. There are two crucial components in our algorithm: an unsupervised uncerlainty-based random sampling scheme, and a localized loss minimization paradigm. We have described the strong motivation of using these techniques, and our empirical study serves as a clear evidence that our algorithm drastically mitigates the demand of labels on large-scale datasets. We highlight that our approach is not essentially tied to MRC, and expect that it can be extended to other label-denanding problems in natural language
10

---

Published in Transactions on Machine Learning Research (12/2022)
processing and computer vision. Since our framework is in online fashion, it is interesting to explore its application in continual learning for multiple tasks.
Acknowledgments
We would like to thank Dean Foster and Bing Xiang for their insightful leedback and support. We appreciate reviewers' valuable and encouraging comments. Jie Shen supported by NSF-TTS-1948133 and startup funding from Stevens Institute of Technology.
References
Naoki Abe and Philip M Long. Associative reinforcement learning using linear probabilistic concepts. In Proceedings of the Sizteenth International Conference on Machine Learning, pp. 3-11, 1999.
Dana Angluin and Philip D. Laird. Learning from noisy examples. Machine Learning, 2(4):343 370, 1988.
David Arthur and Sergei Vassilvitskii. k-means+++: the advantages of careful seeding. In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027 1035, 2007.
Jordan Ash and Ryan P Adams, On warm-starting neural network training. Advances in Neurol Information Processing Systems, 33:3884-3894, 2020.
Jordan Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langlord, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In Proceedings of the 8th International Conference Learning Representations, 2020.
Pranjal Awasthi, Maria-Florina Balcan, and Philip M. Long. The power of localization for efficiently learning linear separators with noise. Journal of the ACM, 63:6)500:15 2017.
Maria-Florina Balcan and Philip M. Long. Active and passive learning of linear separators under log- lon-concave distributions. Proceedings of the 26th Annual Conference on Learning Theory, pp. 288-33 088-11,,50 2013.
Maria-Florine Balcan, Andrei Broder, and Tong Zhang. Margin based active learning. Tn International Conference Computational Learning Theory, pp. 35-50. Springer, 2007.
Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright. Robust principal component analysis? Journal of the ACM (JACM), 58(3):1-33 37, 2011.
Xi Chen, Christos H. Papadimitriou, and Binghui Peng. Memory bounds for continual learning. In IEEE Symposium on Foundations of Computer Science, 2022.
Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, and Sanjiv Kumar. Batch active learning at scale. Advances in Neural Information Processing Systems, 11111993-11944 2021.
Aron Culotta and Andrew McCallum. Reducing labeling effort for structured prediction tasks. In Proceedings of the 20th AAAI Conference on Artificial Intelligence, pp. 746-751, 2005.
Sanjoy Dasgupta. Coarse sample complexity bounds for active learning. In Proceedings of the 19th Annual Conference Neural Information Processing Systems, pp. 235-2422 2005.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidi- rectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171 4186, 2019.
Yonatan Geifman and Ran El- Yaniv. Deep active learning with neural architecture search. In Annual Conference Neural Information Processing Systems, pp. 5974-5984, 2019.
11

---

Published in Transactions on Machine Learning Research (12/2022)
Daniel Gissin and Shai Shalev-Shwarte Discriminative active learning. orXiv preprint arXiv:1907.06347, 2019.
Steve Hanneke. Rates of convergence in active learning. The Annals of Statistics, 39(1) (0(1):3333300) 361, 2011.
Lynette Hirschman, Mare Light, Eric Breck, and John D. Burger. Deep read: A reading comprehension system. In Proceedings of the 27th Annual Meeting of the Association for Computational Linguistics, pp. 325 332, 1999.
Ajay J. Joshi, Fatih Porikli, and Nikolaos Papanikolopoulos. Multi-class active learning for image classifica- tion. In Proceedings of the 2009th IEEE Computer Socieby Conference on Computer Vision and Pattern, pp. 2372 2379, 2009.
Hao Karen. 10 breakthrough technologies 2019. MIT Technology Review, 2019.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-BBrwwinsker et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-335 2017.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming- Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural ques- tions: benchmark for question answering rescarch. Transactions of the Association of Computational Linguistics, 2019.
Angeliki Lazaridou, Elena Gribovskaya, Wojeiech Stokowiec, and Nikolai Grigorev. Internet-augmentee language models through few-shot. prompting for open-domain question answering. arXiv preprinl arXiv:2203.05114 2022.
James L McClelland, Bruce L McNaughton, and Randall C O'Reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learming and memory. Psychological review, 13:41), 1995.
Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the Workshop on Cognitive Computation:
Integrating neural and symbolic approaches, volume 1773, 2016.
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zel Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp. 2227 2237, 2018.
Alec Radford, Karthik Narasimhan, Tim Salimans, and llya Sutskever. Improving language understanding with unsupervised learning. Technical report, OpenAI, 2018.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100, 000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2883-2392, 2016.
Roi Reichart, Katrin Tomanck, Udo Hahn, and Ari Rappoport. Multi-task active learning for linguistic annotations. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pp. 861-869, 2008.
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32, 2019.
Dan Roth and Kevin Small. Margin- based active learning for structured output spaces. In Proceedings of the 17th European Conference on Machine Learning, volume 4212, pp. 413 -24444, 2006.
12

---

Published in Transactions on Machine Learning Research (12/2022)
Burr Settles. Active learning literature survey. Technical report, University of Visconsin-Madison Depart- ment of Computer Sciences, 2009.
Jie Shen. Sample-optimal PAC learning of halfspaces with malicious noise. In Proceedings of the 38th International Conference Machine Learning, pp. 9515-9524, 2021.
Jie Shen and Chicheng Zhang. Attribute-efficient learning of halfspaces with malicious noise: Near-optimal label complexity and noise tolerance. In Proceedings of the 32nd International Conference on Algorithmic Learning Theory, pp. 1072-1113, 2021.
Yanyao Shen, Hyokun Yun, Zachary C. Lipton, Yakov Kronrod, and Animashree Anandkumar. Deep active learning for named entity recognition. In Proceedingss of the 6th. International Conference on Learning Representations, 2018.
Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William W. Cohen. Open domain question answering using early fusion of knowledge bases and text. In Proceedings of the 2018 Conference on Empirical Methods in Natural Longuage Processing, pp. 4231 4242, 2018.
Adam Trischler, Tong Wang. Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. Newsqa: A machine comprehension dalaset. In Proceedings of the 2nd Workshop Represen- tation Learning for NLP, pp. 191- 191-200, 2017.
Gökhan Tür, Dilek Hakkani- Tür, and Robert E. Schapire. Combining active and semi-supervised learning for spoken language understanding. Speech Commun., 45(2):171-186, 2005.
Leslie G. Valiant. A theory of the learnable. Communications of the ACM, 27(11):1134 1142 1984.
Dan Wang and Yi Shang. A new active labeling method for deep learning. In Proceedings of the 2014 International Joint Conference on Neural Networks, pp. 112-119, 2014.
Jing Wang, Jie Shen, and Ping Li. Provable variable selection for streaming features. In International Conference Machine Learning, pp. 5171-5179. PMIR, 2018.
Chicheng Zhang, Jie Shen, and Pranjal Awasthi. Efficient active learning of sparse halfspaces with arbitrary bounded noise. In Proceedings of the 34th Annual Conference on Neural Information Processing Systems, pp. 7184-7197, 2020.
Ye Zhang, Matthew Lease, and Byron C Wallace. Active discriminative text representation learning. In Proceedings of the 31st AAAT Conference on Artificial Intelligence, 2017.
13


Reviews:
[{"review_id": "dI2zzYTx7E", "reviewer": "Reviewer_hf2S", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n  1. The problem of using active learning to reduce fewer samples for NLP is important and well-motivated.\n\n  2. The two ideas in proposed ALBUS algorithm, uncertainty-based sampling and localized regularization, are intuitive and are shown to be effective in experiments.\n\n  3. The writing is mostly clear and easy to follow.\n\nWeaknesses:\n\n  1. Ablation study is missing. The proposed algorithm has two major components, uncertainty-based sampling and localized regularization. They should be evaluated separately in ablation experiments to understand and justify the improvement of each component. \n\n  2. The incremental update using newly sampled data in the proposed method is interesting but not well justified. There should be a tradeoff between model retraining and incremental update in terms of performance and efficiency -- usually model training gives better performance than incremental update. The authors are suggested to include discussion and evidence on this design.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper studied uncertainty-based active learning for machine reading comprehension (MRC). The authors proposed to sample most uncertainty/informative examples based on current model's prediction. In the training part, the authors introduced a localized regularizer to limit current model not changing too much from previous iteration. Experiments on SQuAD and NewsQA showed the proposed active learning method requires fewer samples to achieve similar performance than baselines.\n", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "1. Ablation study on the two components.\n\n2. Discuss and justify the incremental update.\n\nPlease see weaknesses for details.\n", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "qdLNv4Tsmy", "reviewer": "Reviewer_zxTy", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper can be interesting to active learning (AL) researchers as an application of AL. The setup is interesting and could also be applied to AL in image segmentation, for example. The introduction is very well written.\n\nHowever, the claims in the paper in regards to novelty seem misleading as such I cannot recommend acceptance unless there are significant changes to the paper.\n\n1. The paper strenuously avoids placing itself within an active learning context and calls what it does \"adaptive learning\", which is no different to active learning. The paper also ignores deep active learning and only refers to AL works pre-2010 (later works are mentioned in one sentence in the related works section without substantial explanation why they are ignored).\n\n2. The paper proposes marginal sampling, batch acquisitions, and L2 regularization to previous weights (ie. EWC) as novel contributions when they are clearly not. The paper could suggest its novelty as combining all these methods, but would then lack necessary ablations.\n\n3. The paper ignores the possibility of aleatoric uncertainty in the valid answers: it defines high uncertainty and informative samples as the model being unsure about which text passage contains the proper answer, not addressing the possibility that multiple passages might contain valid answers (a sort of aleatoric uncertainty).\n\n4. The paper mentions that L2 regularization and warm starting are beneficial. However, https://arxiv.org/abs/1910.08475 \"On Warm-Starting Neural Network Training\" calls that into question. Ablations or further evidence is warranted.\n\n5. The paper does not explicitly explain what the full NN model is that it uses. From cross-reading, it seems to be using a linear model (logistic regression) on top of pre-trained BERT that is fine-tuned, too? As such, the paper could compare to AL of linear models with a fixed BERT-base, with plenty of prior art from the 1990s, and it warrants the question: why was a Bayesian logistic regression not attempted as baseline on top of fixed BERT embeddings? \n\n    See e.g. MacKay, David JC. \"Information-based objective functions for active data selection.\" Neural computation 4.4 (1992): 590-604.\n\n \n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes active learning for machine reading comprehension. It uses batch acquisitions with random sampling using margin sampling as an acquisition function and uses L2 regularization similar to EWC (for example, https://arxiv.org/abs/1612.00796).", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "1. Rewrite the paper to make clear it's about active learning and change the title?\n2. Reduce the novelty claims and make clear that it is about the combination of 3 known techniques. The only contribution in this reviewer's opinion are the applied results for MRC using pre-trained BERT features.\n3. Ablations for warm-starting vs cold-starting.\n4. Ablation for the model only use logistic regression on top of fixed BERT embeddings? Maybe also Bayesian logistic regression and BALD/EIG/Query-by-Committee (different names, same thing) to use epistemic uncertainty? What is the full model actually used?\n5. Above eq (4), there is a $\\mu$ mentioned. Is that supposed to be $B_t$?\n6. The results seem to report single trials. AL is very noisy in general and multiple trials (3-5?) ought to be run with confidence intervals or std dev reported.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "zVVQ4yNGSn", "reviewer": "Reviewer_Q3Cq", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "$\\textbf{Strengths:}$\n\n1. The authors propose a label-efficient technique to train a machine reading comprehension model, which only uses the most informative labeled samples for training the model.\n\n2. The proposed adaptive loss function jointly minimizes the error for fitting the data and the degree of updating model parameters in consecutive steps.\n\n3. Results on benchmark MRC datasets show the efficacy of the proposed algorithm.\n\n$\\textbf{Weakness:}$\n1. The notion of identifying highly informative samples is unclear. The current definition limits the choice of new samples using prediction confidence irrespective of their correctness.\n\n2. The regularization loss in Equation 5 may limit the generalization capability of the underlying model. The use of regularization seems counterintuitive. First, you want your model to learn harder instances by sampling instances with higher information, whereas the $\\ell_{2}$ loss function restricts the model to its current weight parameter space using the weight regularizer.\n\n3. The experimental results lack ablation studies to show the utility of the individual loss functions in Equation 5.\n\n$\\textbf{Small edits:}$\n1. The $\\mathbf{w}$ in Equation 7 should be the weight at a given time step $t$.\n\n2. It would be great if the authors add standard deviation to their results as the current empirical analysis show a small increase in the benchmarking results.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes uncertainty-based adaptive learning for machine reading comprehension (MRC) tasks. More specifically, the authors introduce Albus (Adaptive Learning By Uncertainty-Based Sampling), which combines data annotation and model updating to mitigate the demand for labeling. The proposed algorithm's primary goal is to identify the most informative samples which can be queried to generate ground-truth labels. Further, it introduces an adaptive optimization problem to simultaneously fit the model with a given set of informative data samples and control the level of updating the model parameters. Finally, empirical results show that Albus achieves on-par or better results on benchmark MRC datasets using 25% less labeled samples.", "claims_and_evidence_yn": "", "claims_explanation": "", "audience_yn": "", "audience_explanation": "", "requested_changes": "In addition to the points mentioned in the weakness, it would be great if the authors can address the following:\n\n1. The authors talk about identifying the most informative samples. It would be great to add more details about what exactly determines a sample as informative. Does informativeness relate to harder or easier examples? Intuitively, since the samples are identified with respect to the current learned model, they should pertain to harder instances.\n\n2. In Line 4 of Algorithm 1, we measure the informativeness of a sample using the model's answer with the highest confidence and that with the second highest using the model weights $\\mathbf{w}_{t-1}$ with an adaptive threshold. However, will this cater to examples that are also misclassified with higher confidence?", "broader_impact_concerns": "Not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper aims to improve sample efficiency of machine reading comprehension. In this regards, the author propose an uncertainty-based active learning technique. The algorithm shows significant improvement empirically. We thank the authors and reviewers for actively engaging in discussion and taking steps towards improving the paper including clarity and providing additional experiments. The paper has come a long way since initial submission.
Decision: Accept (poster)