Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training
Authors: Zhisong Zhang
Emma Strubell
Eduard H Hovy
Venue: EMNLP 2023 Findings (2023)
Keywords: Active Learning
Structured Prediction
Partial Annotation
Self-training
TL;DR: Exploring Better Active Learning Strategies for Structured Prediction

Abstract: In this work we propose a pragmatic method that reduces the annotation cost for structured label spaces using active learning. Our approach leverages partial annotation, which reduces labeling costs for structured outputs by selecting only the most informative sub-structures for annotation. We also utilize self-training to incorporate the current model’s automatic predictions as pseudo-labels for un-annotated sub-structures. A key challenge in effectively combining partial annotation with self-training to reduce annotation cost is determining which sub-structures to select to label. To address this challenge, we adopt an error estimator to adaptively decide the partial selection ratio according to the current model’s capability. In evaluations spanning four structured prediction tasks, we show that our combination of partial annotation and self-training using an adaptive selection ratio reduces annotation cost over strong full annotation baselines under a fair comparison scheme that takes reading time into consideration.

Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training
Zhisong Zhang, Emma Strubell, Eduard Hovy Language Technologies Institute, Carnegie Mellon University zhi songzêcs. cmu edu, strubell@cmu edu, hovy@cmu. edu
Abstract
Adddel Presiittion cosffoo
Annotation for
Antiggious Parti
this work we propose a pragmatic method that reduces the annotation cost for structured label spaces using active learning. Our ap- proach leverages partial annotation, which re- duces labeling costs for structured outputs by selecting only the most informative sub- structures for annotation. We also utilize self- training incorporate the current model's tomatic predictions as pseudo-labels for un- annotated sub-structures. key challenge in effectively combining partial annotation with self-training to reduce annotation cost is deter- mining which sub- structures to select to label. To address this challenge, we adopt error estimator to adaptively decide the partial selec- tion ratio according the current model's capa- bility. In ovaluations spanning four structured prediction tasks, we show that our combination of partial annotation and self-training using adaptive selection ratio reduces annotation cost over strong full annotation baselines under fair comparison scheme that takes reading time into consideration.
He saw the with backpack
Figure Example partial annotations of dependency tree. Manual annotation is requested only for the uncer- tain sub-structures (red), whereas model predictions can be used annotate the highly-confident edges (bluc).
To mitigate such data bottlenecks, active learn- ing (AL), which allows the model to select the most informative data instances to annotate, has been demonstrated to achieve good model accu- racy while requiring fewer labels (Settles, 2009). When applying AL structured prediction, natural strategy is perform full annotation (FA) for the output structures, for example, annotating a full sequence of labels or a full syntax tree. Due to its simplicity, FA has been widely adopted in AL approaches for structured prediction tasks (Hwa, 2004; Settles and Craven, 2008; Shen et al., 2018). Nevertheless, structured object can usually be de- composed into smaller sub-structures having non- uniform difficulty and informativeness. For exam- plc, as shown in Figure in dependency tree, edges such as functional relations are relatively easy to learn, requiring fewer manual annotations, while prepositional attachment links may be more informative and thus more worthwhile to annotate.
1 Introduction
Structured prediction (Smith, 2011) 1) is fundamen- tal problem in NLP. wherein the label space con- sists of complex structured outputs with groups of interdependent variables. It covers a wide range of NLP tasks, including sequence labeling, syntactic parsing and information extraction (IE). Modern structured predictors are developed a data-driven way, by training statistical models with suitable annotated data. Recent developments in neural models and especially pre-trained language models (Peters al., 2018; Devlin et al., 2019; Liu et al., 2019; Yang et al.. 2019) have greatly improved sys- tem performance on these tasks. Nevertheless, the success of these models still relies on the availabil- ity of sufficient manually annotated data, which is often expensive and tome-consuming to obtain.
The non-uniform distribution of informative sub- structures naturally suggests AL with partial an- notation (PA), where the annotation budget can be preserved by only choosing portion of informative sub bbbstrucctrrs to annotate rather than laboriously labeling entire sentence structures. This idea has been explored in previous work, covering typical structured prediction tasks such as sequence label- ing (Shen et al., 2004; Marcheggiani and Artières, 2014 4; Chaudhary et al., 2019; Radmard et al., 2021)

---

Algorithm 1
AL Procedure.
cle. With evaluations on four benchmark tasks for structured prediction (named entity recognition, de- pendency parsing, event extraction, and relation extraction), we show that PA can obtain roughly the same benefits as FA with the same reading cost but less sub-structure labeling cost, leading to bet- ter data efficiency. We also demonstrate that the adaptive partial selection scheme and self-training play crucial and complementary roles.
Input: Sced dataset Lo, dov dataset D, unlaheled pool 14 to- tal budget t, batch selection size b, annotation strategy. Output: Final labeled datasct L. trained model M.
1: L + Lo # Initialize
2: while t > 0 do # Until out of budget
3: M < train(C, ()) Model training
4: S <- sentence--uery((.. (M. (1) # Sentence selection
5: if stralegy = "partial" then
6:
+- S D) # Decide adaptive ratio
7:
partial-annotat(SS.rr r) # Partial annotation
8: else
full-annotate(()) # Full annotation 10: u +- U-S: L LUS; t + t-b
9:
2 Method
11: MM < train(C, U) Final model training
return L, .M
2.1 AL for Structured Prediction
We adopt the conventional pool-based AL setting, which iteratively selects and annotates instances from an unlabeled pool. Please refer to Settles (2009) for the basics and details of AL; our main illustration focuses more specifically on applying AL to structured prediction.
and dependency parsing (Sassano and Kurohashi, 2010; Mirroshandel and Nasr, 2011; Flannery and Mori, 2015; Li et al., 2016). Our work follows this direction and investigates the central question in AL with PA of how to decide which sub-structures to select. Most previous work uses pre-defined fixed selection criterion, such threshold or ra- tio, which may be hard to decide in practice. In this work, we adopt a performance predictor to estimate the error rate of the queried instances and decide the ratio of partial selection accordingly. In this way, our approach can automatically and adaptively adjust the amount of partial selection throughout the AL process.
Algorithm illustrates the overall AL process. We focus on sentence-level tasks. In FA, each sen- tence is annotated with full structured object (for example, a label sequence a syntax tree). In PA. annotation granularity is at the sub-structure level (for example, a sub-sequence of labels or a partial tree). We adopt a two-step selection approach for all the strategics by first choosing a batch of sen- tences and then annotating within this batch. This approach is natural for FA since the original aim is to label full sentences, and it is also commonly adopted in previous PA work (Mirroshandel and Nasr, 2011; Flannery and Mori, 2015; Li et al., 2016). Moreover, this approach makes it casier control the reading context size for fair compar- isons of different strategies as described in $3.2. Without loss of generality, we take sequence labeling as an example and illustrate several key points in the AL process. Other tasks follow similar treatment, with details provided in Appendix A.
Another interesting question for AL is how to better leverage unlabeled data. In this work, we investigate a simple semi-supervised method, self-training (Yarowsky, 1995), which adopts the model's automatic predictions on the unlabeled data as extra training signals. Self-training natu- rally complements AL the typical pool- baseb setting where we assume access to a pool of unla- beled data (Settles, 2009). It is particularly compat- ible with PA-based AL since the un-selected sub- structures are typically also highly-confident under the current model and likely to be predicted cor- rectly without requiring additional annotation. We revisit this idea from previous work (Tomanek and Hahn, 2009; Majidi and Crane, 2013) and investi- gate its applicability with modern neural models and our adaptive partial selection approach.
Model. We adopt a standard BERT-based model with a CRF output layer for structured output modeling (Lafferty al., 2001), together with the BIO tagging scheme.
Querying Strategy. We utilize the query-by- uncertainty strategy with the margin-based based met- ric, which has been shown effective in AL for structured prediction (Marcheggiani and Artières, 2014; Li et al., 2016). Specifically, each token obtains an uncertainty score with the difference between the (marginal) probabilities of the most
We perform a comprchensive empirical investi- gation on the effectiveness of different AL strate- gies for typical structured prediction tasks. We perform fair comparisons that account for the hid- den cost of reading time by keeping the context size the same for all the strategies in each AL cy-

---

and second most likely label. We also tried sev- eral other strategies, such as least-confidence or max-entropy, but did not find obvious benefits." !
sub-structures to select. Typical solutions in previ- ous work include setting an uncertainty threshold (Tomanek and Hahn, 2009) or specifying a selec- tion ratio (Li et al., 2016). The threshold or ratio is usually pre-defined with a fixed hyper-parameter
Sentence selection. For both FA and PA, se- lecting batch of uncertain sentences is the first querying step. We use the number of total tokens to measure batch size since sentences may have variant lengths. The sentence-level uncertainty is obtained by averaging the token-level ones. This length normalization heuristic commonly adopted to avoid biases towards longer sentences (Hwa, 2004; Shen et al., 2018).
This fixed selecting scheme might not be an ideal one. First, it is usually hard to specify such fixed values in practice. If too many sub-structures are selected, there will be little difference between FA and PA, whereas if too few, the annotation amount is insufficient to train good models. Moreover, this scheme is not adaptive to the model. As the model is trained with more data throughout the AL pro- cess, the informative -tuusstuuttrrs become less dense as the model improves. Thus, the number of selected sub-structures should be adjusted accord- ingly. To mitigate these shortcomings, we develop dynamic strategy that can decide the selection in an automatic and adaptive way.
Token selection. In PA, subset of highly uncer- tain tokens is further chosen for annotation. One important question is how many tokens select. Instead of using a pre-defined fixed selection cri- terion, we develop an adaptive strategy to decide the amount, as will be described in $2.2.
We adopt the ratio-based strategy which enables straightforward control of the selected amount. Specifically, we rank the sub-structures by the un- certainty score and choose those scoring highest by the ratio. Our decision the selecting ratio is based on the hypothesis that a reasonable ratio should roughly correspond to the current model's error rate on all the candidates. The intuition is that incorrectly predicted sub- sub-structures are the most in- formative ones that can help to correct the model's mistakes.
Sequence labeling is usually adopted for tasks involving mention extraction, where annotations are over spans rather than in- dividual tokens. Previous work explores sub- sequence querying (Chaudhary al., 2019; Rad- mard al., 2021), which brings further complex- ities. Since we mainly explore tasks with short mention spans, we adopt a simple annotation pro- tocol: Labeling the full spans where any inside token is queried. Note that for annotation cost measurement, we also include the extra labeled tokens in addition to the queried ones.
Annotation.
Since the queried instances come from the unla- beled pool without annotations, the error rate not be directly obtained, requiring estimation. We adopt simple mne-dimeniionl logistic regression model for this purpose. The input to the model is the uncertainty score and the output is a binary pre- diction of whether its prediction is confidently cor- reet' or not. The estimator is trained using all the sub-structures together with their correctness on the development set' and then applied to the queried candidates. For each candidate sub-structure s. the estimator will give it correctness probability. We
Model learning. For FA, we adopt the standard log-likelihood as the training loss. For PA, we follow previous work (Scheffer et al., 2001; Wan- varie et al., 2011; Marcheggiani and Artières, 2014) and adopt marginalized likelihood to learn from incomplete annotations (Tsuboi et al., 2008; Greenberg et al., 2018). More details are pro- vided in Appendix C.
2.2
Adaptive Partial Selection
PA adopts second selection stage to choose highly uncertain sub-structures within the selected sen- tences. One crucial question here is how many
Directll using uncertainty is another option, but the main trouble is that the model is not well-calibrated. We also tried model calibration by temperature scaling (Guo et al., 2017). but did not find better results.
'Please refer to Appendix D.I for more results. Note that our main focus is on AL for structured prediction, where AL selection involves not only what instances to select (acqui- sition function), but also at what granularity to select and annotate. In contrast with most AL work that focuses on the first aspect (and classification tasks), we mainly investigate the second one and explore better partial selection strategies. Exploring more advanced acquisition functions is mostly or- thogonal to our main focus and is left to future work.
"We transform the input with logarithm, which leads to hetter estimation according to preliminary experiments.
that the arg max prediction matches the gold one and its margin is greater than O.5. Since neural models are usually over-confident. it hard decide confidence threshold. Nevertheless, we find 0.5 a reasonable value for ratio decision here.
"The specific criterion
We re-use the development set for the task model training.

---

estimate the overall error rate as one minus the aver- age correctness probability over all the candidates in the query set Q (all sub-structures in the selected sentences), and set the selection ratio r as this error rate:
ton et al., 2015). This choice is because we want avoid the potential negative influences of ambigu- ous predictions (mostly in completely unlabeled instances). One way to mitigate this is to set an uncertainty threshold and only utilize the highly- confident sub-structures. However, it is unclear how to set proper value, similar to the scenarios in query selection. Therefore, we take the model's full output predictions the training targets with- out further processing.
r r=
Σ p(correct = 1[s) SCQ
=1[s)
In this way, the selection ratio be set adap- tively according to the current model's capability. If the model is weak and makes many mistakes, we will have larger ratio which can lead more dense annotations and richer training signals. As the model is trained with more data and makes fewer errors, the ratio will be tuned down corre- spondingly to avoid wasting annotation budget on www.ectty-prreeicceeeccc
Specifically, our self-training objective function is the cross-entropy between the output distribu- tions predicted by the previous model m' before training and the current model m being trained:
C= Σ Pmc log Pm (u|2))
sub-structures. As we will see in later experiments, this adaptive scheme is suitable for AL ($3.3).
DEY
Several points are notable here: 1) The previous model is kept unchanged, and we can simply cache its predictions before training: 2) Over the instances that have partial annotations, the predictions should reflect these annotations by incorporating corre- sponding constraints at inference time: 3) For tasks with CRF based models, the output space y is usu- ally exponentially large and infeasible to explicitly enumerate; we utilize special algorithms (Wang et al., 2021) to deal with this, and more details are presented in Appendix C.
2.3 Self-training
Better utilization of unlabeled data is promising direction to further enhance model training in AL since unlabeled data are usually freely available from the unlabeled pool. In this work, we adopt self-training (Yarowsky, 1995) for this purpose.
The main idea of self-training is to enhance the model training with pseudo labels that are predicted by the current model on the unlabeled data. It has been shown effective for various NLP tasks (Yarowsky, 1995; McClosky et al., 2006; He et al., 2020; Du et al., 2021). For the training of AL mod- els, self-training can be seamlessly incorporated. For FA, the application of self-training is no dif- ferent than that in the conventional scenarios by applying the current model to all the un-annotated instances in the unlabeled pool. The more inter- esting case is on the partially annotated instances in the PA regime. The same motivation from the adaptive ratio scheme ($2.2) also applies here: We select the highly-uncertain sub-structures that are error-prone and the remaining un-selected parts are likely to be correctly predicted; therefore we can trust the predictions on the un-selected sub- structures and include them for training. One more enhancement to apply here is that we could further perform re-inference by incorporating the updated annotations over the selected sub-structures, which can enhance the predictions of un-annotated sub- structures through output dependencies.
Finally, we find it beneficial to include both the pseudo labels and the real annotated gold labels for the model training. With the gold data, the original training loss is adopted, while the KD objective is utilized with the pseudo labels. We simply mix these two types of data with a ratio of 1:1 in the training process, which we find works well.
3 Experiments
3.1 Main Settings
Tasks and data. Our experiments are conducted over four English tasks. The first two are named entity recognition (NER) and dependency parsing (DPAR), which are representative structured pre- diction tasks for predicting sequence and tree struc- tures. We adopt the CoNLL-2003 English dataset (Tjong Kim Sang and De Meulder, 2003) for NER and the English Web Treebank (EWT) from Uni- versal Dependencies v2.10 (Nivre al., 2020) for DPAR. Moreover, we explore two more complex IE tasks: Event extraction and relation extraction.
In this work, we adopt a soft version of self- training through knowledge distillation (KD; Hin-
"Our implementation is available at https: thub. com/ com/

---

Each task involves two pipelined sub-tasks: The first aims to extract the event trigger and/or entity mentions, and the second predicts links between these mentions as event arguments or entity rela- tions. We utilize the ACE05 dataset (Walker et al., 2006) for these IE tasks.
3.2 Comparison Scheme
Since FA and PA annotate at different granularities, need a common cost measurement compare their effectiveness properly. A reasonable metric is the number of the labeled sub-structures; for instance, the number of labeled tokens for sequence labeling or edges for dependency parsing. This metric is commonly adopted in previous PA work (Tomanek and Hahn, 2009; Flannery and Mori, 2015; Li et al., 2016; Radmard et al., 2021).
AL. For the AL procedure, we adopt settings fol- lowing conventional practices. We use the original training set as the unlabeled data pool to select in- stances. Unless otherwise noted, we set the AL batch size (for sentence selection) to 4K tokens, which roughly corresponds to 2% of the total pool size for most of the datasets we use. The initial seed training set and the development set are randomly sampled (with FAA using this batch size. Unless otherwise noted, we run 14 AL. cycles for each ex- periment. In each AL cycle, we re-train our model since we find incremental updating does not per- form well. Following most AL work, annotation is simulated by checking and assigning the labels from the original dataset. In FA, we annotate all the sub-structures for the selected sentences, In PA, we first decide the selection ratio and apply to the selected sentences. We further adopt a heuristic that selects the union of sentence-win wise uncertain sub-structures as well as global ones since both may contain informative sub-structures. Finally, all the presented results are averaged over five runs with different random seeds.
Nevertheless, evaluating only by sub-structures ignores a crucial hidden cost: The reading time of the contexts. For example, in sequence labeling with PA, although not every token in the sentence needs to be tagged, the annotator may still need to read the whole sentence to understand its meaning, Therefore, if performing comparisons only by the amount of annotated sub-structures, it will be unfair for the FA baseline because more contexts must be read to carry out PA.
In this work, we adopt simple two-facet com- parison scheme that considers both reading and labeling costs. We first control the reading cost by choosing the same size of contexts in the sen- tence selection step of each AL cycle (Line 4 in Algorithm 1). Then, we further compare by the sub-structure labeling cost, measured by the sub- structure annotation cost. If PA can roughly reach the FA performance with the same reading cost but fewer sub-structures annotated, it would be fair to say that PA can help reduce cost over FA. A better comparing scheme should evaluate against uni- fied estimation of the real annotation costs (Settles et al., 2008). This usually requires actual annota- tion exercises rather than simulations, which we leave to future work.
Model and training. For the models, we adopt standard architectures by stacking task-specific structured predictors over pre-trained RoBERTa Ta (Liu al., 2019) and the full models are fine-tuned at each training iteration. After obtaining new an- notations each AL cycle, we first train model based on all the available full or partial annota- tions. When using self-training, we further ap- ply this newly trained model to assign pseudo soft labels to all un-annotated instances and combine them with the existing annotations train another model. Compared to using the old model from the last AL cycle, this strategy can give more accu- rate pseudo labels since the newly updated model usually performs better by learning from more an- notations. For PA, pseudo soft labels are assigned to both un-selected sentences and the un- in-annotated sub-structures in the selected sentences.
3.3 NER and DPAR
Settings. We compare primarily three strategies: FA, PA, and a baseline where randomly selected sentences are fully annotated (Rand). We also in- clude a supervised result (Super.) which is obtained from a model trained with the full original training set. We measure reading cost by the total number of tokens in the selected sentences. For labeling cost, we further adopt metrics with practical consid- erations. In NER, lots of tokens, such functional words, can be easily judged as the 'O' (non-entity) tag. To avoid over-estimating the costs of such casy tokens for FA, we filter tokens by their part- of-speech (POS) tags and only count the ones that
"This heuristic will increase the actual selecting ratio, but it will only he slightly larger since there are large overlaps be- tween sentence-wise and global highly-ranked sub-structures.

---

NER Reading Cost
NER Labeling Cost
92
Rand
Rand
Rand IST
Rand IST
PA
FA+ST
PA
PA
88
PAIST
PAIST
Super
Super.
87
86
86-
4000
12000
20000
28000
56000
44000
52000
60000
2500
5000
7500
10000
12500
15000
Token Count
Sub-stmcctoee Crunt
DPAR Reading Cost
DPAR Labeling Cost
90
90
Rand+ST
RandeST
1.45%
88
1.A.5%
FA1ST
FA ST
86
BA+ST
PA+ST
Super.
1000
12000
20000
26000
36000
44000
52000
60000
25000 50000 75000 100000 125000 150000 175000 1750000000000 200000
Taken Count
Sub- Subb-stucture Couss
Figure 2: Comparisons according to reading and labeling cost. Each node indicates one AL cycle. For r-axis, reading cost (left) is measured by token numbers, while labeling cost (right) is task-specific ($3.3). NER is evaluated with labeled F1 scores CoNLL-2003, while DPAR is with LAS scores on UD-EWT. Results are averaged over five runs with different seeds, and shaded areas indicate standard deviations. The overall unlabeled pool contains around 200K tokens. Using AL. good performance can be obtained with less than 30% (60K) annotated.
are likely to be inside entity mention. 8 For PA. we still count every queried token. For the task of DPAR, similarly, different dependency links can have variant annotation difficulties. We utilize the surface distance between the head and modifier the dependency edge as the measure of label- ing cost, considering that the decisions for longer dependencies are usually harder.
Ratio Analysis. We further analyze the effec- tiveness of our adaptive ratio scheme with DPAR as the case study. We compare the adaptive scheme to schemes with fixed ratio r, and the results are shown in Figure For the fixed-ratio schemes, if the value is too small (such as 0.1), although its improving speed is the fastest at the beginning, its performance lags behind others with the same reading contexts due to fewer sub-structures anno- tated. If the value is too large (such as 0.5), it grows slowly, probably because too many uninformative sub-structures are annotated. The fixed scheme with 0.3 seems good choice; however, it is unclear how to find this sweet spot in realistic AL processes. The adaptive scheme provides a reason- able solution by automatically deciding the ratio according to the model performance.
Main Results. The main test results shown in Figure 2, where the patterns on both tasks are similar. First, AL brings clear improvements over the random baseline and can roughly reach the fully supervised performance with only a small portion of data annotated (around 18% for CoNLL-2003 and 30% for UD-EWT). Moreover, self training (+ST) is helpful for all the strategies, boosting per- formance without the need for extra manual annota- tions. Finally, with the help of self-training, the PA strategy can roughly match the performance of FFA with the same amount of reading cost (according to the left figures) while labeling fewer sub-structures (according to the right figures). This indicates that PA can help to further reduce annotation costs over the strong FA baselines.
Error and Uncertainty Analysis. We further alyze the error rates and uncertainties of the queried sub-structures. We still take DPAR as a case study and Figure shows the results along the AL cycles in PA mode. First, though adopting simple model. the performance predictor can give reasonable es- timations for the overall error rates. Morcover, by further breaking down the error rates into selected
"The POS tags are assigned by Stanza (Qi et al., 2020). For CoNII 2003, we filter by PROPN and ADI, which cover more than 95% of the entity tokens.
"err, we use self-training (+ST) for all the strategics.

---

DPAR Reading Cost
pred
crron(S)
margin
error
erro(())
margin(S)
0.8
0.8
LA5%
88
Adapive
0.6
0.6
Super
84
0.4
0.4
4000
12000
20000
280000
36000
44000
$2000
60000
Tokan Count
0.2
0.2
DPAR Labeling Cost
0.50
0.0
0.0
13
11
0.45
[+)
LAS5
88
0.40
Figure 4: Analyses of emor rates and uncertainties (mar- gins) of the DPAR sub-structures in the queried sen- tences along the AL cycles (rr-axis). Here, 'pred' de- notes the predicted error rate, "error' denotes the actual error rate and 'margin' denotes the uncertainty (margin) scores. For the suffixes, '(S)' indicates partially selected sub-structures, and '(N)' indicates non-selected ones. "Margin(N)' is omitted since is always close to
Adaptive
86
Supar
0.30
0.25
84
0
20000
40000
60000
S0000
100000
120000
Sub-sttracture Coure
Figure 3: Comparisons of different strategies to decide the partial ratio. The first three utilize fixed ratio r. while "Adaptive" adopts the dynamic scheme. The grey curve (corresponding to the right y-axis) denotes actual selection ratios with the adaptive scheme.
approach by initializing the model from the one trained with the source data and further fine-tuning with the target data. Since the target data size is small, we reduce the AL batch sizes for BTC and Tweebank to 2000 and 1000 tokens, respec- tively. The results for these experiments in Figure 5. In these experiments, we also include the no-transfer results, adopting the "FA+ST" but without model transfer. For NER, without transfer lcarning, the results are generally worse, especially in early AL stages, where there is small amount of annotated data to provide training signals. In these cases, knowledge learned from the source domain can provide extra information to boost the results. For DPAR, can see even larger benefits of using transfer learning; there are still clear gaps between transfer and no-transfer strategies when the former already reaches the supervised performance. These results indicate that the benefits of AL and transfer learning can be orthogonal, and combining them can lead to promising results.
(S) and non-selected (N) groups, we can see that the selected ones contain many errors, indicating the need for manual corrections. On the other hand, the error rates the non-selected sub-structures are much lower, verifying the effectiveness of us- ing model-predicted pseudo labels on them in self- training. Finally, the overall margin of the selected sentences keeps increasing towards 1, indicating that there are many non-ambiguous sub-structures even in highly-uncertain sentences. The margins of the selected sub-struee uub-structures much lower, sug- gesting that annotating them could provide more informative signals for model training.
shown
Drmain-transfer Experiments. We further in- vestigate a domain- transsfer scenario: in addition to unlabeled in-domain data, we assume abundant out- of-domain annotated data and perform AL on the target domain. We adopt tweet texts as the target domain, using Broad Twitter Corpus (BTC; Der- czynski et al., 2016 6) for NER and Tweebank (Liu et al., 2018) 8) for DPAR. We assume we have models trained from a richly-annotated source domain and continue performing AL on the target domain. The source domains are the datasets that we utilize in our main experiments: CoNLLO3 for NER and UD- EWT for DPAR. We adopt a simple model-transsee
3,4 Information Extraction
We further explore more complex IE tasks that in- volve multiple types of output. Specifically, we investigate event extraction and relation extraction. We adopt a classical pipelined approach, which splits the full task into two sub-tasks: the first per- forms mention extraction, while the second exam- ines mention pairs and predicts relations. While
Plcccse refer to Appendix for more task-specific details.

---

NER Reading Cost
NER Labeling Cost
82-
Rand
Rand
FA
P15
PA
78
FA
PA
PA
NoTtandder
Super.
74
2000
6000
10000
14000
18000
22000
26000
30000
2000
4000
6000
KO00
100000
2000
Token Gount
Subsstructore Count
DPAR Reading Cost
DPAR Labeling Cost
Rand
Rand ST
Rand-ST
1.45%
FA
FAIST
PA
PA-ST
NuTrandrr
NoToraft
Super
Super
78
1000
2000
5000
7000
9000
11000
13000
15000
10000
20000
30000
40000
50000
Token Count
Sub-smucuue Couant
Figure 5: AL results in DPAR). Notations are the same as in Figure 2, except that there is one more curve of "NoTransfer" denoting the setting where transfer learning applied (FA+ST alone).
settings (CoNLLO3 BTC for NER and UD-EWT Tweebank for
EventArgument Reading Cost
EventArguuent Labeling Cost
50
Rand
Rand1ST
Rand
611
BA
FA
FA-ST ST
FA+ST ST
PA
PA-ST
PA
Super.
Supci.
4000
12000
20000
28000
16000
44000
52000
60000
2000
4000
6000
8000
10000
12000
14000
Figure 6: Results event argument extraction on ACE05, Notations are the same Figure
previous work investigates multi-task AL with FA (Reichart et al., 2008; Zhu et al., 2020; Rotman and Reichart, 2022), this work is the first to explore PA this challenging setting.
relational sub-task depends the mentions tracted from the first sub-task, we utilize predicted mentions and view each feasible mention pair as a querying candidate. A special annotation protocol is adopted to deal with the incorrectly predicted mentions. For each queried relation, we first exam- ine its mentions and perform corrections if there are mention errors that can be fixed by matching the gold ones. If neither of the two mentions be corrected, discard this query.
We extend our PA scheme to this multi-task sce- nario with several modifications. First, for the sentence-selection stage, obtain sentence-we wise uncertainty score UNC(x) with a weighted combi- nation of the two sub-tasks' uncertainty scores:
UNC(2) = 3 UNC-Mention(z)
Finally, compensate for the influences of er- rors in mention extraction, adopt further heuris- tics of increasing the partial ratio by the estimated percentage queries with incorrect mentions, as well as including second annotation stage with queries over newly annotated mentions. Please refer to Appendix A.2 for more details.
+ - (3) UNC-Relation
Following Rotman and Reichart (2022), we set / to relatively large value (0.9), which is found to be helpful for the second relational sub-task.
Moreover, for partial selection, we separately se- lect sub-structures for the two sub-tasks according to the adaptive selection scheme. Since the second
We show the results event argument extraction in Figure 6, where the overall trends are similar to

---

those in NER and DPAR. Here, labeling cost is simply measured as the number of candidate ar- gument links. Overall, self-training is helpful for all AL strategies, indicating the benefits making better use of unlabeled data. If measured by the labeling cost, PA learns the fastest and costs only around half of the annotated arguments of FA to reach the supervised result. On the other hand, PA is also competitive concerning reading cost and can generally match the FA results in later AL stages. There is still gap between PA and FA in the earlier AL stages, which may be influenced by the errors produced by the first sub-task of mention extrac- tion. We leave further investigations on improving early AL stages to future work. The results for the relation extraction task share similar trends and are presented in Appendix D.2, together with the results of mention extraction.
NLP, such as sequence labeling (Settles and Craven, 2008; Shen et al., 2018), parsing (Hwa, 2004), se- mantic role labeling (Wang et al., 2017; Myers and Palmer, 2021) and machine translation (Haffari et al., 2009; Zeng et al., 2019). While most previ- ous work adopt FA, that is, annotating full struc- tured objects for the inputs, PA can help to further reduce the annotation cost. Typical examples of PA sub-structures include tokens and subsequences for tagging (Marcheggiani and Artières, 2014; Chaud- hary et al., 2019; Radmard et al., 2021), word wise head edges for dependency parsing (Flannery and Mori, 2015; Li et al., 2016) and mention links for coreference resolution (Li et al., 2020; Espeland et 2020).
Conclusion
In this work, investigate better AL strategies for structured prediction in NLP, adopting a perfor- mance estimator to automatically decide suitable ratios for partial sub-structure selection and utiliz- ing self-training to make better use the available unlabeled data pool. With comprehensive experi- ments on various tasks, we show that the combi- nation of PA and self self-training can be more data- efficient than strong full AL baselines.
4 Related Work
Self-training. Self-training is a commonly uti- lized cumi-supervised method to incorporate unla- beled data. It has been shown effective for a variety of NLP tasks, including word sense disambigua- tion (Yarowsky, 1995). parsing (McClosky al., 2006), named entity recognition (Meng et al., 2021; Huang et al., 2021), text generation (He et al., 2020; Mehta et al., 2022) as well as natural language understanding (Du et al., 2021). Moreover, self- training can be especially helpful for low-resource scenarios, such as in few-shot learning (Vu et al., 2021; Chen et al., 2021). Self-training has also been a commonly adopted strategy to enhance ac- tive learning (Tomanek and Hahn, 2009; Majidi and Crane, 2013; Yu al., 2022).
Limitations
This work has several limitations. First, the AL experiments in this work are based on simulations with existing annotations, following previous AL work. Our error estimator also requires a small development set and the proper setting of a hyper- parameter. Nevertheless, we tried our best to make the settings practical and the evaluation fair, espe- cially taking reading time into consideration. Sec- ond, in experiments, we mainly focus inves- tigating how much data is needed to reach the fully- supervised results and continue the AL cycles until this happens. In practice, may be interesting to more carefully examine the early AL stages, where most of the performance improvements happen. Fi- nally, for the IE tasks with multiple output types, we mainly focus on the second relational sub-task and adopt simple weighting setting to combine the uncertainties of the two sub-tasks. More explo- rations on the dynamic balancing of the two sub- tasks in pipelined models (Roth and Small, 2008) would be an interesting direction for future work.
PA. Learning from incomplete annotations has been well-explored for structured prediction. For CRF models, taking the marginal likelihood as the objective function has been one of the most uti- lized techniques (Tsuboi et al., 2008; Täckström et al., 2013; Yang and Vozila, 2014; Greenberg et al., 2018). There are also other methods to deal with incomplete annotations, such as adopt- ing local models (Neubig and Mori, 2010; Flan- nery et al., 2011), max-margin objective (Fernan- des and Brefeld, 201 1), learning with constraints (Ning et al., 2018, 2019; Mayhew et al., 2019) and negative sampling (Li et al., 2022).
AL for structured prediction. AL has been in- vestigated for various structured prediction tasks in

---

References
Eraldo R Fernandes and Ulf Brefeld. 2011. Learn- ing from partially annotated sequences. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 407-422. Springer.
Leonard E Baum, Ted Petrie, George Soules, and Nor- man Weiss. 1970. A maximization technique occur- ring in the statistical analysis of prohabilistic func- tions of markov chains. The annals of mathematical stafistics, 41(1):164-17 171.
Daniel Flannery, Yusuke Miayo, Graham Neubig, and Shinsuke Mori. 2011. Training dependency parsers from partially annotated corpora. In Proceedings of Sth International Joint Conference on Natural Lan- guage Processing, pages 776-784, Chiang Mai, Thai- land. Asian Federation of Natural Language Process- ing.
Aditi Chaudhary, Jiateng Xie, Zaid Sheikh, Graham Neubig, and Jaime Carbonell, 2019. A little annota- tion does lot of good: A study in bootstrapping low- resource named entity recognizers. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5164-5174, Hong Kong, China. Association for Computational Linguistics.
Daniel Flannery and Shinsuke Mori. 2015. Combin- ing active learning and partial annotation for domain adaptation of Japanese dependency parser. In Pro- ceedings of the 14th International Conference on Parsing Technologies, pages 11-19, Bilbao, Spain. Association for Computational Linguisties.
Yiming Chen, Yan Zhang, Chen Zhang, Grandee Lee, Ran Cheng, and Haizhou Li. 2021. Revisiting self- training for few-shot learning of language model. In Proceedings of the 2021 Conference on Empir- ical Methods in Natural Language Processing, pages 9125-9135, Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Nathan Greenberg. Trapit Bansal, Patrick Verga, and An- drew McCallum. 2018. Marginal likelihood training of BiLSTM- CRF for biomedical named entity recog- nition from disjoint label sets. In Proceedings of the 2018 Conference on Empirical Methods in Natural
Leon Derczynski, Kalina Bontcheva, and Ian Roberts. 2016. Broad Twitter corpus: A diverse named entity recognition resource. In Proceedings of COLING 2016, the 26th International Conference on Compu- tational Linguistics: Technical Papers, pages I 1169- 1179, Osaka, Japan. The COLING 2016 Organizing Committee.
Language Processing, pages 2824-2829, Brussels, Belgium. Association for Computational Linguistics.
Chuan Guo, Gcoff Pleiss, Yu Sun, and Kilian Q Wein- berger. 2017. On calibration of modern neural net- works. In International confe conference on machine learn- ing. pages 1321-1330. PMLR.
Gholamreza Haffari, Maxim Roy, and Anoop Sarkar, 2009. Active learning for statistical phrase-based machine translation. In Proceedings of Human Lan- guage Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 415-423, Boulder, Colorado. Association for Computational Linguis- tics.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long and Short Pupers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Junxian He, Jiatao Gu, Jiajun Shen, and Marc' Aurelio Ranzato. 2020. Revisiting self-training for neural sequence generation. In International Conference on Learning Representations.
Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency pars- ing. In ICLR.
Geoffrey Hinton, Oriol Vinyals, Jeff Dean, et al. 2015. Distilling the knowledge in neural network. arXiv preprint 503.00.1003.0531..22 2(7).
Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaud- hary, Onur Celebi, Michael Auli, Veselin Stoyanov, and Alexis Conneau. 2021. Self-training improves pre-training for natural language understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 4408-5418, Online. Association for Computa- tional Linguistics.
Neil Houlsby, Ferene Huszár, Zoubin Ghahramani, and Máté Lengyel. 2011. Bayesian active learning for classification and preference learning. arXiv preprint arXiv:
Jiaxin Huang, Chunyuan Li. Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng. Jianfeng Gao, and Jiawei Han. 2021. Few- shot named entity recognition: An empirical baseline study. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 10408-10423, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Vebjørn Espeland, Beatrice Alex, and Benjamin Bach. 2020. Enhanced labelling in active learning for coreference resolution. In Proceedings of the Third Workshop on Computational Models of Ref- erence, Anaphora and Coreference, pages 111-121, Barcelona, Spain (online). Association for Computa- tional Linguistics.

---

Rebecca Hwa. 2004, Sample selection for statistical parsing. Computational Linguistics, 30(3):253 276..
Proceedings of the 13th International Conference on Parsing Technologies (IWPT 2013), pages 18-105, Nara, Japan. Assocation for Computational Linguis- tics.
Terry Koo, Amir Globerson, Xavier Carreras, and Michael Collins. 2007. Structured prediction models via the matrix-tree theorem. In Proceedings of the 2007 Joint Conference on Empirical Methods in Nat- ural Language Processing and Computational Nat- ural Language Learning DEMNLP-CONLLY pages 141-150, Prague. Czech Republic. Association for Computational Linguistics.
Diego Marcheggiani and Thierry Artières. 2014. An experimental comparison of active learning strategies for partially labeled sequences. In Proceedings of the 2014 Conference on Empirical Methods Natu- ral Language Processing (EMNLP), pages 898 906. Doha, Qatar. Association for Computational Linguis- tics.
John D Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Proba- bilistic models for segmenting and labeling sequence data. Proceedings of the Eighteenth International Conference on Machine Learning, pages 282-289.
Stephen Mayhew, Snigdha Chaturvedi, Chen-Tse Tsai, and Dan Roth. 2019. Named entity recognition with partially annotated training data. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 645-655, Hong Kong. China. Association for Computational Linguis- tics.
Belinda 7. Li, Gabriel Stanovsky, and Luke Zettlemoyer. 2020, Active learning for coreference resolution us- ing discrete annotation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pages 8320-8331, Online, Association for Computational Linguistics.
David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Proceed- ings of the Human I Language Technology Conference of the NAACL, Main Conference. pages 152- 159. New York City, USA. Association for Computational Linguistics.
Yangming Li, Lemao Liu, and Shuming Shi. 2022. Re- thinking negative sampling for handling missing en- tity annotations. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 7188-7197, Dublin, Ireland. Association for Computational Lin- guistics.
Ryan McDonald and Giorgio Satta. 2007. On the complexity of non-projective data-driven dependency parsing. In Proceedings of the Tenth International Conference on Parsing Technologies, pages 121 121-1 Prague, Czech Republic. Association for Computa- tional Linguistics.
Zhenghua Li, Min Zhang, Yue Zhang, Zhanyi Liu, Wen- liang Chen, Hua Wu, and Haifeng Wang. 2016. Ac- tive learning for dependency parsing with partial an- notation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol- ume I: Long Papers). pages 344-354, Berlin, Ger- many. Association for Computational Linguistics.
Sanket Vaibhav Mehta, Jinfeng Rao, Yi Tay, Mihir Kale, Ankur Parikh, and Emma Strubell. 2022. Improving compositional generalization with self-training for data-to-text generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 4205- 4219, Dublin, Ireland. Association for Computational Linguistics.
Ying Lin, Heng Ji. Fei Huang, and Lingfei Wu. 2020. A joint neural model for information extraction with global features. In Proceedings of the 58th Annual Meeting of the Association for Computational Lin- guistics, pages 7999-8009, Online. Association for Computational Linguistics.
Yu Meng. Yunyi Zhang, Jiaxin Huang. Xuan Wang. Yu Zhang, Heng and Jiawei Han. 2021. Distantly- supervised named entity recognition with noise- robust learning and language model augmented self- training. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, pages 10367-10378, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yijia Liu, Yi Zhu, Wanxiang Che, Bing Qin, Nathan Schneider, and Noah A. Smith. 2018 Parsing tweets into Universal Dependencies. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume 1 (Long Pu- pers), pages 965-975, New Orleans, Louisiana. As- sociation for Computational Linguistics.
Seyed Abolghasem Mirroshandel and Alexis Nasr. 2011. 1. Active learning for dependency parsing using par- tially annotated sentences. In Proceedings of the 12th International Conference on Parsing Technolo- gies, pages 140-149, Dublin, Ireland. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1 16692.
Skatje Myers and Martha Palmer. 2021. Tuning deep active learning for semantic role labeling. In Pro- ceedings of the 14th International Conference on Computational Semantics (IWCS), pages 212-221,
Saeed Majidi and Gregory Crane. 2013. Active learning for dependency parsing by a committee of parsers. In

---

Groningen, The Netherlands (online). Association
861-869, Columbus, Ohio, Association for Compu- tational Linguistics.
for Computational Linguistics.
Graham Neubig and Shinsuke Mori. 2010. Word- based partial annotation for efficient corpus construction. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC' 10), Valletta, Malta. European Language Resources Asso- ciation (ELRA).
Dan Roth and Kevin Small. 2008. Active learning for pipeline models. In AAAI. pages 683-688.
Guy Rotman and Roi Reichart. 2022. Multi-task active learning for pre-train preitraind transformer- based models. Transactions of the Association for Computational Linguistics. 10: 1209-1228.
Qiang Ning. Hangfeng He, Chuchu Fan, and Dan Roth. 2019. Partial or complete, that's the question. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. Volume (Long and Short Papers). pages 2190-2200 -2000 Minneapolis, Minnesota. Association for Computa- tional Linguisties.
Manabu Sassano and Sadao Kurohashi. 2010. Using smaller constituents rather than sentences in active learning for Japanese dependency parsing. In Pro- ceedings of the 48th Annual Meeting of the Associa- tion for Computational Linguistics, pages 356-365, Uppsala, Sweden. Association for Computational Linguistics.
Tobias Scheffer, Christian Decomain. and Stefan Wro- bel. 2001. Active hidden markov models for infor- mation extraction. In International Symposium on Intelligent Data Analysis, pages 309-318. Springer.
Qiang Ning, Zhongzhi Yu, Chuchu Fan, and Dan Roth. 2018. Exploiting partially annotated data in temporal relation extraction. In Proceedings of the Seventh Joint Conference Lexical and Computational Se- mantics, pages 148-153, New Orleans, Louisiana. Association for Computational Linguistics.
Burr Settles. 2009. Active learning literature survey.
Burr Settles and Mark Craven. 2008. An analysis of active learning strategies for sequence labeling tasks. In Proceedings of the 2008 Conference on Empiri- cal Methods in Natural Language Processing, pages 1070-1079, Honolulu, Hawaii. Association for Com- putational Linguistics.
Joakim Nivre, Marie-Catherine de Marneffe, Filip Gin- ter, Jan Hajic. Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Lnguage Resources and Evaluation Conference, pages 4034-4043, Marseille, France. European Language Resourees Association.
Burr Settles, Mark Craven, and Lewis Friedland. 2008. Active learning with real anmotation costs. In Pro- ceedings of the NIPS workshop on cost-sensitive learning, volume 1.
Matthew E. Peters, Mark Neumann, Mohit lyyer, Matt Gardner, Christopher Clark. Kenton Lee, and Luke Zetlemoyer. 2018 Deep contextualized word repre- sentations. In Proceddiggs of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long Papers), pages 2227-2237, New Orleans, Louisiana. Association for Computa- tional Linguisties.
Dan Shen, Jie Zhang, Jian Su, Guodong Zhou, and Chew-Lim Tan. 2004. Multi-criteria-assed active learning for named entity recognition. In Proceed- ings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-004, pages 589- 596, Barcelona. Spain.
Yanyao Shen, Hyokun Yun, Zachary C. Lipton, Yakov Kronrod, and Animashree Anandkumar. 2018. Deep active learning for named entity recognition. In Inter- national Conference on Learning Representations.
Peng Qi. Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics: System Demonstrations, pages 101-108, Online. As- sociation for Computational Linguistics.
David A. Smith and Noah A. Smith. 2007. Probabilistic models of nonprojective dependency trees. Pro- ceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Com- putational Natural Language Learning (EMNLP- CoNLL), pages 132-140, Prague, Czech Republic. Association for Computational Linguistics.
Puria Radmard, Yassir Fathullah, and Aldo Lipani. 2021. Subsequence based deep active learning for named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pupers), pages 4310-4321, Online. Association for Computational Linguistics.
Noah A Smith, 2011. Linguistic structure prediction.
Synthesis lectures on human language technologies, 4(2):1-274.
Oscar Täckström, Dipanjan Das, Slav Petrov, Ryan Mc- Donald, and Joakim Nivre. 2013. Token and type constraints for cross-lingual part-of-speech tagging. Transactions of the Association for Computational Linguistics, 1:1-12.
Roi Reichart, Katrin Tomanek, Udo Hahn, and Ari Rap- poport. 2008 Multi-task active learning for linguistic annotations. In Proceedings of ACL-08: HLT, pages

---

Erik Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the CoNLL-2003 shared task: I sanguage-indeeende
Xinet: Generalized autoregressive pretraining for lan- guage understanding. Advances in neural informa- tion processing systems, 32.
named entity recognition. In Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142- 147.
David Yarowsky. 1995. Unsupervised word sense dis- ambiguation rivaling supervised methods. In 33rd Annual Meeting of the Association for Computa- tional Linguistics, pages 189-196, Cambridge, Mas- sachusetts, USA. Association for Computational Lin- guistics.
Katrin Tomanek and Udo Hahn. 2009. Semi-supervised active learning for sequence labeling. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Confer- ence on Natural Language Processing of the AFNLP. pages 1039-1047, Suntec, Singapore. Association for Computational Linguistics.
Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang, and Chao Zhang. 2022. AcTune: Uncertainty- based active self self-training for active fine-tuning of pretrained language models. In Proceedings of the 2022 Con- ference of the North American Chapter of the As- sociation for Computational Linguisties: Human Language Technologies, pages 1422-1436, Seattle, United States. Association for Computational Lin- guistics.
Yuta Tsuboi, Hisashi Kashima, Shinsuke Mori, Hiroki Oda, and Yuji Matsumoto. 2008. Training condi- tional random fields using incomplete annotations. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 897-904, Manchester, UK. Coling 2008 Organizing Committee.
Xiangkai Zeng, Sarthak Garg, Rajen Chatterjee, Ud- hyakumar Nallasamy, and Matthias Paulik. 2019. Empirical evaluation of active learning techniques for neural MT. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resoure NLP (DeepLo 2019), pages 84 93, Hong Kong, China. Association for Computational Linguisties.
Tu Vu, Minh-Thang Luong, Quoc Le, Grady Simon, and Mohit lyyer. 2021. STraTA: Self-training with task augmentation for better few-shot learning. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 5715- 5731, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Hua Zhu, Wu Ye, Sihan Luo, and Xidong Zhang. 2020. A multitask active learning framework for natural language understanding. In Proceedings of the 28th International Conference on Computational Linguis- rics, pages 4900-4914, 4,, Barcelona, Spain (Online). International Committee on Computational Linguis- tics.
Christopher Walker, Stephanie Strassel. Julie Medero, and Kazuaki Maeda. 2006. ACE 2005 multilingual training corpus. Linguistic Data Consortium, 57.
Chenguang Wang, Laura Chiticariu, and Yunyao Li. 2017. Active learning for black-box semantic role labeling with neural factors. In IJCAI.
Xinyu Wang, Yong Jiang, Zhaohui Yan, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, and Kewei Tu. 2021. Structural knowledge distillation: Tractably distilling information for struc- tured predictor. In Proceedings of the 59th Annual Meeting of the Association for Computational Lin- guistics and the 1 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 550-564, Online. Association for Computational Linguistics.
Dittaya Wanvarie, Hiroya Takamura, and Manabu Oku- mura. 2011. Active learning with subsequence sam- pling strategy for sequence labeling tasks. Informa- tion and Media Technologies, 6(3):680-700.
Fan Yang and Paul Vozila. 2014. Semi- supervised Chi- nese word segmentation using partial-label learning with conditional random fields. In Proceedings of the 2014 Conference Empirical Methods in Natural Language Processing (EMNLP),, pages 90-98, Doha Qatar. Association for Computational Linguistics.
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Car- bonell, Russ R Salakhutdinov, and Quoc V Le. 2019.

---

A More Details of Task Settings
layers as the shared encoding module whose out- put representations are used for both sub-tasks. Each sub-task further adopts a private encoder that is initialized with the remaining pre-traines layers and is trained with task-specific signals. We simply set N to 6, while the results are gen- erally not sensitive to this hyper-parametee Final task-specific predictors are further stacked upon the corresponding private encoders. We adopt CRF layer for mention extraction and a pairwise local predictor with a biaffine scorer for relation or argument extraction.
A.1 DPAR
Model. Similar to NER, we utilize BERT- based module to provide contextualized repre- sentations. We further stack a standard first- order non-projective graph-based parsing mod- ule based on biaffine scorer (Dozat and Man- ning. 2017). The marginals for each token's s head decision can be feasibly calculated by the Matrix- -ree algorithm (Koo et al., 2007; Smith and Smith, 2007; McDonald and Satta, 2007).
Query and Selection. Following previous works (Flamnery and Mori, 2015; Li et al., 2016), we view DPAR as head-word finding problem and regard each token and its head decision as the sub-structure unit. In this case, the query and selection for DPAR are almost identical to the NER task because of this token-wise decision scheme. Therefore, the same AL strategies in NER can be adopted here.
Sentence selection. For an unlabeled sentence, there is an uncertainty score for each sub-task. For mentions, the uncertainty is the average mar- gin as in the NER task. For relations, we find that averaging uncertainties over all mention pairs has a bias towards sentences with fewer mentions. To mitigate such bias, we first aggregate un- certainty score for each mention by taking the maximum score within all the relations that link and then averaging over all the mentions for sentence- scores. Finally, the scores of the two sub-tasks are lincarly combined to form the sentence-leve uncertainty.
Annotation. In DPAR, there are no special span- based annotations as in NER; thus, we simply annotate in a word-based scheme.
Model learning. Similar to NER, we adopt the ligeleiiloood of the gold parse tree as the train- ing loss in FA and marginalized likelihood PA (Li et al., 2016).
Partial selection. For PA selection, the two sub- tasks are handled separately according to the adaptive ratio scheme. We further adopt two heuristics for the relational task compensate for errors in the mention extraction. First, since there can be over-predicted mentions that lead to discarded relation queries, we adjust the PA ratio by estimating how many candidate rela- tions contain such errors in the mentions. We again train a logistic regression model to pre- dict whether a token is NIL (or 'O' in the BIO scheme, meaning not contained inside any gold mentions) based on its NIL probability. Then for each candidate relation, we calculate the proba- bility that any token within its mentions is NIL. By averaging this probability of all the candi- dates, we obtain a rough estimation of the per- centage of problematic relations, which we call it a. Finally the PA selection ratio is adjusted by: Tadjust a- -pophllmm + (1-aa) -Trigin- Here, Torigin denotes the original selection ratio obtained from the adaptive scheme, and rproblem denotes the se- lection ratio of problematic relations, which we conservatively set to I. Secondly, since there can also be under predictee mentions, we add a
A.2 IE
Tasks. We tackle event extraction (EE) and rela- tion extraction (RE) using a two-step pipelined approach. The first step aims to extract entity mentions for RF, and entity mentions and event triggers for EE We adopt sequence labeling for mention extractions in the NER task. Based on the mentions extracted in the first step, the second step examines each feasible candidate mention pair (entity pair for RE and event-entity pair for EE) and decides the relation (entity relation for RE and event argument relation for EE) for them. Since event argument links can be regarded as relations between event triggers and entities, for simplicity we will use the relational sub-task to refer to both relation and argument extraction.
Model. We adopt a multi-task model similar to the one utilized in (Rotman and Reichart, 2022). With a pre-trdined encoder, we take the first N

---

Data
Split
#Sent.
#Token
#Event
#Entity
#Argument
#Relation
train
14.0K
203.6K
23.5K
CONLLO3
dev
3.3K
51.4K
5.9K
test
3.5K
46.4K
5.6K
train
12.5K
204.6K
UD-EWT
dev
2.0K
25.1K
test
2.1K
25.1K
train
14.4K
215.2K
3.7K
38.0K
5.7K
6.2K
ACE05
dev
2.5K
34.5K
0,5K
6.0K
0.7K
0.8K
test
4.0K
61.5K
I.IK
10.8K
1.7K
1.7K
Table 1: Data statistics.
second stage of querying and annotation in each AL cycle based on the annotated mentions in the first stage. This extra stage only selects rela- tions that involve the newly added or corrected mentions. We simply reuse the selection ratio determined from the first stage and apply it to each sentence that contains such mentions. In this way, the second stage is lightweight and only requires relatively cheap re-inference for each queried sentence individually.
and mention tasks. We follow Lin al. (2020) for the pre-processing' of the ACE dataset. For the IE tasks on ACE, we find that the conventional test set contains only newswire documents while the training set consists of various genres (such as from conversation and web). Such mismatches between the AL pool and the final testing set are nontriv- ial to handle with the classical AL protocol, and thus randomly re-split the ACE dataset (with ratio of 7:1:2 for training, dev, and sets, respec- tively). Table I shows data statistics. For each AL experiment, we take the original training set as the unlabeled pool, down-sample a dev set from the original dev set, and evaluate on the full test set.
Annotation. The annotation of the mentions is the same as in the NER task, while for the annota- tion of relational queries, their mentions are first examined and corrected if needed, as explained in $3.4. We measure the labeling cost by the fi- nal annotated items; thus, these extra examined mentioned will also be properly counted.
More Settings. All of our models are based the contextual- ized encoder. We further fine-tune it with the task-specific decoder in all the experiments. The number of model parameters is roughly 124M for single-output tasks and around 186M for multi-task IE tasks. For other mostly follow common practices. Adam is utilized for optimization, with initial learning rate of le- 5 for NER and 2e-5 for DPAR and IE. The learning rate linearly decayed 10% of the initial value throughout the training process. The models are tuned for 10K steps with a batch size of roughly 5 112 tokens. We evaluate the model on the dev set every IK steps to choose the best checkpoint. The experiments are run with one 2080Ti GPU. The training of one AL cycle usually takes only or two hours, and the full simulation of one AL run can be finished within one day. We adopt stan- dard evaluation metrics for the tasks: labeled Fi score for NER, labeled attachment score (LAS) for DPAR, labeled argument and relation Fl score for event arguments and relations (Lin et al., 2020).
the pre-trained RoBERTasse
Model learning. For the mention extraction sub- task, the training objective is the same as in NER. For the relational sub-task, we simply adopt local pairwise model with the standard cross- entropy loss. Since the relation model is local, special treatment needed for PA.
settings, we
B Data Statistics and More Settings
Data. Our main experiments are conducted us- ing the CoNLL-2003 English dataser (Tjong Kim Sang and De Meulder, 2003) for NER, the English Web Treebank (EWT) from Universal De- pendencies v2.10 (Nivre et al., 2020) for DPAR, and English portion of ACE20055 (Walker al., 2006) for IE. We utilize Stanza (Qi et al., 2020) to assign POS tags for cost measurement NER
"https://ww/ wwtp://www..l....uu.aeeeeeree ner/ "httus:/uuiveeralloeeddddee
wantwerpen. wwwnterreen.e//cool222222 be/con112803/
Phttpss//ccttalo..dd. Idc. wtt..mmmeeee/LD222887700
/ntttp:/blendder cs. wwwiniiiiis....... edu/ swware// oneie/
"https: github. cifuub.ioo io/stanza/

---

C Details of Algorithms
labeling (Baum al., 1970) Matrix-tree for non-projective dependency parsing (Koo al., 2007; Smith and Smith, 2007; McDonald and Satta, 2007).
this section, we provide more details of the al- gorithms for CRF-styled models (Lafferty et al., 2001). For an input instance x (for example, a sen- tence), the model assigns a globally normalized probability to each possible output structured ob- ject y (for example, tag sequence parse tree) in the target space y:
Learning with incomplete annotations. Follow- ing previous works (Tsuboi et al., 2008; Li et al., 2016; Greenberg et al., 2018), for the instances with incomplete annotations, we utilize the loga- rithm of the marginal likelihood as the learning objective:
p(ylz) = exp s(ylx)
EVe exp (1
exp EEy s sf(z)
C = -log
Σ p(21z)
EVEy E, Eeey" s (f)
BEDc
exp s(y(z)
Here, s(ylzr) denotes the un-normalized raw scores assigned to y. which is further factorized into the sum of the sub-structure scores (f(lr)) 16 In plain likelihood training for CRE, we take the negative log- peg-rrbbabilty as the training objective:
log
Σ
Zyey
exp ((yla)
VEVo
log Σ
exps(y)x)) + log (x)
DE
Here, V' denotes the constrained set of the output objects that agree with the existing partial annota- tions. In this objective function, the second item is exactly the same as in standard CRF, while the first one can be calculated a modified way (Tsuboi et al., 2008).
L = log p(y(z))
- (ylxx) ) + log Σ exp (t/(()
VEY
For brevity, in the remaining, we use log 2 (x) ()) denote the second term of the log partition function. For model training, we need to calculate the gradi- ents of the model parameters to the loss function. The first item is easy to deal with since it only in- volves one structured object, while log (z) needs some reorganization according the factorization:
to
Knowledge distillation. As described in the main context, we adopt the knowledge distillation objective for self-training with soft labels. For brevity, we denote the probabilities from the last model as p' (y(()r) and keep using p(y[æ) to denote the ones from the current model. Following Wang et al. (2021), the loss can be calculated by:
Ve log 2 Evey exp s(y syyl://))oos(///l Expesexps(y""ll plu (()ll))Vos((1l) s'c
C == = Σ p' (2l logp(u)>)
Σ
VEY
= Σ p' (((1(z)s(()11)) + log 2(x)
Σ plvll0) Σ Vos(s(1x)
NEY
VEY
f'ey'
Σ v'(()x)) Σ >(((x)) + -og ((x)
- P
Σ p(/1))
VEY
N'Ev'
/'E
= Σ ((()))
Σ
p'(/1x)) log 2(x)
The last step is obtained by swapping the order of the two summations, and finally, the problem is re- duced to calculating each sub-structure's 's marginal probability Evey, p()/ Here, D'yr denotes all the output structured objects that contain the sub-structure f", and the marginals can usually be calculated by classical structured prediction al- gorithms such as forward-backwatt for sequence
The loss function is broken down into two items whose gradients be obtained by calculating marginals according to the last model or the current one, respectively.
In our implementation. we adopt a simple method to en- force the constraints by adding negative-inlimite to the scores of the impossible labels. In this case, the structures that vio- lates the constraints will have score of negative-infinite (and a probability of zero) and will thus he excluded.
16 "Such as unary and pairwise scores for sequence laheling or token-wise edge scores for dependency parsing.

---

DAPR Reading Cost
DAPR Labeling Cost
RAND
RAND
DA-M
TA-
LAS%
PA
LASS
PA
88
PA-LC
PA-E
PA-E
PA-B
Super.
Super.
84
4000
12000
20000
25000
56000
44000
52000
60000
25000
50000
75000 100000
125000 150000 75000 200000 Sub-stncctoee Couet
Token Count
Figure 7: Comparisons of different acquisition functions partial annotation: "-M" denotes margin-based, "-LC." denotes least-confident, "-E" denotes entropy based, and "-B" indicates BALD.
DAPR Reading Cost
tions, such as those considering representativeness.
D.2 IE Experiments
90
RAND
In this section, we present more results of the IE experiments. First, Figure 9 shows the mention extraction results for the event extraction task. The overall trends are very similar to those in NER: PA can obtain similar results to FA with the same reading texts and less mention labeling cost, In Figure 10, we show the results for mention and relation extractions. the ACE dataset, relations are very sparsely annotated, and around 97% of the entities are linked with less or equal to two rela- tions. Considering this fact, we measure the cost of FA relation extraction by two times the anno- tated entities, while PA still counts the number of the queried relations. The relation results sim- ilar the patterns for event argument extraction, showing the benefits of selecting and annotating with partial sub-structures. Notice that in some the mention extraction results, there seems to be less obvious differences between the AL strategies over the random baseline. This may be due to our focus on the second sub-task for relations (or event arguments), directly reflected by its high weight (3) in calculating sentence uncertainty. It will be interesting to explore better ways to enhance both sub-tasks, probably with an adaptive combination scheme (Roth and Small, 2008).
FAJLC
FA-B
86
Siper.
4000
12000
20000
28000
3600U
44000
52000
60000
Token Count
Figure 8: Comparisons of different acquisition functions for full annotation. Notations of the methods are the same as in Figure 7.
D Extra Results
D.I Using Different Acquisition Functions
In the main experiments, our acquisition function is based on margin-based uncertainty, that is, se- lecting the instances that have the largest marginal differences between the most and second-most -oot con- fident predictions. Here, we compare it with var- ious other acquisition functions, including least- confident (-LC), max-entropy (-E) and BALD (- B) (Houlsby al., 2011). We take DPAR as the studying case and the results for full annotation and partial annotation are shown in Figure 8 and 7, respectively. Generally, there are no large differ- ences between the adopted querying methods and the margin-based method can obtain the overall best results. Notice that regardless of the adopted acquisition function, we can see the effectiveness of our partial selection scheme: it requires lower labeling cost than full annotation reach the upper bound. This shows that our method is extensible different AI. querying methods and it will be in- teresting to explore the combination of our method with more complex and advanced acquisition func-

---

Mentionnatity Reading Cost
MentioEattit Labeling Cost
86
RandiST
56-
RandAST
$315
FA+ST
FA+ST
844
BAIST
PAIST
Super
Supee
80
4000
12000
20000
250000
36000
441130
$2000
60000
5000
100011
1.5000
200100
25000
30000
Token Count
Sub-structure Count
Trigger Reading
Mention Trigger Labeling
70
Rand
70
Rand
RandiST
RindeST
BAIST
1-1%
FAIST
60
PAYST
PA+ST
Super
Super
55
4000
12000
20000
26000
36000
44000
52000
60000
5000
10000
15000
20000
25000
30000
Token
Count
Figure 9: Results (FI) of mention extraction (entities and event triggers) for the event extraction task on ACE05 (argument results are shown in Figure 6).
MentionEntiity Reading Cost
MentionEntiiy Labeling Cost
Rand Rand+ST
Rand
Rand-ST
FA
FA
84
BA
PA
Super
Super.
60
20000
26000
3nd00
44000
52000
60000
10000
20000
Taken
Relation Reading Cost
Relation Labeling Cost
60
Rand
Rand
55
FA
FA
FA+ST
BA
PA
45
40
Super
Super.
4000
12000
20000
28000
36000
52000
5000
15000
Token Count
Sub-structure Couns
Figure 10: Results (F1) of the extraction entity mentions and relations for the relation extraction task on ACE05.


Reviews:
[{"review_id": "FcTNCZDpRF", "reviewer": "Reviewer_8arV", "summary": "", "questions": "See above.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the helpful comments!\n\nValidation Set: Our method does not require a large validation set to obtain reasonable estimations, since the estimator is a simple logistic regression model and does not need much data to fit. Notice that we utilize the same dev set for both the training of our main model and the estimator; in this way, there is no extra cost.\n\nEstimated error rate: Our motivation for using error rate is that: if a sub-structure is wrongly predicted, we should pick it for annotation; whereas if the model prediction is correct, then there is no need to select. In this way, the ideal ratio should be the error rate if we want all the sub-structures' labels to be correct. Uncertainty can be an indicator, but the main trouble is that the model is not calibrated. Most of the predictions seem to be confident (for example, margin > 0.9) according to the model's output, but many can be wrong. This will also lead to a small selection ratio and as Figures 3 and 9 show, the model will be under-trained if there are not enough training instances.\n\nSelf-training: We agree that using self-training is not a novel idea as we have already acknowledged (for example, Line 98). Nevertheless, we think its application in the context of partial AL is very natural and motivates our adaptive ratio scheme (annotate the wrongly predicted ones and trust the correct ones). We think the combination of AL and self-training with this adaptive ratio scheme is a novel idea.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is clear and easy to read.\n- The training of a calibration model, from uncertainty to correctness, to adaptively determine the selection ratio is interesting.\n- I appreciate the inclusion of reading cost and labeling cost to evaluate the methods from more angles.", "reasons_to_reject": "- The main contribution of the work is the proposed method that trains the error estimator to determine the selection ratio for sub-structure. While the idea is interesting, it is quite straightforward. My main concern for this method is the requirement for a validation set in order to train the error-estimator. It is not clear how the performance is sensitive to the size of the validation set.\n- There also doesn't seem to have a strong motivation for the proposed method. For instance, why do we need to use the estimated error rate, instead of simply using the current uncertainty? Is there any theoretical reason for this or empirical evidence to support?\n- The self-training scheme is more of a direct application here. It's great that the authors explore this direction in the paper but the contribution here is relatively limited.\n", "questions_to_authors": "See above.", "paper_topic_and_main_contributions": "The paper tackles active learning with partial annotation problem for structured prediction. It introduces a method that adaptively determines the selection ratio for sub-structures to annotate. Coupled with self-training, the proposed method provides improvement over existing full-annotation baselines.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Vk81FbHxUq", "reviewer": "Reviewer_gpGM", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the helpful comments!\n\nAdaptive Partial Selection: We think that our analyses in Figures 3 and 9 provide empirical evidence for the effectiveness of our scheme: the adaptive selection scheme is the overall best performing. Moreover, another way to view our motivation is that the ideal goal of the annotation is to obtain fully annotated instances, in this way, if the prediction is incorrect, we should select and correct the corresponding sub-structure.\n\nDerivative of the partition function with restricted sub-structures: In the case that some of the sub-structures are restricted to pre-determined labels, we can still adopt dynamic-programming styled algorithms as in the normal scenario, but force the labels for the restricted sub-structures. Actually, in our implementation, we adopt a simple method to enforce this by adding -inf to the scores of the impossible labels. In this case, the Y that violates the constraints will have a score of -inf (a probability of 0) and will thus be excluded. We will include more details on this in later versions.\n\nPlease let us know if there are any other places where you think there are needs of more explanations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The motivation of this work is very clear and meaningful.\n2. The proposed Adaptive Partial Selection provides the idea for addressing real annotation difficulties.\n3. The writing is well-organised.\n\n", "reasons_to_reject": "Fail to provide solid support for the design details. For example, in Adaptive Partial Selection, the assumption is based on the intuition that incorrectly predicted sub-structures are the most informative ones that can help to correct the model’s mistakes. Also, for optimizing self training objective, regarding the derivative of partition function with respect to \\theta, is there any explanation for the transformation to the restricted sub-structure?\n\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper provides a framework to combining partial annotation with self-training to reduce annotation cost for structured prediction using active learning. \n\nThis paper addressed the annotation for structured prediction and to reduce annotation cost, within the framework of active learning, the developed Adaptive Partial Selection could be more flexible for diverse structures. ", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "64beOkRkt3", "reviewer": "Reviewer_GDw5", "summary": "", "questions": "A) What is a “query set” $\\mathcal{Q}$? Is that the same as “sentence selection” $\\mathcal{S}$ in Algorithm 1? If yes, please agree on the nomenclature and stick to it. If not, please elaborate on what is a query set since it is only mentioned once on line 242.\n\nB) Line 257 – “We further adopt a heuristic that selects the union of sentence-wise uncertain sub-structures as well as global ones since both may contain informative sub-structures.” – Could you please elaborate on the heuristic used?\n\nC) Line 403 - “We first control the reading cost by choosing the same size of contexts in the sentence selection step of each AL cycle (Line 4 in Algorithm 1).” - Does that mean the same sentence length? How do you ensure that different methods choose the same size of context?\n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the helpful comments!\n\nTitle and domain-transfer experiments: Thanks for the suggestions! We will modify the title and move more domain-transfer results to the main content.\n\nHyperparameter and dev set: Our motivation to add a confidence threshold is that we do not want to trust uncertainly correct ones (maybe correct by chance). This threshold is set to an intuitive value of 0.5 throughout our experiments without any tuning. For the usage of a dev set, as also discussed in our response to Reviewer 8arV, our method does not need a very large dev set since we only need to fit a simple logistic regression model. Moreover, we reuse the dev set that is used in the task model training; therefore, there will be no extra cost. We will mention these in the Limitations section. \n\nQuestion A: S denotes the selected sentences and Q denotes all the sub-structures in S. We will add more details for this.\n\nQuestion B: Assuming we have a selection ratio, for example, 0.3, we can rank all the sub-structures globally and take the first 30 percent, or, we can rank each sentence's sub-structures individually and select 30 percent within each sentence. We find that combining both leads to more robust results and thus simply take the union.\n\nQuestion C: This is measured by the total number of tokens (total sentence length), as denoted by the x-axis of all the “Reading Cost” figures. For example, in Figure 2, we select sentences whose total length is around 4K in each AL cycle. In the sentence selection stage, we rank all the sentences according to their sentence-level uncertainty (as discussed starting from Line 159) and select until hitting the 4K budget.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Well-written and content-dense paper.\n\n- This relatively simple yet effective and practical idea could be helpful in real-world scenarios.\n\n- Usually, when authors emphasize their comparison is “fair,” it immediately sounds fishy since all comparisons should be fair; otherwise, what is the point of comparing? However, the comparison authors propose does seem more suitable than the standard comparison that only considers labeling cost, and it is harsher to their proposed PA method than to the FA baselines.\n\n- I liked the Ratio Analysis and the Error and Uncertainty Analysis paragraphs. It is always essential to do a deeper analysis of the newly proposed methods and to get an idea of how they behave and why they work.\n", "reasons_to_reject": "- “Data-efficient Active Learning” in the title sounds like a pleonasm. Is it not the point of AL to be data-efficient? I believe the selling point of the paper is the adaptive ratio rather than data efficiency, and it would be better to emphasize it in the title.\n\n- The Authors often criticize other PA approaches for reliance on hyperparameters, but their method also requires a hyperparameter. In footnote 3 the authors mention they use a confidence threshold, which is a hyperparameter, and that they set it to 0.5. The proposed method also requires a development set to train the logistic regression to estimate the error rate. Still, having a development set in practice is not often feasible. It would be good to reiterate these limitations in the Limitations section. \n\n- I suggest commenting a bit on the results of Domain-transfer Experiments in the main part of the paper when given an extra page for the CR version.\n", "questions_to_authors": "A) What is a “query set” $\\mathcal{Q}$? Is that the same as “sentence selection” $\\mathcal{S}$ in Algorithm 1? If yes, please agree on the nomenclature and stick to it. If not, please elaborate on what is a query set since it is only mentioned once on line 242.\n\nB) Line 257 – “We further adopt a heuristic that selects the union of sentence-wise uncertain sub-structures as well as global ones since both may contain informative sub-structures.” – Could you please elaborate on the heuristic used?\n\nC) Line 403 - “We first control the reading cost by choosing the same size of contexts in the sentence selection step of each AL cycle (Line 4 in Algorithm 1).” - Does that mean the same sentence length? How do you ensure that different methods choose the same size of context?\n", "paper_topic_and_main_contributions": "This paper tackles the problem of active learning (AL) for structured prediction in NLP. The authors focus on developing a partial annotation (PA) schema with an adaptive selection ratio and combining it with self-training. The authors estimate the model’s error rate using logistic regression and then calculate the expected error rate, which they use as the ratio for a given AL step. To compare their method with the full annotation baseline (FA), the authors developed a fair evaluation considering both reading and labeling costs. While the proposed method is competitive with its FA counterpart when accounting for reading cost, it outperforms the baselines when considering labeling cost. The proposed method is thoroughly studied on several structured prediction tasks. The behavior of different aspects of the proposed method, e.g., change in adaptive ratio and the error rates throughout the AL cycle, are analyzed in detail.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper suggests partial annotation coupled with self-training for active learning of structured prediction. The paper is clearly-written, and ideas are interesting, addressing issues in real annotations. However, in the term of self-training, the contribution is limited. In addition, the design details should be provided with more explainations.
Decision: Accept-Findings