SlowGuess/NAACL_2022_OCR huggingface dataset:
ACTUNE: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models
Yue Yu $^{1}$ , Lingkai Kong $^{1}$ , Jieyu Zhang $^{2}$ , Rongzhi Zhang $^{1}$ , Chao Zhang $^{1}$ $^{1}$ Georgia Institute of Technology, Atlanta, GA  $^{2}$ University of Washington, Seattle, WA {yueyu, lkkong, rongzhi.zhang, chaozhang}  $@$  gatech.edu, jieyuz2@cs.washington.edu
While pre-trained language model (PLM) fine-tuning has achieved strong performance in many NLP tasks, the fine-tuning stage can be still demanding in labeled data. Recent works have resorted to active fine-tuning to improve the label efficiency of PLM fine-tuning, but none of them investigate the potential of unlabeled data. We propose ACTUNE, a new framework that leverages unlabeled data to improve the label efficiency of active PLM fine-tuning. ACTUNE switches between data annotation and model self-training based on uncertainty: it selects high-uncertainty unlabeled samples for active annotation and low-uncertainty ones for model self-training. Under this framework, we design (1) a region-aware sampling strategy that reduces redundancy when actively querying for annotations and (2) a momentum-based memory bank that dynamically aggregates the model's pseudo labels to suppress label noise in self-training. Experiments on 6 text classification datasets show that ACTUNE outperforms the strongest active learning and self-training baselines and improves the label efficiency of PLM fine-tuning by  $56.2\%$  on average. Our implementation is available at https://github.com/yueyu1030/actune.
Fine-tuning pre-trained language models (PLMs) has achieved much success in natural language processing (NLP) (Devlin et al., 2019; Liu et al., 2019; Brown et al., 2020). One benefit of PLM fine-tuning is the promising performance it offers when consuming only a few labeled data (Bansal et al., 2020; Gao et al., 2021). However, there are still significant gaps between few-shot and fully-supervised PLM fine-tuning in many tasks. Besides, the performance of few-shot PLM fine-tuning can be sensitive to different sets of training data (Bragg et al., 2021). Therefore, there is a
crucial need for approaches that make PLM finetuning more label-efficient and robust to selection of training data, especially for applications where labeled data are scarce and expensive to obtain.
Towards this goal, researchers have recently reported to active fine-tuning of PLMs and achieved comparable performance to fully-supervised methods with much less annotated samples (Ein-Dore et al., 2020; Margatina et al., 2021a,b; Yuan et al., 2020). Nevertheless, they usually neglect unlabeled data, which can be useful for improving label efficiency for PLM fine-tuning (Du et al., 2021). To incorporate unlabeled data into active learning, efforts have been made in the semi-supervised active learning literature (Wang et al., 2016; Rottmann et al., 2018; Simeoni et al., 2020). However, the query strategies proposed in these works can return highly redundant samples due to limited representation power, resulting in suboptimal label efficiency. Moreover, they usually rely on pseudo-labeling to utilize unlabeled data, which requires greater (yet often absent) care to denoise the pseudo labels, otherwise the errors could accumulate and hurt model performance. This issue can be even more severe for PLMs, as the fine-tuning process is often sensitive to different weight initialization and data orderings (Dodge et al., 2020). Thus, it still remains open and challenging to design robust and label efficient method for active PLM fine-tuning.
To tackle the above challenges, we propose ACTUNE, a new method that improves the label efficiency and robustness of active PLM fine-tuning. Based on the estimated model uncertainty, ACTUNE tightly couples active learning with self-training in each learning round: (1) when the average uncertainty of a region is low, we trust the model's predictions and select its most certain predictions within the region for self-training; (2) when the average uncertainty of a region is high, indicating inadequate observations for parameter learning, we actively annotate its most uncertain
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1422 - 1436 July 10-15, 2022 ©2022 Association for Computational Linguistics
samples within the region to improve model performance. Different from existing AL methods that only leverage uncertainty for querying labels, our uncertainty-driven self-training paradigm gradually leverages unlabeled data with low uncertainty via self-training, while reducing the chance of error propagation triggered by highly-uncertain mislabeled data.
To further boost model performance for AC-TUNE, we design two techniques to improve the query strategy and suppress label noise, namely region-aware sampling (RS) and momentum-based memory bank (MMB). Inspired by the fact that existing uncertainty-based AL methods often end up with choosing uncertain yet repetitive data (Eindor et al., 2020; Margatina et al., 2021b), we design the region-aware sampling technique to promote both diversity and representativeness by leveraging the representation power of PLMs. Specifically, we first estimate the uncertainties of the unlabeled data with PLMs, then cluster the data using their PLM representations and weigh the data by the corresponding uncertainty. Such a clustering scheme partitions the embedding space into small sub-regions with an emphasis on highly-uncertain samples. Finally, by sampling over multiple high-uncertainty regions, our strategy selects data with high uncertainty and low redundancy.
To rectify the erroneous pseudo labels derived by self-training, we design a simple but effective way to select low-uncertainty data for self-training. Our method is motivated by the fact that fine-tuning PLMs suffer from instability issues — different initializations and data orders can lead to large variance in model performance (Dodge et al., 2020; Zhang et al., 2020b; Mosbach et al., 2021). However, previous approaches only select pseudo-labeled data based on the prediction of the current round and are thus less reliable. In contrast, we maintain a dynamic memory bank to save the predictions of unlabeled samples for later use. We propose a momentum updating method to dynamically aggregate the predictions from preceding rounds (Laine and Aila, 2016) and select low-uncertainty samples based on aggregated prediction. As a result, only the samples with high prediction confidence over multiple rounds will be used for self-training, which mitigates the issue of label noise. We highlight that our active self-training approach is an efficient substitution to existing AL methods, requiring little extra computational cost.
Our key contributions are: (1) an active self-training paradigm ACTUNE that integrates self-training and active learning to minimize the labeling cost for fine-tuning PLMs; (2) a region-aware querying strategy to enforce both the informativeness and the diversity of queried samples during AL; (3) a simple and effective momentum-based method to leverage the predictions in preceding rounds to alleviate the label noise in self-training and (4) experiments on 6 benchmarks demonstrating ACTUNE improves the label efficiency over existing self-training and active learning baselines by  $56.2\%$ .
2 Uncertainty-aware Active Self-training
We study active fine-tuning of pre-trained language models for text classification, formulated as follows: Given a small number of labeled samples  $\mathcal{X}_l = \{(x_i,y_i)\}_{i = 1}^L$  and unlabeled samples  $\mathcal{X}_u = \{x_j\}_{j = 1}^U (|\mathcal{X}_l| \ll |\mathcal{X}_u|)$ , we aim to fine-tune a pre-trained language model  $f(\pmb{x};\theta): \mathcal{X} \to \mathcal{Y}$  in an interactive way: we perform active self-training for  $T$  rounds with the total labeling budget  $b$ . In each round, we aim to query  $B = b / T$  samples denoted as  $\mathcal{B}$  from  $\mathcal{X}_u$  to fine-tune a pre-trained language model  $f(\pmb{x};\theta)$  with both  $\mathcal{X}_l, \mathcal{B}$  and  $\mathcal{X}_u$  to maximize the performance on downstream text classification tasks. Here  $\mathcal{X} = \mathcal{X}_l \cup \mathcal{X}_u$  denotes all samples, and  $\mathcal{Y} = \{1,2,\dots,C\}$  is the label set where  $C$  is the number of classes.
2.2 Overview of ACTUNE Framework
We now present our active self-training paradigm ACTUNE underpinned by estimated uncertainty. We begin the active self-training loop by finetuning a BERT  $f(\theta^{(0)})$  on the initial labeled data  $\mathcal{X}_L$ . Formally, we solve the following optimization problem
$$
\min  _ {\theta} \frac {1}{| \mathcal {X} _ {L} |} \sum_ {\left(\boldsymbol {x} _ {i}, y _ {i}\right) \in \mathcal {X} _ {L}} \ell_ {\mathrm {C E}} \left(f \left(\boldsymbol {x} _ {i}; \theta^ {(0)}\right), y _ {i}\right). \tag {1}
$$
In round  $t$ $(1\leq t\leq T)$  of active self-training, we first calculate the uncertainty score based on a given function  $a_{i}^{(t)} = a(\pmb{x}_{i},\theta^{(t)})^{1}$  for all  $\pmb{x}_i\in \mathcal{X}_u$ . Then, we query labeled samples and generate pseudolabels for unlabeled data  $\mathcal{X}_u$  simultaneously to facilitate self-training. For each sample  $\pmb{x}_i$ , the pseudo-label  $\widetilde{y}$  is calculated based on the current
<sup>1</sup>Note that ACTUNE is agnostic to the way uncertainty score  $a_{i}^{(t)}$  is computed.
Algorithm 1: Training Procedures of ACTUNE.
Input: Initial labeled samples  $\mathcal{X}_l$ ; Unlabeled samples  $\mathcal{X}_u$ ; Pre-trained LM  $f(\cdot; \theta)$ , number of active self-training rounds  $T$ .
// Fine-tune the LM with initial labeled data.
1. Calculate  $\theta^{(0)}$  based on Eq. (1).
2. Initialize the memory bank  $g(\pmb{x}; \theta^t)$  based on the current prediction.
// Conduct active self-training with all data.
for  $t = 1,2,\dots ,T$  do
1. Run weighted K-Means (Eq. (3), (4)) until convergence.
2. Select sample set  $\mathcal{Q}^{(t)}$  for AL and  $S^{(t)}$  for self-training from  $\mathcal{X}_u$  based on Eq. (11) or (13).
3. Augment the labeled set  $\mathcal{X}_L = \mathcal{X}_L\cup \mathcal{Q}^{(t)}$
4. Obtain  $\theta^{(t)}$  by finetuning  $f(\cdot ;\theta^t)$  with  $\mathcal{L}_{\mathrm{ST}}$  (Eq. (14)) using AdamW.
5. Update memory bank  $g(\pmb {x};\theta^t)$  with Eq. (10) or (12).
Output: The final fine-tuned model  $f(\cdot ;\theta^T)$ .
model's output:
$$
\tilde {y} = \underset {j \in \mathcal {Y}} {\operatorname {a r g m a x}} \left[ f (\boldsymbol {x}; \theta^ {(t)}) \right] _ {j}, \tag {2}
$$
where  $f(\pmb {x};\theta^{(t)})\in \mathbb{R}^C$  is a probability simplex and  $[f(\pmb {x};\theta^{(t)})]_j$  is the  $j$  -th entry. The procedure of ACTUNE is summarized in Algorithm 1.
2.3 Region-aware Sampling for Active Learning on High-uncertainty Data
After obtaining the uncertainty for unlabeled data, we aim to query annotation for high-uncertainty samples. However, directly sampling the most uncertain samples gives suboptimal results as it tends to query repetitive data (Ein-Dor et al., 2020) that represent the overall data distribution poorly.
To tackle this issue, we propose region-aware sampling to capture both uncertainty and diversity during active self-training. Specifically, in the  $t$ -th round, we first conduct the weighted K-means clustering (Huang et al., 2005), which weights samples based on their uncertainty. Denote by  $K$  the number of clusters and  $\pmb{v}_i^{(t)} = \mathrm{BERT}(\pmb{x}_i)$  the representation of  $\pmb{x}_i$  from the penultimate layer of BERT. The weighted K-means process first initializes the center of each cluster  $\pmb{\mu}_i(1 \leq i \leq K)$  via K-Means++ (Arthur and Vassilvitskii, 2007). Then, it jointly updates the centroid of each cluster and assigns each sample to cluster  $c_i$  as
$$
c _ {i} ^ {(t)} = \underset {k = 1, \dots , K} {\operatorname {a r g m i n}} \| \boldsymbol {v} _ {i} - \boldsymbol {\mu} _ {k} \| ^ {2}, \tag {3}
$$
$$
\boldsymbol {\mu} _ {k} ^ {(t)} = \frac {\sum_ {\boldsymbol {x} _ {i} \in \mathcal {C} _ {k} ^ {(t)}} a \left(\boldsymbol {x} _ {i} , \theta^ {(t)}\right) \cdot \boldsymbol {v} _ {i} ^ {(t)}}{\sum_ {\boldsymbol {x} \in \mathcal {C} _ {k} ^ {(t)}} a \left(\boldsymbol {x} _ {i} , \theta^ {(t)}\right)} \tag {4}
$$
where  $\mathcal{C}_k^{(t)} = \{\pmb{x}_i^{(t)}|c_i^{(t)} = k\} (k = 1,\dots ,K)$  stands for the  $k$  -th cluster. The above two steps in Eq. (3), (4) are repeated until convergence. Compared with vanilla K-Means method, the weighting scheme increases the density of the samples with high uncertainty, thus enabling the K-Means methods to discover clusters with high uncertainty. After obtaining  $K$  regions with the corresponding data  $\mathcal{C}_k^{(t)}$  , we calculate the uncertainty of each region as
$$
u _ {k} ^ {(t)} = U \left(\mathcal {C} _ {k} ^ {(t)}\right) + \beta I \left(\mathcal {C} _ {k} ^ {(t)}\right) \tag {5}
$$
$$
U \left(\mathcal {C} _ {k} ^ {(t)}\right) = \frac {1}{\left| \mathcal {C} _ {k} ^ {(t)} \right|} \sum_ {\boldsymbol {x} _ {i} \in \mathcal {C} _ {k} ^ {(t)}} a \left(\boldsymbol {x} _ {i}, \theta^ {(t)}\right), \tag {6}
$$
is the average uncertainty of samples and
$$
I \left(\mathcal {C} _ {k} ^ {(t)}\right) = - \sum_ {j \in C} f _ {j} ^ {(t)} \log f _ {j} ^ {(t)} \tag {7}
$$
is the inter-class diversity within cluster  $k$  and  $f_{j}^{(t)} = \frac{\sum_{i}\mathbb{1}\{\widetilde{y}_{i} = j\}}{|\mathcal{C}_{k}^{(t)}|}$  is the frequency of class  $j$  on cluster  $k$ . Notably, the term  $U(\mathcal{C}_k^{(t)})$  assigns higher score for clusters with more uncertain samples, and  $I(\mathcal{C}_k^{(t)})$  grants higher scores for clusters containing samples with more diverse predicted classes from pseudo labels since such clusters would be closer to the decision boundary.
Then, we rank the clusters in an ascending order in  $u_{k}^{(t)}$ . A high score indicates high uncertainty of the model in these regions, and we need to actively annotate the member instances to reduce uncertainty and improve the model's performance. We adopt a hierarchical sampling strategy: we first select the  $M$  clusters with the highest uncertainty, and then sample  $b' = \lfloor \frac{B}{M} \rfloor$  data with the highest uncertainty to form the batch  $\mathcal{Q}^{(t)}$ .
$$
\mathcal{K}_{a}^{(t)} = \operatorname *{top - M}_{k\in \{1,\ldots ,K\}}u_{k}^{(t)},
$$
$$
\mathcal {Q} ^ {(t)} = \bigcup_ {k \in \mathcal {K} _ {a} ^ {(t)}} \mathcal {C} _ {a, k} ^ {(t)} \text {w h e r e} \mathcal {C} _ {a, k} ^ {(t)} = \underset {\boldsymbol {x} _ {i} \in \mathcal {C} _ {k} ^ {(t)}} {\operatorname {T o p - b}} ^ {\prime} a (\boldsymbol {x} _ {i}, \theta^ {(t)}). \tag {8}
$$
We remark that such a hierarchical sampling strategy queries most uncertain samples from different regions, thus the uncertainty and diversity of queried samples can be both achieved.
2If the number of samples in the  $i$ -th cluster  $\mathcal{C}_i$  is smaller than  $b^{\prime}$ , then we sample all the data within  $\mathcal{C}_i$  and increase the budget for the  $(i + 1)$ -th cluster by  $b^{\prime} - |\mathcal{C}_i|$ .
2.4 Self-training over Confident Samples from Low-uncertainty Regions
For self-training, we aim to select unlabeled samples which are most likely to have been correctly classified by the current model. This requires the sample to have low uncertainty. Therefore, we select the top  $k$  samples from the  $M$  lowest uncertainty regions to form the acquired batch  $S^{(t)}$ :
$$
\begin{array}{l} \mathcal {C} _ {s} ^ {(t)} = \bigcup_ {k \in \mathcal {K} _ {s} ^ {(t)}} \mathcal {C} _ {k} ^ {(t)} \text {w h e r e} \mathcal {K} _ {s} ^ {(t)} = \operatorname {b o t t o m - M} u _ {k} ^ {(t)}, \tag {9} \\ \mathcal {S} ^ {(t)} = \underset {\boldsymbol {x} _ {i} \in \mathcal {C} _ {s} ^ {(t)}} {\text {b o t t o m - k}} a (\boldsymbol {x} _ {i}, \theta^ {(t)}). \\ \end{array}
$$
Momentum-based Memory Bank for Self-training. As PLMs are sensitive to the stochasticity involved in fine-tuning, the model suffers from the instability issue — different weight initialization and data orders may result in different predictions on the same dataset (Dodge et al., 2020). Additionally, if the model gives inconsistent predictions in different rounds for a specific sample, then it is potentially uncertain about the sample, and adding it to the training set may harm the active self-training process. For example, for a two-class classification problem, suppose we obtain  $f(\pmb{x}; \theta^{(t-1)}) = [0.65, 0.35]$  for sample  $\pmb{x}$  the round  $(t-1)$  and  $f(\pmb{x}; \theta^{(t)}) = [0.05, 0.95]$  for the round  $t$ . Although the model is quite 'confident' on the class of  $\pmb{x}$  when we only consider the result of the round  $t$ , it gives contradictory predictions over these two consecutive rounds, which indicates that the model is actually uncertain to which class  $\pmb{x}$  belongs.
To effectively mitigate the noise and stabilize the active self-training process, we maintain a dynamic memory bank to save the results from previous rounds, and use momentum update (He et al., 2020; Laine and Aila, 2016) to aggregate the results from both the previous and current rounds. Then, during active self-training, we will select samples with the highest aggregated score. In this way, only those samples that the model is certain about over all previous rounds will be selected for self-training. We design two variants for the memory bank, namely prediction-based and value-based aggregation.
Prediction based Momentum Update. We adopt an exponential moving average approach to aggregate the prediction  $g(\pmb{x}; \theta^{(t)})$  on round  $t$  as
$$
g (\boldsymbol {x}; \theta^ {(t)}) = m _ {t} f (\boldsymbol {x}; \theta^ {(t)}) + (1 - m _ {t}) g (\boldsymbol {x}; \theta^ {(t - 1)}), \tag {10}
$$
where  $m_t = (1 - \frac{t}{T})m_L + \frac{t}{T} m_H$  ( $0 < m_L \leq m_H \leq 1$ ) is a momentum coefficient. We gradually increase the weight for models on later rounds,
since they are trained with more labeled data thus being able to provide more reliable predictions. Then, we calculate the uncertainty based on  $g(\pmb{x};\theta^{(t)})$  and rewrite Eq. (9) and (2) as
$$
\begin{array}{l} \boldsymbol {S} ^ {(t)} = \underset {\boldsymbol {x} _ {i} \in \mathcal {C} _ {s} ^ {(t)}} {\text {b o t t o m - k}} a \left(\boldsymbol {x} _ {i}, g (\boldsymbol {x}; \theta^ {(t)}), \theta^ {(t)}\right) \tag {11} \\ \widetilde {y} = \operatorname * {a r g m a x} _ {j \in \mathcal {Y}} \left[ g (\boldsymbol {x}; \theta^ {(t)}) \right] _ {j}, \\ \end{array}
$$
Value-based Momentum Update. For methods that do not directly use prediction for uncertainty estimation, we aggregate the uncertainty value as
$$
g (\boldsymbol {x}; \theta^ {(t)}) = m _ {t} a (\boldsymbol {x}; \theta^ {(t)}) + (1 - m _ {t}) g (\boldsymbol {x}; \theta^ {(t - 1)}). \tag {12}
$$
Then, we use Eq. (12) to sample low-uncertainty data for self-training as<sup>3</sup>
$$
\begin{array}{l} \mathcal {S} ^ {(t)} = \underset {\boldsymbol {x} _ {i} \in \mathcal {C} _ {s} ^ {(t)}} {\text {b o t t o m - k}} g (\boldsymbol {x} _ {i}, \theta^ {(t)}), \\ \widetilde {y} = \underset {j \in \mathcal {Y}} {\operatorname {a r g m a x}} \left[ f (\boldsymbol {x}; \theta^ {(t)}) \right] _ {j}. \tag {13} \\ \end{array}
$$
By aggregating the prediction results over previous rounds, we filter the sample with inconsistent predictions to suppress noisy labels.
2.5 Model Learning and Update
After obtaining both the labeled data and pseudolabeled data, we fine-tune a new pre-trained BERT model  $\theta^{(t + 1)}$  on them. Although we only include low-uncertainty samples during self-training, it is difficult to eliminate all the wrong pseudo-labels, and such mislabeled samples can still hurt model performance. To suppress such label noise, we use a threshold-based strategy to further remove noisy labels by selecting samples that agree with the corresponding pseudo labels. The loss objective of optimizing  $\theta^{(t + 1)}$  is
$$
\begin{array}{l} \mathcal {L} _ {\mathrm {S T}} = \frac {1}{| \mathcal {L} ^ {(t)} |} \sum_ {\boldsymbol {x} _ {i} \in \mathcal {L} ^ {(t)}} \ell_ {\mathrm {C E}} \left(f \left(\boldsymbol {x} _ {i}; \theta^ {(t + 1)}\right), y _ {i}\right) \tag {14} \\ + \frac {\lambda}{| \mathcal {S} ^ {(t)} |} \sum_ {\widetilde {\boldsymbol {x}} _ {i} \in \mathcal {S} ^ {(t)}} \omega_ {i} \ell_ {\mathrm {C E}} \left(f (\widetilde {\boldsymbol {x}} _ {i}; \theta^ {(t + 1)}), \widetilde {y} _ {i}\right), \\ \end{array}
$$
where  $\mathcal{L}^{(t)} = \mathcal{X}_L\cup \mathcal{Q}^{(t)}$  is the labeled set,  $\lambda$  is a hyper-parameter balancing the weight between clean and pseudo labels, and  $\omega_{i} = \mathbb{1}\{\left[f(\pmb{x}_{i};\theta^{(t + 1)})\right]_{\widetilde{y}_{i}} > \gamma \}$  stands for the thresholding function.
Complexity Analysis. The running time of AC-TUNE is mainly consisted of two parts: the inference time  $O(|\mathcal{X}_u|)$  and the time for K-Means clustering  $O(dK|\mathcal{X}_u|)$ , where  $d$  is the dimension of the BERT feature  $\pmb{v}$ . For self-training, the size
3Other choices such as soft pseudo label (Xie et al., 2020; Liang et al., 2020) is also applicable.
Dataset	Label Type	# Class	# Train	# Dev	#Test
SST-2	Sentiment	2	60.6k	0.8k	1.8k
AG News	News Topic	4	119k	1k	7.6k
Pubmed	Medical Abstract	5	180k	1k	30.1k
DBPedia	Wikipedia Topic	14	280k	1k	70k
TREC	Question	6	5.0k	0.5k	0.5k
Chemprot	Medical Abstract	10	12.8k	0.5k	1.6k
Table 1: Dataset Statistics. For DBPedia, we randomly sample  $20\mathrm{k}$  sample from each class due to the limited computational resource.
of the memory bank  $g(\pmb{x};\theta)$  is proportional to  $|\mathcal{X}_u|$ , while the extra computation of maintaining this dictionary is ignorable since we do not inference over the unlabeled data for multiple times in each round as BALD (Gal et al., 2017) does. The running time of ACTUNE will be shown in section C.
Tasks and Datasets. In our main experiments, we use 4 datasets, including SST-2 (Socher et al., 2013) for sentiment analysis, AGNews (Zhang et al., 2015) for news topic classification, PubmedRCT (Dernoncourt and Lee, 2017) for medical abstract classification, and DBPedia (Zhang et al., 2015) for wikipedia topic classification. For weakly-supervised text classification, we choose 2 datasets, namely TREC (Li and Roth, 2002) and Chemprot (Krallinger et al., 2017) from the WRENCH benchmark (Zhang et al., 2021) for evaluation. The statistics are shown in Table 1.
Active Learning Setups. Following (Yuan et al., 2020), we set the number of rounds  $T = 10$ , the overall budget for all datasets  $b = 1000$  and the initial size of the labeled  $|\mathcal{X}_l|$  is set to 100. In each AL round, we sample a batch of 100 samples from the unlabeled set  $\mathcal{X}_u$  and query their labels. Since large development sets are impractical in low-resource settings (Kann et al., 2019), we keep the size of development set as 1000, which is the same as the labeling budget<sup>4</sup>. For weakly-supervised text classification, since the datasets are much smaller, we keep the labeling budget and the size of development set to  $b = 500$ .
Implementation Details. We choose RoBERTabase (Liu et al., 2019) from the HuggingFace codebase (Wolf et al., 2020) as the backbone for AcTUNE and all baselines except for Pubmed and Chemprot, where we use SciBERT (Beltagy et al., 2019), a BERT model pre-trained on scientific cor
pora. In each round, we train from scratch to avoid overfitting the data collected in earlier rounds as observed by Hu et al. (2019). More details are in Appendix B.
Hyperparameters. The hyperparameters setting is in Appendix B.5. In the  $t$ -th round of active self-training, we increase the number of pseudolabeled samples by  $k$ , where  $k$  is 500 for TREC and Chemprot, 3000 for SST-2 and Pubmed-RCT, and 5000 for others. For the momentum factor, we tune  $m_{L}$  from  $[0.6, 0.7, 0.8]$  and  $m_{H}$  from  $[0.8, 0.9, 1.0]$  and report the best  $\{m_{L}, m_{H}\}$  based on the performance of the development set.