Improving Single-round Active Adaptation: A Prediction Variability Perspective
Authors: Xiaoyang Wang
Yibo Jacky Zhang
Olawale Elijah Salaudeen
Mingyuan Wu
Hongpeng Guo
Chaoyang He
Klara Nahrstedt
Sanmi Koyejo
Venue: Accepted by TMLR ()

Abstract: Machine learning models trained with offline data often suffer from distribution shifts in online environments and require fast adaptation to online data. The high volume of online data further stimulates the study of active adaptation approaches that achieve competitive adaptation performance by selectively annotating only 5%-10% of online data and using it to continuously train a model. Despite the reduction in data annotation cost, many prior active adaptations assume a multi-round data annotation procedure during continuous training, which hinders timely adaptation. In this work, we study a single-round active adaptation problem with a minimum data annotation turnaround time but require the selected subset of data samples to help the entire continuous training procedure until convergence. In our theoretical analysis, we find that the prediction variability of each data sample throughout the training is crucial, in addition to the conventional data diversity. The prediction variability measures how much the prediction could possibly change during the continuous training procedure. To this end, we introduce a novel approach called feature-norm scaled gradient embedding (FORGE), which incorporates prediction variability and improves the single-round active adaptation performance when combined with standard data selection strategies (e.g., k-center greedy). In addition, we provide efficient implementations to construct our FORGE embedding analytically without explicitly backpropagating gradients. Empirical results further demonstrate that our approach consistently outperforms the random selection baseline by up to 1.26% for various vision and language tasks while other competitors often underperform the random selection baseline.

Published in Transactions on Machine Learning Research (10/2025)
Improving Single-round Active Adaptation: A Prediction Variability Perspective
Xiaoyang Wang' University of Nlinois Urbana- Champaign
rw28@illiniss.o edu
Yibo Jacky Zhang Stanford University
yiboz@stanford. edu
Olawale Salaudeen Massachusetts Institute of Technology
olwwale@mit. edu
Mingyuan Wu
mw84.0illmooe edu
University of Blinois Prebna-CCamppnn
Hongpeng Guo
hg5 edu
University of Nlinois Urbuna- Channaag
Chaoyang He TensorOpera, Inc.
ch@leensorpperr.com
Klara Nahrstedt
klara@illinois. edu
University of Blinois Urbana- Chammpignnn
Sanmi Koyejo Stanford University
aammi@@cs.stafford.eee
Reviewed on OpenReview:
Abstract
Machine learning models trained with offline data often suffer from distribution shifts in online environments and require fast adaptation to online data. The high volume of online data further stimulates the study of active adaptation approaches that achieve competitive adaptation performance by selectively annotating only 5%-10% of online data and using to continuously train a model. Despite the reduction data annotation cost, many prior active adaptations assume multi-round data annotation procedure during continuous training, which hinders timely adaptation. In this work, study single-round active adaptation problem with minimum data annotation turnaround time but require the selected subset of data samples to help the entire continuous training procedure until convergence. In our theoretical analysis, we find that the prediction variability each data sample throughout the
Work partially performed while TensorOpera (formerly FedMI. and done before XW started Amazon.

---

Published in Transactions on Machine Learning Research (10/2025)
training is crucial, in addition to the conventional data diversity. The prediction variability measures how much the prediction could possibly change during the continuous training procedure. To this end, we introduce a novel approach called feature-norm scaled gradient embedding (FORGE), which incorporates prediction variability and improves the single- round active adaptation performance when combined with standard data selection stralegies (e.g., k-center greedy). In addition, we provide efficient implementations to construct our FORGE embedding analytically without explicitly backpropagating gradients. Empirical results further demonstrate that our approach consistently outperforms the random selection baseline by up to 1.26% for various vision and language tasks while other competitors often underperform the random selection baseline.
1 Introduction
The data in production environments can shift. away from what is used for training the model. For example, vision model inside a camera of a surveillance or autonomous driving system may see new images that are different from its offline training images every day. A fraud detection model may process emails and transactions from new users adversaries that try to penetrate novel attacks to bypass the detection. A language model in a chat application also receives new and time-based questions over time. Such ubiquitous distribut ion shifts are among the major causes of performance degradation in machine learning models Huyen 2022p. The consequence of performance degradation caused by distribution shifts can be quite severe: a failure of vision model may cause traffic accidents, penetrating a novel fraud can cause financial loss to company, and language model generating incorrect answers to new questions can raise concerns in mission-critical applications such medical diagnostic and healthcare.
One of the most effective ways to address distribution shift problems is contimuously training a model using online data (Huyen, 2022). Given a large amount of online data and the subsequent annotat ion cost, a few recent works (Prabhu et al. 2021 Xie et al. 2023 explore active adaptation and show that carefully curating a subset of ouline data is an effective means to achieve superior adaptation performance while siguificantly reducing the data annotation cost. Despite lowering the annotation cost by a factor of ten or twenty, existing active adaptation methods assume multi-round data annotation procedures. Under these multi-round settings, we issue multiple queries that soquentially request labels for selected data samples from data annotators and continuously train a model between adjacent queries for a few iterations. However, these sequentially issued queries may hinder timely adaptation, causing user dissatisfaction.
Language models interact with users and continuously improve their performance based on user feedback (e.g., preference annotations). Repeatedly asking users to annotate their preferences without delivering a customized experience can hurt their satisfaction. Besides, for a conventional supervised learning problem (e.g., fraud detection), issuing K mini-batches of queries can incur ~ K times more turnaround time than selecting data samples once and issuing single batch of queries because modern data annotation systems (e.g., MTurk (Crowston] 2012)) are optimized for throughout (Haas et al. 2015 Difallah et al. 2015). The throughput-oriented systems are good at handling a large batch instead of multiple mini-batches. To this end, we propose a single-round active adaptation problem, where we select once and contimuously train a model for many iterations until convergence (Figure D.
The single-round adaptation problem requires us to study whether the selected subset is helpful throughout the continuous training procedure with many iterations. To this end, we start with pair-wise loss reduction gap that measures how much learning one selected sample can help learn another unselected sample. We also show that reducing this gap can improve adaptation performance. Then, we show that the loss reduction gap depends on (1) gradient distance, which is important for the first few continuous training iterations, and (2) prediction variability, which can dominate the latter iterations. This prediction variability is estimated using the norm of a tangent feature of lincarized noural network, which plausible model for adaptation tasks with a few fine-tuning epochs (Malladi et al.) 2023).. The tangent feature characterizes how the prediction varies during continuous training, and the norm of the tangent feature can upper bound the variability.
2

---

Published in Transactions on Machine Learning Research (10/2025)
Round
Round 2
Round
76
75.19
74.87
74
73.74
Pre-shift
CR
Post-shift
CL
Pre-shift
2
Post-shift
70
Random
BADGE
FORGE
(a) Multi-round Setting.
(b) Single-round Setting.
Comparison.
Figure 1: Prior works apply active learning to reduce the data annotation cost of adapting machine learning models to shifting distributions. However, many assume multi-round setting (a), which incurs multiple turnaround times and unnecessary delays. We study single-round adaptation setting (b) and develop an improved approach called feature-norm scaled gradient embedding (FORGE) (c).
Based the theoretical analysís, we develop feature-norm scaled gradient embedding (FORGE) improved active adaptation approach that considers the prediction variability and achieves better single-round active adaplation performance. One interesting observation from our analysis is that certain data samples may have small gradient distances and show a small loss reduction gap in the first few iterations. However, their loss reduction gap may increase significantly with high prediction variability. Following this observation, we aim to discover data samples with small gradient distances but high variability. Specifically, our approach first represents each data sample using their gradient embeddings, defined as the gradient of the loss w.r.t. the parameters and can help represent the gradient distance. We then re-scale the gradient embedding according to the sample-wise prediction variability, measured by the tangent feature norm, and construct feature-norm scaled gradient embedding. With the re-scaling operation, high-variability samples are represented by "long" embedding vectors. Therefore, they are more likely to be picked by diversity-basd data acquisition methods such as k-center greedy that primarily look for long vectors far away from the selected oncs (Ash et al. 2020 We outline suffucient conditions under which our neatrre-nom scaled embeddings outperform an approach without feature-norm scaling.
We derive an efficient analytical implementation for FORGE embedding that eliminates the need for expensive gradient backpropagation. We also extend the derivation to various vision and language tasks, including image classification, sentence classification, span-based question and answering, and reward modcling. Extensive empirical evaluation further demonstrates the advantage of our approach for single-round active adaptation tasks. Our main contribution is listed as follows:
We comprehensively analyse single-round adaptation problem, outlining two key conditions: gradient. distance and prediction variability.
We develop an improved active adaptation approach by incorporating prediction variability.
We provide efficient implementations and extensively evaluate them with various tasks.
2 Related Work
Active learning. Common active learning approaches are based data diversity and data uncertainty.
Diversity- based active learning (Sener & Savarese 2018 Ash al. 2020 Shen et al. 2022 aims to select a subset of diverse data samples that can best represent the full datasct in the input space, an embedding space, or gradient embedding space. Uncertainly-based active learning (Balcan el al. 2007 Gal & Ghahramani 2016 Ban et al.) 2022 2024) prioritizes the selection of data samples where the model prediction uncertain. Such uncertainty often requires entropy estimation and carcful calibration. A few recent works also show that incorporating training dynamics can improve convergence speed (Wang et al. 2022 Mohamadi al., 2022). Most of these approaches interleave the data selection and the model training procedure and perform
3

---

Published in Transactions on Machine Learning Research (10/2025)
selection every few training iterations. In contrast, our work studies single-round data selection with many training iterations for timely adaptation. Chen et al.
(2022) Wang et al. (2023) studied similar single-round problems to ours but assumed learning from scratch instead of adaptation. Several recent works apply active
learning to foundations model training (Zhang et al. 2023 Bhatt et al. 2024 Shen et al. 2025). and we will show a case with reward modeling as part of our experiments.
Domain adaptation.
Li et. al. (2021) Zhao et al. (022)
demonstrated the difficulty of unsupervised
adaptation. Hence, recent works have focused on incorporating external supervision (Su et al. 2020
Prabhu
ct al. 2021 Li al., 2021 Zhou et al. 2022 Xic et al. 2023
Alabdulmohsin ct al., 2023 Tsai et al. 2024
However, most of the works on active adaptation focus on improving the performance of conventional active
learning techniques via, for example, balancing diversity and uncertainty (Prabhu al.
2021 improving
uncertainty calibration (Xie et al. 2023). but few aim to minimize the annotation turnaround time.
3
Preliminaries
We list the notation and introduce an active adaptation algorithm framework before proceeding with the technical discussion. Appendix A provides a table of notations to ease the reading further.
3.1 Notation
A pair of input X' and its label y E 3' is a data sample. So : X -X>+ y is a model (e.g,, neural network) that is parameterized by 0. f: V x y + R is a loss function. x is the last hidden state of a neural network (ie.. input to the last linear layer). We use f(xx, V; fo) to simplify the notation of {(fo(z), y), which denotes the loss of a function fe at a given data sample (xx y). ro-r(w y; fo) = f(x, yi fa,) {(x, ys foz) is the amount of loss reduction on a data sample (x. y) after the model parameter evolves from 6n to 0r after T training iterations. Vefe(x) is the gradient of the model output fa( w.r.t. the model parameter 0, which is also called a tangent feature (Jacot et al.] 2018 Lee et al.) 2019) the following sections. Vol(x, y; fo) is the gradient of the loss value w.r.t. the model parameter 0. - denotes the L2 norm. 8 denotes selected subset of indexes from the full set. |-| denotes the size of set. [n]] denotes set of n natural numbers. S is set of selected data samples {æ;; Vi i E $) S' is set of unselected data samples {æ;, Vs E [n]] \ 8) and S is set of unsclected data samples and its closet selected neighbor {æ,, y/. zz Vy E [n]] \ s.j arg min, ll ()x;)) o(2,))1}- ø is an embedding function. Cat(:,)) is a vector concatenation operator.
3.2 Algorithm Framework
Our approach operates under a conventional two-step active learning procedure (Sener & Savareso 2018 Ash
et al. 2020):: (1) construct data representations using an embedding function á and (2) perform diversity- diversity-based data selection. We use the k-center greedy algorithm (lines 3-6) an example in the algorithm framework (Algorithm [] Then, we continuously train model using the selected data samples. The k-center greedy a algorithm can effectively minimize the maximum distance between an unselected data sample x' and its closest selected neighbor æ. which can be formulated as an objective function max] minjes || - æyy [Sener& & Savarese 2018). Intuitively, this k-center greedy algorithm selects data samples that are far from others. Our framework differs from prior ones (Ash et al. 2020)) regarding the continuous training step (line 7), which was often placed inside the data selection loop (lines 3-6).
4 Analysis
This section introduces our main insights into the single-round active adaptation problem. Our goal is to select a subset of data samples; once a model is trained upon them for many iterations, the model performance is comparable to that model trained over the full dataset. This goal requires us to study whether learning with selected data samples can also help learn the unselected ones, quantified by loss reduction gap. We show that such loss reduction gap vital in learning the full dataset via a subset. However, estimating the loss reduction gap after many training iterations is non-trivial because the model's training dynamics remain
4

---

Published in Transactions on Machine Learning Research (10/2025)
Algorithm 1 Algorithm framework
Input: A set of data samples "E......NN embedding function o, and model fe.- Steps:
1: Construct data representations (ő(z1),...) ((œN)) using an embedding function o;
2: Initialize a set of selected indices = {si} with a random 81 ~ uni((O,
3: for & + to K do
8k = arg [[]]] minjex llo(;;;) (
8 8 8 {
end for
Continuously training model feo by minimizing Tーx E.= ([a folifo) for T iterations; Return Jor-
unknown at the selection step. To this end, we introduce a novel method estimate an upper bound of the loss reduction gap incorporating (1) gradient distance term and (2) a prediction variability term under unknown training dynamics.
By obtaining a loss reduction gap upper bound, we observe that minimizing the upper bound has a sample- wise difficulty, depending not only on the gradient distance between samples but also on a sample-wist prediction variabilily. Moreover, the impact of the prediction variability can grow quadratically as the model parameter deviates from its initialization during training. The quadratic growth of the prediction variability can dominate the term associated with the gradient distance, which only grows lincarly. Such a result goes beyond the prior result (Sener & Savarese 2018) on the distance term and will further guide our algorithm design in Section
Technically, we employ the lincarization of a non-lincar neural network with a mcan squared error loss function under a neural tangent kernel (NTK) regime (Jacot et al., 2018 Lee et al. 2019 Malladi et al.) 2023); a standard tool in deep learning theoretical analysis (Ren & Sutherland] 2025 Notably, this approach does not require assuming linear models. The mean squared error loss function the standard choice in the NTK regime (Jacot et al., 2018; Lee et al.] [001 Malladi et al.) (023), which produces clear theoretical results and behaves closely to the cross-entropy loss function (Hui & Belkin 2021). This NTK regime is increasingly
employed in modern active learning rescarch (Awasthi et al. 2021
Mohamadi et al.
2022
Wang al. 2022
4.1 Objective Function
In an active adaptation problem, we hope that learning one selected data sample æ can also help learning another unselected data sample x'. To this end, we introduce an objective function called loss reduction gap between a pair of selected and unselected data samples.
Definition 1. (Loss reduction gap) Let ro-rr(ar, y; So) l(x, y: Soo) Elar y: Soy) be the loss reduction on data sample (x, y) after the model parameter evolves from Oo to Or after T training iterations we define loss reduction gap between æ and x':
Fo-r fo) To-r (2' fo).
(1)
The loss reduction gap includes the loss reduction (o-2 ys fo) on selected data, sample x and the loss reduction ro-r(a"' 5//: fo) on an unselected data sample x'. In our active adaptation problem, for given amount of loss reduction roduces {(x', v/: ). Note that the loss reduction is an objective, and we do not assume that the reduction is positive. The following proposition further illustrates the role of the loss reduction gap in minimizing the expected loss reduction Ep(ro-r((2 y; f)] with a given data distribution D.
selected z, hope that the model for parameterized by Or also learns z' and
Proposition 2. (Decomposition of expected loss reduction) Let x be the closest selected neighbor of on unselected x' and w; a where 6; is the frequency of each x appears the closest neighbor, the expected
5

---

Published in Transactions on Machine Learning Research (10/2025)
x y2. ||40/|1
Vol(x,)
vol(<','''o.)
Objective
x y² ||40-11
1 Case 1: Few iterations.
-07
Loss reduction gap
Case 2: Many iterations.
1|A8
S <11Vg(x, - 1|48,11
+ 2max (xtaff,,,,,,)11,)))) (1.0))ll110///,c33,""") ||A0,11
Figure The gradient similarity Vol(x, yi fo.) - Vol(a', can indicate loss reduction gap in few training iterations sctting (casc 1). With an increasing amount of training iterations (casc 2), we must also consider the prediction variability (i.e., ? max(I (x)),, V
loss reduction Ep(ro-T(xx, yi fe)] with a given data distribution D can be decomposed and upper bounded by (1) training loss reduction, (2) marimum loss reduction gap, and (3) a generalization gap:
N woo-T(EL Uss fo) + EEplro-r(e, v; 1E8
Σ
ro v6: f)
N ==1
(2)
Weighted training loss reduction
Generakeatton gop
Iro-T(I, fo) 1o
N w' y' EyES
Mazimum loss reduction gup
The expected loss reduction Ep(ro-T(C y, fe)] on the left- seft-hand-side of Equation 2 is what we want to maximize but lacks direct cstimation of it. Thanks to the lower bound in Proposition 2 we obtain additional insights suggesting that the expected loss reduction lower bounded by (1) how much does the loss on the selected samples reduce, (2) the standard generalization gap between the data distribution and the data samples and (3) the loss reduction gap between the selected and unselected data samples in an active learning procedure. In what follows, will present is helpful because reducing the upper bound of the loss reduction gap can help increase the lower bound of the expected loss reduction Ep/ro-Ti (a y. fe)].
upper bound of the loss reduction gap. The upper bound
4.2 Main Result
Estimating the loss reduction gap non-trivial because the model parameter Or at iteration T is unknown the data selection step. Therefore, derive an upper bound of the loss reduction gap applicable to unknown 0r. Our result suggests that for arbitrary Or, the upper bound depends on the gradient similarity between pair of data samples and their prediction variability. The gradient similarity term complements prior practice (Ash et al.) 2020), and the prediction variability term will guide an improved algorithm design. Before proceeding to the upper bound, we first introduce the NTK regime that is used in our analysis.
Assumption 3. (NTK regime) [Jacot al. 2018 Lee et al. 2019) Let for : X + a non-linear model at training slep T, we assume that its oulput for (x)) are governed by linear model flin X oblained

---

large loss Published in Transactions on Machine Learning Research (10/2025)
from the first-order Taylor expansion of the non-linear model for around its initial porameter 8u:
(0)(w) (ax) [ao (2x)
(o) AOT,
(3)
Tangent feature
where A0r 0g denotes the parameter deviation throughout training iterations.
The NTK regime is first studied using infinite-wide neural networks (Jacot et al. 2018 Lee al. 2019) and
is later applied to model fine-tuning (Malladi et al. 2023). In active adaptation, only continuously train
model for a few epochs, matching the model fine-tume the-tuning setting (Malladi et al., 2023). Under Assumption
it is easy to see that the prediction variability at a given data sample æ throughout the training procedure is upper bounded by its tangent feature norm || Vofo (a))l and the magnitude parameter derivation:
1 feu (x) S
(e)e |
(4)
Prediction variability
Feature norm
Variability upper bound
Although the model linearized, the loss function remains non-linear. We further introduce an upper bound of the loss reduction gap that is composed of (1) a term that associated with gradient distance and (2) variability upper bound based maximum feature norm.
Theorem 4. (Loss reduction gap upper bound) Let E(x, fo) = [1fo(z)) yll be a mean square error (MSE)
loss function, with defovitions in Section and Assumption 3 we have:
ro-T(w (2; fb mm - TO-I (a', (2'.3/:10)) f0m) S V yi fo.) - Vol(x', ,/' fao]ll |
Gradient distance
2 max (a:) oo, (x (1)) A0
(5)
Maz feature norm
Variability upper bound
The upper bound in Equation 5 is casy to interpret: (1) the first term with gradient distance captures first-order similarity between loss reductions of and x' and (2) the variability upper bound indicates the difficulty of maintaining higher-order loss reduction similarity during many training iterations. An interesting observation is that the high-order variability term relies on the square of a first-order tangent feature instead of high-order gradients. Figure 2 illustrates the main result.
Implications. The gradient distance term in our main result complements previous works on active learning with gradient embedding (Ash et al. 2020). In addition, the variability upper bound term further implics that minimizing the gradient distance can be insufficient when adaptation training contains many iterations. Without explicitly considering the variability term, active learning algorithm may neglect the data samples that are very difficult to learn by learning its close neighbors. According to Proposition reduction gap may result in diminished expected loss reduction. To this end, we will present an improved approach that considers the prediction variability and improves the single-round active adaptation with many training iterations in the adaptation procodure.
5 Approach
The goal of our algorithu design is to avoid neglecting the high-variabillly data samples so that we can directly maximize the training loss reduction over them while keeping a small loss reduction gap small (Proposition 2 in Section 4p. In bird's eye view, our approach operates under the conventional two-step active adaptation framework (Section 3.2p. where the first step is to represent each data sample by its embedding and then conduct data acquisition (e.g., k-center greedy) in embedding space. This framework is similar to prior

---

Published in Transactions on Machine Learning Research (10/2025)
Selected.
Selected.
Unselected.
Unselected
(a) Low-variabilily.
(b) High-variabillty.
Figure 3: When data samples have low-variability (a), selecting one data sample can be sufficient to represent a cluster of data samples. However, in a high-variabilityy setting (b), data samples can hardly represent each other, even they are close (c.g., the left cluster with red dots).
work in terms of using gradient embedding in the first step but also differs in the sense that it adds a variation to the gradient embodding to encourage the direct selection and learning of high-variability samples, which may lead to a large loss reduction gap, following our theoretical analysis in Section | Figure 3 provides intuitive example where we select all the high-variability samples from one cluster and selectively pick few representative low-variability samples from other clusters. We also derive efficient implementations for various tasks, including image and text classification, question-answering, and reward modeling.
5.1 neature-norm Scaled Gradient Embedding (FORGE)
In the previous section, the theoretical analysis shows that achieving good active adaptation performance requires minimizing loss reduction gap (Theorem 2), which further depends
gradient distance
between a pair of samples and their variability upper bound (Thcorem 4 As is detailed in the preliminary (Section 3 we perform diversity-based data selection in a gradient space, i.e., 6(0, y, Sên, E) = Vol(a, y; foo), which encourage a k-center greedy algorithm (Section 3.2) effectively minimizes the gradient distance term, // yi fa,) for minimizing the variability upper bound term, max(I|Vof (x)]l. Vo (x)) variability upper bound can lead to sub-optimal selection results because we will miss certain samples with small gradient distance from their selected neighbors but deviate from their neighbors after many training iterations due to high prediction variability (Figure 3). To achieve a low prediction variability and small gradient distance simultaneously, we develop a new embedding approach called peature-nom scaled gradient embedding (FORGE):
fe.)). However, diversity sampling over gradient space is insufficient Neglecting the
Definition 5. (FORGE) A feature-norm scaled gradient embedding function & is defined as:
Veee (x)]l Vol(ar yi foo)
$0x v. S0
Vol(a, Snc)
Fenture norm re-scoling
Gradient embedding
Our FORGE embedding is ro-scaled version of the gradient embedding Vol(x, ys fo,), where the fcature Ja, (z)] is the re-scaling factor and decides the magnitude of the FORGE embedding. Since the feature norm also decides the variability upper bound under unknown training dynamics, we assign high-magnitunn embodding vectors to nigh-variability samples. This strategy is effective because diversity-basod data selection methods seek data samples far away from selected ones. High-magnittuee vectors are often further away from others compared low-magnitude vectors, which usually fall into few clusters around 0. Figure 4 further illustrates the advantage of our FORGE embedding and makes comparisons. By re-scaling the embedding vector according to the prediction variability, avoid neglecting data samples with low gradient embedding magnitude but high prediction variability. We further investigate under which condition our FORGE approach with re-sacling operation can outperform the baseline BADGE approach without re-scaling.
Theorem 6. Let "BaDgE and "FORGE be the marimum feature norm of any data sample in ŚBadGE and SFORGE, respeclively. TRADOR is an upper bound of the loss reduction gap in Equation 2 max s..ySSSDDDS (0-T((2,0
(x', y/ fe)l < PBADGE, and T is also an upper bound of max a....uESPO WO,T((2VV Mi fe) (o- fo)]. If the FORGE embedding helps select large feature norm samples such that "BADGE > "FORGE, when the parameter deviation is large such that ||rr]]
8

---

Published in Transactions on Machine Learning Research (10/2025)
Selected.
Selected.
Unselected,
Unselected.
Overlooking high variability sample.
Re-scaling eases the selection of high variability samples.
(a) Without FORGE re-scaling.
(b) With FORGE re-scaling.
Figure The re-scaling operation FORGE scales gradient embedding according to their prediction variability and intentionally "pops out" high-variability samples in a gradient embedding space. This re- scaling helps bbvers data acquisition methods (e.g., k-center greedy) pick high-variability samples.
(o-1)ictO
Vot( II and O max w',y S..ESOORR Vol(a', f [ala((
Piymanon - "RoRGe] where o WESORcE
[a (x')l
Vol(x' v/: foo)
are constants, we have
Volo (x)]
I FORGE < T BADGE
(7)
This theorem implies that if the re-scaling operation in FORGE embedding can help selecting data samples with large feature norm and making TFORGE the maximum feature norm among unselected data samples and their corresponding selected neighbor SFORGE smaller than TADDGE the loss reduction gap upper bound FORGE of the FORGE embedding is provably smaller. Combining with Proposition 2 and taking the training loss reduction into consideration, we can further see that if the parameter deviation is large such that
(o-1)ic+O+ [[]]))+[]] -BYBABBE-
E woro (woovr(R..::.0))
E.ERHANGR Wi (B
||
4["HADGR DGB="POOGE)) our FORGE approach achieve a higher lower bound of the expected loss reduction.
5.2 Efficient Implementation
Computing the tangent feature Vale, (æ) and the gradient embedding Val(æ, yi fe,)) are expensive. To alleviate the computational overhead, prior work (Ash et al., 2020) showed that using the last layer's gradient embedding of a neural network achieves competitive performance. We extend this result and provide analytical constructions of FORGE embodding using only a single forward pass of a neural network. Such single-pass construction applies various vision and language tasks. Our analytical construction involves the last hidden state (ie., the input to the last linear layer) x, the sigmoid activation function a, and the pseudo-label (i.e., the prediction) ÿ. The pscudo-label 0 is a common surrogate of the true label (Ash et al. 2020 which is not yet available in the data selection step. Notably, all the following analytical constructions require only a forward pass through the neural network and are, therefore. computationally efficient.
Classification task. In binary classification task, we have Φ(x, G. fe, 2) [[( لل
(fo, (x) - 0)..
In sequence classification tasks where each input token has corresponding hidden state, use the hidden state #CLS of the [CLS] token. Concatenating the FORGE embedding vector of each class extends our approach to multi-class cases.
Span-based QA task. Span-based question-answering (QA) task requires a model to predict the starting index 9. and the end index de of an answer in sequence, using two separated linear layers, For given [ sequence with L tokens, we have d(2., Us. fe..s) = 1. Ete [|o((fg.. (m):
((f,,,(()).. - 0z,i)2,,i Then, we concatenate the staring and ending FORGE embeddings: Cat (0(x, Der few.s z.). pl0, De, fe..e: ze))
Reward modeling task. The loss function of a reward modeling task E(x", fo) = log o (fo( Jo(x")) where x is the preferred winning sample and at is the other loss sample. Note that the subtraction
9

---

Published in Transactions on Machine Learning Research (10/2025)
15.
|Loss reduction gapl
|Loss reduction gapl
|Loss reduction gapl
10
5
5
0
0
0.04
0.06
0.08
0.10
15
20
25
30
1.9
2.0
Embedding distance
Feature norm
Embedding distance
(a) BADGE
(b) Feature norm in BADGE
(c) FORGE
Figure 5: The loss reduction gap can be large even if the gradient embedding (BADGE) distance is small (a). The feature norm explains the large loss reduction gap (b). Incorporating the feature norm term in FORGE embedding alleviates this issuc (c).
g-1, where 8-1 is the parameter of the last linear layer. Therefore, we can consider xx as the input to the last layer, ignore the gradient (e=" (e="To-i-="" -' 0-1 + +1)-1 of the log-sigmoid function because it is positive scalar, and have ,(2", ,11) z² z!.
fo(x")) - fo(")) within the sigmoid function a equals to z T= 2¹ TTT__1,
6 Experiments
We present the empirical verification of our approach and make comparisons with strong baselines.
6.1 Setup
Tasks and datasets. In the image classification task, we use the VL.CS dataset (Gulrajani & Lopez-Par 2021) and the VisDA dataset (Peng al. 2017). The sentiment classification task operates over the Amazon and Yelp review datasets (McAuley al.) 2015 Zhang al.) 2015). The span-based question-answering
(QA) task employs the Squad and News datasets Raajpurkar et al. 2016 Trischler et al., 2017p. The reward modcling task utilizes the Anthropic-hh-rIhf dataset (Bai et al. 2022). We split the data evenly across sources during training, which consistently stabilizes training. For example, in VLCS (4 domains), when the source domain is Caltech and the total batch size is 16, we allocate samples to the source domain each batch.
Model architecture. We the Resnet-50 model (He et al. 2015 for the image classification task. The sentiment classification and the span-based QA task adopt the distilled-Bert models (Devlin et al. 2019 Sanh et al. 2019) with a classification head and QA head, respectively. We consider the GPT-2-medium model Radford et al. 2019) for reward modeling tasks.
Hyper-parameters. We use the SGD optimization for the Resnet-50 model, the Adam optimizer for the distilled-Berr models, and the GPT-2 model. The initial learning rate le-4 for all adaptation tasks, and use linear decay scheduling for the GPT-2 model in reward modeling task. The number of epochs for adaptation tasks is 4, and we train the reward model for epoch. The batch size for the Resnet-50 model is 64, the distilled-Ban Bert model is 16, and the GPT-2 model is 4.
6.2 Baselines
We include random selection, uncertainty-basel selection: margin Balcan et al. 2007) and DUC Xic et al. 2023), diversity-based selection: CORESET (Sener & Savarese 2018). BADGE AAsh et al. 2020 TypiClust (Hacohen et al. 2022), and ProbCover (Yehuda et al.) 2022), ) and hybrid selection: CLUE (Prabhu et al.) 2021 and BiLAF (Lu et al. 2024). Approaches such DynamicAL, (Wang al. 2022) and DULO (Wang ct al. 2023) are omitted due to their prohibitive computational overhead one requires computing neural tangent kernel, and the other needs training 4000 proxy models.
10

---

Published in Transactions on Machine Learning Research (10/2025)
Table 1: Accuracy decomposition of active adaptation methods with 5% labels.
Method
Selected Train
Unselected Train
Validation
Test
Random
100.0 +: 0.00
74.30 = 0.71
74.52 = 0.65
73.89 0.76
CORESET
100.0
73.68
73.67
72.99
BADGE
100.0
75.18
74.15
73.28
CLUE
100.0
75.90
73.77
72.97
FORGE (ours)
100.0 4 0.00
76.13
0.00
75.19 à 0.20
74.94
0.08
Note: The numbers are average accuracy over three runs. Variance is rounded up.
6.3 Visualizing Loss Reduction Gap
We investigate the correlation between the embedding distance and the absolute loss reduction gap in Proposition 2 This correlation is important because diversity-based data selection methods aim to minimize the distance ween pairs on selected and unselected samples (lines 3-6, Algorithm D
In these experiments, we use the Caltech (C) and VOC (V) datasots from the VLCS dataset with classos. We first fine-tune pre-trained Resnet-50 on the Caltech dataset obtain a source model. Then, we use CORESET, BADGE, and our approaches to select 5% data, samples from the VOC dataset (i.e., target domain). The source model is further fine-tuned the target domain to get the final accuracy, where each batch is split evenly for source and target data. For the VLCS dataset, we report the average accuracy on the V, I,, and S datasets. We always find using source data stabilizes and improves adaplation.
In Figure 3 we first plot the correlation between the gradient distance between each unselected sample and its closest select neighbor æ, [|ónADGF((E, y. fao: e) éBADGE fa., e)ll, and their absolute loss reduction gap, |ro- y; fe) ro-K (a', y/: fe)l. With the gradient embedding (BADGE), the loss reduction gap significantly inercases with minor increase in the embedding distance, diminishing their correlation and hurting the diversity-based selection performance. Then, show that the diminished correlation can be explained by the (tangent) feature norm in Figure 5b Adopting FORGE, which explicitly considers prediction variability via the feature norm term, recovers a strong correlation between the embedding distance and the loss reduction gap.
Table 2: Accuracy of active adaptation methods.
Image-CLS CLS on VLCS
Image- CLS on VisDA 5%
Image-CLS
Method
Average
5%
10%
10%
Random
74.87
# 0.74
75.33
0.76
81.91 4 0.01
84.28
0.01
79.10
Margin
61.43
1 0.17
64.32
0.18
81.08 1 0.01
83.59
1 0.01
72.61
DUC
68,07
72.78 78.
81.59
85.20
76.91
CORESET
73.50
74.45
81.04
84.85
78.46
BADGE
73.74
74.82
81.37
84.54
78.62
CLUE
74.56
75.56
0.08
81.42
84.82 0.02
79.09
TypiClust
74.94
0.59
75.23
1.50
81.75 +: 0.68
85.05 0.81
79.24
ProbCover
73.26 + 0.58
73.69
4 0.70
81.55 4 0.66
84.88 1: 0.34
78.35
BiLAF
74.87
0.26
75.02
0.38
81.38 + 0.57
84.44
0.49
78.93
FORGE (ours)
75.19
0.01 76.06
+ 0.07
82.28 + 0.01
85.45 0.02
79.75
Note: The numbers average accuracy three Variance is rounded up.
11

---

Published in Transactions on Machine Learning Research (10/2025)
Table 3: Accuracy of active adaptation methods.
Sentiment-Chi CCS
Span- QA
Method
Average
5%
10%
5%
10%
Random
50.53 0.01
51.66
38.27 0.25
38.84 +
44.83
Margin
44.41 0.02
48.57 = 0.01
33.01 0.11
35.92 i
30.48
DUC
48.21
50.66
33.59
0.21
37.82
42.57
CORESET
51.34
51.33
37.57
38.68
44.73
BADGE
51.26 0.01
52.28
0.0a
38.25
0.09
38.91
0.07
45.18
CLUE
50.90 0.01
51.65
0.01
38.21
0.09
38.45
0.06
44.80
TypiClust
51.38
0.02
51.94
0.02
38.38
0.01
38.87
0.01
45.14
ProbCover
50.90 i 0.02
51.58 = 0.01
38.10
0.02
38.41
1 0.03
44.77
BiLAF
50.47
0.03
50.78
= 0.
37.94 0.02
38.12 +: 0.02
44.31
FORGE (ours)
51.79
0.01
52.23
=
38.67 1 0.13
39.06 0.18
45.44
Note: The numbers average accuracy three
Variance is rounded up.
6.4 Performance Evaluation
We further show that FORGE, which recovers a strong correlation between the embedding distance and the loss reduction gap (Section 6.3). improves active adaptation performance. Table I lists the accuracy decomposition of active adaptation mothods in an image classification task (C to V adaptation in VLCS) with labeling budget of 5%. Our approach improves the accuracy the unselected training set and achieves better test performance. Such advantage further extends to another image classification (C to VLS), sentiment classification, question answering, and reward modeling tasks with labeling budgets of 5%, 10%, and 20%.
the image classification experiments, we consider a more challenging setting. The Caltech (C) dataset remains the source domain, and we use mix of the VOC (V), LabcIME (L), and SUN (S) as the target domains. We select data samples evenly from each target domain (e.g., 5% from each target domain). In VisDA, consider the synthetic to real transfer. The source domain for sentiment classification Amazon review, and the target domain is Yelp review. Both datasets have 5 sentiment classes. For the span-QA task, we directly use fine-tuned distilled- Bert on the Squad dataset D and use News as the target domain. The adaptation procedures follow the previous loss reduction gap experiment. and always include source domain data in target domain adaptation. We report the model performance on target domains. For the QA task, we report the exact match performance. Our approach achieves the best performance among out of the settings and beats the random selection baseline across all settings. In the sentiment classification task with 5% labeling budget, we observe our approach achieves 1.26% higher accuracy than the random selection bascline. In contrast, the BADGE approach without the re-scaling operation only beats the random selection baseline in 4 out of the 8 settings.
In the reward modeling task, we first fine-tune a GPT-2- medium model on 50% of the Anthropic-hh-rlhf dataset us- ing the "chosen" response. Then, we employ the warmup strategy in LESS (Xia et al. (2024) and train the super- vised fine-tuned (SFT) model using 5% of the remaining data samples, aiming to help the SFT model capture better sentence-level data representations instead of the token-level data representations. We only use the SFT model with warmup in the data selection step. The re- ward model training starts with the SFT model without any warm-up using the Anthropic-hh-rile dataset. We report the accuracy of hold-out test. set. The accuracy of reward model measured by the percentage of
Table 4: Reward model accuracy.
Reward Modeling 20%
Method
Random
63.01
CORESET
63.89 0.19
BADGE FORGE (ours)
63.12 0.24
64.45 à 0.14
12

---

Published in Transactions on Machine Learning Research (10/2025)
the preferred score being greater than the umpreferred score, fo(x") > fo(x²). On average of three different runs, we find that our approach outperform the BADGE approach, which was the second best approach in language tasks (Table 30, by 1.33%.
7 Ablation Study
We conduct an ablation study to evaluate the performance of our approach under metrics other than exact match. For the QA benchmark, we report the F1 score in Table 5 and show that our model remains the top performer. Another ablation study (Table 6 on VLCS dataset includes the norm scaling factor in our FORGE embedding construction, considering the 0, 1 1, and powers of [[aa (w))
Equation
Table 5: Fl score of adaptation methods.
Table 6; Accuracy of norm scaling.
Sentiment- CLS
Image- CLS on VLCS 5%
Method
Method
5%
10%
10%
Random
54.03 0.r
54.73 + 0.14
norm 0
73.01 0.15 74.91 + 0.09
CORESET
53.08 0.17
54.15 i 0.1
norm
74.63
0.17 75.16
BADGE
54.12
0.18
54.70 + 0.07
norm
75.19
0.02
76.06 0.07
FORGE (ours)
54.20 1 0.04
54.79
0.04
norm²
74.25
0.25 75.83 + 0.18
8 Conclusion and Future Work
This work studies single-round active adaptation problem, aiming to reduce the annotation turnaround time and promote timely adaptation to distribution shifts. A single-round adaptation problem requires selecting subset of data samples for many training iterations. Through thcoretical analysis, show that selecting for many iterations requires considering the prediction variability of each data sample, which is highly correlated with a tangent feature norm. Then, we introduce an improved approach called feature-non norm scaled gradient embedding (FORGE) that incorporates prediction variability into the data selection process. Extensive empirical results with various vision and language tasks demonstrate the effectiveness of our approach.
In the future, would be interesting to study the prediction variability in the pre-training stage (Chen et al, 2023 Tirumala al.) 2024) and the pseudo-label bias in gradient embedding construction in the context of learning with AI feedback (Taori & Hashimoto 2023
Panickssery et al. 2024
References
Ibrahim Alabdulmohsin, Nicole Chiou, Alexander D' D'Amour, Arthur Grelton, Sanmi Koyejo, Matt J Kusner, Stephen Pfohl, Olawale Salaudeen, Jessica Schrouff, and Katherine Tsai. Adapting latent subgroup shifts via concepts and proxies. In International Conference Artificial Intelligence and Statistics, pp. 9637-9661. PMLR, 2023.
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy. John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations, 2020. URL https://opeerrvien wttpriiwwww.eeeeeeeeeeeeeeeeeeee hwt.://..o......irrg.........
Pranjal Awasthi, Christoph Dann, Claudio Gentile, Ayush Sckhari, and Zhilei Wang. Neural active learning with performance guarantees. In Newral Information Processing Systems, 2021. URL https: //api
semanticscholar. httg/CooppsID::25... 235358351
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Dassarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfiel-DDodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown,
13

---

Published in Transactions on Machine Learning Research (10/2025)
Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan. Training helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, abs/2204.05862, 2022.
URL httpss///pp...mmmanicccccchhal www.aarpus/CCCC.C....oo :tgg/CorpusDD:24111888 248118878
Maria-Florina Balcan, Andrei Broder, and Tong Zhang. Margin based active learning. In International Conference on Computational Learning Theory, pp. 35-50. Springer, 2007.
Yikun Ban, Yuheng Zhang, Hanghang Tong, Arindam Banerjee, and Jingrui He. Improved algorithms for neural active learning. Advances in Neural Information Processing Systems, 35:27497-27509, 2022.
Yikun Ban, Ishika Agarwal, Ziwci Wu, Yada Zhu, Kommy Weldemariam, Hanghang Tong, and Jingrui He. Neural active learning beyond bandits. In The Twelfth International Conference on Learning Representalions, 2024. URL /ttpp:///pperrevier net/ wwt/fforrnee www.foooruuuggiooo -or.umiiigg177223FG.....
Gantavya Bhatt, Yifang Chen, Arnav M. Das, Jifan Zhang, Sang T. Truong, Stephen Mussmann, Yinglun Zhu, Jeff Bilmes, Simon Shaolei Du, Kevin Jamicson, Jordan T. Ash, and Robert Nowak. An experimental design framework for label-efficient supervised fineturing of large language models. ArXiv, abs/2400.0666) 2024. URL https: /ttt:s:/aapi.seeeee http:/ppp..smannicssccccccccoiel
org/CorpusID:26690 266977362
Si Chen, Tianhao Wang, and Ruoxi Jia. Zero-round active learning, 2022. URL https://oppnneveee..ee. forum?id=-0 90091mmmm2mmm
Yilan Chen, Wei Huang, Hao Wang, Charlotte Loh, Akash Srivastava, Lam M. Nguyen, and Tsui- Wei Weng.
Analyzing generalization of neural networks through loss path kernels. In Thirty-sewenth Conference Newral Information Processing Systems, 2023. URL htttps://pperreviorr http://www....ettroooooooaaaaaaa
Kevin Crowston. Amazon mechanical turk: research tool for organizations and information systems scholars. In Shaping the Future of ICT Research, 2012. URL https://aptl www.nticscholar.ccooooo www.:esshhoaaarrr//ooooo org.oopuuu
910853
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional
transformers for language understanding. In North American Chapter of the Association for Computational Linguistics, 2019. URL https:ttsss:. //apt.. www.sstessmtsnnnoooeeeeeee hwgpehhorr.rrooeeeeoooooo :orppsID::29611 52967399
Djellel Eddine Difallah, Michele Catasta, Gianluca Demartini, Panagiotis G. Ipcirotis, and Philippe Cudré- Mauroux. The dynamics of micro-task crowdsourcing: The case of amazon mlurk. In Proceedings of the 24th International Conference on World Wide Web, WWW '15, pp. 238 238-247, Republic and Canton of Geneva, CHE, 2015. International World Wide Web Conferences Steering Committee. ISBN 9781450334693. doi: 00.1145/273677..77425
10. URL https:///oi...... wtt..o......///1177777..771 1145/27362777 2741685
Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Maria Florina Balcan and Kilian Q. Weinberger (eds.), Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, mlr
pp. 1050-1059, New York, New York, USA, 20-22 Jun 2016. PMLR. URL https::/proceeeeee press/v48/gal hatp///gaall.... html.
Ishaan Gulrajani and David Lopez-Paz Pazz In search of lost domain generalization. In International Conference on Learning Representations, 2021. URL
Daniel Haas, Jiannan Wang, Eugene Wu, and Michael J. Franklin. Clamshell: speeding up crowds for low-latency data labeling. Proc. VLDB Endow,, 974 373-33,,, dec 2015. ISSN 2150-8097. doi: 10.14778/ 0856318.2856331. URL https::/doli http:::://oooor//11.../ 1rg///..1778///218 147111883177788881118888 2856331
Guy Hacohen, Avihu Dekel, and Daphna Weinshall. Active learning on budget: Opposite strategies suit high and low budgets. orXiv preprint arXim:2202.0277 2022.
Kaiming Hc, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual Icarning for image recognition. 2016
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2015. URL
https://api. www.ssacaaamnlla.... wttp:/aaaaaaattcaa..ooooooooo//e http://ccccccatteeero/o/opeaaaaa
14

---

Published in Transactions on Machine Learning Research (10/2025)
Like Hui and Mikhail Belkin. (EVALUATION] fof} (neural) (architectures) {trained} {with] {square} {in) (classification) (tasks). In International Conference on Learning
(loss} {vs} Representations, 2021. URL
Chip Huyen. Designing machine learning systems. O'RReill Media, Inc.", 2022.
Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural tangent kernel: convergence and generalization in neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp. 5580-8589, 2018.
Jachoon Lee, Lechao Xiao, Samuel S. Schocnholz, Yasaman Bahri, Roman Novak, Jascha Narain Sohl- Dickstein, and Jeffrey Pennington. Wide neural networks of any depth evolve as linear models under gradient descent. Journal of Statistical Mechanics: Theory and Experiment, 2020, 2019.
Bo Li, Yezhen Wang, Shanghang Zhang, Dongsheng Li, Kurt Keutzer, Trevor Darrell, and Han Zhao. Learning invariant representations and risks for semi-supervised domain adaptation. In 2021 IEEE/CVE Conference Computer Vision and Pattern Recognition (CVPR), pp. 1104-1113, 2021. 2021.00116.
doi: 1L110//CVR444377..
Han Lu, Yichen Xie, Xiaokang Yang, and Junchi Yan. Boundary matters: A bi-level active finetuning method. In The Thirtt-eighth Anrual Conference on Neural Information Processing Systems, 2024.
Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, and Sanjeev Arora. A kesnel-based view of language model fine-tuning. In Proceedings of the 40th International Conference on Machine Learning, ICML/23. JMLR.org, 2023.
Julian McAuley, Christopher Targett, Javen Qinfeng Shi, and Anton van den Hengel. Image-based recommen- dations styles and substitutes. Proceedings of the 38th International ACM STGIR Conference on Research and Development in Information Retrieval, 2015. URL httpps://api..smmatiiio 1012652
org/ Serg/CrppuIDD'
Mohamad Amin Mohamadi, Wonho Bae, and Danica J. Sutherland. Making look-ahead active learning strategies feasible with neural tangent kernels. In NewrIPS, 2022.
Arjun Panickssery, Samuel R. Bowman, and Shi Feng. LLM evaluators recognize and favor their own generations. In The eightt Annual Conference on Neural Information Processing Systems, 2024. URL https::/ http://www....eeeeee.oomaaaaaaaa
Xingchao Peng, Ben Usman, Necla Kaushik, Judy Hoffman, Dequan Wang, and Kate Sacnko. Visda: The visual domain adaptation challenge. arXiv preprint (Tfi::17100699. 2017.
Viraj Prablu, Arjun Chandrasekaran, Kate Saenko, and Judy Hoffman. Active domain adaptation via clustering uncertainty-weiihtedd embeddings. 2021 IEEE/CVE International Conference on Compuler Vision (ICCV), pp. 8485 8494, 2021. URL httppe///pp..smmmtttcccooooooo.p
semanticscoollnrr http:/hhhlaarooo/rrrooeoeo :
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Language models are unsupervised multitask Iearners. 2019.
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383 23922, Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10.18653 10.18653/v1/D16- 1264. URL http:://aclamtaooogy.oor/// www.:oog...r///--11011 1264
Yi Ren and Danica J. Sutherland. Learning dynamics of LLM finctuning. In The Thirteenth International Conference on Learning Representations, 2025. URL latts:://pperrevieu nct.//orrriiiiii0-PPHOOOO
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. ArXio, 10001190000100, 2019.
15

---

Published in Transactions on Machine Learning Research (10/2025)
Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Represenlations, 2018. URL httpp:///ppmmerriie.ne///rrrrrrr
Hialuk-Bi RW
Maohao Shen, Bowen Jiang, Jacky Yibo Zhang, and Oluwasanmi Koyejo. Batch active learning from the perspective of sparse approxination. arXiv preprinl arXiv:2211.00240 2022.
Yunyi Shen, Hao Sun, and Jean-François Ton. Reviving the classics: Active reward modeling in large language model alignment. arXiv preprint ar.ie::202 04354, 2025.
Jong-Chyi Su, Yi-Hsuan Tsai, Kihyuk Sohn, Buyu Liu, Subhransu Maji, and Manmohan Chandraker. Active adversarial domain adaptation. In 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 728-737, 2020. doi: 10.1109 WWACV 909.999390.
Rohan Taori and Tatsunori Hashimoto. Data feedback loops: Model-driven amplification of dataset biases. In International Conference on Machine Learning, pp. 33883 33920. PMLR, 2023.
Kushal Tirumala, Daniel Simig, Armen Aghajanyan, and Ari Morcos. D4: Improving Ilm pretraining via document de-duplication and diversification. Advances in Neural Information Processing Systems, 36, 2024.
Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. Newsqa: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pp. 191-200, 2017.
Katherine Tsai, Stephen R Pfohl, Olawale Salaudeen, Nicole Chiou, Matt Kusner, Alexander D Amour, Sanmi Koyejo, and Arthur Gretton. Proxy methods for domain adaptation. In International Conference PMLR, 2024.
on Artificial Intelligence and Stalislics, pp.
Haonan Wang, Wei Huang, Ziwei Wu, Hanghang Tong, Andrew J Margenot, and Jingrui He. Deep active learning by leveraging training dynamics. S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 25171 25184. Curran Associates, Inc., 2022.
Jiachen T. Wang. Si Chen, and Ruoxi Jia. One-round active learning through data utility learning and proxy models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https: /ttpss///openrvvait net//forumndd-BHQOOM
Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. Less: Sclecting influential data for targeted instruction luning. arXio preprint arxiv:2402. 04333, 2024.
Mixue Xie, Shuang Li, Rui Zhang, and Chi Harold Liu. Dirichlet-based uncertainty calibration for active domain adaptation. In International Conference on Learning Representations (ICLR), 2023.
Ofer Ychuda, Avihu Dekel, Guy Hacohen, and Daphna Weinshall. Active Learning Through a Covering Lens. arXiv preprint ar Xrxi:2200.11 11320, 2022.
Jifan Zhang, Yifang Chen, Gregory Canal, Stephen Mussmann, Arnav M. Das, Gantavya Bhatt, Yinglun Zhu, Simon Shaolei Du, Kevin Jamieson, and Robert. Nowak. Labelbench: A comprehensive framework for benchmarking adaptive label-efficient learning. 2023. URL https://api. www.nticscoolaa....... org/
1D:
Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. Character-level convolutional networks for text classification. In Neural Information Processing Systems, 2015. URL https://app httppppps//ppp...naatcccoooooooo semaaniccccntn wrt.laarrrogg/opuuuuiiii 368182
Han Zhao, Chen Dan, Bryon Aragam, Tommi Jaakkola, Geoffrey J. Gordon, and Pradeep Ravikumar. Fundamental limits and tradcoffs in invariant representation learning. Journal of Machine Learning
Research, 23(340):1-49) 2022. URL http:///wii..rrrpppppppss//////0 html
16

---

Published in Transactions on Machine Learning Research (10/2025)
Shiji Zhou, Han Zhao, Shanghang Zhang, Lianzhe Wang, Heng Chang, Zhi Wang, and Wenwu Zhu. Online continual adaptation with active self-training. In Gustau Calps-Valls, Francisco J. R. Ruiz, and Tsabel Valera (eds.). Proceedings of The 25th International Conference on Artificial Intelligence and Stotistics, volume 151 of Proceedings of Machine Learning Research, pp. 8852-8888 8883. PMLR, 28-30 Mar 2022. URL https://proceedingg mwr. halr.prees/v115/2hoo222 html
17

---

Published in Transactions on Machine Learning Research (10/2025)
A
Table of Notations
Table 7: Table of Notations
Symbol
Description
2, y
A data sample
N
The number of data samples
A selected subset of data samples
0
The parameters of model
0
The parameters of model at the tth training iteration
The number training iterations
To-r (x,, ys fo)
The loss reduction ((x, y; fo.) - f(x, yi for) after training iterations The L2 norm of a vector The size of a set A set of N natural numbers
-I
1-1
[N]
An embedding function
Cat(s))
A vector concatenation operator
A set of selected data samples {æ,, vi 8)
S
A set of unselected data samples {z;, yr E in] \ s}
A set of unselected data samples and its closet selected neighbor {a;, Vi, Tz Vj | E [n] \ s, = arg min, ||)(x;) <(x;)]1 The maximumn feature norm, maxx(|/eell (x) VoJo (x))ll)
y
B Proofs
Proposition 2. (Decomposition of expected loss reduction) Let be the closest selected neighbor of unselected x' and +1 where is the frequency of each a appears as the closest neighbor, the expected loss reduction Ep Ep[ro--((x,, yi fe)] with a given data distribution D can be decomposed and upper bounded by (1) a training loss reduction, (2) marimum loss reduction gap, and (3) a generalization gap:
Ep Ep/ro-r(2, y: fo)]
wT Mes So) + (2, so))
ro To M6i J)
N
(8)
Weighted training lass reduction
Generatization gap
(1 ra (a' :
max S
Mazimum loss reduction gap
18

---

Published in Transactions on Machine Learning Research (10/2025)
Proof. Let (Zg,s, (s.i) be the closest selected neighbor of an unselected (o), yo), when i [M] 8, and c₁ be the frequency of each (z) appears as closest selected neighbor, we have:
E Ep f) EDIro- (o,
To-T (Z 91:
N
Σ
To- Mi; f)
Σ
ro-
N
:E|N]}
iE#
1
(2,
To-T (1, yk:
N
Σ
(oo-T((I; M:: - ro v (e
N
=E|N]I
To-7 (2-,
1
Σ
(9)
N
N
SEB
EE
1 Σ
(c: + M6: ED
N
10-
LE|N]L
1 N w.ro-T(2H () (o)rr))))(ooo7(88 yi
ro-T (2,, 36
( fo) ro-T(=' -'o'r
N o' &.ES
Theorem 4. (Loss reduction gap upper bound) Let f(xr, : fo =])((()) y11 be a mean square error (MSE)
loss function, with definitions in Section 3 and Assumption we have:
70-T(x2,1 y: m TO- (a', / 1) S Vol( yi fe.) Vol(x', g/: foo) |
Gradient distance
2 max (x) V (x (z'))]] |1) A0T
(10)
Maz feature
Variabiläty upper bound
Proof. Capturing the loss reduction f(x, y: fo,) ((x, yS (lP)) during training is non-trivial because the loss function & remains non-linear if the model flim is linearized. Therefore, resort to the Lagrange mean value theorem and show that the loss reduction depends
interpolated gradient:
70-T(2, flin) ((xx, ys fe.) ((x, y; flip) V Vol(x, flin T AOT,
(11)
where flim = fo, Vofo Adr and a E [o. 1] is an interpolatoin factor. With some ro-arrangement of terms, we quantify the deviation between the interpolated gradient and the gradient embedding, which depends on the interpolation factor and the tangent. feature (o)o.
VoE(flin (x), = (z) (x) +a Volo, A0
(12)
Gradlent embedding
Interpolation deviation
19

---

Published in Transactions on Machine Learning Research (10/2025)
With this analysis of gradient deviation and the abbreviation Qu,T, = Vofoo AB Vofoo we have the following upper bound on pair-wise interpolated gradient distance:
Vol(<, yi fer. Vol(x', ./: (0
$ (foo (x) - y) voVoo (a) - (f)) Voloo
(13)
Gradient distance
+ |1a Qa,T. a' @e.T,a)) l Qu.T. - a' I
Interpolation distance
Feature distance
where the first term quantifies the contribution gradient embedding, For the latter two terms, we further present upper bounds that grow w.r.t. the feature norms [[)fo(>)]] or [[Vefu<>')]. For the interpolation distance, we have:
||a QT,T a Qu.T [
(14)
For the feature distance, we have:
aa QR.T. a' 2x'T,x))
VVaeo (x) Vefo, (<' (z') max( Vo (x')ll
(15)
max V Vafo. (x) Vefo, (z)) Ar
Plugging Equations 14 and 15
into
13
we have:
Vol(a, yi plin Vol(w' allll''v v/: fom
V
(16)
Vol(x, fll Vol(æ', v/: {b + max (e)l (z))))
Combining Equations 11 and
we complete the proof:
ro-r( ys ) ro-rr vv: flu)
Vol(xx yi fulm Vol(a' v/ for Aer
(17)
< Vol(x, yi fo. Vol(a', v/: |lll)
All
+ Tmax([]Veefe (x) IV
Thcorem Let YBADGE and "FoRGE be the mazimum feature norm of any data sample in SBADGE and SFORCE, respectively. TEADGE is an upper bound of the loss reduction gap Equation 2 Iro-T (D-T(wWV1 (x, fo) ro-r(ww', v': I TBADGE; and TronGE is also upper bound of max v vvSESORGE (1-T((1,0) fe) To-T(22 (x', (3'. fo)l. If the FORGE embedding helps select large feature norm samples such that "BADGE > |FORGE, when the parameter deviation is large such that || (o-1)-e+O
max "'
2[rRADCE "FFORE) where maX [oJo(())
[Volla.v.fo.)] and O max ' R.SESORGG
Vol(x', fe.)
oVolla'w''da..ll I. Vol(a', are constunts, we have
Voooa (x)]
I FFORGE < T BADGE.
(18)
20

---

Published in Transactions on Machine Learning Research (10/2025)
Proof. By the FORGE embedding definition, we have
| y, fe,, E) - "'''''y',for flll
VoJo, (a=)
(Vooe,
Vo foo)
Vol(a, y: foo)ll
Vol((' s': foo
(o (=))
(19)
Vol(a, vi f0.)
Vol(x', feo)
Vol(( vi So
|V V yı fo,)1l
Vo.a. (x)')l
Vol(a' v:
Soo)
VoJo, (x)
V foo
Rearranging some terms, we get a gradient distance term of the FORGE embedding distance:
Vol(x, yi fao) Vol(a' //: feo) in lower bound
(1 y, f e) <(x', v/ foo. E)
fa. (=)]l
Vol(a, vi Soo) - Vol(x',s's :oo)
Vol xr y; Jo,)
(20)
[VoS,, (a=)
[Vo((a, y5 fao
(ar'("))
Volla's
Vol(('' illar',' V: foo)ll Vol(x', y/: smo)
Vol((2 fo.
VoJo, (x)1
With Equation 20 if we achieve e-cover over FORGE embeddings, |1 Ja 6) 6(a', v/ S00: 811l S the
gradient embedding also has bounded coverage:
Volla, Son) - Vol(w', s/: [mo) S so.e+0
(21)
where max PESORGE
Vodle. and O = max x' s w. (o
Vollx',''s
[2))
V vel(x',v's foo)
Recalling the upper bound of the loss reduction gap, if we use the BADGE embedding, the upper bound TBADGE is
To- ) ro- "/' fi
< E AT| +
([Vofe, (x')))
(22)
ar' SEEDESB
With FORGE embedding, the upper bound l'FORCE is
ro-7(1 yi sein) To-r(aa', (2':''/'''''"""" fmm)
$ + O)l + max
([Volo, (Vooo, (a')
(23)
xr' SeEEESE
I7 BBADGE > TFORGE; the upper bound FFORGE with FORGE embedding smaller when
e 21BADGE A0T 0. + O A0
(o - +
(24)
||00T||
2(7BADGR ?FORGE)
C
Experiments
C.1 Baselines
Random Randomly selecting a subset of samples without repetition.
21

---

Published in Transactions on Machine Learning Research (10/2025)
Margin Selecting the data samples that are closest to their decision margin and have high uncertainty. The distance to the decision margin is measured by the difference between the largest logit and the second-largest logit.
2023 The authors first utilize Dirichlet-based uncertainty calibration (DUC) to mitigate mis-calibration of neural networks under distribution shifts. Then, they use two-round procedure to select data samples with high distribution uncertainty and high data uncertainty. Distribution uncertainty helps identify data samples that out of the source domain, and data uncertainty captures discriminative samples.
DUC Xie et al.
CORESET (Sener & Savarese 2018 Selecting diverse data samples using k-center greedy algorithm in embedding space. They use the last hidden state to construct embeddings.
BADGE Ass et al. 2020) Selecting diverse data samples using a k-means algorithm in an embedding space. They use the last. layer's gradient to construct embeddings.
CLUE Prabbhu et al. 2021) They use the last hidden state to construct embeddings and then run k-means clustering in an embedding space. The uncertainty, measured predictive entropy, serves as the weight of the k-means clustering. The data samples that are closest to each clustering center are selected.
C.2 FORGE Embedding Computation
practice, we compute the feature norm (RHS of Equation which serves as a principled upper bound on prediction variability for fixed parameter update. The feature norm is particularly suitable for data selection because the actual parameter update not yet known at that stage. As detailed in Section 5.2 estimate it by extracting last-layer embeddings from neural networks across tasks, and use these to construct the FORGE embedding.
C.3 Dataset Setup
Our method assumes pool-based scenario, where the full set of online data is available prior to selection for labeling. This set titting is common in many real-world applications where data can be collected in batches before annotation begins, such as autonomous driving and document classification where datascts are periodically updated. We split the data evenly across domains during training, which consistently stabilizes training, For example, in VLCS (4 domains), when the source domain is Caltech and the total batch size 16, we allocate samples to the source domain each batch.
C.4
Computational Efficiency
We compared our forward-only approach for constructing FORGE embeddings using Resnet-50 and the VLCS dataset against a baseline requiring a backward pass (batch size = 16, 100 batches). Our method reduces embedding construction time from 37.1 seconds 12.7 seconds 2.9 speedup.
D Limitation
Although this study illustrates the potential benefits of incorporating prediction variability into active adapta- tion problems, several important limitations remain unaddressed. In particular, outliers and class imbalance frequently occur in live traffic scenarios, yet the current formulation does not. explicitly account for these factors. A thorough consideration of such challenges along with corresponding mitigation strategies-wool substantially enhance the robustness and practical applicability of the proposed method.
22


Reviews:
[{"review_id": "Nbzh1o0VTP", "reviewer": "Reviewer_qHgt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "-  The paper introduces an active adaptation algorithm framework by performing diversity based data selection using k-center greedy algorithm. Each data is represented by a novel embedding function called FORGE(Feature-norm Scaled Gradient Embedding). FORGE embedding achieves low prediction variability and a small gradient distance simultaneously.\n\n- It considers both high and low-variability data. Does it take into account the exception cases like outliers?\n\n- The paper evaluates the performance of FORGE embeddings by using diversity based data selection only. However, its results utilizing uncertainty based selection remains unknown.\n\n-  It performs the analysis on a single round active adaptation problem by formulating the expected loss reduction objectives as minimizing the upper bound of a maximum loss reduction gap. The upper bound loss reduction gap is estimated by encompassing a gradient distance term and prediction variability term. Further analysis also shows that a given approach achieves a higher lower bound of the expected loss reduction.\n- The study mentions an interesting observation regarding higher-order variability terms relying on first order tangent features instead of higher-order gradients. Are there any experiment analyses to support this fact?\n\n- It also provides analytical constructions of FORGE embedding using only a single forward pass of a neural network. It incorporates last layer gradient embedding instead of computing initial tangent feature and gradient embedding to alleviate the computational overhead. However, It doesn’t provide any quantitative analysis to support the fact. \n\n- Extensive results on various vision and language tasks verify the effectiveness of the proposed approach for active learning. However, the experiment relies on source data to finetune the model first and later adapt it into target domain using 5 to 10% data selected using active learning. It is not discussed how much source data is enough to stabilize training and improved adaptation.\n\n- This paper lacks additional ablation studies to fully support the experiment. Furthermore, it only takes accuracy as the evaluation measures to compare the performance despite having several metrics for assessment. Additionally, how the proposed active learning strategy mitigates the effects of class imbalance during query selections should also be addressed?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a novel active learning framework tailored for the **single-round active adaptation** setting, where a model selects a subset of unlabeled data once and then trains continuously. Unlike traditional multi-round strategies, this formulation is better suited for online or dynamic learning scenarios. The paper proposes a principled approach for data selection that jointly considers **gradient distance** (important in early training) and **prediction variability** (which becomes dominant in later stages). To capture both aspects, it develops a new embedding technique called **FORGE (Feature-norm Scaled Gradient Embedding)**, which scales gradient embedding by the prediction variability of samples. This promotes the selection of **high-variability samples**—often underrepresented in prior diversity-based methods. The paper presents theoretical analyses showing that reducing the upper bound of  **loss reduction gap** between selected and unselected samples leads to improved expected loss reduction. The **upper bound loss reduction gap** is estimated by encompassing a gradient distance term and prediction variability term.  It also proves that FORGE achieves a **tighter lower bound** on this quantity compared to existing embedding techniques such as BADGE. It further provides an efficient implementation of FORGE using only single forward pass, making it computationally practical across diverse tasks. The proposed method is evaluated across multiple domains—including vision, sentiment classification, span-based QA, and reward modeling—demonstrating consistent improvements over strong baselines such as CORESET, BADGE, and uncertainty-based strategies.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- In page number 11, first paragraph, second line, the authors use only selected samples to calculate gradient distance for BADGE. Maybe it is a notation error. It should account for both selected and unselected samples.\n\n- There are also some typos or misspellings.\n\n- Ablation studies need to be done.\n\n- Major shortcomings of the proposed approach should be discussed.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "oRJd2EJvWX", "reviewer": "Reviewer_tV17", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The theoretical analysis is sound and clearly motivates the proposed approach.\n\t- The theoretical analysis is based on estimating the upper bound of the loss reduction gap.\n\t- The paper shows that the loss reduction gap is upper bounded by the sum of gradient distance and the variability upper bound.\n\t- The variability upper bound implies that without considering it, an active learning method may neglect the samples that are difficult to learn by learning its close neighbors.\n\t- This variability upper bound term is then used to construct the proposed embedding used for selection. The embedding contains a feature norm factor that captures the variability upper bound under unknown training dynamics.\n- The evaluation is extensive and shows that the proposed approach works well for a wide range of tasks.\n\t- The evaluation is performed on four tasks and seven datasets where the proposed method outperforms other approaches.\n- The paper is clearly written and easy to follow.\n\n**Weaknesses**\n- The evaluation could be improved.\n\t- Why does Table 1 not show the results of Random or other methods, such as CLUE?\n\t- Why does Table 4 compare only with Random and BADGE and not with other methods such as CLUE, CORESET, etc?\n\t- The paper should include a comparison with more models, e.g., TypiClust [1], ProbCover [2], and BiLAF [3].\n- The presentation of the paper could be improved.\n\t- In Algorithm 1 line 4 there should be $i$ in argmax, i.e., $i \\in [N] \\setminus \\mathbf{s}$.\n\t- The paper flips between using $n$ and $N$ for the number of samples. E.g., Algorithm 1 uses $N$ while Proposition 2 uses $n$. This should be made consistent throughout the paper.\n\t- It is unclear what the meaning of the red line in Figure 5 is.\n\t- In the proof of Proposition 2 $c_i$ should be $c_j$ and in the last line of the proof $1/n$ should be $1/N$. Additionally, using index $j$ both for looping over sums and to mean the closest selected neighbor makes it harder to read the proof.\n\t- In the proof of Theorem 4 $l(\\mathbf{x},y;f^{\\text{lin}}_{\\theta_T}) - l(\\mathbf{x},y;f_{\\theta_0})$ should be $l(\\mathbf{x},y;f_{\\theta_0}) - l(\\mathbf{x},y;f^{\\text{lin}}_{\\theta_T})$.\n- Typos:\n\t- Section 5.2. last sentence, \"neural network and is, therefore. computational efficient.\" should be \"neural network and is, therefore, computationally efficient.\"\n\t- Section 6.1. \"Model architecture. we use the ...\" should be \"Model architecture. We use the ...\"\n\n*References*\n\n[1] Guy Hacohen, Avihu Dekel, Daphna Weinshall, \"Active Learning on a Budget: Opposite Strategies Suit High and Low Budgets\", In ICML, 2022\n\n[2] Ofer Yehuda, Avihu Dekel, Guy Hacohen, Daphna Weinshall, \"Active Learning Through a Covering Lens\", In NeurIPS, 2022\n\n[3] Han Lu, Yichen Xie, Xiaokang Yang, Junchi Yan, \"Boundary Matters: A Bi-Level Active Finetuning Framework\", In NeurIPS, 2024", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a method for active adaptation (fine-tuning) of pre-trained models based on a single-round selection approach. The method is based on calculating an embedding for each sample and then performing selection using the k-center greedy approach. The embedding is constructed by rescaling the gradient embedding using a feature norm. This choice of embedding is based on theoretical derivation that shows that such embedding allows for selecting samples with high prediction variability, which is crucial for good selection. The proposed method is evaluated on four different tasks and seven datasets, where it consistently shows improvements over the random selection and outperforms existing active adaptation approaches.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "*Critical*\n- The paper should extend the evaluation to include additional active learning methods and include random and CLUE into all tables.\n\n*To strengthen the work*\n- Improving the presentation of the paper. E.g., making the use of $n$ or $N$ consistent throughout the paper, clarifying the meaning of things in plots, etc.", "broader_impact_concerns": "The work does not have any ethical concerns that would need to be addressed in a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mJqEyz0RdC", "reviewer": "Reviewer_JXb8", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "strength: This paper addresses an important problem in active learning, how to minimize the time human annotators need to spend (aka single round active adaptation), which is particularly relevant in large-scale systems like large language models, where repeated querying and annotation are costly due to high inference latency and data volume. The authors make a compelling case for reducing annotation turnaround time without compromising adaptation quality. A major strength of the work lies in its theoretical depth: the paper introduces prediction variability as a critical yet previously underappreciated factor in data selection for adaptation, complementing traditional diversity-based approaches. This idea is both novel and well-supported by analytical insights and empirical evidence. Furthermore, the proposed method, FORGE, is designed with computational efficiency in mind and is shown to be robust across a wide range of tasks, including image classification, sentiment analysis, and reward modeling.\n\nWeakness: The paper is rather technical and lacks accessible, real-world examples that could help readers. Additionally, the work would benefit from a deeper analysis of the trade-offs between computational cost and performance improvement, particularly in terms of annotation budget and model training time. Another limitation is the minimal ablation or sensitivity analysis of key components such as the impact of feature norm scaling or alternative embedding formulations. Without this, it is difficult to fully assess the robustness and generalizability of the approach across different settings.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the problem of single-round active adaptation, where a model must select a subset of online data in a single step and use it for continuous training to adapt to distribution shifts, thereby minimizing annotation turnaround time. \nThe authors mention that traditional data selection methods focusing solely on diversity are insufficient for this setting, and introduce the concept of prediction variability—the degree to which a sample’s prediction changes during training—as a critical factor. \nThey develop a theoretical framework centered on the loss reduction gap and show that minimizing this gap requires considering both gradient distance and prediction variability. \nTo address this, they propose FORGE (Feature-norm scaled Gradient Embedding), a method that rescales gradient embeddings by each sample's variability, enabling diversity-based selection algorithms to better prioritize impactful data. \nFORGE is computationally efficient, requiring only forward passes, and is applicable across various tasks, including image classification, sentiment analysis, question answering, and reward modeling. \nExtensive experiments demonstrate that FORGE consistently outperforms both traditional baselines and, surprisingly, even random selection, highlighting the importance of accounting for prediction variability in single-round adaptation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "It would be helpful if the authors could elaborate further on how the prediction variability is calculated in practice. While the paper provides a theoretical formulation using tangent feature norms, a more concrete explanation or step-by-step procedure—especially in the context of their analytical approximation—would improve clarity and reproducibility. Additionally, the paper should clarify the data access assumptions in the single-round setting: does the algorithm have access to the full pool of online data before selecting a subset to be labeled, or is the data arriving in a streaming fashion? If the latter, estimating prediction variability on unseen future examples could pose significant challenges. Addressing how FORGE could operate under streaming constraints—or explicitly stating that it assumes a pool-based scenario—would strengthen the applicability of the method.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)