Test-Time Adaptation with Binary Feedback
Authors: Taeckyung Lee
Sorn Chottananurak
Junsu Kim
Jinwoo Shin
Taesik Gong
Sung-Ju Lee
Venue: ICML 2025 poster (2025)
Keywords: test-time adaptation
domain adaptation
deep learning
machine learning
Primary area: general_machine_learning->online_learning_active_learning_and_bandits
TL;DR: We propose a novel test-time adaptation setting that uses few binary feedback (correct/incorrect). Our dual-path optimization algorithm, BiTTA, combines guided adaptation on uncertain samples with self-adaptation on confident predictions.

Abstract: Deep learning models perform poorly when domain shifts exist between training and test data. Test-time adaptation (TTA) is a paradigm to mitigate this issue by adapting pre-trained models using only unlabeled test samples. However, existing TTA methods can fail under severe domain shifts, while recent active TTA approaches requiring full-class labels are impractical due to high labeling costs. To address this issue, we introduce a new setting of TTA with binary feedback, which uses a few binary feedbacks from annotators to indicate whether model predictions are correct, thereby significantly reducing the labeling burden of annotators. Under the setting, we propose BiTTA, a novel dual-path optimization framework that leverages reinforcement learning to balance binary feedback-guided adaptation on uncertain samples with agreement-based self-adaptation on confident predictions. Experiments show BiTTA achieves substantial accuracy improvements over state-of-the-art baselines, demonstrating its effectiveness in handling severe distribution shifts with minimal labeling effort.

Teme Adaptation with Binary Feedback
Taeckyung Lee' Sorn Chottananurak Junsu Kim
Jinwoo Shin Taesik Gong
Sung-Ju Lee'
Abstract
conditions that differ from the training data (Sakaridis et al., 2018).
Deep learning models perform poorly when do- main shifts exist between training and test data. Test-time adaptation (TTA) is a paradigm to mit- igate this issue by adapting pre-tree pre-trained models using only unlabeled test samples. However, ex- isting TTA methods can fail under severe domain shifts, while recent active TTA approaches re- quiring full-class labels are impractical due to high labeling costs. To address this issue, we in- troduce a new setting of TTA with binary feed- back. This setting uses a few binary feedback inputs from annotators to indicate whether model predictions are correct, thereby significantly re- ducing the labeling burden of annotators. Under the setting, propose BiTTA, novel dual-path optimization framework that leverages reinforce- ment learning to balance binary feedback-guided adaptation on uncertain samples with agreement- based self-adaptation on confident predictions. Experiments show BiTTA achieves 13.3%p ac- curacy improvements over state-of-the-ard base- lines, demonstrating its effectiveness in handling severe distribution shifts with minimal labeling effort. The source code is available at https: TTA. BBTTTA
Test-time adaptation (TTA) (Wang et al., 2021) dynamically adapts the pre-trained models in real time using only unla- beled test samples. Hence, it is a viable solution to domain shifts. However, without ground-truth labels, existing TTA methods are vulnerable adaptation failures or suboptimal accuracies in severe distribution shifts (Gong al., 2022; Niu al., 2023; Gong et al., 2023b; Lee et al., 2024b; Press et al., 2023). This is largely due to relying on self-supervised metrics such as entropy or confidence (Wang et al., 2021; Gong et al., 2023b) that could be unreliable when adaptation fails (Lee et al., 2024b).
This limitation motivates the potential value of incorpo- rating human feedback into the adaptation loop. In lan- guage models, reinforcement learning from human feed- back (RLHF) (Ouyang et al., 2022) has emphasized the critical role of human feedback in aligning large language models with human intent. The key challenge in TTA is making human feedback practical and scalable. While full labeling of test samples is prohibitively expensive (Joshi et al., 2010), recent studies on language models have shown that binary feedback (c.g., thumbs up/down) can effectively guide model behaviors (Shankar et al., 2024).
/ //github. com tanckyung/kgaacc
Inspired by this, we introduce the TTA with binary feed- back setting (Figure 1), which uses binary feedback on the model prediction (correct or incorrect) to guide adaptation while maintaining efficiency. Our binary-feedback approach requires only minimal label information, significantly reduc- ing labeling costs compared with full-class active TTA (Gui et al., 2024), mitigating the interaction bottlenecks to enable real-world applications.
1. Introduction
Deep learning has revolutionized various fields, including computer vision (Deng et al., 2009), speech recognition (Gu- lati et al., 2020), and natural language processing (Brown et al., 2020). However, deep models often suffer from do- main shifts, where discrepancies between training and test data distributions lead to significant performance degra- dation. For example, autonomous driving systems might struggle with new types of vehicles or unexpected weather
As a solution, we propose BITTA, a dual-path optimization framework for TTA with binary feedback that incorporates both binary feedback and unlabeled samples. Motivated by the recent reinforcement learning studies that show effective- ness in incorporating human feedback in the optimization process (Ouyang et al., 2022; Fan et al., 2023; Black et al., 2024), BITTA leverages reinforcement learning to effec- tively balance two complementary adaptation strategies (Fig- ure 3): Binary Feedbuck freddbck-guided Adaptation (BFA) on uncer- tain samples and Agreement- Based relf-Adaptation (ABA) confident samples. Using Monte Carlo dropout (Gal &
"Corresponding authors. 'KAIST PUNIST. Correspondence to: Taesik Gong <taesik.gong knr>, Sung-Ju Lee <profsj@ kc.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada, PMLR 267, 2025. Copyright 2025 by the author(s).

---

perspective, Time Adaptation with Binary Feedback
Horse a m Class probobility
Unsupervised Entropy min.
Incorrect
90
Original TTA
Accuracy (%)
85
True label: Dog
Acc: 53%
80
Pre-adopt
Post-odopt
Correct
Acc: 87%
75
TTA with binary feedback
70
Few binary b feedbocks model output
TENT
EATA
COTTA*
ROTTA*
SOTTA
BITTA
Figure I: Overview of TTA with binary feedback. Traditional TTA algorithms often fail under severe distribution shifts due to the risk of unlabeled-onle adaptation. Our proposed TTA with binary feedback addresses this challenge by offering a few binary feedback (correct or incorrect) on selected model predictions. TTA with binary feed- a modified algorithm to utilize bi- back significantly improves the adaptation performance with minimal labeling effort, nary feedback. The dotted line is enabling practical and scalable TTA paradigm for real-world applications.
Figure 2: Accuracy (%) of TTA methods with binary feedback on CIFARIÓ-C. The asterisk indicates
full-class active TTA (SimATTA).
Ghahramani, 2016) for policy estimation and uncertainty assessment, we select uncertain samples for binary feedback in BFA while utilizing samples with high prediction agree- ment in ABA. This dual approach enables BiTTA to adapt to new uncertain patterns (via BFA) while maintaining con- fidence in correct predictions (via ABA), achieving robust performance improvements.
2. Test-Time Adaptation with Binary Feedback
We propose TTA with binary feedback, novel TTA setting for adapting pre- pre-trained models during test time using binary feedback from an oracle, which indicates whether a model's 's predictions are correct (Figure 1). This real-time feedback seamlessly integrates into the adaptation process, enabling continuous refinement of the model's 's performance.
We evaluate BiTTA under the TTA with binary feedback setting with various test-time distribution shift scenarios, including three image corruption datasets (CIFARIO-C, CIFAR100-C, C, and Iny--mmaeNett-C) and two domain gen- eralization scenarios (domain-wise and mixed data streams). Comparisons with TTA and active TTA methods demon- strate that BiTTA achieves an accuracy improvement of 13.3%p on average. Moreover, BiTTA outperforms ac- tive TTA utilizing full-class feedback from the oracle (the ground-trum label, Figure 2) and the state-of-the-art founda- tional model (GPT-40, Figure 5), despite using only binary feedback. These results highlight the importance and effec- tiveness of BiTTA and TTA with binary feedback, enabling robust adaptation with minimal labeling effort.
TTA with binary feedback addresses the critical challenge of adapting pre-trained models to domain shifts with mini- mal labeling effort. From an information- full-class labeling requires log class) times more bits than binary feedback to encode the same informa- tion (MacKay, 2003). Empirical studies on human anno- tation also validate the efficiency of binary feedback: full- class labeling 50-class takes 11.7 seconds average with 12.7% error rate, whereas binary comparison requires only 1.6 seconds with 0.8% error rate (Joshi et al., 2010).
Although binary feedback provides only single bit, the feedback is based on the adapted model's prediction, which is typically better than chance. This makes the feedback more informative and allows it to directly guide model be- haviors. Therefore, TTA with binary feedback an efficient and practical framework for weal-world TTA applications.
Our main contributions are as follows:
We propose a lightweight and scalable setting of TTA with binary feedback (Figure 1). offering correct or incorrect feedback selected model predictions.
Notation. Let z denote a test sample selected for binary- feedback labeling, and y arg max, ,, fo(y|ar) is the class prediction output of the model parameterized by A. The binary feedback B(x, y) is defined as:
We develop BiTTA, a dual-path optimization strategy combining two complementary signals from binary bredback-guided adaptation and agreement-basee self- adaptation with reinforcement learning framework.
{
1
if y is correct,
B(x, v)
(I)
if y is incorrect.
We perform extensive experiments that show BiTTA outperforms TTA and active TTA baselines by 13.3%p. Comparisons with full-class active TTA indicate the effectiveness of BiTTA and TTA with binary feedback.
Accordingly, each binary sample is represented as the tuple (x, y², B(x, y²)), consisting of the input instance, the model's 's predicted label, and the binary feedback.
2

---

Test-Time Adaptation with Binary Feedback
(1) Binary Feedback-Guided Adaptation (BFA)
Ne (y" [s): Probobility on model prediction y"
Uncertoin sampies
Human feedback reward
Reward
correct
incorrect
Sample selection
(2) Agreement-Baal Based Self-Adaptation (ABA)
Somples with prediction agreement
Self-feedback reward
Unlabeled Test stream
x
Prediction probability
Backward
Forword
Reward (o
agreement
else
estimation
{x yy
argmaxy H. Or(2))
Policy gradient
Figure 3: Overview of BiTTA algorithm. BiTTA implements reinforcement learning- based dual-path optimization that estimates prediction probabilities using MC-dropout. It computes policy gradients from complementary signals: (1) Binary Feedback-guided Adaptation (BFA) uncertain samples, using binary rewards of =1, and (2) Agreement- self-Adaptation (ABA) confident, unlabeled samples, using reward 1. By jointly optimizing both paths, BiTTA enables robust adaptation under dynamic distribution shift scenarios.
3. BiTTA: Dual-path Optimization to Learn with Binary Feedback
continuous adaptation to the evolving test distribution.
As binary feedback employ the REINFORCE algorithm (Williams, 1992), also known the derivvvvv trick". This method allows us to estimate the gradient of the expected reward with respect to the model parameters:
con-differentiable function, we
Motivation. Recent advancements in reinforcement learn- ing from human feedback (RLHF) (Ouyang al., 2022) have demonstrated the effectiveness of incorporating sparse feedback signals in large language model training. Inspired by this, we propose BiTTA, a reinforcement learning (RL) based approach for TTA with binary feedback that effec- tively adapts to distribution shifts using minimal labeling effort (Figure 3). BiTTA leverages binary feedback as reinforcement signal, offering several key advantages for TTA. (i) Binary feedback be seamlessly incorporated non-differentiable rewards in the RL framework, enabling the model to learn from minimal supervision (Zoph & Lc, 2017: Yoon et al., 2020). (ii) The RL framework allows for integrating binary feedback and unlabeled samples into a single objective function optimized through policy gradient methods. By combining sparse binary-feedback samples with unlabeled data, BiTTA provides a robust framework with minimal labeling effort, making TTA more feasible for weal-world applications.
V ( = (y [ftr y) Va log Na (y(l)))] (3)
By using this gradient estimator, we effectively optimize model parameters using stochastic gradient ascent.
To estimate the policy тe. adopt Monte Carlo (MC) dropout (Gal & Ghahramani, 2016). practical Bayesian approximation technique that enables robust uncertainty esti- mation without architectural changes. MC-dropout performs multiple stochastic forward passes with dropout enabled test time, allowing the model to capture epistemic uncer- tainty in its predictions.
Formally, we approximate the policy (u(()² as the mean of softmax outputs across N stochastic forward passes:
Policy gradient modeling. Given a batch of test samples (........ } ), our goal to adapt the model parame- ters 0 to improve performance the test distribution. We formulate the test-time adaptation process as RL problem by assigning test-time input ~ B as a state, the model prediction y* = fe(x)) as an action, and the corresponding prediction probability no To(l) as a policy, which objective maximizing the expected reward, defined as:
no T
(vlz),
(4)
N
where fú denotes the model with dropout applied during inference, and N' is the number samples drawn.
This approach is crucial for test-time adaptation under dis- tribution shift. Unlike standard softmax outputs, which often miscalibrated and overconfident on out-of-distribution (OOD) samples, MC-dropout provides well-calibrated con- fidence estimates. These improved uncertainty estimates are key to BiTTA'S dual-path strategy: selecting the uncertain samples for feedback (BFA) and identifying confidently
((O) =
s)),
where R(x, y) represents the reward function defined later. This optimization performed for each test batch, allowing
3

---

Test-Time Adaptation with Binary Feedback
predicted ones for self-adaptation (ABA). We empirically validate the calibration benefits of MC-dropout in Section 4.
Then, we select the set of k samples get the binary feed- back, noted SEFA- One straightforward strategy is select the least confident samples:
Dual-path optimization strategy. With the proposed RL framework, BiTTA addresses the challenge of TTA with binary feedback (Figure 3), utilizing (1) few samples with ground-truth binary feedback and (2) many unlabeled sam- ples with potentially noisy predictions by two complemen- tary strategies:
SEFA argsort (C(x)):: (C()) Al.
(6)
We further discuss the BFA sample selection strategy and its impact in Appendix B.
Reward function design. For these selected samples, we query the binary feedback (xx, y) (correct/incorrect) and define the reward function REFA for feedback samples as:
1. Binary Feedback-guidee Adaptation on uncertain sam- ples (BFA, Section 3.1): ): This strategy focuses on en- hancing the model's areas of uncertainty. By selecting samples where the model is least confident and obtain- ing binary feedback on these, BiTTA efficiently probes the boundaries of the model's current knowledge.
1
if is correct, otherwise.
REFL v) B(z,
This binary-feedback reward scheme provides clear signal for model adaptation, encouraging the prediction probability of correct predictions and penalizing incorrect ones. By se- lectively applying this reward function to the most uncertain samples, BFA efficiently utilizes the limited labeling budget, maximizing the contribution of each queried label.
2. Agreement-Basee self-Adaptation on confident sam- ples (ABA, Section 3.2): To complement the guided adaptation strategy, BíTTA also leverages the model's existing knowledge through self-adaptation on confi- dently predicted samples. Without requiring additional feedback, ABA identifies confident samples by the agreement between the model's s standard predictions and those obtained via MC-dropout.
3.2. Agreement-Based Self-Adaptation
To complement the binary feedback- adaptation on uncertain samples, we propose leveraging the model's 's con- fident predictions on the remaining many unlabeled sam- ples. This approach, which we call Agreement-Base Adaptation (ABA), aims to reinforce the model's current knowledge without requiring additional oracle feedback.
The synergy between BFA and ABA enables BiTTA to ef- fectively utilize both labeled and unlabeled samples. BFA drives exploration and adaptation to new patterns in the test distribution through binary feedback on uncertain samples. Concurrently, ABA maintains and refines existing knowl- edge through self sulf-super adaptation on confident pre- dictions. This dual-path optimization allows for effective adaptation across diverse challenging conditions.
self-
Sample selection. The key idea behind ABA is to iden- tify samples where the model's standard prediction agrees with its MC-dropout prediction. We consider these samples "confident" and use them for self-adaptation. Formally, we define the set of confident samples SABA as:
3.1. Binary Feedback-Guides Adaptation
In TTA with binary feedback settings where binary feedback is limited and costly, selecting samples to query and using them for effective model adaptation becomes crucial. To address this challenge, we propose Binary Feedback-guided Adaptation on uncertain samples (BFA). This approach re- fines the model's decision boundaries and improves its un- derstanding of challenging data points through binary feed- back guidance, enabling robust and efficient adaptation in test-time distribution shifts.
SABA = {x E B SEFA yy = arg max To (y|z)),
(8)
where B is the entire batch of test samples, SEFA is the set of samples selected for active feedback, y² is the original class prediction, and T T (z) is the MC-dropout softmax.
Unlike existing TTA methods that rely on fixed confidence thresholds (Niu et al., 2022; 2023; Gong et al., 2023b), our approach can dynamically select confident samples based on the agreement between standard and MC-dropout pre- dictions. Figure 4a illustrates the dynamic nature of pre- diction confidences during distribution shifts-tighllighting the need for dynamic sample selection. To demonstrate the effectiveness of ABA further, we compare our agreement- based approach with thresholding strategies (Figure 8 in Appendix B). The results motivate our dynamic agreement- based selection over static confidence thresholding.
Sample selection. To guide the adaptation, we focus on the uncertain samples, often the most informative for model improvement (Settles, 2009). We quantify the sample-wise (un) (un)ccrta C(x) the MC-dropout softmax of the origi- nal model predicted class:
C(x) To(y"
(5)
where y" = arg max, fo (y) (r) is the deterministic class prediction and no (y) is MC-dropout softmax confidence.
Furthermore, our method effectively identifies confident samples for self-adaptation. Figure 4b demonstrates the

---

Test-Time Adaptation with Binary Feedback
CIFARIO-C CIFARIDO
Tiny-lmageNet-C
Samples with agreement
Samples with disagreement
1.0-
100
Confidence
0.8-
Accuracy (%)
80
60-
0.6
40-
0.4
200
400
600
800
1000
200
400
600
800
1000
Online Batch
Online Batch
(a) Average sample-wise confidence in three image corruption datasets.
(b) Average sample-wise accuracy of samples with agreement (Swa) and disagreement (B (ABA).
Figure 4: Analysis of confidence and accuracy during online adaptation. (a) Average sample- wise confidence over time and dataset, showing dynamic changes that challenge fixed thresholding methods. (b) Average sample-wise accuracy for samples with prediction agreement and disagreement on CIFARIO-C, demonstrating the effectiveness of agreement-based selection for confident samples.
stable accuracies in samples with agreement, while sam- ples with disagreement show unstable and low accuracies. This originates from the prediction agreement of indicat- ing robustness and reliability via the consistency in model outputs across different dropout masks. By leveraging this consistency, ABA can reliably select confident samples for effective self-adaptation.
rithmic formulation and its practical implementation using memory-based sample buffers.
Combined objective. We formulate the overall objective a weighted sum of expected rewards from BFA and ABA:
J(0) = a Sur [Ruaa (I, v)] + SE [RABA (Ir y)]
(10)
where a and 3 balance BFA and ABA. We set a = B = in all experiments and analyze the impact of these hyperpa- rameters in Figure 10 (Appendix B).
Reward function design. We now incorporate these sam- ples into our reinforcement learning framework. We intro- duce self-feedback reward function RABA for unlabeled samples. This reward encourages the model to maintain its predictions on confident samples while discarding the adaptation on unreliable ones. Formally, we define RABA as:
The policy gradient is estimated via REINFORCE (Williams, 1992):
VoJ(O) = aFFESSEX [RIEA (I, Vo log no [x]]
1 if x E SABA otherwise.
+ -.E
(Ve log Ta (v|[z)]
(1I)
RASA (I, u)
(9)
where To(y)z) is approximated using MC-dropout.
By incorporating this adaptive prediction agreement strat- egy, ABA enhances the learning process by maintaining the knowledge of confident predictions. While prediction disagreement suggests uncertainty, our analysis shows dis- agreement samples exhibit unstable accuracy rather than consistent errors (Figure 4b). Therefore, ABA assigns zero rewards to them rather than penalizing (as BFA), prevent- ing potentially harmful adaptation from noisy signals. This conservative approach is especially valuable in TTA scenar- ios where distribution shift may be partial or gradual, where most of the model's existing knowledge remains relevant.
Practical implementation. We implement the policy gra- dient in Equation 11 using two FIFO memory pools: Me. for correct predictions and MM for incorrect ones. Each memory stores up to batch-sized number of samples for computational stability and efficiency. Additionally, we use SABA, confident unlabeled samples identified via agreement the current batch.
For each input x with predicted label y", we optimize the MC-dropout softmax probability To(y" (c:) by minimizing or maximizing the cross-entropy depending on its type. The overall loss is defined as:
3.3. BiTTA Algorithm
CBTTA = a |Meel
1
Σ ZeMo
(- (olo Voloooo(''' (T
Our proposed BiTTA algorithm unifies binary feedback- guided adaptation (BFA, Section 3.1) and agreement- self-adaptation (ABA, Section 3.2) into a dual-path opti- mization framework. This design enables robust and effi- cient test-time adaptation under distribution shifts while maintaining stability across time. Here, we detail the algo-
1
a MM
(+ lo
aEM:
+ B. SABAAl
Σ
(- Iolog ((()))))))) (12)
DESABA
5

---

Test-Time Adaptation with Binary Feedback
Algorithm 1 BiTTA Algorithm
maximizing cross-entropy, respectively, and (ii) ABA over unlabeled samples with agreement by minimizing cross- entropy. The final objective CELTTA combines both compo- nents and is optimized via gradient descent. Notably, the policy gradient Vo log To (o(aa)) is implemented implicitly via backpropagation through cross-entropy on MC-dropout outputs, enabling a simple and effective optimization.
I: Input: Model Ju. MC-dropout ne, batch B, correct memory Mc. incorrect memory Mr, budget k epochs E, learning rate 1,. balancing hyperparameters o
2: # Binary Feedback Sample Selection
3: Y* + arg max fa(* [B) // Deterministii prediction // Equation 6 BFA samples
4: IEFA
5: SEFA + 1(( v() (To [B)]|: k]
4. Experiments
6: for each (z,y) in SaFA do
7: B(r, t Binary feedback from oracle
We present our experimental setup and results across various scenarios. Additional experiments, results, and details are provided in Appendices B, C, and D.
8: if B(z,y) then
9: Mc.update(z.8)
// Store correct sample
10: else
Baselines. We evaluated BiTTA against a comprehen- sive set of baselines, including source validation VSriValid) and seven state-of-the-art TTA methods: BN-Stats (Nado et al., 2020), TENT (Wang et al., 2021), EATA (Niu et al., 2022), SAR (Niu et al., 2023), COTTA (Wang et al., 2022), RoTTA (Yuan et al., 2023), and SoTTA (Gong et al., 2023b), To ensure a fair comparison, we incorporate an equal num- ber of random binary-feedback data into TTA baselines by adding correct-sample loss (cross-entropy) and incorrect- sample loss (complementary label loss Kim et al. (2019)). We also included SimATTA (Gui et al., 2024) as an active TTA baseline, adapting it to use binary-feedbar data by in- corporating complementary loss for negative samples. The non-active and full-class active TTA accuracies are reported in Appendix C.
11: M,.upaate(s
// Store incorrect sample
12: end if
13: end for
14: # Test-Time Adaptation
15: Update BN stats with and freeze them
16: for = to E do
17:
# BFA (Section 3.1)
18:
Xc,Yo Mo
// Get correct samples
19:
&r,Yi+M MM
// Get incorrect samples
20:
LBFA + (<a(TT(( (c)), Yc) - (=x X1), Yr)
21:
# ABA (Section 3.2)
22:
Xu + Bi SaFA
// Get unlabeled samples
23:
RABA t 1argg max fo (+1 (x) arg max To( Xx)]
24:
XADA + fo E Xu RABA(R (x) 1}.
To evaluate the robustness of BiTTA across var- ious domain shifts, we used standard image corruption datasets CIFARIO-C. CIFAR100-C, and Liny-ImageNet C (Hendrycks & Dietterich, 2019). Additionally, we con- ducted experiments on the PACS dataset (Li et al., 2017), which is commonly used for domain adaptation tasks. To closely simulate real-world scenarios with evolving distribu- tion shifts, we implemented a continual TTA (Wang et al., 2022) where corruption continuously changes.
Dataset.
YABL+ + arg max fo XABA)
I1 Get SABA
25:
LABA + leu(To (TO(* | XABA), YABA)
26:
# Final Update (Section 3.3)
27:
LEITTA c- a LeFa + B LABA
28:
0 +- 0 VOLTTTA
29:
end for
Settings and hyperparameters. We configured BiTTA to operate with minimal labeling effort, using only three binary feedback samples within each 64-sample test batch, accounting for less than 5%. We utilize a single value of balancing hyperparameters a = and , for BiTTA in all experiments. A comprehensive list of settings and hyperparameters is provided in Appendix D.
Based the unified objective LITTTA Algorithm outlines the full adaptation procedure. The algorithm proceeds in two phases. In the first phase, we select the top-k uncer- tain samples (or random samples in large-scale additional studies) from the current batch based on MC-dropout confi- dence scores over the predicted class. We then query binary feedback from an oracle and update the corresponding FIFO memory: Mc (correct) or MM (incorrect). This separation ensures balanced and stable adaptation, particularly during early rounds when labeled feedback is scarce or skewed.
Overall result. As shown in Table BiTTA consistently outperformed all TTA and active TTA baselines, showcasing its effectiveness in the proposed TTA with binary feedback setting. Existing TTA methods, even when adapted to binary feedback, showed suboptimal results, as their fixed filtering strategies (e.g., EATA, SAR, SoTTA) struggle to cope with dynamic uncertainties under continuous distribution shifts.
In the second phase, we perform E' epochs of test-time adap- tation. We first freeze the batch normalization (BN) statistics using the current batch. We then compute two types of adap- tation losses: (i) BFA over Me and M, by minimizing and
6

---

Test-Time Adaptation with Binary Feedback
Table I: Accuracy (%) comparisons with TTA and active TTA baselines with binary feedback in corruption datasets (severity level 5). Notation indicates the modified algorithm to utilize binary-feedback samples. B: TTA with binary feedback. Results outperforming all other baselines are highlighted in bold fonts. Averaged over three random seeds. Comparison with non-active TTAs and full-class active TTA are in Table 12 in Appendix C.
Noise
Blur
Weather
Digital
Label
Method
Gau.
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Cont.
Elas.
Pix.
JPEG
Avg.
SreValid
25.97
13.19
24.71
56.73
52.02
67.37
64.80
77.97
67.01
74.14
91.51
33.90
76.62
46.38
73.23
57.23
BN-Stats
66.96
69.04
60.36
87.78
65.55
86.29
87.38
81.63
80.28
85.39
90.74
86.88
76.72
79.33
71.92
78.42
TENT
75.25
80.71
73.21
85 $5.42
68.89
82.95
85,48
81.96
K2.99
83.18
88.88
86.78
75.37
80.77
75.59
80.49
EATA*
73.22
74.16
65.72
76.50
62.47
74.62
79.65
76.42
77.31
79.70
85.37
83.66
69.52
77.12
72.16
75.17
SAR*
71.57
78.62
73.33
88.98
73.30
87.98
89.72
86,00
87.09
87.86
92.46
90.00
82.07
86.69
80.96
83.78
COTIA
66.97
69.04
60.35
87.77
65.54
86.29
87.38
81.63
80.28
90.73
86.87
76.74
79.35
71.92
78.42
RoTTA*
64,49
69.69
63.89
86.29
69.48
86. 78
89.23
85.21
85.39
87.45 .848
92.02
87.09
81.76
85.66
80.18
80.98
SoTTA
71.39
79.27
70.58
85.07
68.39
84.03
87.27
83.47
84.90
55
90.81
86.18
78.26
83.41
76.94
81.03
SiMATTA*
70.21
81.67
71.49
79.59
69.41
82.15
87.28
83,90
86.89
86. 49
91.51
83.40
77.94
83.81
82.25
81.09
76.78
78.75
87.51
91.36
89.42
90.30
92.62
92.42
(a) CIFARIO-C.
Noise
Blur
Weather
Digital
Label
Method
Gam.
Shot
Tmp.
Def.
Gla.
Mot.
7oom
Snow
Fro
Fog
Brit.
Cont.
Elas.
Pix.
JPFG
VacValdd
10.63
12.14
7.17
34,86
19.58
44.09
41.94
46.34
34.22
41.08
67.31
18.47
50.36
24.91
44.56
33.18
BN-Stats
66.14
53.81
58.15
56.63
TENT*
49.70
51.52
39.35
44.63
28.42
27.26
24.74
14.74
10.19
6.44
4.89
3.01
2.99
2.87
2.62
20.89
EATA
1.88
1.37
1.30
1.39
SAR"
46.57
55.41
48.54
66.29
51.01
65.27
68.49
60.72
62.35
63.35
71.08
69,50
$9.82
65.55
56.48
60,70
CoTTA*
39.22
40.76
34,10
66.11
42.48
63.57
64.83
53,80
$3.48
58. 15
68.22
64.45
53.85
56.66
45.17
$3.66
RaTTA*
36.68
40.73
36.00
62.54
44.58
62.90
66.93
$8.09
59.71
60.23
70.33
62.67
59.12
63.55
51.83
$5.86
SoTTA
44,39
38.67
23.70
29:92
20.45
26.89
31,05
26.63
25
33.69
16.99
24.57
27.67
24.82
27.96
SimATTA
28.93
41.54
28.94
39.79
34.83
49.11
55.42
46.59
$1.06
48.83
60.03
34,67
44.27
45.19
46.22
43.69
BITTA
50.12
58.34
52.07
63.27
$2.70
63.80
68.16
62.65
65.39
63.79
71,26
68.97
63,93
69.45
63.38
62.49
(b) CIFAR100-C.
Noise
Blur
Weather
Digital
Method
Gau.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Brit.
Cont.
Elas.
Pix.
JPBG
Avg.
Valld
6.66
8.91
3,68
16.03
10.05
26.62
27.16
29.29
33.40
11.26
30.76
1.96
27.83
40.67
47.96
21.48
BN-Sttas
32.28
34.27
22.21
32.63
22.02
44.56
46.11
39.31
43.27
31.66
46.01
10.19
43,51
52.14
50.62
36.72
TENT*
36.80
35.24
17.71
8.21
3.33
3.30
2.91
2.36
2.15
1.99
1.88
1.94
1.88
8.21
EATA"
35.19
37.98
25.35
35.82
24.84
47.04
48.19
42,10
45.53
36.67
48.79
9.16
44.71
53.43
$1.60
39.09
33.90
27.84
26.70
41.54
45.49
37.25
14.85
46.63
52.49
B
CaTTA*
32.27
34.26
22.21
32.62
22.04
44,56
46.11
39.30
43.27
31.67
46.01
10.18
43.51
52.18
50.60
36.72
RoTTA*
31.54
40.70
33.07
B
SaTTA
34.24
13.48
18.21
17.96
10.63
15.41
12.75
9.46
7.35
5.97
7.70
1.64
5.96
6.13
5.70
12.84
S
16.76
14.51
14.72
30.63
17.91
35.13
BITTA
34.84
39.88
28.56
35.37
26.65
48.41
49.57
43.62
47.90
39.53
$0.95
12.27
47.18
54.01
54.06
40.85
(c)
SimATTA, an active TTA baseline, also underperformed due to hard-three terd-thresholdng and clustering with incorrect samples.
In Figure 5, replacing an oracle feedback with a founda- tional model (GPT-4o) significantly degraded SimATTA's performance due to GPT-4o's high classification error (40% on average). This result underlines the current limitations of automatic feedback: while easily accessible, it is unreliable under distribution shifts. In contrast, approach leverages lightweight yet trustworthy binary feedback from an oracle, offering robust and cost-efficient alternative for real-world test-time adaptation.
To fairly evaluate binary feedback under labeling constraints, further compared BiTTA against full-class active TTA in two scenarios: (1) equal labeling cost adjusted for information gain (Figure 5), and (2) equal number of full- hewu/binar--eeddbac samples (Table 12). Scenarios are fur- ther explained in Appendix 1. In both scenarios, BiTTA (with binary feedback) consistently outperformed SimATTA (with full-class labels), demonstrating that binary feedback alone can drive more effective adaptation than full-class supervision under limited interactions. With equal labeling cost, BiTTA achieved up 32% higher accuracy, high- lighting its superior cost-efficiency and robustness.
We also evaluated BiTTA on broader distribution shifts in domain (continual TTA (Wang et al., 2022)) and mixed- stream scenarios (Gui et al., 2024) for a domain generaliza- tion task. Table 2 shows that BiTTA achieved the highest average accuracy across all baselines, demonstrating gener-

---

Test-Time Adaptation with Binary Feedback
Table 2: Accuracy (%) comparisons with TTA and active TTA baselines with binary feedback in PACS. The domain-wise data stream is a continual TTA setting, and the mixed data stream shuffled all domains randomly, where we report the cumulative accuracy at four adaptation points. Notation * indicates the modified algorithm to utilize binary-feedhare samples. B: TTA with binary feedback. Results outperforming all other baselines are highlighted in bold fonts. Comparison with non-active TTAs and full-class active TTA are in Table 13 in Appendix C.
Domain-wise data stream
Mixed data stream
Label
Meihod
Art
Cartoon
Sketch
Avg
25%
50%
75%
(00%)
STe
59.38
27.94
42.96
43.43 200)
42.74
42.80
42.64
42.77 a0.0
BN Stats
67.87 18
63.48 a
$4.07 1075
61.81 aous
59.09 10.N
58.28 in
58.05 u022
57.82 40.30
B
TENT'
71.26
$1.57
63.51 2033
sh.71
57.54 a0.33
56.21 at:15
B
EATA*
68.67 .0.78
65.31 aox
59.05 102
64,34 10u22
59.58 10.10
59.15 u0.55
59.26 a0.36
59.39 unnw
B
SAR"
67.95 10
-051
55.35 10.19
62.32 4011
59.17 4014
m0.0
58.41 -008
B
COTTA*
67.87 2018
63.47 20:00
54.07 20.36
61,80 sb.19
59.10 s0.32
58.29 s0.0
58.06 s0.23
57.83 a0
B
RoTIA*
64.73 20.20
55.14 21.91
56.05 #072
58.64 20.50
55.50 st.30
52.68 20.64
51.45 20.56
50.10 an:3
B
SoTTA*
69.73 mn
42.48
46.07
52.76
54.33
52.89
53.09
53.37
B
SimATTA
55.83
59.68 4788
72.40 +1.11
62.63 1961
59.34 4771
63.81 10.0
67.09 40:54
69:21 40:11
B
73,86 6336
76.03
75.57 28:33
59.65 20.00
69.23 20.17
72.18 20.8
100
+9%
90
90-
Accuracy (%)
Accuracy (%6
Accuracy (%)
+30%
80
80
60
Full-class
SimATTA oracle
+32%
70
70-
SimATTA GPT-40
BiTTA
BiTTA SiMATTA
Binary feedback
60
SimATTA
60
BITTA
10
20
30
0
C10-C
C100-C
T-IN-C
Feedback error ratio (%)
Feedback samples per batch k
Figure 5: Accuracy (%) with full-class feedback (SimATTA) and binary-feedback (BiTTA) and under the equal total label- ing cost. GPT-4o is used foundational model provide a full-class label.
Figure 6: Accuracy (%) varying the feedback error in CIFARIO-C. Binary feedback is flipped (correct ++ incor- rect) when a feedback error occurs.
Figure 7: Accuracy (%) varying the number of binary-feedbach samples per batch (k) in CIFARIO-C. Zero feedback sample equivalent to con- ventional TTA.
alization beyond corruptions.
fident unlabeled samples with labeled samples to reduce the impact of labeling errors, thus outperforming SimATTA.
Overall, these results show that TTA with binary feed- back is a practical and scalable TTA paradigm, and BITTA delivers state-of-the-art performance under min- imal supervision, outperforming full-class approaches in both accuracy and efficiency. Additional results, includ- ing large-scale and synthetic datasets (e.g., ImageNet- C (Hendrycks & Dietterich, 2019), ImageNet-R (Hendrycks et al., 2021). ColoredMNIST (Arjovsky et al., 2019). VisDA- 2021 (Bashkirova et al., 2022), and DomainNet (Peng et al., 2019)) and challenging settings (e.g., non-iid and single- sample), are available Appendix C.
Impact of number of feedback samples. We evaluated how the number of binary-feedback samples per batch (k) influences adaptation performance. As illustrated in Fig- ures 7 and 11, BiTTA maintains high accuracy even with sparse active samples. The performance improves creases, showcasing effective utilization of additional binary feedback. SimATTA shows similar trend of increasing accuracy with more active samples, but the overall perfor- mance is consistently lower than BiTTA. This suggests that BiTTA can effectively leverage additional feedback while maintaining stability in low budgets, indicating its potential for deployment in scenarios with varying labeling budgets.
in-
Impact of feedback errors. We assumed the binary feed- back provided by the oracle contained no labeling errors. In practice, user feedback might include labeling errors by shifting the binary feedback between correct and incorrect. We examine the impact of binary-feedback error compared with the active TTA baseline. Figure 6 shows that SimATTA has consistently low accuracy under labeling errors by re- lying on the noisy labeled samples without utilizing the unlabeled samples. In contrast, BiTTA combines many con-
Synergistic effect of adaptation strategies. We com- pared BiTTA against its components: Binary Feedback- guided Adaptation (BFA) and Agreement-based self- Adaptation (ABA). In CIFARIO-C, BFA-only adapta- tion achieved 58.90% and ABA-only adaptation achieved 82.64%, whereas BiTTA achieved on average 87.20% ac- curacy, consistently outperforming entire continual corrup-
8

---

Test-Time Adaptation with Binary Feedback
tions. The superior performance of the combined approach (BiTTA) indicates that BFA and ABA complement each other to achieve robust accuracy.
nence for fine-tuning large language models, and Shankar et al. (2024) demonstrates how lightweight binary feedback (e.g., thumbs up/down) can effectively guide model behavior. Similar approaches have been explored in vision and multi- modal research (Fan et al., 2023; Black et al., 2024; Le et al., 2022; Pinto et al., 2023). Recently, Reinforcement Learning with CLIP Feedback (RLCF) (Zhao et al., 2024) has been proposed to adapt fision-lanuuaae models. RLCF relies on the pre-trained CLIP model reward function, which may not be available or suitable for all domains or tasks. In contrast, our approach provides more general and flexible approach for test-time adaptation by effectively guiding the adaptation without relying on specific pre-traine trained models.
Uncertainty calibration of MC-dropout. Beyond accu- racy, we also analyze the calibration quality of MC-dropout compared with deterministic softmax outputs. Reliable cal- ibration is essential for effective sample selection in both BFA and ABA. We compute the expected calibration error (ECE) across CIFARION I0-C corruptions and find that MC- dropout achieves a substantially lower ECE of 0.062, com- pared with 0.100 from softmax-based confidence, a 38% reduction in miscalibration. We further compare with var- ious uncertainty estimation methods (e.g., augmentation, ensemble, and deterministic softmax confidence) in Table 6 (Appendix B), showcasing the importance of MC-dropout for the policy gradient modeling.
Active learning. Active learning (Cohn et al., 1994; Set- tles, 2009) involves an oracle (e.g., human annotator) in the machine learning process to enable efficient annotation and training. The active learning framework has been widely studied in active (source-free) domain adaptation (Ash et al., 2019; Prabhu et al., 2021; Li et al., 2022; Wang et al., 2023; Du & Li, 2024; Ning et al., 2021) and active TTA (Gui et al., 2024). Compared with active domain adaptation, active TTA focuses on the non-regrettable active sample selection on the continuously changing data stream without access to source data. Using binary feedback related to the active learning with partial feedback problem (Hu et al., 2019), which seeks to recursively obtain partial labels until a definitive label is identified. Joshi et al. (2010) proposed an active learning setup where users compare two images and report whether they belong to the same category. In contrast, our approach leverages single-step binary feedback on the model's 's current batch sample output without requiring additional data. This simplifies the process and reduces the labeling effort.
5. Related Work
Test-time adaptation. Test-time adaptation (TTA) im- proves model accuracy on distribution shift on the pre- trained model with only unlabeled test samples (Wang et al., 2021). Existing TTA focused on robust adaptation (Niu et al., 2023; Gong et al., 2022; Yuan et al., 2023; Wang
et al., 2022; Boudiaf et al., 2022; Niu et al., 2022; Gong
et al., 2023b; Park et al., 2024) across various types of dis- tribution shifts (Niu et al., 2023; Gong et al., 2022; Wang al., 2022: Gong et al., 2023b; Press et al., 2023). However, existing TTA methods suffer from adaptation failures dur- ing lifelong adaptation (Press et al., 2023), highlighting the need for a few-sample guide for robust adaptation. Active test-time adaptation (ATTA) (Gui et al., 2024) introduced a foundational analysis of active TTA setting, with a su- pervised learning scheme (SimATTA) using low-entropy source-like sample pseudo-labeling and active labeling from an incremental clustering algorithm. However, SimATTA is sensitive to the pre-trained model and selected active sam- ples, as it does not leverage most unlabeled samples and only utilizes a few labeled samples. In contrast, TTA with binary feedback utilizes a large set of unlabeled samples while guiding adaptation with beearc-feedback samples, per- forming stable adaptation.
6. Conclusion
We proposed test-time adaptation with binary feedback to address the challenge of adapting pre-trained models to new domains with minimal labeling effort. Our approach leverages binary feedback on the model predictions (correct or incorrect) from an oracle to guide the adaptation process, significantly reducing the labeling cost compared to methods that require full-class labels. Our method, BiTTA, uniquely combines binary feedback-guides adaptation on uncertain samples with agreement- based self-adaptation on confident samples in a reinforcement learning framework, balancing between a few labeled samples and many unlabeled samples. Through extensive experiments on distribution shift datasets, we demonstrated that BiTTA outperforms state-of -tet-art test-time adaptation methods, showcasing its effectiveness in handling continuous distribution shifts. Overall, test-time adaptation with binary feedback represents a significant step forward in test-time adaptation. offering a practical balance between performance and labeling efficiency.
Reinforcement learning for model tuning. Reinforce- ment learning (RL) has been successfully applied in various domains to incorporate non-differentiable rewards in the optimization process (Zoph & Le, 2017: Yoon et al., 2020: Ouyang et al., 2022; Fan al., 2023; Black et al., 2024). For example, Zoph& & Le (2017) and Yoon et al. (2020) em- ploy the REINFORCE algorithm to use the accuracy of the validation dataset as a mnon-iifferentialle) reward in neural architecture search or data valuation. In the domain of natu- ral language processing, reinforcement learning with human feedback (RLHF) (Ouyang et al., 2022) has gained promi-

---

Test-Time Adaptation with Binary Feedback
Acknowledgements
Boudiaf, M., Mueller, R., Ben Ayed, I., and Bertinetto, L. Parameter-free online test-time adaptation. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2022.
This work was supported by Institute of Information & com- munications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2025- 02215122, Development and Demonstration of Lightweight Al Model for Smart Homes), Institute of Information &
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert- Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Advances in Neural Information Processing Systems, 2020.
communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS- 1925, Detection and Prediction of Emerging and Undiscovered Voice Phishing), and Institute of Informa- tion & communications Technology Planning & Evalua- tion(IITP) grant funded by the Korea governmen((MSIT) (No. RS-2020-11201336, Artificial Intelligence Graduate School Program(UNISTT)
Chen, D.. Wang, D., Darrell, T., and Ebrahimi, S. Con- trastive test-time adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
Impact Statement
TTA with binary feedback and BiTTA offer significant so- cietal and practical benefits while presenting some impor- tant considerations for implementation. The use of binary feedback substantially reduces labeling costs, making ac- tive test-time adaptation more accessible to the end-users. This enables real-time adaptation in critical applications like autonomous driving, healthcare diagnostics, and medical applications; thereby improving safety, efficiency, and user experience. While the system enhances model performance in diverse and changing environments with minimal label- ing effort, it requires careful consideration of potential risks, including privacy concerns in surveillance applications and biased decision-making in applications like targeted adver- tising.
Cohn, D., Ghahramani, Z., and Jordan, M. Active learning with statistical models. Advances in neural information processing systems, 7, 1994.
Deng, J., Dong, W., Socher, R., Li, L.-J,, Li. K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference Computer Vision and Pat- lern Recognition, 2009.
Du, Z. and Li, J. Diffusion-based probabilistic uncertainty estimation for active domain adaptation. Advances in Neural Information Processing Systems, 2024.
Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, M., Boutilier, C., Abbeel, P., Ghavamzadeh, M., Lee, K., and Lee, K. Rein- forcement learning for fine-tuning text-to-image diffusion models. In Advances in Neural Information Processing Systems, 2023.
References
Arjovsky, M., Bottou, L., Gulrajani, I., and Lopez-Paz, D. Invariant risk minimization. arXiv preprint weXiv:197.002890 2019.
Foret, P., Kleiner, A., Mobahi, H., and Neyshabur, B. Sharpness-aware aware minimization for efficiently improving generalization. In International Conference on Learning Representations, 2021.
Ash, J. T., Zhang. C., Krishnamurthy, A., Langford, J., and Agarwal, A. Deep batch active learning by diverse, uncer- tain gradient lower bounds. In International Conference on Learning Representations, 2019.
Gal, Y. and Ghahramani, Z. Dropout as a bayesian approxi- mation: Representing model uncertainty in deep learning. In Proceedings of The 33rd International Conference on Machine Learning. 2016.
Bashkirova, D., Hendrycks, D., Kim, D., Liao, H., Mishra, S., Rajagopalan, C., Saenko, K., Saito, K., Tayyab, B. U., Teterwak, P., et al. Visda-2021 competition: Universal domain adaptation to improve performance on out-of- distribution data. In NeurlPS 202/ Competitions and Demonstrations Track. PMLR, 2022.
Gong, T., Jeong. J., Kim, T., Kim, Y., Shin, J., and Lee, S.- J. NOTE: Robust continual test-time adaptation against temporal correlation. In Advances in Neural Information Processing Systems, 2022.
Black, K., Janner, M., Du, Y., Kostrikov, I., and Levine, S. Training diffusion models with reinforcement learning. In International Conference on Learning Representations, 2024.
Gong, T., Jang, S. Y., Acer, U. G., Kawsar, F., and Min, C. Collaborative inference via dynamic compo- sition of tiny ai accelerators on mcus. arXiv preprint arXiv:240I.088. 283.,77, 2023a.
10

---

Test-Time Adaptation with Binary Feedback
Gong, T., Kim, Y., Lee, T., Chottananurak, S., and Lee, S.-J. SoTTA: Robust test-time adaptation on noisy data streams, In Thirty-seventh Conference on Neural Infor- mation Processing Systems, 2023b.
Le, N., Rathour, V. S., Yamazaki, K., Luu, K., and Savvides,
M. Deep reinforcement learning in computer vision: comprehensive survey, Artificial Intelligence Review, pp. 1-87, 2022.
Gui, S., Li, X. and Ji, Active test-time adaptation: Theo- retical analyses and algorithm. In International Con- ference Learning Representations (ICLR), 2024.
Lee, J., Jung, D., Lee, S., Park, J., Shin, J., Hwang. U., and Yoon, S. Entropy is not enough for test-time adaptation: From the perspective of disentangled factors. In The Twelfth International Conference on Learning Represen- lations, 2024a.
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang. S., Zhang. Z., Wu, Y., et al. Con- former: annvouutin-aaugeentee transformer for speech recognition. arXiv preprint arXiv:2005. 08100, 2020.
Lee, T., Chottananurak, S., Gong, T., and Lee, S.-J. AETTA: Label-free accuracy estimation for test-time adaptation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Puttern Recognition (CVPR), 2024b.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEFE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Li, D., Yang, Y., Song, Y.-Z., and Hospedales, T. M. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, 2017.
Hendrycks, D. and Dietterich, T. Benchmarking neural network robustness to common corruptions and perturba- tions. In International Conference on Learning Represen- tations, 2019.
Li, X., Du, Li, J., Zhu, L., and Lu, K. Source-frree active domain adaptation via energy -aaeey- locality preserving transfer. In Proceedings of the 30th ACM international conference multimedia, 2022.
Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al. The many faces of robustness: A critical analysis of cattof-diitriibtiinn generalization. In Proceedings of the IEEE/CVF international conference computer vision, 2021.
Li, Y., Xu, X., Su, Y., and Jia K. On the robustness of open-world test-time training: Self-training with dynamic prototype expansion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Hong, J., Lyu. L.. Zhou, J., and Spranger, M. MECTA: Memory-economin continual test-time model adaptation. In The Eleventh International Conference Learning Representations, 2023.
Liberis, E. and Lane, N. D. Differentiable neural network pruning to enable smart applications on microcontrollers. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(4):1-19, 2023.
Hu, P., Lipton, Z., Anandkumar, A., and Ramanan, D. Active learning with partial feedback. In International Confer- ence on Learning Representations, 2019.
Liu, W., Wang, X., Owens, J., and Li, Y. Energy-baser based duttif-distributtion detection. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems, 2020.
Jang, M., Chung, S.-Y., and Chung, H. W. Test-time adapta- tion via self- self-training with nearest neighbor information. In The Eleventh International Conference on Learning Representations, 2022.
MacKay, D. J. Information theory, inference and learning algorithms. Cambridge university press, 2003.
Nado, Z., Padhy, S., Sculley, D., D' Amour, A., Lakshmi- narayanan, B., and Snoek, J. Evaluating prediction-ting time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006. 1000066.0900 10963, 2020.
Joshi, A. J., Porikli, F., and Papanikolopoulos, N. Break- ing the interactive bottleneck in multi-class classification with active selection and binary feedback. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010.
Ning, M., Lu, D., Wei, D., Bian, C., Yuan, C., Yu, S., Ma, K., and Zheng, Y. Multi-ancho active domain adaptation for semantic segmentation. In Proceedings of the IEEF/CVE International Conference on Computer Vision, 2021.
Kim, Y., Yim, J., Yun, J., and Kim, J. Ninl: Negative learn ing for noisy labels. In Proceedings of the IEEE/CVF international conference on computer vision, 2019.
Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao, P., and Tan, M. Efficient test-time model adaptation with- out forgetting. In Proceedings of the 39th International Conference on Machine Learning, 2022.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.

---

Test-Time Adaptation with Binary Feedback
Niu, S., Wu, J., Zhang, Y., Wen, Z.,, Chen, Y., Zhao, P., and Tan, M. Towards stable test-time adaptation in dynamic wild world. In The Eleventh International Conference Learning Representations, 2023.
Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C. A., Cubuk, E. D., Kurakin, A., and Li, C.- L. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. In Advances in neural infor- mation processing systems, 2020.
Ouyang. L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. In Advances in neural information processing systems, 2022.
Song. J., Lee, J., Kweon, S., and Choi, S. Ecotta: Memory- efficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Putlern Recognition, 2023.
Park, H., Hwang, J., Mun, S., Park, S., and Ok, J. MedBN: Robust test-time adaptation against malicious test sam- ples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Torch maintainers and contributors. Torchvision: Py- torch's computer vision library. https://github. com httpppyttottiiiiioeeeeooep 2016.
Wang, D., Shelhamer, E., Liu, S., Olshausen, B., and Darrell, T. Tent: Fully test-time adaptation by entropy minimiza- tion. In International Conference on Learning Represen- lations, 2021.
Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., and Wang, B. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision, 2019.
Wang. F., Han, Z., Zhang, Z., He, R., and Yin, Y. Mhpl: Minimum happy points learning for active source free domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Pinto, A. S., Kolesnikov, A., Shi, Y., Beyer, L.,, and Zhai, X. Tuning computer vision models with task rewards. In International Conference Machine Learning, 2023.
Prabhu, V., Chandrasekaran, A., Saenko, K., and Hoffman, J. Active domain adaptation via clustering uncertainty- weighted embeddings. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2021.
Wang, Q.. Fink, O., Van Gool, L., and Dai, D. Continual test- time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Press, O., Schneider, S., Kümmerer, M., and Bethge, M. Rdumb: A simple approach that questions our progress in continual test-time adaptation. In Thirty-sevnnth Con- ference on Neural Information Processing Systems, 2023.
Williams, R. J. Simple statistical gradient- foolowin algo- rithms for connectionist reinforcement learning. Machine learning, 8:229-256, 1992.
Rusci, M., Capotondi, A., and Benini, L. Memory-driven mixed low precision quantization for enabling deep net- work inference microcontrollers. Proceedings of Ma- chine Learning and Systems, 2:326-335 2020.
Yoon, J., Arik, S., and Pfister, Data valuation using reinforcement learning. In International Conference on Machine Learning, 2020.
Yuan, L., Xie, B., and Li, S. Robust test-time adaptation in dynamic scenarios. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Saito, K., Kim, D.., Sclaroff, S., and Saenko. K. Universal domain adaptation through self supervision. In Advances in neural information processing systems, 2020.
Sakaridis, C., Dai, D., and Van Gool, L. Semantic foggy scene understanding with synthetic data. International Journal of Computer Vision, 126:973-992 99209999999999999999919999 2018
Zhang, M., Levine, S., and Finn, C. Memo: Test time ro- bustness via adaptation and augmentation. In Advances in Neural Information Processing Systems, 2022.
Settles, B. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin- Madison, 2009.
Zhao, S., Wang. X., Zhu, L., and Yang, Y. Test-time adap- tation with clip reward for zero-shot generalization in vision-language models. In The Twelfth International Conference on Learning Representations, 2024.
Shankar, S.. Lamfirescu-Pereine J., Hartmann, B., Parameswaran, A., and Arawjo, I. Who validates the validators? aligning ilm-assisted evaluation of Im out- puts with human preferences. In Proceedings of the 37th Annual ACM Symposiun on User Interface Software and Technology, 2024.
Zoph, B. and Le, Q. Neural architecture search with re- inforcement learning. In International Conference on Learning Representations, 2017.
12

---

Test-Time Adaptation with Binary Feedback
A. Discussion
Despite the promising results, the TTA with binary feedback setting and BiTTA method have limitations. First, the reliance on binary feedback, while reducing labeling effort, may still require substantial oracle involvement in scenarios with high data variability or rapid domain shifts. Although BiTTA robustly performs across various labeling scenarios (Figures 7, 11, and 12), future work may explore reducing oracle involvement by developing more advanced and dynamic sample selection strategies. Second, the computational overhead introduced by Monte Carlo dropout (Table 3) could be further reduced by efficient TTA (Hong et al.. 2023; Song et al., 2023) and on-device machine learning (Liberis & Lane, 2023; Rusci et al., 2020; Gong et al., 2023a). Finally, although our algorithm robustly outperformed with feedback errors (Figure 6), designing a method for specifically handling noisy or incorrect feedback remains an area for future research.
B. Additional experiments
88
80
90-
Accuracy (%)
Accuracy (%)
Accuracy (%)
80
84
60
70-
82
CIFAR1O-C
60
PACS
40
Ours
th
th
.5
th=0.9
th
th=0.99
80
Ours
Random
Cont
Entropy
Energy
50
0.5
1.0
1.5
2.0
Hyperparameter B
Figure& 8: Accuracy (%) comparison with hard thresholding (th). Ours dy- namically selects confident samples via agreement between the model and MC-dropout. Averaged over three random seeds.
Figure 9: Accuracy (%) varying the binary-feedback sample selection strategy in CIFARIO-C. Ours lever- ages MC-dropout to select the most uncertain samples. Averaged over three random seeds.
Figure 10: Accuracy (%) varying the balancing hyperparameter (5) in CIFAR10- and PACS. Another bal- ancing hyperparameter is set to 1. Averaged over three random seeds.
90
90
87.4
Accuracy (%)
Accuracy (%)
Accuracy (%)
85
85
BiTTA
87.2
80
SimATTA*
80
BiTTA
87.0
75
75
SimATTA
70
70
86.8
3
2
3 4 5
6
Number of skipped batches
Number of delayed batches
MC-Dropout Iterations (N)
Figure 11: Accuracy (%) varying the labeling skip in CIFARIO-C. Aver- aged over three random seeds.
Figure 12: Accuracy (%) varying the labeling delay (labeled samples arrive after certain batches). Averaged over three random seeds.
Figure 13: Accuracy (%) varying the number of dropouts in MC-dropout. Averaged over three random seeds.
Comparing prediction agreement with confidence thresholding. To assess the effectiveness of our prediction agreement method for confident sample selection, we compared it against fixed confidence thresholding approaches. We evaluated thresholds ranging from 0.8 to 0.99, with 0.99 being the value used in SOTTA (Gong et al., 2023b). Figure illustrates the performance of these approaches on unlabeled-only TTA in the continual CIFAR setting. Our prediction agreement method consistently outperformed all fixed thresholding approaches, which exhibited high variance and instability. This result demonstrates the superiority of our dynamic sample selection strategy, particularly in scenarios with continuously changing corruptions, highlighting the importance of adaptive confidence assessment in test-time adaptation.
13

---

Test-Time Adaptation with Binary Feedback
Table 3: Average wall-clock time per batch (s) comparisons with TTA and active TTA baselines with binary feedback in Mnn--mmagNeetC.. Notation indicates the modified algorithm to utilize binary-feedback samples. Averaged over three random seeds.
Valld
BN-Stats
TENT
EATA
SAR*
CaTTA
RoTTX*
SoTTA*
SimATTA*
BITTA
Avg.
0.18 2012
0.33 w0.00
1.0
0.98
1.02
26.63
1,68
1.25
45.45
4.19
Table 4: Average GPU memory consumption (MB). MECTA and gradient accumulation (GA) are applied to BiTTA. Notation * indicates the modified algorithm to utilize binary-feedback samples.
SreValid
BN-SSatt
TENT
EATA"
SAR*
CoTTA*
ROTTA*
SeTTA*
SimATTA*
BITTA
+MECTA
+GA
Avg.
2081
2696
3246
3239
3244
2966
3038
3229
2824
8304
6724
Impact of sample selection. We examined the impact of sample selection, including our MC-dropout certainty approach with random selection, maximum entropy (Saito et al., 2020), minimum confidence (Sohn et al., 2020), and minimum energy (Liu et al., 2020). In Figure 9, our method outperforms others by leveraging MC-dropout to estimate epistemic uncertainty. In contrast, naive methods may struggle with overconfidence in test-time scenarios, failing to prioritize samples that offer the most valuable information for model improvement. On the other hand, we observe that for large datasets (c.g.. ImageNet-C/F and VisDA-2021 I in Table 7), most predictions are already unconfident and incorrect, random selection is sufficient to capture uncertain samples for effective adaptation.
Sensitivity to balancing hyperparameter a. 3. We investigated the sensitivity of BiTTA to the balancing hyperparameter while fixing a = 1.0, which controls the trade-off between binary feedback-guided adaptation and agreement- baase self-adaptation. Figure 10 illustrates the overall accuracy across various values for both image corruption and domain adaptation datasets. The results demonstrate that BiTTA maintains consistent performance across wide range of 33 values, indicating robustness to this hyperparameter choice. This stability suggests that BiTTA can effectively deploy across different scenarios without extensive hyperparameter tuning.
Impact of intermittent and delayed labeling. To further understand the impact of the annotator's labeling budget, we conduct an experiment scenario where annotators skip delay the labeling of few batches (e.g., labeling only 1 out of consecutive batches). In Figures 11 and 12, we observe whereas the active TTA baseline (SimATTA) shows high accuracy degradation with batch skips and consistent suboptimal performance with delayed batches. The result supports the robustness of BiTTA in varying labeling scenarios, enabling a practical and scalable TTA.
BiTTA shows stable performance with minimal degradation,
Runtime analysis. To assess the practical applicability of BiTTA, we conducted a comprehensive runtime analysis by measuring the average wall-clock time per batch across different methods on the Tiny-ImageN dataset. Our results in Table 3 show that BiTTA requires 4. 49 #0.06 seconds per batch, positioning it between simpler TTA methods (0.33- and more complex approaches like CoTTA (26.63s) and SimATTA (45.45s). The runtime profile demonstrates that BiTTA achieves a favorable balance between computational cost and performance, particularly considering its significant accuracy improvements over faster baselines while maintaining substantially lower processing time than methods like SimATTA.
Memory analysis. We report the average GPU memory consumption across all CIFARIO-C corruptions (severity 5) in Table 4. Compared to baseline methods, BiTTA shows higher memory usage due to repeated forward passes for MC- dropout and the reinforcement learning formulation. To mitigate this, we evaluate two variants: BiTTA+MECTA (applying the memory-efficient continual test-time adaptation from MECTA (Hong et al., 2023)) and BiTTA+GA (using gradient accumulation to divide batch into multiple mini-batches). BiTTA+GA reduces memory usage up to 60% while retaining BiTTA's 's performance, demonstrating practical strategy for deployment memory
settings.
Impact of the number of epochs. To understand the BiTTA's 's performance under time-constrained environments, we examined how reducing training epochs affects adaptation accuracy on CIFARIO-C. We compared our standard 3-epoch configuration against reduced 1- and 2-epoch settings, adjusting learning rates proportionally (x3 and x1.5) to compensate for fewer update steps. Results in Table 5 show that BiTTA maintains robust performance even with fewer epochs. This consistent performance across epoch configurations demonstrates that BiTTA can effectively adapt to distribution shifts even under stricter computational constraints, offering flexibility in real-world deployment scenarios where faster adaptation may be preferred.
14

---

Test-Time Adaptation with Binary Feedback
Table 5: Accuracy (%) comparisons with varying epochs in CIFARIO-C (severity level 5). B: TTA with binary feedback. Averaged over three random seeds.
Noise
Blur
Weather
Digital
Lahel
Method
Imp.
Del:
Gla.
Fog
Brit.
Cout.
Avg.
B
BiTTA (spoch = 3)
76.78
84.24
78.75
87
77.39
88.38
91.36
89.42
90.72
90.30
94.65
92.62
86.15
92.42
87.24
87.20
B
spoch
76.92
84.29
78.61
86.99
77.20
88.36
91.51
89.31
90.58
90.30
94.51
92.70
85.77
92.08
87.50
87.11
spoch
90.37
87.15
Table Accuracy (%) comparisons with replacing MC-dropout uncertainty estimation with (1) sample augmentation, (2) ensemble method, and (3) original confidence in CIFARIO-C (severity level 5). B: TTA with binary feedback. Averaged over three random seeds.
Noiso
Blur
Weather
Digical
Label
Method
Gau.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Brii.
Cont.
Elos.
Pix.
JPEG
Avg.
B BiTTA
76.78
84,24
78.75
87.51
77.39
88.38
91.36
89.42
90.72
90.30
94,65
92.62
86.15
92.42
87.24
87.20
Augunettation
16,99
Ensemble
74.08
81.60
75.57
88.16
74.15
88.50
91.02
87.93
89.71
88.92
94.05
92.01
83.28
89.83
83.64
85.50
B Confidence
74.00
82.61
76.54
87.12
75.13
87.83
90.92
88.14
89.90
89.20
94,33
92.28
84.08
90.93
85.54
85.90
MC-dropout configuration. We used 4 dropout inferences (N = 4) for policy estimation in BÎTTA. To evaluate sensitivity to the number of MC-dropout inferences, we conducted an ablation study varying N from 1 to 6. As shown in Figure 13, BiTTA performs consistently well for all N > 1, demonstrating robustness to this hyperparameter. Even at = 1, MC-dropout is still active, introducing stochasticity essential for both core components: (1) BFA relies on uncertainty to select feedback samples, and (2) ABA leverages agreement between stochastic and deterministic predictions. contrast, removing dropout entirely leads 2.56% accuracy drop. Although gains from increasing N are modest, higher N improves uncertainty calibration, reducing the expected calibration error (ECE) from 0.142 at N = 1 to 0.064 at N 4. We thus adopt N 4 as balanced choice. In practice smaller N may be preferred for low-latency scenarios, while larger N may benefit reliability-critical deployments.
Replacing MC-dropout uncertainty estimation. To further understand the importance of MC-dropout uncertainty estimation, we compare BiTTA with replacing MC-dropout with augmentation-baser estimation (as in Wang et al. (2022); Zhang et al. (2022)), ensemble-based estimation (Jang et al., 2022), and simple deterministic softmax confidence. Results in Table 6 suggest that augmentation- based uncertainty appears less stable and overfits in the early adaptation stage, leading to suboptimal performance.
C. Additional results
Results on additional datasets. We conduct an additional experiment to evaluate the scalability of BiTTA across various datasets covered in recent works (Lee et al., 2024a; Niu et al., 2023; Gui et al., 2024; Chen et al., 2022): ImageNet- (Hendrycks Dietterich, 2019), ImageNet-R (Hendrycks et al., 2021), ColoredMNIST (Arjovsky et al., 2019), VisDA- 2021 (Bashkirova et al., 2022), and DomainNet (Peng et al., 2019). Results in Table demonstrate superior performance of BiTTA, especially on large-scale datasets such ImageNet-C. The key insight is that BiTTA formulates both binary feedback and unlabeled sample adaptation a single reinforcement learning objective, where the reward signals seamlessly guide the model's adaptation. Also, the use of MC-dropout provides a robust uncertainty estimate, while optimizing on MC-dropout prevents the TTA model from overfitting, therefore showing a stable adaptation in large-scale datasets.
Results on additional scenarios. Recent TTA works suggest a new scenario of (1) imbalanced/non-id label distribution, where ground-truth labels are temporally correlated (Niu et al., 2023; Gong et al., 2022), (2) and batch size (Niu et al., 2023). Note that SimATTA's clustering algorithm for sample selection is not applicable in scenarios where the memory capacity is limited only one image. Experiment results on CIFAR10-C (Table 8) suggest the robustness of our method over imbalanced label distribution and batch size by effectively utilizing reward signals from the binary feedback and unlabeled samples.
Comparison with recent baselines We incorporated new comparisons with recent TTA methods: DeYO (Lee et al., 2024a), OWTTT (Li al., 2023), and TAST (Jang et al., 2022). As shown in Tables and 10, BiTTA consistently
15

---

Test-Time Adaptation with Binary Feedback
Table 7: Accuracy (%) comparisons with TTA and active TTA baselines in additional datasets. Notation indicates the modified algorithm to utilize umary--feddbar samples. Results outperforning all other baselines are highlighted in bold fonts. Averaged over three random seeds.
Datasct
SreValid
BN-Stats
TENT
EATA*
SAR*
CoTTA"
RoTTA*
SoTTA*
SimATTA
BITTA
ImageNet-C
14.43
26.88
0.93
30.87
35.15
26.80
22.55
36.02
17.50
36.59
ImageNer-R
33.05
35.08
29.10
37.14
36.64
35.02
34.35
31.00
35.63
38.59
VisDA-201 2021
27.36
26.46
20.38
27.82
27.41
26.46
27.23
27.71
22.80
29.30
DomainNet
54.82
54.41
18,80
59.49
57.78
54.40
56.41
54.82
58.41
611.85
ColoredMNIST
50.49
45.59
44.92
45.59
45.74
45.60
48,90
59.45
93.66
96.75
Table 8: Accuracy (%) comparisons with TTA and active TTA baselines with binary feedback in online CIFAR 10-C (severity level 5) with additional scenarios. Notation indicates the modified algorithm to utilize binary-feedback samples. B: TTA with binary feedback. Results outperforming all other baselines are highlighted in bold fonts. Averaged over three random seeds.
Noise
Digital
Label
Method
Gam.
Shot
Def.
Gla.
Mot.
7aom
Snow
Fr.
Fog
Brit.
Cont.
Elas.
Pix.
JPFG
SreValid
24.01
30.91
22.36
55.00
53.44
66.99
63.74
78.01
68.41
73.92
91.34
34.30
76.77
46.26
73.05
57.70
StatStas
25.21
22.91
TENT*
20.00
21.27
19.56
26.77
19.19
26.54
25.76
24.94
24.66
26.50
28.03
26.66
22.14
23.88
20.98
23.79
EATA"
16.24
16.52
13.73
18.82
15.97
18.87
18.79
16.87
17.62
19.30
20.34
17.85
18.02
17.87
16.29
17.54
SAR*
22.95
23.57
21.36
30.06
21.44
29.52
28.81
21.38
28. 10
28.48
31.40
30.69
24.90
26.46
23.31
26.63
CoTTA"
22.76
23.36
21.14
29.99
21,42
29.48
28.84
28.10
29.43
31.34
30,85
24.92
26.43
23.22
26.58
RoTTA*
41.83
44.60
37.97
58.54
41.14
$7.40
57.79
$2.54
51.86
56.87
62.27
53.20
48.41
50.65
44.84
50.66
SoTTA*
67.03
71.31
61.84
83,96
66.01
82.23
84.47
78.62
78.48
82.94
87.74
77.29
74.07
76.94
72.12
76.34
SimATTA
59.05
68.67
44.43
84.96
67.46
83.36
84.99
81.75
82.87
83.83
89.11
72.28
76.15
81.90
73.41
75.62
BETTA
82.32
84,02
75.77
90.39
79.05
90.73
90.93
90.71
89.09
92.22
95.36
82.16
87.56
87.40
85.91
86.91
(a) Imbalanced (non (non-iid) label distribution.
Noise
Blur
Weatber
Digital
Label
Method
Gau.
Sbot
Imp.
Del:
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Brit.
Cont
Elus.
Pix.
JPBG
Avg.
VreValid
25.96
52.02
64.80
77.97
74.14
33.90
76.61
73.23
BN-Sttats
20.53
21.09
18.15
32.45
20.72
33.45
30.49
28.76
29.29
33.34
36.96
40.55
24.20
25.95
21.43
27.82
TENT*
10.50
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10,04
EATA
20.53
21.09
18.15
32.45
20.72
33.45
30.49
28.76
29.29
33.34
36.96
40.55
24.20
25.95
21.43
27.82
SAR*
20.56
21.12
18.29
32.51
20.86
33.59
30.67
29.12
29.51
33.68
37.52
41.15
24.70
26.57
21.98
28.12
CoTTA
20.54
21.09
18.15
32.44
20.70
33.45
30.49
28.75
29.28
33.33
36.95
40.55
24.20
25.95
21.42
27.82
ROTTA"
11.70
10.23
10.03
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.01
10.14
SOTTA
17.02
15.32
13.00
79.00
18.17
57.44
63.30
51.26
49.67
61.47
50.27
53.56
42.18
$2.14
45,92
BETTA
62.14
64.01
55.13
82.07
59.64
79.22
83.26
75.84
71.26
81.92
86. 13
31.94
71.34
73.80
67.73
70.17
(b) Batch size 1.
outperforms these methods under equal binary feedback conditions.
Results on additional architectures. To further examine the applicability of BiTTA in various model architectures, we experimented with ResNet50 and ViT-Base. Table 11 shows the overall result, where BiTTA still outperformed the baselines all corruptions, demonstrating the applicability of BiTTA over diverse models.
Comparison with original TTA and active TTA. In Tables 12 and 13, we compare BiTTA with original TTA (without binary-feedback samples) and original active TTA (with full-labeling) baselines. We first observe that comparing to TTA with binary feedback setting, unsupervised adaptation shows 1.14%p accuracy degradation on aver- age; showcasing the importance of binary-feedback for guiding model adaptation. Also, the experiment results demonstrate superior performance BiTTA, even outperforming the active TTA baseline (SimATTA, (Gui al., 2024)), showing the effectiveness of our RL-based adaptation with binary-foedback adaptation and agreement- adaptation. We consider this the drawback of SimATTA's strategy using source-like confident samples. Even with tuning the hyperparameters. the accuracy of source-like like samples is highly dependent the source- sennee-petrained model. This results in noisy predictions, hindering its applicability in various datasets and scenarios.
Comparison with enhanced TTA. Following the setting of SimATTA (Gui et al., 2024), we compare BiTTA with an enhanced TTA setting, which is unsupervised TTA baselines adapting on the fine-tuned model, which is tuned with an
16

---

Test-Time Adaptation with Binary Feedback
Table 9: Average accuracy (%) comparisons. Notation indicates the modified algorithm to utilize binary-feedback samples. Averaged over three random seeds.
Table 10: Average accuracy (%) comparisons under the OWTTT pre-trained model. Notation indicates the modi- fied algorithm to utilize binary-feedback samples. Averaged over three random seeds.
Label Method
CI0-C
C10D-C
T-N-CC
DeYO
84.41
40.67
Lahel
Method
C10-C
C100-C
TAST-BN*
75.53
29.13
17.05
OWTTT
54.63
29.10
40.85
B
OWTTT
31.24
3.39
B
BITTA
89.89
Table 11: Accuracy (%) comparisons with TTA and active TTA baselines with binary feedback in CIFARIO-C (severity level 5) with additional architectures. Notation * indicates the modified algorithm to utilize binary-feedback samples. B: TTA with binary feedback. Results outperforming all other baselines are highlighted in bold fonts. Averaged over three random seeds.
Noise
Blur
Weather
Digital
Gau.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Tog
Brit.
Cont.
Elas.
Pix.
JPEG
Avg.
SreValid
22.56
27.66
21.49
46.91
43.23
$5.29
54.62
66.90
53.91
61.31
84,94
24.24
65.29
41.03
65.35
48.98
BN-Stats
60.20
62.13
55.50
82.21
58.39
80.01
81.65
75.67
73.78
78.92
86.14
81.86
69.56
73.34
67.23
72.44
TENT
67.91
72.96
63.60
72.68
56.98
62.43
65.48
60.95
58.81
56.47
66.26
64.45
55.80
61.30
57.70
61.58
EATA"
75.19
80.89
73.29
81.65
67.68
76.30
79.09
75.80
77.09
76.19
82.23
79.64
68.67
74.07
70.10
75.62
63.51
70.85
65.95
85.07
84.06
82.
83.24
84.02
90.46
86,74
78.53
79.10
CoTTA*
60.20
62.13
55.50
82.20
58.40
80.01
81.65
75. 68
73.78
78.92
86. 14
81.87
69.55
73.36
67.20
72.63
79.68
81.19
SOTTA*
71.06
80.72
73.98
82.02
67.78
79.96
83.85
81.16
81.96
80.95
87.10
82.77
74.12
78.02
75.80
78.29
SiMATTA
49.99
62.69
77.35
80.75
79.95
72.59
78.84
BETTA
75.72
83.25
78.58
85.41
75.75
86.14
89.82
87.28
89.55
88.83
93.67
92.04
84,93
91.91
88.38
86.08
(a) ResNet50,
Noise
Blur
Weather
Digital
Label
Method
Gou.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Brit.
Cont.
Elas.
Pix.
JPEG
Avg.
VreValid
55.54
58.42
43.48
20.83
68.84
66.77
66.92
TENT
34.98
37.63
28.47
55.19
37.90
55.62
63.70
53.38
56.36
48.73
62.57
20.46
60.85
63.05
62.21
49.41
EATA"
55.23
61.37
51.56
63.45
62.71
69.75
74.29
67.57
68.86
58.67
72.04
29.22
68.66
70:93
69.05
62.89
SAR*
40.74
44.40
36.36
57.77
58.43
63.49
66.87
67.01
62.74
52.41
72.93
22.30
71.71
62.83
68.83
$6.59
CoTTA
39.84
41.83
34.89
58.54
56.56
58.43
60.38
57.69
43.46
72.36
20.82
68.90
66.77
66.94
$4.01
RoTTA*
38.89
38.69
30.53
61.44
$2.76
66.21
71.65
61.72
54.29
59.08
74.23
29.24
71.30
58.42
67.70
55.74
SOTTA*
54.59
62.22
51.71
63.62
61.67
67.45
72.72
66.28
67.09
58.19
70.80
30.92
67.18
70.71
67.34
62.17
SiMATTA
54.08
60.09
47.30
65.07
57.94
66.37
70.90
64.23
66.15
56.46
71.65
27.98
67.25
68.31
67.40
61.26
BITTA
53.83
62.40
52.23
66.77
63.88
72.27
77.66
69.97
72.40
65.10
76.07
30.70
73,80
74.84
72.65
65.64
(b) BaseB.
equal amount of binary-feedback samples before the adaptation phase. Table 14, we observe that BiTTA still outperforms over enhanced TTA baselines. The result necessitates the superiority of online adaptation binary feedback samples.
D. Experiment details
We conducted all experiments with three random seeds [0, 1, 2] and reported the mean and standard deviation values. The experiments were mainly conducted on NVIDIA RTX 3090 and TITAN GPUs.
D.1. Settings
Dataset. We utilized the corruption dataset (CIFARIO-C, CIFAR100-C, Tiny-ImageNet-CC (Hendrycks & Dietterich, 2019)) and domain generalization baselines (PACS (Li et al., 2017)). CIFAR COTTARO-C/CIFARI I0-CCCTiy----maaeeeeee-eee is a
10/100/200-class dataset of total of 150,000 images in 15 types of image corruptions, including Gaussian, Snow, Frost, Fog, Brightness, Contrast, Elastic Transformation, Pixelate, and JPEG Compression. PACS is 7-class dataset with 9,991 images in four domains of art painting, cartoon, photo, and sketch.
17

---

Test-Time Adaptation with Binary Feedback
Table 12: Accuracy (%) and standard deviation comparisons with original TTA and full-label active TTA baselines in corruption datasets (severity level 5). Full-label feedback active TTA, B: TTA with binary feedback. Results that outperform all baselines are highlighted in bold font. Averaged over three random seeds.
Noise
Blur
Weather
Digital
Label
Method
Gam.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Brit.
Coni.
Elas.
Pix.
JPEG
Avg.
Valid
$2.02
91.51
73.23
BN-Statt
66.96
69.04
60.36
87.78
65.55
86.29
87.38
81.63
80.28
855 K5.39
90.74
86.88
76.72
79.33
71.92
78.42
TENT
74.34
77.30
65.86
74.12
54.40
58.08
58.89
53.49
50.45
46.76
46.23
34.78
34.37
29.62
53.42
RATA
76.45
77.33
64, 70
77.51
62.31
71.91
78. 34
75.29
75.24
78.56
84.68
83.19
68.81
70.97
67.18
16.16
SAR
67.94
62,82
87.79
66.18
86.31
87.38
81.63
80.28
85.39
90.74
86.88
76.72
79.33
71.98
78.72
CoTTA
66.97
69.04
60.37
87.78
65.55
86.30
87.38
81.63
80.27
85.39
90.74
86.88
76.72
79.33
71.92
78.42
RoTTA
65.21
71.11
64.77
85.11
69.73
87.44
89.95
86.05
86.60
87.98
92.73
88.00
82.53
85,49
81.11
81.59
SOTTA
74.59
81.22
74.55
84.74
71.41
83.33
87.86
83.68
84.63
85.51
90.34
83.09
78.87
82.88
77.99
81.65
SimATTA
73.89
82.45
73.36
79.97
72.14
84.13
88.95
86.22
89.01
87.94
92.81
85,21
80.94
85,93
83.97
83.13
BITTA
76.78
84.24
78.75
87.51
77.39
88.38
91.36
89.42
90.72
90.30
94.65
92.62
86.15
92.42
87.24
87.20
(a) CIFARIO-C.
Digital
Label
Method
Imp.
Gla.
Zoom
Fro.
Fog
Brit.
Elas.
JPEG
Arg.
VacVa
10.63
12.14
7.17
34.86
19.58
44.09
41.94
46.34
34.22
41.08
67.31
18.47
50.36
24.91
44.56
33.18
BN-Sttats
39.23
40.75
34.10
66.14
42.46
63.57
64.82
53.81
58.15
68.
64.48
53.88
45.17
66
TENT
49.71
51.12
38.34
42.40
24.86
21.51
17.21
9.39
5.84
4.24
3.87
2.56
2.74
2.40
2.36
18.57
2.50
2.21
2.17
SAR
46.45
55,24
48.53
66.27
50.93
65.35
68.49
60.73
62.36
63.37
71.12
69.48
$9.76
65.34
56.33
60.65
53.87
RoTIA
35.63
40.04
35.55
60.32
42.09
62.76
67.53
58.54
60.60
60.72
71.58
64.08
$9.50
63.13
$4.49
$5.77
SoTTA
52.31
$7.80
48.30
48.82
63.45
68.17
61.69
62.62
69.73
57.40
56.67
SimATTA
42.86
$4.18
44,18
53.98
46.64
60.51
65.54
57.01
62.73
57.25
68.38
$2.17
$4.53
61.10
56.88
$5.86
BITTA
50.12
$2.07
52.70
65.39
63.79
71.26
63.93
63.38
(b) CIFAR100-C
Noise
Blur
Weather
Digital
Label
Method
Gou.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fro.
Fog
Brit.
Cont.
Elas.
Pix.
JPEG
Ave.
VacValid
6.99
8.93
5.09
15.18
9.65
26.50
26.33
29.77
33.0 33.64
12.34
31 31.80
2.34
27.71
34.99
46.97
21.22
BN-Staas
31.45
33.28
23.55
32.33
22.30
44.30
45.04
38.89
42.6
29.97
46.55
8.46
43,70
52.53
49.50
36.30
TENT
35.97
33.92
18.12
8.67
2.93
2.84
2.57
2.35
1.87
1.86
1.86
1.33
1.57
1.63
1.58
7.94
EATA
34.53
36.80
26.46
36.49
25.69
47.83
48.33
41.88
44.98
35.83
49.62
6.86
44,86
53.79
50.95
38.99
SAR
33.35
38.03
28.94
35.83
27.12
47.13
48.39
41.36
45.09
36.79
50.24
13.46
46.45
52.44
50.52
39.68
CoTTA
31:45
33.29
23.54
32.35
22.27
44.33
44.99
38.94
42.67
29.99
46.57
8.67
43,74
52.58
49.45
36.32
RoTTA
31.13
34.94
25.71
31.74
25.01
46.18
47.47
41.40
45.13
31.38
48.01
8.92
45.07
50.77
49.69
37.50
SoTTA
37.62
40.91
31.72
33.55
26.75
41.50
44. 84
37.72
41 42
38.75
47.04
7.46
34,88
44.08
45.04
36.89
SiMATTA
23.70
33.82
26.11
23.55
23.36
16
43.41
30.22
41.84
26.42
40.72
2.88
41.37
49.21
52.85
33,31
BITTA
34.84
39.88
28.56
35,37
26.65
48.41
49.57
43.62
47.90
39.53
50.95
12.27
47.18
54.01
$4.06
40.85
(c) Tiny-ImagNNetCC.
Source domain pre-training. We closely followed the settings and utilized the pre-trained weights provided by SoTTA (Gong al., 2023b) and SimATTA (Gui al., 2024). As the backbone model, we employ the ResNet18 (He et al., 2016) from Torch Vision (TorchVision maintainers and contributors, 2016). For CIFARIO-C/CIFAR 100-C/Tiny- Mo0-C/Tin-IImaeNeeee ImageNet-C. we trained the model with the source data with a learning rate of O. 1..1/.110..00 and a momentum of 0.9, with cosine annealing learning rate scheduling for 200 epochs. For PACS, we fine-tuned the pre-trained weights from ImageNet on the selected source domains for 3,000 iterations using the Adam optimizer with a learning rate of 0.0001.
Scenario. For the number of binary- samples, we used & = 3 samples from 64-sample test batch, accounting for less than 5% of the total data size. For the binary version of TTA basclines, we added cross-entropy loss (for correct samples) combined with complementary loss (for incorrect samples, (Kim et al., 2019)), maintaining an equal budget size to our method. To implement, we replace the original TTA loss lTTA with /TTA + Ice lacE, where Ice is cross-entropy loss on y/k log(1 (1- fo(k)=:) is the complementary cross-entropy loss (Kim et al., 2019) on
correct samples and loag > num class incorrect samples.
k=1
For enhanced TTA, we used the same binary version loss with an SGD optimizer with a learning rate of 0.001 and a batch size of 64. The number of fine-tuning epochs was set to 150 for PACS, 150 for CIFAR-10, 10, 150 for CIFAR-100, and 25
18

---

Test-Time Adaptation with Binary Feedback
Table 13: Accuracy (%) and standard deviation comparisons with original TTA and full-label active TTA baselines in PACS. The domain-wist wise data stream a continual TTA setting (Wang et al., 2022), and the mixed data stream shuffled all domains randomly, where we report the cumulative accuracy at each of the four adaptation points. F: Full-label feedback active TTA, B: TTA with binary feedback. Results outperforming all other baselines are highlighted in bold fonts. Averaged over three random seeds.
Domain-wise data stream
Mixed data stream
Label Meihod
Art
Cartoo-
Sketch
Avg
2.5%
50%
7.%
100%i(Avgg)
SicValid
59.38 at:0
27.94
42.96
43.43
42.74 s1.13
42.80 a112
42.64 20.90
42.77 ancl
BN Stats
67.87 -8.18
63.48 10
54.07 10:36
61.81 18
$9.09 10.24
58, 38.28 .n.8
58.05 10.22
57.82 a0:10
TENT
71.61 40.10
s0.51
56.75
53.22 20.57
49.64 s0.50
BATA
68.44 .m.11
64.90 un
58.58 in.s
63.97 un
59.60 10.15
58. 98 a115
$9.10 ac.35
59.24 ince
67,90
m083
-0.4
59.13 +0.11
58 58.49
58.32 aa
58.25 -no7
COTTA
67.87 20.18
63.48 20
54,06 s1.35
61.81 20
59.10 s0.32
58. 29 20.00
58.06 s0.23
51.83 sh:22
RoTTA
64.39 20.59
38.27 20
40.80 21.64
47.82 20
52.64 20.25
49.01 =0.85
46.87 29.55
45.75 2l:4
SoTTA
69.86
32.02
23.66
41.84
51.96
49.85
48,09 16
47.06
F
SimATTA
77.13 -0.76
71.46 47.47
78.80 48.53
75.80 40.7
68.27 41.24
72.67 40.45
75.41 +0.30
77.47 40.4
73.86 4376
20.17
72.18 a1s5
Table 14: Accuracy (%) comparisons with enhanced TTA on fine-tuned model (Gui et al., 2024) and TTA with binary feedback baselines on source model, in CIFARIO-C (severity level 5). Notation indicates the modified algorithm utilize binary-feedback samples. E: Enhanced TTA, B: TTA with binary feedback. Results outperforming all other baselines highlighted in bold fonts. Averaged over three random seeds.
Blur
Weather
Digital
Label
Method
Gau.
Shot
Imp.
Def.
Gla.
Mot.
Zoom
Snow
Fog
Cont.
Elas.
Pix.
JPEG
Avg.
SreValid
76.17
77.48
67.54
82.24
71.89
79.90
83.44
82.67
84.36
81.18
88.74
75.12
77.53
80.66
80.24
79.28
RN-Stars
77.90
79.66
71.76
86.52
85.26
84.07
90.10
86.70
79.39
84.76
78.98
E
TENT
77.52
76.94
63.79
68.35
52.67
56.00
55.58
52.93
49.02
45.02
43.94
33.46
32.12
31.39
29.27
51.20
77.18
75.32
70.73
75.81
69.52
E
SAR
77.90
79.66
71.76
86.52
73.53
85.26
86.77
84.66
85.27
84.07
90.10
86.70
79.39
84.76
78.98
82.36
COTTA
77.90
79.66
71.77
84.06
79.39
RoTTA
78.93
81.00
74,28
86.56
75.45
86. 18
88. 63
86.85
87.71
86.73
91.36
88.06
82.41
87.19
82.42
84.25
SoTTA
79.19
81.45
74.23
82.67
70.73
81.99
85.41
82.78
83.69
02.02
89.40
84.41
78.41
83.44
78.94
81.45
SimATTA*
70.21
81.67
71.49
79.59
69.41
82.
87.28
83.90
86.89
86.49
91.51
83.40
77. 77.94
83.81
82.25
81.09
76.78
78.75
87.51
88.38
90.30
92.62
92.42
for -eageNeCa Note that the hyperparameters were selected to maximize accuracy on the test data stream, which unrealistic since test data stream accuracy is not accessible during the fine-tuning process.
Comparison with active TTA. To compare BiTTA with full-label feedback methods, we propose two scenarios: (1) an cqual labeling cost and (2) an equal number of active samples. To compare with an equal labeling cost, we formulate the labeling cost with Shannon information gain (MacKay, 2003) log(p-') where p is the probability of selecting label. We assume the probability of each feedback strategy p 2-1 (correct/inorreet) and p num class (select in the entire class set). The final labeling cost for binary feedback is 1 for binary feedback and log(num class) ) for full-label feedback. Therefore, we utilize (enm_mmmlllasss times more feedback samples for TTA with binary feedback setting compared to active TTA.
D.2. TTA Baselines
TENT. For TENT (Wang et al., 2021), we utilize an Adam optimizer (Kingma & Ba, 2015) with learning rate LR = 0.001, aligning with the guidelines outlined in the original paper and active TTA paper (Gui et al., 2024). The implementation followed the official code.!
EATA. For EATA (Niu et al., 2022), we followed the original configuration of LR = 0.001, entropy constant Eg = 0.4 x In C, where C' represents the number of classes. Additionally, we set the cosine sample similarity threshold e = 0.5. trade-off parameter 3 2, 000, and moving average factor a 0.1. The Fisher importance calculation involved 2,000
Phttps://gt thub. com www.o//egaaaaagggttteeeeeeeeeeee
19

---

Test-Time Adaptation with Binary Feedback
samples, as recommended. The implementation followed the official code.?
SAR. For SAR (Niu et al., 2023), we set learning rate of LR = 0.00025, sharpness threshold = 0.5, and entropy threshold Fgg 0.4 x InC,, following the recommendations from the original paper. The top layer (layer for ResNet18) was frozen, consistent with the original paper. The implementation followed the official code."
CoTTA. For COTTA (Wang et al., 2022), set the restoration factor p 0.01, and exponential moving average (EMA) factor = 0.999, For augmentation confidence threshold Pth- we followed the previous implementation (Gui et al., 2024)
as Prh 0.1. The implementation followed the official code.* +
RoTTA For RoTTA (Yuan et al., 2023), we utilized the Adam optimizer (Kingma & Ba, 2015) with a learning rate of LR 0.001 and B 0.9. We followed the original hyperparameters, including BN-statistic exponential moving average
updating rate a 0.05, Teacher model's exponential moving average updating rate v 0.001, timeliness parameter X 1.0, and uncertainty parameter X 1.0. The implementation followed the original code.
SoTTA For SoTTA (Gong et al., 2023b), we utilized the Adam optimizer (Kingma & Ba, 2015), with a BN momentum of m = 0.2 and a learning rate of LR = 0.001. The memory size was set to 64, with the confidence threshold Co = 0.99. The L22norm constraint p was set to 0.5, aligning with the suggestion (Foret et al., 2021). The top layer was frozen following the original paper. The implementation followed the original code.
SiMATTA. We follow the original implementation of SimATTA (Gui et al., 2024). Since SimATTA queries active samples at dynamic rate, we set the centroid increase number to k = 3 and limit the budget per batch to 3, ensuring an equal active sample budget compared to BiTTA. For the adaptation objective, we add the complementary loss (incorrect samples, (Kim et al., 2019)) to the original cross-entropy loss for correct samples. For CIFAR- 10 and CIFAR-I00, we performed grid search to find the optimal hyperparameters. We found the optimal hyperparameters to be LR 0.0001/0.0004 eh 0.001 /0.001, and et 0.0001 0.0001/0.0000 /0.00001 for the CIFAR-10 and CIFAR-100 datasets, respectively. The implementation is based on the original code.?
BiTTA. During adaptation, we update all parameters, including BN stats, with an SGD optimizer with a learning rate/epoch of 0.001/3 (PACS), 0.0001/3 (CIFARIO-C CIFARI00-C), and 0.00005/5 (Tiny- (nny-ImmagNet-C) on the entire model. We applied weight decay of 0.05 to PACS and 0.0 otherwise. We applied stochastic restoration (Wang et al., 2022) in Tiny-ImageNet-C to prevent overfitting. We update batch norm statisties with momentum 0.3 on the unlabeled test batch, and freeze the statisties during adaptation, following Gui et al. (2024). We apply the dropout layer after residual blocks, following the previous work on TTA accuracy estimation (Lee et al., 2024b). With 4 dropout instances, we apply dropout rate of 0.3 for small-scale datasets (e.g., CIFARIO-C, CIFARI00-C, PACS) and 1 for large-scale datasets (e.g., Tiny-ImageNet-C. ImageNet-C).
E. License of assets
Datasets.
(Creative Commons Autribution 4.0 International), and Tiny-ImageNet-CC dataset 2Ap. The license of PACS dataset is not specified.
Torchvision for 8 (Apache 2.0), the official repository of TENT (MIT License), the official repository of EATA (MIT License), the official repository of SAR (BSD 3-Clause License), the official repository of CoTTA (MIT License), the official repository of RoTTA (MIT License), the official repository of SoTTA (MIT License), and the official repository of SimATTA (GPL-3.0 License).
Codes.
20


Reviews:
[{"review_id": "xsH8hKoeNl", "reviewer": "Reviewer_wnVe", "summary": "This paper explores a new setting of test-time adaptation, in which the authors introduce binary human feedback for test-time learning. The authors introduce MC-dropout for samples’ confidence estimation and then devise a unified test-time RL learning framework to exploiting both Human Feedback Rewards (for unconfident samples) and Self-Agreement Rewards (for confident samples). Experiments demonstrate the effectiveness and superiority of the proposed method. My detailed comments are as follows.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer wnVe for the thoughtful evaluation of our work and recognition of our novel problem setting and technically sound framework. \n\n**Clarification on Implementation Details.** While we provided the source code and additional implementation details in $\\text{\\color{blue}Appendix D}$, we will review them again to include all details.\n\n**Computational Complexity, Q8: Runtime Memory Comparison.** Thank you for the suggestions. During the rebuttal, we analyzed the memory consumption in $\\text{\\color{red}Table R1}$. While MC-dropout increases computational load, the overhead remains manageable (also acknowledged by Reviewer SHhn). Efficient techniques like MECTA ($\\text{\\color{blue}Hong et al., 2023}$) or gradient accumulation reduce runtime memory up to  60% with maintained accuracy.\n\n**Table R1**. Average GPU memory consumption (MB) in CIFAR10-C. MECTA and gradient accumulation (GA) are applied to BiTTA.\n\n| Method          | SrcValid    | BN-Stats | TENT* | EATA* | SAR* | CoTTA* | RoTTA* | SoTTA* | SimATTA* | BiTTA | BiTTA+MECTA | BiTTA+GA |\n|-----------------|-------------|----------|-------|-------|------|--------|--------|--------|----------|-------|----------------|----------------|\n| Avg.   | 2081     | 2696     | 3246  | 3239  | 3244 | 2966   | 3038   | 3229   | 2824     | 8304  | 6724        |  2841   |\n\n**Q1(1): Relation to Prior Work.** Prior work penalizes disagreement samples to lower their predictive confidence; we discard these instead, given their inherently unstable yet moderate accuracies ($\\text{\\color{blue}Figure 4(b)}$). \n\n**Q1(2): Advantages of using RL-based training.** In this rebuttal, we experimented with directly minimizing prediction disagreement via backpropagation, which dropped accuracy by  1.08%p. This highlights our RL-based formulation, which naturally leverages prediction agreement as a reward. By doing so, the model adaptively reinforces learning from confident samples while effectively disregarding unstable samples with disagreement, achieving stable and effective adaptation.\n\n**Q2: Ablation Study without Binary Feedback.** We did investigate this in “Synergistic effect of adaptation strategies” ($\\text{\\color{blue}Section 4, line 391}$). Adaptation without binary feedback achieved lower accuracy (82.64%) compared to BiTTA (87.20%), highlighting the importance of binary feedback.\n\n**Q3: Combination with Existing TTA Losses.** Thanks for the suggestion. During the rebuttal, we experimented by replacing the ABA loss with entropy minimization. Our RL framework (87.20%) outperformed the modified one (85.98%) by balancing the gradients in a unified framework.\n\n**Q4: Evaluation on Larger Datasets.** We did conduct experiments on ImageNet-C in $\\text{\\color{blue}Appendix B}$ ($\\text{\\color{blue}Table 6}$). BiTTA (36.59%) outperforms all baselines, including TENT (0.93%) and SimATTA (17.5%). Due to page limits, we initially placed them in $\\text{\\color{blue}Appendix C}$ but will move them to the main manuscript. Also, in this rebuttal, we ran additional baselines (e.g., DeYO), finding BiTTA consistently superior when introducing binary feedback ($\\text{\\color{red}Rebuttal for SHhn: Additional results on DeYO, OWTTT, and TAST}$). \n\n**Q5: MC-Dropout Configuration.** We used 4 dropout iterations (N=4) for policy estimation. During the rebuttal, we conducted an abalative study, which showed the robustness of BiTTA to the dropout iterations when N>1 ($\\text{\\color{red}Table R2}$).\n\n**Table R2**. Average accuracy (%) comparisons with varying MC-dropout inferences (N) in CIFAR10-C.\n\n| N   | 1     | 2     | 3     | 4     | 5     | 6     |\n|-----|-------|-------|-------|-------|-------|-------|\n| Avg.  | 86.93 | 87.19 | 87.18 | 87.20 | 87.29 | 87.25 |\n\n\n**Q6: Modification to Baseline Algorithms.** Baseline algorithms are modified to incorporate binary feedback by adding a cross-entropy loss on correct samples and complementary cross-entropy loss on incorrect samples. Details are in $\\text{\\color{blue}Appendix D}$.\n\n**Q7: Dropout Pre-training.** Models were not dropout pre-trained; we injected dropout at test time. During the rebuttal, we conducted a calibration analysis that MC-dropout has a lower calibration error than the original softmax ($\\text{\\color{red}Rebuttal for SHhn: MC-dropout vs. softmax probabilities and uncertain sample selection}$). Also, during the rebuttal, we experimented on replacing the MC-dropout with the softmax, which resulted in 2.56%p drop, further demonstrating the importance of MC-dropout.\n\n**Q9: Comparison with Fully Labeled TTA.** BiTTA outperforms fully labeled TTA (SimATTA) primarily because our approach effectively combines both binary feedback and unlabeled data, whereas SimATTA is highly dependent on source-like samples and prone to overfitting without unlabeled data.\n\nWe will incorporate this rebuttal in the final manuscript. **We hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.**", "claims_and_evidence": "Yes. The claims are well-supported.", "methods_and_evaluation_criteria": "Yes. The overall designs are reasonable.", "theoretical_claims": "No theoretical claims were provided.", "experimental_designs_or_analyses": "Please refer to my Questions section.", "relation_to_broader_scientific_literature": "The problem studied is a fundamental challenge in machine learning with good potential for practical applications.", "essential_references_not_discussed": "Please refer to my Questions section.", "other_strengths_and_weaknesses": "++Pros:\n\nThe setting of binary feedback-guided test-time adaptation is novel and interesting. The devised overall learning framework is technically sound, simple, and effective. \n\n--Cons:\n\nMany implementation details of the proposed method and modifications to baseline algorithms are difficult to find in the manuscript. Could the authors clarify these further?\n\nThe computational complexity is higher than that of conventional TTA methods like TENT.", "other_comments_or_suggestions": "No minor suggestions.", "questions_for_authors": "Q1 Relation to Prior Work: The agreement-based self-adaptation strategy, which maximizes the consistency between MC-dropout predictions and the original predictions, has been explored in prior work [Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting]. A more detailed discussion of the connections and differences with this related work would be beneficial. \n\nAdditionally, what are the performance implications or advantages of using RL-based training instead of directly minimizing prediction inconsistency via backpropagation?\n\nQ2 Ablation Study: Is there a clear ablation study evaluating the proposed method without Binary Feedback?\n\nQ3 Combination with Existing TTA Losses: How does the method perform when Binary Feedback is applied to unconfident samples, while conventional TTA losses (e.g., TENT, EATA, DEYO) are used for confident samples?\n\nQ4 Evaluation on Larger Datasets: Could the authors provide direct experiments on ImageNet-C in the main paper? Results on CIFAR-10, CIFAR-100, and Tiny-ImageNet are not sufficiently convincing for me. Moreover, in Tables 1-3, reporting results for the original baselines (including DEYO) without Binary Feedback would help to better understand the effectiveness of introducing Binary Feedback.\n\nQ5 MC-Dropout Configuration: For MC-Dropout, do the authors apply only a single dropout iteration to calculate confidence and agreement loss? Could the number of dropout samples impact the performance of the method?\n\nQ6 How are the baseline algorithms modified to incorporate Binary Feedback? \n\nQ7 Are all the adopted models pre-trained with Dropout? If not, I am concerned that the confidence score  C(x)  from MC-Dropout predictions may be very low. Would this impact the performance of the proposed method?\n\nQ8 Could the authors provide some runtime Memory Comparison?\n\nQ9 Why does the proposed method outperform fully labeled TTA? A more in-depth explanation of this phenomenon would be valuable.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mx5D1vkVuR", "reviewer": "Reviewer_QyDK", "summary": "The paper introduces BiTTA, a novel test-time adaptation (TTA) framework that leverages binary feedback (correct/incorrect) from annotators to address domain shifts. The key contribution is a dual-path optimization strategy combining reinforcement learning (RL)-guided adaptation on uncertain samples (BFA) and agreement-based self-adaptation on confident predictions (ABA).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer QyDK for the positive feedback on our work and for recognizing our paper's dual-path optimization strategy and its contributions.\n\n**Delays or inability to obtain timely feedback.** During the rebuttal, we conducted an additional experiment where active adaptation algorithms (BiTTA and SimATTA) receive the labeled samples in delayed batches (e.g., labeled instances arrive 1/2/3 batches later). The results in $\\text{\\color{red}Table R3}$ suggest that the delayed feedback shows a negligible impact on the performance of TTA with binary feedback, further enhancing the practicality of the feedback framework. We will include the result in the final manuscript.\n\n**Table R3**. Accuracy (%) comparisons under delayed feedback in CIFAR10-C. Zero delay is equivalent to the original experiment setting. Notation * indicates the modified algorithm to utilize binary-feedback samples.\n\n| Delay | 0     | 1     | 2     | 3     |\n|-------|-------|-------|-------|-------|\n| SimATTA* | 81.09 | 81.20  | 81.07  | 81.11  |\n| BiTTA | 87.20 | 87.14 | 87.22 | 87.28 |\n\n\nAlso, we did include experiments where annotators skipped labeling for several batches (e.g., labeling only 1 out of 4 consecutive batches, $\\text{\\color{blue}Figure 11}$ in $\\text{\\color{blue}Appendix B}$). The results show that compared with the baseline, BiTTA maintains stable performance even with such intermittent feedback, outperforming SimATTA (active TTA baseline) by 9.22%p. \n\n**Noisy/erroneous feedback scenarios.** We did include BiTTA under noisy/erroneous feedback scenarios in $\\text{\\color{blue}Figure 6}$ in $\\text{\\color{blue}Section 4}$, where BiTTA maintains robust performance even with up to 30% incorrect binary feedback, constantly outperforming SimATTA significantly by 7.81%p. \n\nWe will include these additional results in the final paper to address practical deployment concerns. We believe our TTA with binary feedback framework and proposed BiTTA algorithm is practical in real-world scenarios with delayed or noisy feedback. **Given the strengths highlighted in our submission, we hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.**", "claims_and_evidence": "The central claims are well-supported:\n\n**BiTTA outperforms SOTA methods**: Table 1 shows BiTTA’s accuracy improvements over TTA baselines.\n\n**Dynamic sample selection improves adaptation**: Figure 4 demonstrates ABA’s effectiveness in selecting confident samples via prediction agreement.", "methods_and_evaluation_criteria": "The dual-path RL framework is well-motivated, combining BFA (uncertain samples) and ABA (confident samples). MC dropout for uncertainty estimation aligns with recent TTA literature.", "theoretical_claims": "There is no theoretical claim in the paper.", "experimental_designs_or_analyses": "The experimental design is sound.", "relation_to_broader_scientific_literature": "The work connects to: Active TTA (SimATTA), RL with human feedback (RLHF) and Uncertainty estimation (MC dropout).", "essential_references_not_discussed": "No critical omissions detected in cited literature.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "My main concern is the practicality of the proposed RL-guided framework in real-world scenarios.\nThe proposed RL-guided framework requires real-time binary feedback from annotators.\nHowever, in practical deployment scenarios, there might be delays or inability to obtain timely feedback (e.g., annotator unavailability).\nHow would this affect BiTTA’s performance?\nCould the authors design experiments to simulate delayed/noisy feedback and evaluate robustness?", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OeiySMQCHd", "reviewer": "Reviewer_SHhn", "summary": "The common test-time adaptation methods focus on sample selection through softmax probabilities and further minimize the uncertainty-based loss on the target data. Different from this, the paper proposes to use binary feedback for test-time adaptation to determine adaptation. In contrast to the existing overall process of test-time adaptation, they also use binary feedback instead of full-label access, thereby preventing model adaptation on uncertain samples with agreement-based self-adaptation on confident predictions. Experiments with baselines compared to their setting reveal the effectiveness of their method. The ablation studies indeed provide additional insights for their method.\n\n## update after rebuttal\nThe authors have addressed my concerns about the MC-Dropout, softmax probabilities, and its rationale. Furthermore, the inclusion of comparisons to ensemble techniques significantly enhances the value of the paper, as it was previously incomplete without these comparisons.\n\nI recommend the authors include the text from the rebuttal to the paper, such as (i) MC-dropout vs. softmax probabilities and sample selection, (ii) Detailed details of MC-Dropout, including the text from the follow-up query, (iii) Table R4, Table R5 (both from rebuttal) (iv) justifying the number of MC-dropout iterations. Therefore, in response, I have increased my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer SHhn for the detailed review and thoughtful questions that helped us improve our work. We appreciate your recognition of our novel problem setup and methodology. \n\n**Clarification regarding Fig. 1 caption.** Traditional TTA methods indeed struggle with severe distribution shifts, which have been reported in the literature. In terms of the risk of adapting to incorrect predictions from unlabeled samples, we would tone down and further support with theoretical insights from ATTA ($\\text{\\color{blue}Gui et al., 2024}$), where labeled test instances can enhance the overall performance compared with unlabeled adaptation. We will update the caption to clarify this.\n\n**MC-dropout vs. softmax probabilities and sample selection (Q2).** MC-dropout offers more robust uncertainty estimates than standard softmax probabilities because (1) it captures epistemic uncertainty (model uncertainty) through multiple stochastic forward passes ($\\text{\\color{blue}Gal and Ghahramani, 2016}$) and (2) Softmax probabilities are known to be overconfident on out-of-distribution samples ($\\text{\\color{blue}Lee et al., 2024b}$).\n\nDuring the rebuttal, we compared the expected calibration error (ECE) between MC-dropout and original softmax probabilities. We found that MC-dropout confidence (avg. ECE 0.062) shows 33% lower ECE than the original softmax confidence (avg. ECE 0.100) during all corruptions, showcasing MC-dropout provides a more robust uncertainty estimate.\n\nAlso, in $\\text{\\color{blue}Figure 9}$ in $\\text{\\color{blue}Appendix B}$, we reported that our MC-dropout-based uncertain sample selection outperforms the original confidence-based sample selection, demonstrating a robust uncertainty estimation of MC-dropout.\n\n**Details of MC-dropout.** We used N=4 iterations of dropout inferences to implement MC-dropout. We will open-source the code and integrate details of MC-dropout in the final manuscript.\n\n**Usage of foundational models for feedback.** We agree that current state-of-the-art foundational models (e.g., GPT-4o) are erroneous for active labeling (please note that we used foundational models for generating full-class active labels in $\\text{\\color{blue}Figure 5}$). This necessitates lightweight labeling from an oracle, such as our proposed binary feedback. Please let us know if you need further clarification.\n\n**Questions about EATA performance and noise induction.** EATA's performance decreases on CIFAR-100-C compared to CIFAR-10-C. This phenomenon has also been observed in previous literature ($\\text{\\color{blue}Lee et al., 2024b}$). This explains the vulnerability of unlabeled adaptation; if an early adaptation stage leads to an erroneous direction of adaptation, simple entropy adaptation methods such as EATA and TENT can lead to model failures where most of the predictions become incorrect. Since model failures lead to over-confidence in incorrect samples, simple confidence/entropy thresholding in EATA cannot filter out wrong predictions.\n\n**Experiment setting for the impact of labeling errors.** For evaluating the impact of labeling errors ($\\text{\\color{blue}Figure 6}$), we randomly flipped binary feedback labels (correct↔incorrect) at specified rates (10%, 20%, 30%) to simulate annotation errors. \n\n**Additional results on DeYO, OWTTT, and TAST (Q1).** Thank you for suggesting these recent works. During the rebuttal, we experimented with DeYO and TAST in $\\text{\\color{red}Table R4}$ and OWTTT in $\\text{\\color{red}Table R5}$. We observed that our BiTTA still outperforms the latest baselines.\n\n**Table R4**. Average accuracy (%) comparisons. Notation * indicates the modified algorithm to utilize binary-feedback samples.\n\n|       | CIFAR10C | CIFAR100C | TinyINC |\n|-------|---------|----------|--------|\n| DeYO* | 84.41   | 61.30     | 40.67  |\n| TAST-BN* | 75.53   | 29.13    | 17.05  |\n| BiTTA | 87.20    | 62.49    | 40.85  |\n\n\n**Table R5**. Average accuracy (%) comparisons under the OWTTT pre-trained model. Notation * indicates the modified algorithm to utilize binary-feedback samples.\n\n|        | CIFAR10C | CIFAR100C |\n|--------|---------|----------|\n| OWTTT  | 54.63   | 29.10    |\n| OWTTT* | 31.24   | 3.39     |\n| BiTTA  | 89.89   | 64.06    |\n\n\nIn addition, during the rebuttal, we experimented on replacing MC-dropout with an ensemble structure from TAST. The ensemble method showed 1.7%p lower accuracy than the original MC-dropout-based algorithm. This emphasizes that while our dual-path RL optimization framework is flexible to incorporate ensemble methods (e.g., TAST), our proposed MC-dropout policy estimation shows the best performance.\n\n**Rebuttal on Other Comments or Suggestions.** Thank you for pointing out typos and suggesting improvements. We will update the final manuscript.\n\n\nWe will revise our final manuscript to reflect the rebuttal. **Given the strengths highlighted in our submission, we hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.**", "claims_and_evidence": "– The Fig 1 caption says that Test-time adaptation algorithms fail under severe distribution shifts due to the fundamental risk of adapting to unlabelled samples. While the first half of the statement is reflected in several works, the reasoning for this is not written in the second half. In fact, it is not at all relevant to state it here \n– The claim that MC-dropout offers a robust uncertainty estimate, while softmax probabilities exhibit overconfidence, needs further evidence. It can be agreed that sometimes softmax probabilities are too confident about the wrong predictions and vice versa. However, I am curious to know how MC dropout would mitigate this. Does over-sampling help?", "methods_and_evaluation_criteria": "The usage of foundation models by assuming all the predictions for the binary feedback is erroneous in nature. Perhaps, it's hard to find the oracle solution for all kinds of distribution shifts.", "theoretical_claims": "Yes, the paper doesn't include alot of theory sections.", "experimental_designs_or_analyses": "– With the increase in classes (Cifar-10 to Cifar-100) from Table 1, why does the performance of the baselines such as EATA decrease drastically\n\n– What was the experiment setting for the Impact of labelling errors? How was the noise in labeling induced? I would urge the authors to include more baselines in this\n\n– How many forward passes were used for the MCMC dropout? I would like to see the detailed implementation details for the usage of the dropout.", "relation_to_broader_scientific_literature": "The overall contribution of the problem setup is novel. The proposed methodology contains the usage of techniques that are not novel themselves but novel to the problem setting.", "essential_references_not_discussed": "No. The authors have discussed the related work appropriately. Moreover, the paper's method is novel for the setting of test-time adaptation.", "other_strengths_and_weaknesses": "Strengths: \n– The overall idea of using Reinforcement learning for test-time adaptation sounds interesting. \n\n– I find the idea of using a few samples with ground truth binary feedback innovative!\n\n– I don't find the computational overload-induced due to Monte Carlo to be a significant issue. This can be fixed with efficient sampling optimization techniques. \n\nWeaknesses:\n– I urge to include more recent works in the tables, including DeYo [1] and OWTTT [2]. \n\nReferences:\n[1] Lee, Jonghyun, et al. \"Entropy is not enough for test-time adaptation: From the perspective of disentangled factors.\" arXiv preprint arXiv:2403.07366 (2024).\n[2] ​​Li, Yushu, et al. \"On the robustness of open-world test-time training: Self-training with dynamic prototype expansion.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.", "other_comments_or_suggestions": "Dynamically adopting → Dynamically adapting \n\nPACS dataset is used commonly for domain generalization not domain adaptation. \n\nThe authors cite too many results during the introduction and methodology. While it may provide evidence, it's not reader-friendly. Instead, the key message be stated with hyperlinks", "questions_for_authors": "– Dropout [1], especially with multiple forward passes, acts as an ensemble consisting of multiple networks within the encoder. Therefore, I would like to see comparisons to works that also utilize ensembles for adaptation. For instance, TAST [2] uses ensemble networks for test-time adaptation. \n\n– Using confidence with argsort to obtain samples (Equation 5) might as well face the same issue as common test-time adaptation methods. Since the setting is unsupervised, it is not certain to what extent the predictions can be useful. \n\nReferences: \n[1] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning (Chapter 7.2). Cambridge, MA: MIT Press.\n[2] Jang, Minguk, Sae-Young Chung, and Hye Won Chung. \"Test-time adaptation via self-training with nearest neighbor information.\" arXiv preprint, ICLR 2022", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "c2JlifwqzB", "reviewer": "Reviewer_R67W", "summary": "The paper introduces binary feedback-guided test-time adaptation (BiTTA), a novel TTA framewrok designed to adapt deep learning models to domain shifts at test time using binary feedback from human annotators. The authors address limitations in prior active TTA methods, which suffer from high annotation costs, especially in complex multi-class settings. BiTTA mitigates this challenge by incorporating recent reinforcement learning concepts and agreement-based self-adaptation. It introduces Binary Feedback-guided Adaptation (BFA) for selecting uncertain samples and Agreement-Based Self-Adaptation (ABA) for refining confident ones, effectively enhancing model performance on challenging test distributions. Experiments across multiple datasets demonstrate that BiTTA outperforms existing TTA methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer R67W for the comprehensive review and for highlighting both the strengths and potential areas of improvement in our paper. We appreciate your recognition of our reduced labeling costs and dual-path optimization framework.\n\n**Open-sourcing.** We truly agree with your suggestion. We have already included source code zip files in the supplementary materials in the original submission, and we will open-source the code and include the corresponding repository link in the final version.\n\n**Theoretical Analysis.** We thank the reviewer for highlighting the value of a theoretical comparison to full-label adaptation methods. Our theoretical analysis between binary feedback and full-label feedback from an information-theoretic perspective is in $\\text{\\color{blue}Appendix D.1}$, which shows that binary feedback provides log(num_classes) times less information than full-class labeling per sample, offering lightweight labeling costs. Our empirical results also suggest that binary feedback provides enough information to drive effective adaptation under domain shift. This is further supported by a comparison with full-class labeling under equal labeling cost, where BiTTA shows superior performance over the full-label active TTA baseline ($\\text{\\color{blue}Figure 5}$). These insights imply that BiTTA retains strong adaptation capabilities despite more limited supervision, supporting its practicality in real-world scenarios where annotation time is constrained.\n\n**Scalability and Robustness.** Please note that we did include detailed per-corruption results for ImageNet-C in $\\text{\\color{blue}Table 6}$ in $\\text{\\color{blue}Appendix B}$. We will move them to the main paper in our final draft. In summary, BiTTA achieves 36.59% on ImageNet-C, outperforming all baselines by 12.0%p on average, demonstrating its scalability to larger datasets. \n\nRegarding high model error scenarios, our experiments on challenging datasets (Tiny-ImageNet-C with source accuracy 21.48%) show that BiTTA still provides substantial improvements (+19.37%p), highlighting its robustness to high-error settings. This is because our agreement-based adaptation (ABA) can leverage samples with reliable prediction, complementing many \"incorrect\" labeled samples from binary feedback samples, leading to stable adaptation. \n\n**Frequency of Human Intervention.** We did include additional experiments where we reduced the frequency of human intervention (e.g., labeling only 1 out of 4 consecutive batches) in $\\text{\\color{blue}Figure 11}$ in $\\text{\\color{blue}Appendix B}$. Results show that BiTTA maintains strong performance even with significantly reduced intervention frequency (outperforms the baseline by 9%p on average), suggesting that practical implementations could use less frequent feedback while maintaining performance benefits. During the rebuttal, we further examined the impact of delayed feedback where binary feedback arrives after a few batches ($\\text{\\color{red}Rebuttal for QyDK: Delays or inability to obtain timely feedback}$). The results showed that delayed feedback can derive the same level of adaptation performance. This showcases the practicability of our system.\n\nWe appreciate your constructive feedback and will carefully revise our manuscript accordingly. **Given the strengths highlighted in our submission, we hope our rebuttal has addressed your concerns. Please let us know if you have any further questions.**", "claims_and_evidence": "The authors claims the limitations in prior active TTA methods, which suffer from high annotation costs, especially in complex multi-class settings. BiTTA mitigates this challenge by incorporating recent reinforcement learning concepts and agreement-based self-adaptation. Experiment results show that BiTTA successfully utilizes binary annotations to enhance model performance during test-time, achieving results competitive with methods employing full annotations, validating the central claim.", "methods_and_evaluation_criteria": "BiTTA mitigates this challenge by incorporating recent reinforcement learning concepts and agreement-based self-adaptation. It introduces Binary Feedback-guided Adaptation (BFA) for selecting uncertain samples and Agreement-Based Self-Adaptation (ABA) for refining confident ones, which is reasonable. The authors assessed BiTTA across a range of benchmarks and settings: CIFAR10-C, CIFAR100-C, Tiny ImageNet-C, and continual and imbalanced test-time adaptation scenarios.", "theoretical_claims": "This paper doesn’t include a lot of proofs or theoretical claims. This work prioritizes experimental validation.", "experimental_designs_or_analyses": "I have checked the experimental designs, and think it is reasonable with a range of benchmarks and settings: CIFAR10-C, CIFAR100-C, Tiny ImageNet-C, and continual and imbalanced test-time adaptation scenarios.", "relation_to_broader_scientific_literature": "The authors claims the limitations in prior active TTA methods (e.g. SimATTA), which suffer from high annotation costs, especially in complex multi-class settings. BiTTA mitigates this challenge by incorporating recent reinforcement learning concepts and agreement-based self-adaptation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "Strengths:\n\n1. Reduced Labeling Costs: BiTTA minimizes labeling demands by using binary feedback from human annotators instead of requiring full-class labels. This significantly reduces the annotation burden, making it more feasible for real-world scenarios than current ATTA methods.\n\n2. Dual-Path Optimization with Reinforcement Learning: Binary Feedback-guided Adaptation (BFA) for uncertain samples and Agreement-Based Self-Adaptation (ABA) for confident samples. Introducing reinforcement learning by binary feedback optimization is interesting and novel in TTA.\n\n3. Strong Experimental Results: BiTTA consistently outperforms competing TTA methods and even surpasses the ATTA method with full-class labels in certain settings.\n\nWeaknesses:\n1. Theoretical Analysis: The paper's empirical validation is strong, but a theoretical analysis, particularly comparing BiTTA to full-label methods, would significantly enhance its contribution and provide deeper insights.\n\n2. Scalability and Robustness: While ImageNet-C experiments demonstrate scalability, detailed per-corruption results in Table 1(d) would be beneficial. Furthermore, the paper should have more discussions on how BiTTA addresses robustness in high model error scenarios.\n\n3. Frequency of Human Intervention: The requirement of annotating 3 samples per 64-sample batch implies a high annotation burden and continuous human availability. It means every batch of samples needs interventions. Exploring strategies to reduce the frequency of human intervention would improve the system's practicality.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This work incorporates simple human feedback into test-time adaptation in the form of binary correct/incorrect supervision. This feedback addresses failures of purely unsupervised TTA when shifts are too extreme. While the proposed method requires more information, in the form of this correct/incorrect binary feedback, it does improve adaptation results so there is sufficient return from this requirement. Experiments evaluate the proposed BiTTA on common benchmarks like CIFAR-10-C, CIFAR-100-C, and PACS, but ImageNet-C is notably missing from the main paper (with Tiny-ImageNet-C taking its place). Accuracy improves by 10+ points on CIFAR-100-C even in a comparison that augments baseline TTA methods to also incorporate feedback. Reviewers raised concerns about (1) the requirement of immediate feedback (which is indeed a practical issue for a test-time adaptation method) [QyDK], (2) insufficient implementation detail for the method and augmented baselines [wnVe], (3) missing baselines for recent methods [SHhn], and (4) concerns about lack of theory, scalability, and practicality given the rate of feedback [R67W]. However the author rebuttal and additional results satisfied these concerns: SHhn comments and raises to 4, R67W made an official comment (not rebuttal comment) to confirm maintaining 3, and all reviewers acknowledged the rebuttal. There is consensus for acceptance with scores of 4, 3, 3, 3 and the area chair agrees with acceptance.

The area chair acknowledges the confidential comment by the authors and confirms the decision incorporates all of the submission content, details, and threads.
Decision: Accept (poster)