ICML 2025

Ranked Entropy Minimization for Continual Test-Time Adaptation
Jisu Han 1 Jaemin Na 2 Wonjun Hwang 3
Abstract
Test-time adaptation aims to adapt to realistic en-
vironments in an online manner by learning dur-
ing test time. Entropy minimization has emerged
as a principal strategy for test-time adaptation
due to its efficiency and adaptability. Neverthe-
less, it remains underexplored in continual test-
time adaptation, where stability is more impor-
tant. We observe that the entropy minimization
method often suffers from model collapse, where
the model converges to predicting a single class
for all images due to a trivial solution. We pro-
pose ranked entropy minimization to mitigate
the stability problem of the entropy minimiza-
tion method and extend its applicability to con-
tinuous scenarios. Our approach explicitly struc-
tures the prediction difficulty through a progres-
sive masking strategy. Specifically, it gradually
aligns the model’s probability distributions across
different levels of prediction difficulty while pre-
serving the rank order of entropy.
The pro-
posed method is extensively evaluated across var-
ious benchmarks, demonstrating its effectiveness
through empirical results. Our code is available
at https://github.com/pilsHan/rem
1. Introduction
The real world is non-i.i.d., which demands real-time adap-
tation of AI applications.
Deep learning models have
achieved remarkable progress in recent years; however,
performance degradation caused by distribution shifts be-
tween different domains limits the generalization capabili-
ties (Shimodaira, 2000). Test-time adaptation (TTA) (Wang
et al., 2021) has emerged as a practical approach to address
non-stationary environmental changes by enabling models
trained on source domain to adapt in an online manner to
unlabeled target data during the test-time.
1Ajou University 2Korea Telecom 3Korea University. Corre-
spondence to: Wonjun Hwang <wjhwang@korea.ac.kr>.
Proceedings of the 42 nd International Con...