Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation
Authors: Zixi Wang
Yushe Cao
Yubo Huang
Jinzhu Wei
Jingzehua Xu
Shuai Zhang
Xin Lai
Venue: NeurIPS 2025 poster ()
Keywords: Gradual Domain Adaptation
Adversarial Domain Adaptation
Distribution Shift
Out-of-distribution Generalization
Primary area: general_machine_learning
TL;DR: The paper introduces a dynamic self-training method for gradual domain adaptation that effectively transfers knowledge across domains.

Abstract: In this paper, we propose a new method called \textit{Self-Training with Dynamic Weighting} (STDW), which aims to enhance robustness in Gradual Domain Adaptation (GDA) by addressing the challenge of smooth knowledge migration from the source to the target domain. Traditional GDA methods mitigate domain shift through intermediate domains and self-training but often suffer from inefficient knowledge migration or incomplete intermediate data. Our approach introduces a dynamic weighting mechanism that adaptively balances the loss contributions of the source and target domains during training. Specifically, we design an optimization framework governed by a time-varying hyperparameter $\varrho$ (progressing from 0 to 1), which controls the strength of domain-specific learning and ensures stable adaptation. The method leverages self-training to generate pseudo-labels and optimizes a weighted objective function for iterative model updates, maintaining robustness across intermediate domains. Experiments on rotated MNIST, color-shifted MNIST, portrait datasets, and the Cover Type dataset demonstrate that STDW outperforms existing baselines. Ablation studies further validate the critical role of $\varrho$'s dynamic scheduling in achieving progressive adaptation, confirming its effectiveness in reducing domain bias and improving generalization. This work provides both theoretical insights and a practical framework for robust gradual domain adaptation, with potential applications in dynamic real-world scenarios.

Self-Training with Dynamic Weighting for Robust Gradual Domain Adaptation
Zixi Wang"
Yushe Cao" Tsinghua University Beijing. China
Yubo Huang" Zhenguan Al Lab Shenzhen, Guangdong, China Southwest Jiaotong University Chengdu, Sichuan, China
University of Electronic Science and Technology of China Chengdu, Sichuan, China
Jinzhu Wei Shanghai University
Jingzehua Xu
Tsinghua University
Shanghai, China
Beijing, China xjzh23@mails. ttsaa2mmmails...snnuu....mm edugg
wbiozhu@gmail.com
Shuai Zhang
Xin Lai
New Jersey Institute of Technology Newark, NJ, United States
Southwest Jiaotong University Chengdu, Sichuan, China
Abstract
In this paper, we propose new method called Self-Training with Dynamic Weight- ing (STDW), which aims enhance robustness in Gradual Domain Adaptation (GDA) by addressing the challenge of smooth knowledge migration from the source to target domain. Traditional GDA methods mitigate domain shift through intermediate domains and self-training but often suffer from inefficient knowledge migration or incomplete intermediate data. Our approach introduces dynamic weighting mechanism that adaptively balances the loss contributions the source and target domains during training. Specifically, we design an opti- mization framework governed by time- darevigrying hyperparameter @ (progressing from 0 to 1), which controls the strength domain-specific learning and ensures stable adaptation. The method leverages self-training generate pseudo-labels and optimizes a weighted objective function for iterative model updates, main- taining robustness across intermediate domains. Experiments on rotated MNIST, color-shifted MNIST, portrait datasets, and the Cover Type dataset demonstrate that STDW outperforms existing baselines. Ablation studies further validate the critical role of g's dynamic scheduling in achieving progressive adaptation, confirming its effectiveness in reducing domain bias and improving generalization. This work provides both theoretical insights and practical framework for robust gradual domain adaptation, with potential applications in dynamic real-world scenarios. The code is available at https:: //github wwwmcoo//eeeeeeee honm/Daawwig////
Introduction
"Equal contribution.
1 "Co-Corresponding author. Corresponndig author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Source
Intermodiate
Targot
The phenomenon of domain shift presents fundamental challenge machine learning sys- tems, where the statistical properties of training (source) and deployment (target) domains ex- hibit significant divergence 2 As illustrated in Figure [T, this distributional mismatch fre- quently arises in practical applications where tar- get domain annotations are either prohibitively expensive or fundamentally unavailable. Do- main adaptation (DA) methodologies B have emerged as a critical paradigm for addressing this challenge, particularly through the frame-
C
C
Co
Figure I: Overview of gradual domain adaptation. The classifiers perform accurately in classifying the sample points in the current domain and the neighboring domains, but fail when classifying the samples from the Source directly to the Target domain.
work of unsupervised domain adaptation (UDA) (5] However, conventional UDA approaches demonstrate limited efficacy when confronted with substantial domain discrepancies or when transi- tional domain data is sparse absent.
Recent theoretical and empirical advances have established Gradual Domain Adaptation (GDA) as promising alternative for mitigating these limitations. The GDA framework, formalized by [6] and extended by 77. implements a sequential knowledge migration mechanism that systemati- cally reduces the inter-domain discrepancy through intermediate transitional phases. Although this paradigm demonstrates improved stability compared to direct adaptation approaches (8), current GDA implementations exhibit inefficient information flow between adjacent domains, resulting in gradual performance degradation throughout the adaptation trajectory [9]. Moreover, the sequential nature of GDA introduces compounding approximation errors, particularly when transitioning through numerous intermediate domains [10].
This paper proposed Self-Training with Dynamic Weighting (STDW), identify and address three critical challenges in existing GDA approaches: (1) suboptimal knowledge propagation across do- mains, (2) instability during transitional phases, and (3) limited generalization capability. Our STDW introduces a principled framework that fundamentally rethinks the domain adaptation paradigm through two key innovations:
A theoretically-grouddee optimization framework employing the time-dependent parameter ? E [0, 1J. which orchestrates a smooth transition from source to target domain representation learning. This mechanism ensures controlled knowledge assimilation while maintaining model stability throughout the adaptation trajectory.
An integrated self-training paradigm that synergistically combines pseudo-label refinement with adaptive loss reweighting, effectively addressing the domain shift problem through iterative model purification and confidence-awart sample selection.
Through extensive empirical validation across four benchmark datasets including Rotated MNIST, Color-shifted MNIST, Portrait, and Cover Type datasets we demonstrate that STDW achieves ofateoof-the-at performance, outperforming existing methods by significant margins. Comprehensive ablation studies further verify the critical role of our dynamic weighting mechanism and its superior ability facilitate robust knowledge migration across domains. The proposed framework establishes new theoretical foundations for gradual domain adaptation while providing practical insights for real-world applications.
2 Related Work
Domain Generalization (DG) has emerged as a principled approach to address the fundamental
challenge of distributional shift without requiring access target domain data during training [ The theoretical underpinnings of DG trace back to kernel-based methods for learning comain-invvriinn representations, as formalized by [12. who established important connections between feature space alignment and generalization performance. Subsequent developments have significantly expanded this paradigm through adversarial learning frameworks that explicitly minimize the discriminability of domain-specift features while preserving task-relevant information 13] A parallel line of research has demonstrated the effectiveness data-centric approaches simulating domain variations during training. The work of []4] pioneered sophisticated data augmentation strategies that generate synthetic

---

domain shifts through carefully designed transformations, effectively creating a continuum of virtual training environments.
More recently, the field has witnessed significant advances through the integration of meta-learning principles into the DG framework. [13] and [16] have reformulated the problem
meta- optimization task, where models are trained to rapidly adapt to simulated domain shifts presented in episodic training scenarios. This meta-learning perspective provides formal generalization guarantees and has established new state-of-the-art performance across multiple benchmark datasets. While these approaches demonstrate impressive results in controlled evaluations, our analysis reveals persistent limitations when confronting substantial domain discrepancies, particularly in scenarios where the underlying data manifold undergoes complex nonlinear transformations. This observation motivates our investigation of gradual adaptation strategies that can better preserve model stability across significant distributional shifts.
Gradual Domain Adaptation (GDA) has emerged as a principled approach to address the funda- mental challenge of substantial domain shifts by decomposing the adaptation process into sequence of intermediate, more manageable steps. The theoretical foundations of GDA were established by 77] and [6] demonstrated that gradual transitions between domains can significantly improve adaptation performance compared to direct source-to-targel approaches. This paradigm shift builds upon the key insight that complex real-world domain shifts often evolve gradually rather than occurring abruptly, suggesting that modeling the continuous transformation between domains may better capture the underlying data manifold structure.
Recent advances in GDA have focused on two primary directions: the generation of meaningful intermediate domains and the development of robust adaptation strategies across these domains. Theoretical work by [16] has provided rigorous mathematical frameworks for understanding the benefits of gradual adaptation, while algorithmic innovations have explored various approaches to constructing intermediate distributions. These include gradient flow-based geodesic paths [7] that preserve the intrinsic geometry of the data manifold, style-transfer interpolation techniques TT] that generate visually plausible intermediate samples, and optimal transport methods [9] that minimize the Wasserstein distance between consecutive domains. These methodological advancements collectively represent a significant step forward in addressing large domain gaps, though important challenges remain in scaling these approaches to high-dimensional spaces and ensuring their robustness to noisy or imperfect intermediate domains.
3 Problem Setup
Domain Space Let 2 x y denote the measurable instance space, where X C T is the d-dimensional input space and y = (1,2...., k) is the label space with k denoting the number of classes.
Gradually Shifting Domain Following the gradual domain shift framework 6 we assume a sequence of n + 1 domains {De} defined over Z. Here, Do represents the labeled source domain, D, denotes the unlabeled target domain, and the intermediate domains Dı.... Dn-1 form smooth interpolation between them. Each D, corresponds to a distinct data distribution, with the divergence between consecutive domains assumed to be sufficiently small to enable stable adaptation.
Classification and Empirical Risk Minimization The goal of supervised classification is to learn hypothesis f + ) that minimizes the expected risk under given distribution D,. Formally, given a loss function : y x y Rz0, the optimal parameters A are obtained by solving:
0 arg min E ( (xr 0),
Domain Adaptation In unsupervised domain adaptation (UDA), the model is trained labeled source domain Ds but evaluated on an unlabeled target domain Dy, where (s(x) * Pr(x) while the conditional distribution Pyy xx) is assumed be invariant. The central challenge lies in transferring knowledge across domains despite the distributional shift and the absence of labeled target data.
Gradual Domain Adaptation Gradual domain adaptation (GDA) addresses this challenge by leveraging sequence of unlabeled intermediate domains {Dr that bridge Do and D,.. Starting
3

---

y1
somc
Classifier
Classifier
Sign
CrOS3- entropy
550
H4,
y/
Sign
entrupy
syne
Feature Extractor
Feature Extractor
Self- Training with Dynamic Weighting
Batch Sampling
B
Stepwise
Current
S
S2
S3
S₁
S
Do
D1
D2
DD
D
Ds
Source Domain
Unlabeled Intermediate Domain
Target Domain
The framework of our Frafiirainin with Dynamic Weighting (STDW). It is designed to facilitate a smooth and controlled adaptation of a model from source domain to a target domain. The hyperparameter governs the trade-off between the two domains: when e 0, the model operates exclusively on the source domain, whereas as @ increases toward I, the model progressively shifts its emphasis toward the target domain. Pseudo-labels, generated the model itself, guide the adaptation process, thereby ensuring a gradual and stable transition across domains.
Figure 2:
from model f-; Oo) trained on the source domain Do. adaptation proceeds iteratively: at stage t (I...., m. the current model f(: (:-) generates pseudo-labels for samples from De, which then used to update the model parameters via self-training:
A: = arg min E. (iriD.,..((f (x; 0)) f(: 011))]
(2)
0'
where f(x;; 61-1) arg maxyey fy (2; 11) denotes the hard pseudo-label assigned by the previous model. The resulting model f-; 0,) is then employed to annotate data from the next domain D By constraining the distributional shift between consecutive stages be small, this procedure ensures controlled and stable propagation of knowledge from source to target, thereby mitigating error accumulation and enhancing generalization in the final target domain.
Methodology
This work proposes Self- Training with Dynamic Weighting (STDW), which reconstructs traditional self-training paradigms through triple adaptive mechanism. Fig. illustrates the overall framework of Self- Telfnirainnng with Dynamic Weighting (STDW).
4.1 Pseudo-labeling Dynamic Learning
Existing gradual domain adaptation (GDA) methodologies typically employ a static pseudo-labeling paradigm, wherein the entire unlabeled dataset = { ZN is annotated in a single pass. This process involves a classifier f(; 0) assigning hard labels through:
f(x) = arg max f(x)y:
followed by discarding a fixed proportion (e.g., 10%) low-confidence samples to mitigate noise 6 However, residual mislabeling inevitably persists, leading to compounded errors across successive domains and exponential degradation model accuracy phenomenon empirically validated in prior studies.
To address this limitation, we propose dynamic pseudo-labeling a sequential strategy that alternates between incremental pseudo-label generation and classifier refinement. Specifically, partition

---

D into T disjoint batch B, via:
(BI..... Br}. At iteration f, the model 6,-1 labels samples within
9 arg max f(z; Oc-1)y Va, E Be:
yielding pseudo-labels ((())) ). These labels then drive parameter update by minimizing the cross- entropy loss L over By:
= 0 Vo (1-)),))
(5)
where 1) denotes the learning rate. This iterative process is formalized through batch-wiss update operator U/
0. = (O,1,, t t= I...,,
(6)
By progressively refining pseudo-labels with incrementally improved classifier, our approach circumvents the еπгоr accumulation inherent to static labeling schemes. This dynamic interaction between label generation and model adaptation ensures that each batch benefits from the enhanced discriminative capability acquired from preceding batches, thereby alleviating label noise propagation and fostering more stable domain adaptation. The resultant framework demonstrates superior robustness, as evidenced by our experimental analyses.
4.2 Cyclic Batch Matching Across Neighboring Domains
Data comes from range of domains in the gradual domain adaptation scenario. Consider two neighboring domains D; (left domain) and D, (right domain), which are partitioned as Dj 1B Bi....... B D, [B,,1, Br.m.
To facilitate robust inter-domain adaptation, we define cyclic sequences that allow batches from each domain to be matched in fixed, periodic manner. Specifically, define the sequences (t)) (2oond B, (t) with the mapping given by:
(t( Bt,
(t) Br,)())
where the indexing functions i(t) and (t) are defined as:
i(1) (4 mod n) + +1,
t mod m) 1.
(8)
j(t) (t
This formulation ensures the batches are sampled cyclically over the two domains. The resulting cyclic matching can then be represented by the ordered pair sequence:
(BL, By,5(K)) A20
Consider sequence of n + domains {D:) where each domain D, consists of m data batches denoted by (Br,1, .t....... Br,m(t)): }. We define time-varying classifier f(t,8) to be the model after the k-th batch update within domain D:-1 and D.. Its evolution governed by the incremental update rule:
(lt,k+1) (((tt,) Bi-1,5(k), B(s())) where & is an optimization operator defined via the following objective:
(10)
+ arg min E EawB, a..)) (( f(x;; (): + +EawBr.s [eco ((1:((1)))) (11)
6'
Just like Eq. 5
and Eq. 6, we can update 0 through 4.
4.3 Stepwise Dynamic Osmosis
We formulate the cross-domain adaptation process as the following optimization problem:
((t,++)) = ((((t.k), BL-1 BIJ(K),)) e):
(12)
5

---

= arg min (1 BB Br, ((a()(
fax + Bl [<a(U
(13)
where & E [0, 1] is hyperparaameter that balances the contribution of the current domain D; and the subsequent domain D++1- As the training progresses, increases from 0 to 1, thereby gradually migrating the learned knowledge from the source domain D; to the target domain Di+1-
Algorithm 1 Self- Teaining with Dynamic Weighting (STDW)
1: Input: Source batches { (Bo, l domain sequence (Dr [P initial model f(0.0), Inter-domain
migration steps: 8
2: Output: Adapted model f(nm)
3: g 4 g+11/s
4: for = 1 TO n do
5: while @ < 16 do
6: for k = TO m epochs do
Fetch batch B generate pseudo-labels: 9 t- f(1;6((kk-1
7:
8:
Use Use 12 Eq. 13 update model: g(t,k) + ((((ik--), ( BeJ(M) 0)
BB-1.(())
9:
end for
10: - 0
end while
12: Switch to the next domain: ((++1,0) + (((.m)
end for
Experiments
5.1
Environments Setup
Following established gradual domain adaptation protocols 66 9 we utilize four core datasets: Rotated MNIST [18] and Color-Shift MNIST for controlled synthetic transformations, Portraits Dataset [19] for real-world temporal shifts, and Cover Type Dataset 20) for tabular domain adaptation. The Rotated MNIST dataset, derived from the standard MNIST digits [18), introduces progressive geometric transformation where source images (0 rotation) gradually evolve to target images (45 rotation) through intermediate domains of increasing rotation angles [9]. Similarly, Color-Shift MNIST examines photometric transformations by systematically varying pixel intensity ranges from [0.1] in the source domain [1.2] in the target domain (9) For real-world evaluation, the Portraits Dataset 19 provides a temporal domain shift spanning over a century (1905-2013), partitioned into nine chronological domains with distinct fashion and photographic characteristies (6]. The Cover Type Dataset (20) offers challenging tabular scenario where domains are constructed based on ecological proximity to water sources [6]. To further stress-test our method under severe distribution shifts, we incorporate two additional corruption benchmarks: CIFAR-I0-4 and CIFAR- 100-C (1.. which systematically apply types of image corruptions at varying severity levels, treating the highest severity as the target domain.
Datasets.
Implementation. For image datasets (Rotated MNIST, Color-Shift MNIST, Portraits), we imple- ment a convolutional neural network comprising three 32-channel convolutional layers followed by two 256-unit fully connected layers. The tabular Cover Type dataset utilizes a progressively expanding fully connected architecture (128-256-55 units). All models incorporate ReLU activa- tions, batch normalization (22) and dropout regularization [23], optimized using Adam 2 with carefully tuned hyperparameters. The transport network architecture features sssiuull-connecee generators and three-layer discriminators (128 hidden units per layer) to facilitate domain transitions. Our evaluation systematically varies the number of intermediate domains (0-4) to analyze adaptation trajectory smoothness. For the corruption benchmarks, we employ established robust architectures: Nett 25 for CIFAR-10-C and Rt-2eXt-29 [26] for SIFAR-100-5 following Robust- Bench protocols
to ensure comparable evaluation conditions. All experiments are conducted on NVIDIA RTX 4090 GPUs with identical random seeds for reproducibility.

---

Benchmarks. Our comparative analysis encompasses the following state-of-the-arr domain adapta- tion methods spanning both conventional and gradual adaptation paradigms. The evaluation includes three unsupervised domain adaptation baselines: DANN [29 (adversarial adaptation), DeepCoral 30 (correlation alignment), and DeepJDOT 31 (joint distribution optimal transport). For gradual domain adaptation, we benchmark against five recent approaches: GST [6] (gradual self-training), IDOL (8] (intermediate domain learning), GOAT [ (geodesic optimal transport), GGF [17] (gradual geometric flow), and GNF [32 (gradual normalizing flows). To evaluate the adaptability of the model in slowly evolving environment, we conducted experiments using TTA application methods includ- ing TENT-continal [33], AdaContrast (3), COTTA [35,, GTTA-MIX (36] and the GDA method GST. all the experiments, we took the error rate as the main indicator in order to make a direct comparison with the existing literature and provide clear explanation of the model performance under different adaptation challenges at the same time.
5.2 Results
Table I presents a comprehensive comparison of our method against state-of-the-ane unsupervised domain adaptation (UDA) and gradual domain adaptation (GDA) approaches across multiple bench- mark datasets. Our Prefniraannng with Dynamic Weighting (STDW) consistently achieves the best performance, establishing new state-of-the-art results with substantial margins. Specifically, STDW improves absolute accuracy by 11.2%, 6.5%, 0.94%. and 4.3% over the second-best methods. These gains underscore the effectiveness of STDW in modeling continuous distributional shifts through its dynamic weighting mechanism, in contrast to static alignment strategies (e.g.. DANN) or single-step gradual adaptation schemes. Moreover, among the limited set of domain adaptation frameworks capable of operating across diverse adaptation scenaiio--incluee both abrupt and gradual Shiits-SDDW demonstrates exceptional versatility and robustness, highlighting its broad applicability.
Table 1: Benchmarks Comparison on different datasets, including UDA methods and GDA methods.
UDA/GDA methods
Rotated MNIST
Color-Shift MNIST
Portraits
Cover Type
DANN 37
44.2
56.5
73.8
65.2
DeepCoral 30
49.6
63.5
71.9
66.8
DeepJDOT 31
51.6
65.8
72.5
67.4
GST 6
83.8
74.0
82.6
73.5
IDOL 8
87.5
78.3
85.5
72.1
GOAT
86.4
91.8
83.6
69.9
GGF 17
67.72
73.6
86.1
71.2
CNF 38
62.55
70.4
84.6
70.8
STDW (Ours)
97.6
98.3
87.1
74.2
The experimental results presented in Table demonstrate the effectiveness of our proposed STDW method compared seven ofth-off-hee-art approaches across 15 distinct corruption types severity level 5 achieving superior performance overall, attaining the lowest mean error rate of 15.5% on CIFAR-10-C and 25.8% on CIFAR-100-C. This represents a significant improvement over the source- only baseline (43.5% and 46.4% respectively) and outperforms all comparison methods, including the closest competitor GTTA-MIX (15.6% and 28. 28.9%). Notably, STDW achieves the best performance on 11 out of 15 corruption types for CIFAR- 10-C and 14 out of 15 for CIFAR-100-C, demonstrating remarkable consistency across diverse corruption patterns.
Furthermore, for high-frequency corruptions like Gaussian noise, shot noise, and impulse noise, STDW shows particularly strong performance with error rate reductions of 1.6 percentage points compared to the next best method. This suggests our approach effectively handles high-frequen distortions that typically challenge conventional adaptation methods. For geometric distortions (e.g., zoom, motion) and weather-based corruptions (e.g., snow, frost), STDW maintains consistent advantages. indicating robust performance across both spatial and photometric transformations. The consistent superiority of STDW across both datasets and corruption types validates our key contributions: (1) the effectiveness of progressive domain transport through weighted intermediate steps, and (2) the robustness of our approach to diverse types of domain shifts.
7

---

Table 2: Classification rate (%) the highest damage severity level 5 after gradual adaptation of
1O0C with progressively higher damage. For all method results evaluated on the same ResNeXt-29 that has been trained on the source domain, for STDW fixed the of intermediate steps. We report the average performance of our method over runs. Red indicates the best value, Green indicates the second-best.
Methods
Source only
BN-1
AdaContrast Tont-
COTTA
GTTA-MIX
GST
STDW (ours)
Gradual
X
/
Gaussian
72.3
28.1
24.8
29.1
24.3
23.4
50.0
21.5
Shot
65.7
26.1
20.6
22.5
21.3
18.3
43.9
19.0
Impulse
72.9
36.3
28.6
30.0
26.6
25.5
50.3
27.9
Defocus
46.9
12.8
14.4
14.0
11.6
10.1
20.6
9.6
Glass
54.3
35.3
31.1
32.7
27.6
27.3
51.2
28.3
Motion
34.8
14.2
16.5
14.1
12.2
11.6
17.2
10.3
CIFAR-I04 10-C
Zoom
42.0
12.1
14.1
12.0
10.3
16.7
8.5
Snow
25.1
17.3
19.1
16.6
14.8
14.1
17.5
3.0
Frost
41.3
17.4
18.6
14.9
14.1
13.0
24.3
13.3
Fog
26.0
15.3
18.6
14.4
12.4
10.9
17.5
11.7
Bright
12.2
8.1
7.4
6.9
Contrast
46.7
12.6
20.3
10.0
10.6
9.0
13.2
8.8
Elastic
26.6
23.8
25.7
21.9
18.3
19.4
24.9
19.6
Pixelate
58.5
20.8
17.7
13.4
14.5
39.9
13.4
JPEG
30.3
27.3
24.9
20.0
17.3
19.8
26.6
20.1
Mean
43.5
20.4
20.7
18.5
16.2
15.6
28.1
15.5
Gaussian
73.0
42.1
37.2
42.3
40.1
36.4
49.8
31.0
Shot
35.8
36.8
37.7
32.1
56.7
27.6
Impulse
39.4
42.7
41.7
38.6
39.7
34.0
32.3
26.1
Defocus
29.3
27.6
37.9
27.7
26.9
24.4
22.5
22.1
Glass
54.1
51.2
40.1
38.0
35.2
41.6
31.6
CIFAR- CIFAR-100-C C00-
Motion
30.8
29.7
48.3
29.1
27.9
25.9
25.0
22.9
Zoom
28.8
27.9
48.5
27.5
26.4
23.9
23.3
22.6
Snow
39.5
34.9
58.4
32.9
32.8
28.9
30.3
25.0
Frost
45.8
63.7
30.7
27.5
32.2
25.4
Fog
50.3
41.5
71.1
38.2
40.3
30.9
38.1
25.8
Bright
29.5
26.5
70.4
25.9
24.7
22.6
22.1
21.9
Contrast
55.1
30.3
82.3
28.3
26.9
23.4
27.0
24.3
Elastic
37.2
88.0
33.9
29.4
33.1
27.2
Pixelate
74.7
32.9
88.5
33.3
28.3
25.5
40.8
22.9
JPRG
41.2
41.2
90.4
36.2
33.5
33.3
35.8
30.0
Mcan
46.4
35.4
60.9
33.4
32.5
28.9
33.3
25.8
Table 3: Comparison of accuracy for our STDW method across various datasets, considering different numbers of given intermediate domains (including the source and target domains). Results are averaged over 5 runs, with error bars representing the 95% confidence interval of the mean.
Rotated MNIST
Color- MNIST
Given Domains
# Inter-domain
STDW
# Inter-domain counts STDW
2
81,5+1.3 82.843.5 81.714.4 82.0#6.4
83,6+1.2
86.2+0.1
96.5+0.0
87.243
86.640.0 86.640.1
95.940.4
96.5+0.8
96. 86..3 96.920.1
96.8+0.1
98.140.0
98.240.0
98. 98.240.0 98.3+0.0 98.3+0.0
96.540.2
96.940.1
86.0
84.00
96.810.1
98.3+0.0
98.340.0
98.3+0.0
98.3+0.0 98.3+0.0
96.840.1
96.940.1
96.740.1 96.9+0.0 97.0+0.1
98.3+0.0
98.340.0
98.3+0.0
98.240.0
98.3+0.0
6
96.940.1
96.940.1
96.940.1
97.6*0.0 96.940.0
98.3+0.0
98.3+0.0
98.3+0.0 98.240.0 98.240.0
Portraits
Cover Type
# Given
# Inter-domain counts in STDW
# Inter-domain counts in STDW
Domains
0
3
4
0
3
4
2
83.740.3
84.420.8
85.1+0.6
85.1
85.1
69.7+0.0
70.0+0.0
70.540.1 71.240.0
71.8+0.1
44.1
84.6+0.2
84.8+0.5
84.7+0.1
84.7+0.2
70.1+0.0
72.340.0
73.940.0
74.3+0.0
74.4+0.1
83 33.8
84.0+0.1
84.0+0,2
83.840.3
83.940.1
71.540.0
73.8+0.0
74.2+0.0
74.0+0.0
74.2+0.0
5
84.8+0.4
84.8+0.4
84.8+0.4
84.8+0.2
84.9+0.4
72.5+0.1
74.2+0.0
74.140.0
74.240.1
73,9+0.0
85.3+0.9
86.120.4
85.8+1.0
85.6+188
85.420.9
73.120.0
74.140.0
73.420.0
73.920.0
73.420.0

---

We present a comparative evaluation of the proposed STDW framework across four benchmark datasets: Rotated MNIST, Color-Shift MNIST. Portraits, and Cover Type, with detailed results provided in Table 3 For each dataset, experiments were repeated multiple times under identical conditions, and the reported metries represent the mean performance together with their associated variance intervals (e.g., + one standard deviation). The leftmost column in each table corresponds to the baseline obtained via adversarial training alonc-specifically, domain-adversaria setup without the integration of flow matching-serving serving as a reference point to quantify the contribution of our proposed components.
The experimental results demonstrate the effectiveness of the STDW framework across the four benchmark datasets. For each dataset, we systematically vary two key factors: (i) the total number of available domains (including the source and target), and (ii) the number of inter-domain adap- tation steps employed in the STDW procedure. We evaluate model performance as the number of adaptation steps increases, thereby providing fine-grained analysis of how the granularity of domain interpolation influences the overall efficacy of the adaptation process. Notably, the observed standard deviations across five independent runs remain consistently low (all < 1.2%), underscoring the stability and reliability our method under diverse domain configurations.
5.3
Ablation Study
6.35
3.55
0.5
3.5
0.72
0.35
3.00
0.85
3.5
0.71
0.50
335
0.4
cipe
0.8
Numtber of Domains
Nuttber of Domains
Number of Demains
Number Demains
(a) Rotated MNIST
(b) Color-Shiff MNIST
(c) Portraits
(d) Covertype
Figure 3: Ablation of STDW under different domain configurations and step counts. Accuracy four benchmarks when varying the total number of domains (x-axis) and the inter-domain step counts used by STDW (curves). Each curve labeled "0-4 steps" indicates how many stepwise dynamic osmosis updates are performed per adjacent domain pair.
To investigate the impact of intermediate domains on adaptation performance, we conduct ablation studies on four benchmark datasets under a non-STDW baseline, varying the number of domains from 2 (i.e., source and target only) to 6 (i.c., source, target, and four unlabeled intermediate domains). The results, summarized in Figure 3 reveal consistent trend: the inclusion of intermediate domains leads substantial improvements classification accuracy across most settings, accompanied by notable reduction in standard deviation (computed over multiple runs). These findings strongly support the hypothesis that gradual, stepwise domain adaptation enabled by intermediate distributions pplyssplsssss pivotal role in enhancing both the effectiveness and stability of domain generalization, particularly under significant souree-target shifts.
Table 4: Comparison STDW Method (Ours) with basclines across different domain adaptation steps on Rotated MNIST and Portraits datasets.
Method
Rotated MNIST
Portraits
2
81.3 0.9
86.1
86.9
88.1
82.9
8 85.3
Sorted
84.1 0.8
86.0 0.6
1.7 86.3
0.5
82.7
8..
8.2
81.30.22 0.1
Rand
83.4 0.2
$0.9
7.0
84.
86.3
0.9
8.
82.4
0..
84.00.6
0.6
84
84.3
84.1
Fixed
83.3 0.0 83.7 + 0.0
83.8
+ 0.1 83.8
0.1 84.1
83.9
0.3
83.4 0.0 8.11 84.1
84.7 +4.7
Additionally, we compare our stepwise dynamic osmosis srratey-refferre to as Ours (Equal), which linearly increases the weighting hyperparameter ş in equal increments across adaptation against three representative baselines: Fixed (e 0.5 held constant), Rand (o sampled independently from uniform distribution U(( 1) at each step), and Sorted (a variant of Rand where

---

IEEE Transactions on sampled values are sorted in ascending order to enforce monotonicity). Experiments are conducted on the Rotated MNIST and Portraits datasets under a two-domain setting (source and target only). with the number of inter-domain adaptation steps varied from 0 to 4. Results, reported in Table 4 show that Equal consistently achieves the highest final accuracy- for instance, attaining 88. % on Rotated MNIST' at Step 4, compared to 84. 14.1% % and 86. 1 % for Fixed and Rand, respectively. Moreover, Equal exhibits significantly lower variance across runs (c.g., =1.5 vs. =7.0 at Step I on Rotated MNIST), highlighting its superior stability. These results demonstrate that a controlled, monotonic increase of g-as opposed to static, random, or even sorted-randon Schedules-yieloe more reliable and optimal adaptation performance.
Conclustion
This paper presents Self-Training with Dynamic Weighting (STDW), innovative approach to Gradual Domain Adaptation (GDA) that systematically addresses the fundamental challenge of smooth knowledge migration across evolving domains. The proposed methodology introduces dynamic weighting mechanism governed by the hyperparameter which continuously modulates the relative influence between souree and target domains throughout the adaptation process. This dynamic balancing enables a progressive and efficient knowledge migration within the self-training framework, effectively mitigating domain bias while maintaining model stability. Comprehensive experimental validation across multiple benchmark datasets confirms that STDW achieves superior performance compared to existing state-of-the-ar methods, demonstrating significant improvements in both classification accuracy and model robustness under varying domain shift conditions.
Acknowledgments
The work of Shuai Zhang was supported by National Science Foundation (NSF) #2349879.
References
[1] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan. and Jasper Snoek. Can you trust your model's uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019.
[2] Simno Jialin Pan and Qiang Yang. A survey on transfer learning. knowledge and data engineering, 22(10))1345-1359 2009.
[3] Jingjing Li, Yue Wu, and Ke Lu. Structured domain adaptation. IEEE Transactions on Circuits and Systems for Video Technology, 27(8): 170-1773, 2016
[4] Sicheng Zhao, Xiangyu Yue, Shanghang Zhang, Bo Li, Han Zhao, Bichen Wu, Ravi Krishna, Joseph E Gonzalez, Alberto I. SangiovaniiVViccetellii Sanjit A Seshia, et al. A review of single-source deep unsupervised visual domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 33(2):473-495 2020.
[5] Xiaofeng Liu, Chaehwa Yoo, Fangxu Xing, Hyejin Oh, Georges El Fakhri, Je-Won Kang, Jonghye Woo, et al. Deep unsupervised domain adaptation: A review of recent advances and perspectives. APSIPA Transactions on Signal and Information Processing, 11(1), 2022.
[6] Ananya Kumar, Tengyu Ma, and Percy Liang. Understanding self-training for gradual domain adaptation. In International conference machine learning, pages 5468-5479, PMLR, 2020.
[7] Ali Farshchian, Juan A Gallego, Joseph P Cohen, Yoshua Bengio, Lee E Miller, and Sara A Solla. Adversarial domain adaptation for stable brain-machine interfaces. arXiv preprini 180.001810.000045, 2018.
[8] Hong- You Chen and Wei-Lun Chao. Gradual domain adaptation without indexed intermediate domains. Advances in neural information processing systems, 34:8201-82 2021.
10

---

[9] Yifei He, Haoxiang Wang, Bo Li, and Han Zhao. Gradual domain adaptation: Theory and algorithms. arXiv preprint arXiv:2310.13852. 2023.
[10] Robert A Marsden, Mario Döbler, and Bin Yang. Introducing intermediate domains for effective self-training during test- ast-time. In 2024 International Joint Conference on Neural Networks (IJCNN), pages 1-10. IEEE, 2024.
[11] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International conference on machine learning, pages 10-18. PMLR, 2013.
[12] Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. Domain gener- alization via model-agnostic learning of semantic features. Advances in neural information processing systems, 32, 2019.
[13] Manh-Ha Bui, Toan Tran, Anh Tran, and Dinh Phung. Exploiting domain-specific features to enhance domain generalization. Advances in Neural Information Processing Systems, 34: 21 21189-21201, 2021.
[14] Yue Wang, Lei Qi, Yinghuan Shi, and Yang Gao. Feature-based style randomization for domain generalization. IEEE Transactions on Circuits and Systems for Video Technology, 32 (8):5495-5509, 2022.
[15] Da Li, Yongxin Yang. Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta- learning for domain generalization. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
[16] Haoxiang Wang. Bo Li. and Han Zhao. Understanding gradual domain adaptation: Improved analysis, optimal path and beyond. In International Conference on Machine Learning, pages 22784-22801. PMLR, 2022.
[17] Zhan Zhuang, Yu Zhang, and Ying Wei. Gradual domain adaptation via gradient flow. In The Twelfth International Conference on Learning Representations, 2024.
[18] Li Deng. The mnist database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 29(6):141-142, 2012.
[19] Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexci A Efros. A century of portraits: visual historical record of american high school yearbooks. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1-7, 2015.
[20] Jock Blackard.
Covertype.
UCI Machine
Learning Repository, 1998.
DOI:
[21] Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv: 12261, 2019.
[22] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.0155,..3 2015. URL https
7/arxiv. //arxiv.org/ org/
03167
[23] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 19596);199,1195, 2014. URI. RtRl.httss://wwe amlr.org///.................oo wttpg/appes/vvv//
srivastaval4a. html
[24] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 11Xiv:1412.6980, 2014. https:::: http///xxvvv.//// 11tp://.11...0000
[25] Sergey Zagoruyko and Nikos Komodakis. arXiv:1605.07146. 2016.
Wide residual networks. arXiv preprint
[26] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492-1500 2017.

---

[27] Francesco Croce, Maksym Andriushchenko, Vikash Sehwag, Edoardo Debenedetti, Nicolas Flammarion, Mung Chiang. Prateek Mittal, and Matthias Hein. Robustbench: standardized adversarial robustness benchmark. arXiv preprint arXiv:2010.09670, 2020.
[28] Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. In International conference on machine learning. PMLR, 2020.
pages
[29] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International conference on machine learning, pages 1180-1189. PMLR, 2015.
[30] Baochen Sun and Kate Saenko. Deep coral: Correlation alignment for deep domain adaptation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14, pages 443 450. Springer, 2016.
[31] Bharath Bhushan Damodaran, Benjamin Kellenberger, Rémi Flamary, Devis Tuia, and Nicolas Courty. Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation. In Proceedings of the European conference computer vision (ECCV), pages 447-463, 2018.
[32] Shogo Sagawa and Hidcitsu Hino. Gradual domain adaptation via normalizing flows. arXiv preprint arXiv:2206. 11492 2022.
[33] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. 10726, 2020.
[34] Dian Chen, Dequan Wang. Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 295-305, 2022.
[35] Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 7201-7211, 2022.
[36] Robert A Marsden, Mario Döbler, and Bin Yang. Gradual test-time adaptation by self-training and style transfer. arXiv preprint urXiv:2208.07736 1(5), 2022.
[37] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky. Domain-adversane training of neural networks. Journal of machine learning research, 17(59):1-35, 2016.
[38] Shogo Sagawa and Hideitsu Hino. Gradual domain adaptation via normalizing flows. Neural Computation, pages 1-47, 2025.
12

---

Limitations
While the proposed STDW method demonstrates significant improvements in gradual domain adap- tation, several limitations should be acknowledged. First, the performance of STDW relies heavily on the availability and quality of intermediate domains, which may not always be accessible or well-defined in real-world scenarios. Second, the dynamic weighting mechanism, though effec- tive, introduces additional hyperparameters (e.g., the scheduling of @) that require careful tuning, potentially increasing the computational overhead during training. Third, the method assumes smooth transition between domains, which might not hold for highly discontinuous or abrupt domain shifts. Additionally, the current experiments focus primarily on image and tabular data, leaving its applicability to other data modalities (c.g., text or time-series) unexplored. Finally, the theoretical analysis, while rigorous, relies on assumptions such as bounded Wasserstein distances and Lipschitz continuity, which may not fully capture the complexity of all practical domain adaptation problems. These limitations highlight directions for future work, such automating hyperparameter selection and extending the framework to handle more diverse and challenging domain shifts.
A Supplementary Experimental Results
CIFARI00-1--CIAAR-III and CoreNeettt--mageNett ImageNet-C adaptation task on the highest corruption severity level 5. For CIFAR-10-C the results are evaluated on WideResNett28, for CIFAR- 100-C on ResNeXt-29, and for Imagenet-C, ResNet-50 is used.
Classification correctness (%) for the CIFARIO+t-CIFAR-IIII ImageNet-to
Table 5: Classification correctness (%) on the highest damage severity level 5 after gradual adaptation of
10C with progressively higher damage. For all method results evaluated on the same WedeResNet-28 that has been trained on the source domain, for STDW we fixed the use of 2 intermediate steps. We report the average performance of method over 5 runs.
brightness
Method
Given
Domains
gaussian
sbot
impulse
defocus
glass
motion
200m
SnOW
frost
1og
contrast
elostic
pixclate
jpeg
Mean
Source
27.7
34.3
27.1
53.1
45.7
65.2
58.0
74.9
58.7
74.0
90.7
53.3
73.4
41.5
69.7
56.5
37.0
40.9
33.8
57.5
46.3
67.1
76.7
60.6
76.9
91.9
60.6
74.1
44.8
70.5
59.8
3
38.3
44.0
39.1
73.8
46.5
70.1
68.1
80.2
65.1
79.4
92.6
80.4
71.5
46.4
71.2
64.5
GST
4
41.4
48.7
41.5
76.2
48.2
76.8
75.7
80.0
71.8
81.5
93.0
85.4
75.0
51.9
72.1
68.0
42.5
50.3
45.2
79.2
49.1
78.0
78.5
80.4
72.2
82.0
93.1
86.1
75.0
54.1
73.1
69.2
49.7
79.4
48.8
82.8
82.5
75.7
93.1
75.1
60.1
73.4
71.9
27.7
34.3
27.1
53.1
45.8
65.3
58.1
74.9
58.7
74.0
90.7
53.4
73.5
41.6
69.8
56.5
27.5
34.3
27.0
53.2
45.8
65.4
58.2
74.9
58.8
74.0
90.7
53.5
73.4
41.7
69.8
56.5
GOAT
27.4
34.2
26.9
53.2
45.6
58.2
65.4
75.0
59.0
74.0
90.6
53.4
73.6
41.8
69.8
56.5
27.3
34.2
26.9
53.3
45.6
65.5
58.4
75.1
59.0
74.0
90.7
53.4
73.6
41.8
69.8
56.6
27.3
34.3
27.0
53.3
45.5
65.7
58.5
75.1
59.0
74.0
90.7
53.4
73.6
41.9
69.8
56.6
73.6
75.5
65.4
88.5
66.4
87.5
89.2
84.0
83.5
86.2
92.5
88.8
77.9
82.2
74.0
81.0
68.3
69.2
88.5
85.2
84.7
92.9
76.4
82.5
STDW
76.4
69.6
90.0
69.7
88.9
91.4
85.6
88.2
92.9
90.9
79.7
85.2
77.5
83.5
(Ours)
77.2
80.0
70.9
90.4
70.9
89.4
91.5
86.6
86.3
88.3
92.8
91.2
80.1
86.2
78.7
84.0
78.5
81.0
72.1
90.4
71.7
89.7
91.4
97.0
86.7
88.3
92.7
90.8
80.4
86.6
79.9
84.5
B Theoretical Arguments
B.1 Lyapunov Stability Under Cyclic Matching and Switching Anchors
For completeness, we extend the stability argument to the piecewise-smooth path constructed for discontinuous shifts. Let Or denote the parameters at step & and consider the mixed loss on matched pair mini-batches
Le(0.) e,e(O,, B(")) (1 (.
(14)
13

---

Table 6: Classification correctness (%) the highest damage severity level 5 after gradual adaptation of
with progressively higher damage. For all method results evaluated on the same ResNeXt-29 that has been trained on the source domain, for STDW fixed the use of intermediate steps. We report the average performance of our method over 5 runs.
Method
# Givee Domains
gaussian
shot
impulse
defocus
glass
motion
200m
Snow
Frost
brightness
contrast
elastic
pixelate
ipeg
Mcan
fog
Source
27.0
12.0
60.6
70.7
45.9
69.2
71.2
60.5
54.2
49.7
70.5
44.9
62.8
25.3
58.8
53.6
2
49.4
52.3
62.4
72.4
51.9
70.4
72.6
64.7
63.0
56.5
73.7
61.0
64.0
50.2
60.0
61.6
48.3
50.4
62.8
74.2
54.7
71.8
74.4
66.6
63.8
57.4
74.9
64.0
64.1
49.4
61.1
62.5
GST
49.5
64.2
75.6
56.1
73.0
75.7
67.3
65.6
59.1
76.5
67.2
65.5
50.3
61.9
63.7
50.0
52.0
66.2
76.6
57.9
73.8
76.2
68.1
66.8
60.3
77.2
69.1
66.5
53.7
63.4
65.2
6
50.2
53.3
67.7
77.5
58.6
75.0
76.7
69.7
67.8
61.9
77.9
73.0
66.9
59.2
64.2
66.7
27.1
32.0
60.8
70.9
45.9
69.3
71.3
60.6
54.2
49.9
70.4
44.9
63.0
25.0
58.8
53.6
31.9
60.7
71.0
45.9
69.6
71.3 60.6
54.2
49.8
70.4
45.1
63.1
25.2
58.7
53.6
GOAT
26.8
32.0
60.9
71.1
45.9
69.6
71.2 60.5
54.4
50.0
70.5
45.5
63.0
25.2
58.8
53.7
26.3
32.1 60.9
71.1
46.0
69.7
71.3 60.6
54.3
50.0
70.6
45.9
63.2
25.2
58.6
53.7
26.6
32.1
60.9
71.3
46.2
69.8
71.2
60.7
54.3
50.0
70.6
46.3
63.2
25.7
58.8
53.8
61.5
63.9
60.7
75.4
61.3
73.1
75.3
69.3
68.1
62.4
76.4
72.2
67.6
71.2
62.0
68.0
3
64.0
67.6
64.4
77.1
65.0
74.8
76.9
71.0
70.0
65.5
77.7
74.7
70.1
74.1
64.5
70.5
STDW
4
65.6
69.0
67.6
77.7
66.0
76.1
77.3
72.3
71.6
68.7
78.1
75.5
70.9
76.0
66.4
71.9
(Ours)
66.8
70.5
70.9
77.9
67.6
76.1
77.4
73.1
72.4
71.6
78.0
75.4
72.3
76.4
67.5
72.9
73,9
68.4
77.1
75.0
74.6
74.2
75.7
72.8
77.1
70.0
74.2
where L is cross-entropy and 2: [0, 1]. Assume é is L-smooth and j--ttrongyy convex in a neigh- bourhood of the local joint minimizer 0% for the pair (Da-1, Da). With gradient descent
0,41 0g V La(e))
(15)
and step size E (0,2/L), define the Lyapunov function
(a = 0 07|
(16)
Standard smooth strongly-convex analysis yields
Va(0++1) S Va(O.) - 7 7 LM) 10 0213
(17)
so Va strictly decreases whenever 0, * By and n < 2p/L2.. Cyclic matching enumerates all batch pairs once per epoch without changing smoothness curvature, the decrease persists across iterations. For discontinuous path approximated by anchors Ao, ... A, we obtain switched system with modes E (1,...,,)) 7)- there exists a common quadratic I Lyapunov function (θ)) = _ 1 valid in the union of neighbourhoods of for instance when 10; 0- is bounded and the Hessians share eigenvalue bounds (n, L)- then the same inequality holds for all modes with the same n, ensuring global asymptotic stability under arbitrary switching. When common Lyapunov function is controller:
too conservative, we adopt a dwell-time condition implemented by the discrepancy- move to the next anchor) only after the gradient norm on
the mode can switch (i.e., we advance the current mode falls below tolerance and the consistency gates are met. In both cases, the energy decreases monotonically, preventing divergence even under abrupt transitions.
14

---

NeurIPS Paper Checklist
Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the novel optimization framework Traff-rainnng with Dynamic Weighting) with dynamic hyperparameler the incorporation of self-training for pseudo-labels, and the demonstrated empirical improvements on multi- ple benchmarks; these match the theoretical and experimental developments presented in Sections and of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
[Yes]
Answer: [Yes]
Justification: The paper explicitly discusses limitations in several parts, most notably in the methodological framing and ablation studies. The authors recognize and address three key limitations of prior GDA methods that their approach (STDW) aims to solve: suboptimal knowledge propagation, instability during domain transitions, and limited generalization. Moreover, paper acknowledges potential sources of error such as mislabeling during pseudo-labeling and includes theoretical discussion erгоr propagation and stability (Appendix B). The ablation studies (Section 5.3) further demonstrate awareness of the method's 's performance sensitivity to the number of intermediate domains and adaptation strategies. These discussions reflect a thoughtful and transparent treatment of limitations in both theory and empirical practice.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
15

---

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: All assumptions are clearly stated alongside each theorem in Section (e.g. Assumptions 1-3 preceding Theorem 1), and full, detailed proofs appear in Appendix B of the supplemental material.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The paper provides sufficient detail to reproduce the main experimental results supporting its claims. The authors clearly specify: 'he datasets used (Rotated MNIST, Color-Shift MNIST, Portraits, Cover Type, CIFAR-10-C. CIFAR-100-C) along with their source and transformation details (Section 5.1). Model architectures for each dataset, including convolutional and fully connected networks, well as backbone choices like WideRessNe-288 and ResNeX1-29 (lines 196-198, 204-205). Optimization methods (Adam), hyperparameter tuning, and hardware used (line 206). Detailed description of the algorithm (Algorithm 1 and Section 4), including the update rules, dynamic weighting strategy with the e parameter. and of intermediate domains and adaptation steps, and performance metrics (accuracy, error rate) across multiple baselines (Tables 1-4, Figures 2-4).
procedure. Experimental setup involving the number
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same
16

---

dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
The supplementary materials include all code files necessary to reproduce the experiments, and detailed instructions for data preparation and execution of training scripts. Additionally, all datasets used are publicly available and referenced in the supplemental README.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https ctipssccc/pullcc/eeeccc gwtp://dddeeeeeee httpa///iiggggieeeeeeeeeeeeaaaae
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
17

---

Answer: [Yes]
Justification: All experimental settings-including dataset splits, optimizer (Adam) settings, full hyperparameter tables (learning rates, batch sizes, number of epochs, 2 schedule parameters), and selection criteria- are documented in Appendix C of the supplemental material and are directly instantiated in the provided code files with accompanying README instructions.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The main results tables (e.g., Table 2) explicitly report the 95% confidence interval of the mean across five independent runs, clearly indicating the variability due to random initialization and dataset splits.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The paper provides key information about the compute resources used for the experiments. Specifically, states that all experiments were conducted on NVIDIA RTX 4090 GPUs (line 206), which gives a clear indication of the type of compute hardware used. The authors also mention that identical random seeds were used ensure reproducibility. While detailed metrics such as runtime, memory usage, or total compute cost are not quantified, the provided GPU type, consistent setup, and fixed architecture configurations across datasets give sufficient basis for estimating resource needs and replicating the results. Therefore, the disclosure meets reasonable standard for reproducibility and resource transparency.
18

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics ottiishhttpee//eerrii.cc/ooboob
Answer: [Yes]
Justification: The authors reviewed and adhered to the NeurlPS Code of Ethics, ensuring misuse of data, respecting privacy and consent for all datasets, maintaining anonymity, and avoiding any potential harm or bias in experimental design and reporting.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: The authors highlight that their method, Self-Training with Dynamic Weight- ing (STDW), enhances model robustness under gradual domain shifts, which is especially relevant for real-world dynamic environments (c.g., shifting lighting conditions in visual systems, evolving demographics in user behavior). This robustness has positive socictal implications for improving machine learning reliability in safety-critical or long-term de- ployment scenarios. Although explicit negative impacts are not discussed, the general applicability of domain adaptation methods (e.g., surveillance or profiling systems) raises potential ethical concerns regarding fairness and misuse if applied without safeguards. A fuller, more explicit discussion would strengthen this component, but the paper does touch societal relevance through its goals and benchmarks.
Guidelines:
The answer NA means that there is no socictal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
19

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
The paper does not release any models or datasets that pose high risk for misuse. The research focuses improving domain adaptation using existing benchmark datasets (e.g., Rotated MNIST, CIFAR-1O-C, etc.) and standard model architectures. No new pretrained models, scraped datasets, generative tools with dual-use concerns are introduced or released. Therefore, safeguards for responsible release are not applicable in this context.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper and supplemental README explicitly cite each public benchmark (e.g., 10, Office-31, MNIST) along with version numbers and links to their licenses (e.g., MIT for code dependencies, CC-BY for datasets) and properly credit the original dataset and library authors.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.ppaperswtt
package should be provided. For popular datasets, paperswi thcode. cow/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
The paper does not introduce any new datasets, codebases, or other assets beyond code for reproducing experiments (already provided in supplemental), and relies existing publicly available benchmarks, so this question is not applicable.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The study does not involve any crowdsourcing or research with human partici- pants, relying solely on publicly available datasets without new data collection or participant interaction.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The work uses only pre-existing, publicly available image and tabular datasets and does not involve any new data collection from human participants or crowdsourcing experiments, so IRB considerations not applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
21

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
The core methodology centers on gradual domain adaptation for vision and tabular data using self-training and transport networks, with use of large language models in any part of the algorithm or experiments.
Justification:
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non ntn-standard components.
Please refer to our LLM policy for what should or should be described.
22


Reviews:
[{"review_id": "O0zlS5OXrQ", "reviewer": "Reviewer_nsMJ", "summary": "The authors propose a novel method, Self-Training with Dynamic Weighting (STDW), for the problem of Gradual Domain Adaptation (GDA). This method employs a self-training mechanism with pseudo-labels and introduces a dynamic hyperparameter ϱ, which evolves from 0 to 1 to balance the contributions of the source and target domains in the loss function. This dynamic weighting ensures a smooth and controlled knowledge transfer across intermediate domains during adaptation. This mechanism further incorporates a dynamic, mini-batch-wise pseudo-label generation strategy to mitigate the accumulation of pseudo-labeling errors during continual adaptation. The proposed method is evaluated on multiple image datasets and consistently outperforms both traditional domain adaptation approaches and other state-of-the-art GDA methods. Comprehensive ablation studies further confirm the essential role of the dynamic parameter ϱ and the use of intermediate adaptation steps, which together enhance stability and reduce domain bias.", "questions": "Please refer to weaknesses.", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Introduces a novel optimization framework employing a time-varying hyperparameter ϱ, which flexibly adjusts the relative contributions of the source and target domains during adaptation, thereby ensuring a stable and effective knowledge transfer across domains.\n2. Instead of generating pseudo-labels for the entire dataset in a single pass, the model produces pseudo-labels on a mini-batch basis and updates itself iteratively. This dynamic approach mitigates the accumulation of pseudo-labeling errors during adaptation, thereby enhancing stability and robustness throughout the gradual adaptation process\n\nWeaknesses:\n1. The method has not been compared with the most recent state-of-the-art approaches.\n2. The proposed approach does not demonstrate a significant breakthrough compared to existing methods.\n3. The effectiveness of the method largely depends on the identification and availability of intermediate domains. In practice, intermediate data can sometimes be difficult to identify or incomplete, which limits its applicability.", "was_revised": true, "final_justification": "Although I still think the main solution of the paper using the parameter ϱ varying from 0 to 1 is quite simple, Gradual Domain Adaptation GDA has high practical applicability  and the authors have provided relatively thorough and detailed responses to the various rebuttals. Therefore I am increasing my rating for this paper to 4.", "rebuttal": ""}, {"review_id": "q9NMSel8mu", "reviewer": "Reviewer_pTMZ", "summary": "This paper introduces Self-Training with Dynamic Weighting (STDW), a novel method designed for Gradual Domain Adaptation (GDA) that improves robustness by smoothly migrating knowledge from the source domain to the target domain. STDW generates dynamic pseudo-labels and uses a time-dependent hyperparameter to progressively shift learning emphasis from source to target domains. Empirical results on various benchmarks (demonstrate that STDW significantly surpasses existing GDA approaches.", "questions": "1. Does random sampling perform about the same with cyclic batch matching? If not, can you please offer an explanation why cyclic batch matching work?\n2. While I think gradual domain adaptation is a more practical assumption than traditional UDA, it can be difficult to collect data from gradual domains in practical situations. What is your view of this?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- Paper is clearly written.\n- Performance is strong.\n- Setting is practical.\n\n**Weaknesses**\nOverall, I don't think the contribution from this paper is very novel.\n1. Generating dynamic pseudo-labels is already common in self-training [1]. This is very similar to the approach Section 4.1. FixMatch even has an additional step of selecting high-confidence pseudo-labels.\n2. The paper proposes cyclic batch matching in Section 4.2. But I am confused why data sampling is designed in such a manner. One can simply randomly sample a batch from the left and right domain each time. This simplifies the training pipeline, and I feel it wouldn't impact the performance too much.\n3. While the stepwise dynamic osmosis makes a lot of sense, it feels like an engineering technique (or hyperparameter tweaking) and offers little technical contribution.\n\n[1] FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence", "was_revised": true, "final_justification": "My concerns are resolved after reading authors' response. Hence I will increase my score.\n\nI encourage authors to include the discussions about W1-W2 in the revision, which help clarify the contribution of the paper.", "rebuttal": ""}, {"review_id": "pmYxKuSM8Q", "reviewer": "Reviewer_BwCA", "summary": "The paper introduces Self-Training with Dynamic Weighting (STDW) to Gradual Domain Adaptation (GDA). It addresses the challenge of smooth knowledge migration from source to target domains by introducing a dynamic weighting mechanism controlled by a time-varying hyper-parameter ϱ, which balances the loss contributions of source and target domains during training. The method leverages self-training to generate pseudo-labels and optimizes a weighted objective function for iterative model updates. The authors provide theoretical insights and validate the method through extensive experiments on multiple benchmark datasets, demonstrating that STDW outperforms existing baselines and achieves SOTA performance.", "questions": "1)\tThe paper assumes a smooth transition between domains. How would the method perform in scenarios with highly discontinuous or abrupt domain shifts? Could the authors provide some insights or potential solutions for such cases?\n2)\tThe linear ϱ schedule (0→1) is heuristic. Are there theoretical or data-driven guidelines (e.g., based on domain discrepancy)? Could ϱ be learned?\n3)\tThe paper mentions that the performance of STDW relies on the availability and quality of intermediate domains. What strategies could be employed to generate or select high-quality intermediate domains in cases where they are not readily available?\n4)\tCyclic batch matching and iterative pseudo-labeling may increase training time. How does STDW’s complexity compare to baselines (e.g., GST)?", "limitations": "Yes, The authors have adequately addressed the limitations of their work in the paper.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1)\tFormal analysis (optimal transport, Lyapunov stability) provides error bounds for pseudo-label consistency and convergence (Appx. B). \n2)\tThe paper introduces a novel dynamic weighting mechanism that adaptively balances the loss contributions of source and target domains during training. This innovation effectively addresses the challenges of GDA.\n3)\tExtensive validation across 6 datasets, varying intermediate domains (0–6), and 15 corruption types. Ablation studies confirm criticality of ϱ scheduling and intermediate domains.\n4)\tThe proposed STDW method has demonstrated significant improvements in handling continuous distribution shifts compared to existing methods. \n\nWeaknesses:\n1)\tThe performance of STDW relies heavily on the availability and quality of intermediate domains, which may not always be accessible or well-defined in real-world scenarios. This limitation could restrict the applicability of the method in certain practical situations.\n2)\tThe dynamic weighting mechanism introduces additional hyperparameters (e.g., the scheduling of ϱ) that require careful tuning, potentially increasing the computational overhead during training. \n3)\tThe method assumes a smooth transition between domains, which might not hold for highly discontinuous or abrupt domain shifts. The authors acknowledge this limitation and suggest that future work could explore ways to handle more diverse and challenging domain shifts.", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "OxaYmyBbyh", "reviewer": "Reviewer_KmnV", "summary": "In this paper, the authors propose a novel gradual self-training method named Generalized Dynamic Osmosis (STDW), which largely improves the performance of the model on the gradual domain adaptation problem.\n\nThe method involves three innovations. First, instead of assigning labels for the next whole domain and keeping them unchanged when fine-tuning, STDW uses a batchwise strategy of predicting pseudo-labels for a mini-batch of data so the pseudo-labels are consistently updated by a new model. Second, they use cyclic batch matching to facilitate robust inter-domain adaptation. Third, a weighted loss with a dynamic weight is adopted to stabilize the training process.\n\nThe authors conduct some experiments to validate the efficacy of the method and the effect of some hyperparameters.", "questions": "* In the GST method, a new model is trained from scratch on each domain, while this work trains only one model and fine-tunes the model in each iteration. I am a little curious about whether keeping fine-tuning only one model would also improve the performance of the GST method.", "limitations": "Yes, the authors discuss some limitations of their method in the appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Pros:\n* The method is interesting and effective. First, the pseudo-labeling dynamic learning can make more efficient use of the current model since they predict the pseudo-labels using the up-to-date model. Second, the cyclic batch matching with a dynamic weight can stabilize the training process and decrease the bad effect of the incorrect pseudo-labels. The method is novel to me, and the effectiveness is verified by the experiments.\n\nCons:\n* The writing needs to be polished, especially the math notations. Those conflicting notations largely hinder a smooth reading for the readers and raise some misunderstandings. There are so many conflicting math notations. In Line 107, you use $k$ to denote the class number, while it also represents the batch index later in Line 169. In Line 113, the model is a mapping from $X$ to $Y=\\{1,2,...,k\\}$, while the expressions later in Eq. (2), (3) indicate it is a mapping from $X$ to $[0,1]^k$. In Line 127, you add the $sign$ to denote the final prediction function $\\hat{f}$ but this is only for binary classification---you use a multi-classification setting in Line 107. In Line 146, you use $D$ to denote a domain set. Since it does not have a subscript, I was wondering if this indicates a dataset combined from all domains or if it indicates one specific domain $D_t$? In Eq. (11), $x_i$ or $x$? In Line 176, the domain is denoted as $D_i$, while the index in the previous section is $t$ for the domain (Line 128). I totally understand that the gradual domain adaptation problem, which involves many domains, increases the complexity of the notation system, but those conflicting notations make it difficult for readers to understand the method of this work.\n* The motivations of the method should be explained more clearly. In section 4.2, the motivation of combining two batches from two adjacent domains is not clear. To my understanding, I think the idea is a little bit similar to Momentum. When the weight $\\rho$ is small at the early stage, the model is trained based on the loss on the previous domain rather than the current domain, where the accuracy of the pseudo-label could be high. So this strategy could prevent the model from collapsing due to the incorrect pseudo-labels of the current domain.\n* The framework illustration in Fig. 2 is not clear. What is the meaning of \"sync\" in the figure? To my understanding, you use only one classifier to learn the batches $B_t$ and $B_{t+1}$. I suggest replacing \"sync\" with \"shared weights,\" which is more widely used in papers. Besides, what is $H_t$? In the method, you never use the feature extractor and classifier in a separate way. So is it really necessary to separate the network into three parts—classifier, $H_t$, and feature extractor—in the illustration figure? \n* More ablation studies are needed. I suggest the authors validate the effects of the three components in Section 4 via an ablation study. For example, you could remove the cyclic batch matching and compare the results to show the important role of cyclic batch matching.\n* As the authors note in Line 141, the mislabeling would hurt performance. So it would be more convincing if the average accuracy of the pseudo-labels on intermediate domains can be shown and compared to further validate the effectiveness of your method.\n* Another baseline method. The work of [1] proposes an adversarial self-training method, which significantly improves the performance of GDA. Please consider comparing your method with [1].\n\n[1]: Adversarial Self-Training Improves Robustness and Generalization for Gradual Domain Adaptation. Lianghe Shi, Weiwei Liu. NeurIPS 2023.", "was_revised": true, "final_justification": "The authors have resolved the notation conflicts and added additional experiments in their rebuttal. However, I remain concerned about their reported results for the baseline method AST. I have independently implemented the AST method and found that it can achieve approximately 97% accuracy on the Rotated MNIST dataset, which is very close to the performance reported for their proposed method (97.6%). In contrast, the authors report an accuracy of only 92.5% for AST.\n\nOverall, I remain neutral regarding the acceptance of this work.", "rebuttal": ""}]

Author rebuttal:

Meta-review: This paper presented a novel Self-Training with Dynamic Weighting (STDW) approach for gradual domain adaptation. The method introduced a time-dependent weight parameter to guide the adaptation process in a simple yet effective manner. The proposed approach was empirically validated in extensive experiments compared to baseline methods.

Strengths:
- The paper is well-written and easy to follow.
- The proposed STDW method based on a time-dependent weight parameter is simple yet effective for gradual domain adaptation.
- Extensive experiments demonstrate the effectiveness of STDW over baseline approaches.

Weaknesses:
- The rationale behind the stepwise dynamic osmosis can be further clarified.
- The applications of STDW in challenging scenarios with highly discontinuous or abrupt domain shifts can be explored and validated in the experiments.
- The theoretical analysis of linear schedule can be analyzed.
- The reproducibility of both STDW and the baseline implementations could be improved with clearer experimental details.

Following the rebuttal and discussion, all reviewers have positive opinions on the paper. While some concerns remain regarding theoretical depth and experimental coverage, the contributions are clear and the empirical results are strong. I recommend acceptance, and encourage the authors to address the remaining reviewer comments in the final version.
Decision: Accept (poster)