Title: Parameter Dynamics of Online Machine Learning and Test-time Adaptation
Venue: NeurIPS 2025
Accepted: yes
Keywords: Online machine learning

Test-time adaptation

Probabilistic parameter dynamics

Bayesian deep learning
Abstract: Pre-trained models based on deep neural networks hold strong potential for cross-domain adaptability. However, this potential is often impeded in online machine learning (OML) settings, where the breakdown of the independent and identically distributed (i.i.d.) assumption leads to unstable adaptation. While recent advances in test-time adaptation (TTA) have addressed aspects of this challenge under unsupervised learning, most existing methods focus exclusively on unsupervised objectives and overlook the risks posed by non-i.i.d. environments and the resulting dynamics of model parameters. In this work, we present a probabilistic framework that models the adaptation process using stochastic differential equations, enabling a principled analysis of parameter distribution dynamics over time. Within this framework, we find that the log-variance of the parameter transition distribution aligns closely with an inverse-gamma distribution under stable and high-performing adaptation conditions. Motivated by this insight, we propose Structured Inverse-Gamma Model Alignment (SIGMA), a novel algorithm that dynamically regulates parameter evolution to preserve inverse-gamma alignment throughout adaptation. Extensive experiments across diverse models, datasets, and adaptation scenarios show that SIGMA consistently enhances the performance of state-of-the-art TTA methods, highlighting the critical role of parameter dynamics in ensuring robust adaptation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper models the dynamics of weights during online learning and test-time adaptation as a stochastic differential equation. It uses a discretization to obtain the transition distribution of the parameter evolution. The paper focuses on the transition variance and argues that the log-variance of the transition distribution follows an inverse gamma distribution when adaptation is stable. Based on this argument, the paper proposes an unsupervised adaptation mechanism that aligns the log-variance to an inverse gamma distribution to enable stable adaptation to a non-stationary test stream.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- The continuous-time SDE formulation of SGD updates in online settings, while not novel [3], seems an interesting and suitable theoretical framework for analyzing dynamic adaptation methods.\n- The paper is clearly written and easy to follow.\n\n# Weaknesses\n\n- Most related work is not (sufficiently) discussed. The following papers also model the dynamics of weights in TTA:\n1. Lee, Jae-Hong, and Joon-Hyuk Chang. \"Stationary latent weight inference for unreliable observations from online test-time adaptation.\" ICML 2024.\n\n2. Lee, Jae-Hong, and Joon-Hyuk Chang. \"Continual momentum filtering on parameter space for online test-time adaptation.\" ICLR 2024.\n\n3. Lee, Jae-Hong. “Bayesian Weight Enhancement with Steady-State Adaptation for Test-time Adaptation in Dynamic Environments,” ICML 2025.\n\nIn particular, 3.) also uses an SDE formulation for parameter dynamics and adjusts the step size $\\Delta t$ based on the covariance. Could we see a detailed qualitative and empirical comparison against 3.)? As the paper is very similar both conceptually as well as in the experiments to 3.), yet not cited, could the authors please highlight how the current work differs from or extends 3.)?\n\n- The authors empirically find that the log-variance follows an inverse gamma (IG) distribution in stable adaptation settings and, in a second step, regularize the updates to follow an IG distribution. However, except for empirical correlation, the paper does not explain why the IG is the “holy grail.” What properties of the distribution explain this correlation? What is the intuition or theoretical connection behind this finding?\n\n- If I understand correctly, adapting the model results in modifying the time discretization $\\Delta t$. Does this correspond to changing the learning rate? If so, does this mean only the step size is changed but not necessarily the direction of the gradient?\n\nQUESTIONS:\n- Equation 13 suggests a Bayesian treatment of the parameters. This makes sense, given that the paper focuses on the parameters' variance. However, from the text, it seems that the method is just evaluated at the mean. Could we see results with Monte Carlo sampling as well, incorporating the adjusted variance term?\n\n- How come the adaptation gains of most TTA baselines in the standard setting of Table 1 are close to zero (e.g., for TENT, SAR, RoTTA, EATA)? These results seem to deviate from other papers \\[1, 2], where the adaptation gains on ImageNet-C severity 5 are quite pronounced.\n\n[1] Niu, Shuaicheng, et al. \"Efficient test-time model adaptation without forgetting.\" International Conference on Machine Learning. PMLR, 2022.\n\n[2] Zhao, Hao, et al. \"On pitfalls of test-time adaptation.\" ICML (2023).\n\nLIMITATIONS:\nLimitations are not discussed.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the parameter dynamics of online machine learning and its application in test-time adaptation (TTA). The key findings include: (1) the fitness of inverse gamma (IG) and log-variance serves as a strong predictor of adaptation performance and model collapse, and (2) state-of-the-art TTA methods often implicitly promote IG alignment. Furthermore, the authors propose SIGMA, a method that dynamically estimates an appropriate IG distribution using derivative-free optimization and adjusts the parameter update trajectory to maintain alignment during online adaptation. Experimental results demonstrate the effectiveness of SIGMA.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n* This paper studies the fundamental and important question about parameter dynamics in online machine learning and test-time adaptation.\n\n* Interesting and novel findings about parameter dynamics.\n\n* A plug-and-play and effective method improves the performance of TTA methods.\n\n**Weaknesses**  \nI do not find any major weakness. However, I have some questions and advise, see below.\n\nQUESTIONS:\n* In line 511, authors show that DUC dataset is used. However, the motivation and details of DUC is not introduced and not any reference. DomainNet-126 [3] , a subsect of Domain Net[2] , or full DomainNet is more widely used in related papers [1,4].\n\n* Some references should be cited with published format, such as [40], please check all references.\n \n* How to ensure $c_k \\leq 0$ and $b_k > 1$ (line 244)? use Lagrange multiplier or just clip?\n* How to understand alignment length $\\lambda$ (in line 245)? \n* It is better to show how to solve Eq.(9) and get Eq.(10). And how to get $r_k$ and $T_k$ ?\n\n**Reference**\n1. Robust Test-Time Adaptation in Dynamic Scenarios. CVPR 2023\n2. Moment Matching for Multi-Source Domain Adaptation. ICCV 2019.\n3. Semi-supervised Domain Adaptation via Minimax Entropy. ICCV 2019.\n4. Feature alignment and uniformity for test time adaptation. CVPR 2023.\n\nLIMITATIONS:\nPlease see Questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a probabilistic framework that models the parameter adaptation for test-time adaptation (TTA) using stochastic differential equation (SDE). The authors show empirically that for stable adaptation, the log-variance distribution of the SDE's transition aligns close to an inverse-gamma (IG) distribution and that deviating from this alignment correlates strongly with model collapse.\nThus, they propose the Structured Inverse-Gamma Model Alignment (SIGMA) algorithm, based on this observation.\nSIGMA can be used with existing TTA approaches, and the authors have shown experimental results on the ImageNet-C dataset, showing improved performance for the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n* This paper focuses on the parameter dynamics using SDEs, which is different from existing works on TTA that focus on the loss function, parameter efficiency, pseudo-labeling, or sample selection.\n* The empirical observation that the log-variance of the SDE's transition aligns closely with an IG distribution is an interesting empirical result.\n\n**Weaknesses**\n* The experimental evaluation is limited. The experiments are performed only for the ImageNetC dataset benchmark, whereas existing approaches report their experiments on other benchmarks such as CIFAR10C, CIFAR100C classification, and/or Cityscapes-to-ACDC semantic segmentation.\n* Theorem 1 involves questionable assumptions. The paper's theoretical claim relies on Theorem 1, which is about the discretization of the SDE approximation. This theorem relies on the assumption that both the gradient and the variance term remain approximately constant within each discrete time-step (Appendix A.1, Line 494). This is a strong assumption in the context of the non-i.i.d. setting that the paper focuses on. When the model encounters an abrupt domain shift (say, moving from noise to blur), the loss landscape and, thus, the gradients can change dramatically. This paper does not show how well this assumption holds in practice.\n* The paper shows empirically that IG alignment correlates with stability, but there is no theory on the reason for this correlation. Is the correlation specific to a particular dataset, ImageNetC? Results on more datasets will help in supporting the claims made in the paper.\n\nQUESTIONS:\n1. The experiments are performed only for the ImageNet-C dataset benchmark. Whether the authors have performed experiments on other benchmarks such as CIFAR10C, CIFAR100C, etc.?\n2. How does the proposed approach perform in a more non-i.i.d. setting of continual TTA [1][2]? It is also noteworthy that [2] proposes a probabilistic framework, though it is different from the SDE-based probabilistic framework proposed in this paper.\n3. Could you provide experimental analysis showing how well the assumptions hold when there is a drastic distribution shift? \n4. Related to question 3, the continual TTA setting will be a more realistic test for this assumption, but more broadly, how well do the authors feel that the proposed approach will generalize to the continual TTA setting?\n\n**References**\n1. Wang, Qin, et al. \"Continual test-time domain adaptation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n2. Brahma, Dhanajit, and Piyush Rai. \"A probabilistic framework for lifelong test-time adaptation.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n\nLIMITATIONS:\nThe reliance of Theorem 1 on the strong assumption that both the gradient and the variance term remain approximately constant within each discrete time-step is a limitation that needs to be discussed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors empirically find that during successful test-time adaptation (TTA) the variance of the gradients seem to follow a log-normal distribution (or rather that the log-normal distribution can fit the gradient variance well). They show in various experiments that departure from this empirical phenomenon correlates strongly with bad adaptation performance. Based on this insight, they then design an adaptive update rule that can be combined with other TTA methods in the literature to increase the adaptation performance further. The adaptive rule can scale the learning rate accordingly to improve adaptation stability. It is shown in various experiments that TTA performance improves when this adaptive rule is combined with existing state of the art TTA methods.\n\nSTRENGTHS AND WEAKNESSES:\nI liked the experimental nature of the paper and the results seem quite promising. However, as mentioned below, the paper would benefit from a more thorough related works section and also on the motivation of the heavy tail phenomenon. Also, the continuous time formulation presented in Theorem 1 does not seem to be particularly useful or relevant to the rest of the paper, which is experimental and not theoretical. See below for more detailed comments on these issues.\n\nQUESTIONS:\n- The paper does not concern foundation models in particular, so the mention of foundation models at the start of the abstract is misleading and inappropriate. Moreover, The refs in line 22 have nothing to do with foundation models, they are CNNs for image recognition, of course they can be used with/inside foundation models but foundation models typically mean (sometimes multi-modal) LLMs.\n- Model collapse is not defined in the introduction adequately I think. Performance degradation does mean model collapse (line 157), which rather means catastrophic forgetting of the source I believe.\n- The first paragraph seems to lump two different (but related) topics: domain adaptation (DA) and continual learning (CL). The references seem to be mainly from DA whereas the discussion mostly relates to CL (online ML better translates to CL, especially if the particular setting of unsupervised DA is not mentioned). The two fields have different emphasis: very roughly, CL focuses on catastrophic forgetting, and memory related issues while DA focuses on alignment of distributions to transfer well to unlabeled target distribution. Test-time UDA does focus also on the harder problem of how to align the distributions when the source inputs are not available. From the paragraph's discussion the 'collapse' seems to relate more to the 'catastrophic forgetting' studied in CL, rather than to the 'negative transfer' of UDA methods. Hence I would recommend clarifying the discussion or at least choosing the references more carefully.\n- The next paragraphs seem to clarify somewhat the above distinctions, however the confusion in the first paragraph can be alleviated by explicitly mentioning/separating the UDA/CL fields in 1st paragraph explicitly.\n- Section 2.1 is repetitive: I would recommend streamlining with section 1, as the content is discussed there already. \n- But otherwise, the paper would definitely benefit from a wider and more detailed related works section, some of which should be streamlined with the main text. For instance, there is a whole series of research on the 'heavy tail' phenomenon in deep learning theory and practice (e.g. SGD, Hessians, learned weights etc.), which is not mentioned at all. Were the authors not influenced by this research direction? Also, Bayesian deep learning and the parameter distributions that evolve during SGD-like updates are not mentioned adequately (except briefly in the appendix).\n- Theorem 1 does not seem new at all, moreover (6) is I believe incorrect: Sigma_{k|k-1} should have Sigma_{k-1} added to sigma_k^{2}\\deltat^{2}I. More importantly I believe, Theorem 1 could have been derived just from the discrete case by propagating the Gaussian parameters along the gradient descent step, so it is not clear why the authors present a continuous time formulation. The parametric distributions can be recovered using, e.g., Bayesian or approximate Bayes methods. For instance in variational Bayes approaches, a lot of works present optimizers that include a KL-divergence term as a regularizer (to prevent distributional collapse), and can then formulate various forms of parametric distribution updates.\n- It is not clear why we need the conjugate step in the algorithm presented in section 5.1. The align step already scales the learning rate\n  and hence changes the model f_k right? Why do you need the conjugate step? Do you have ablations that show that this step is needed?\n- In the experiments, it would help to explain the Correlated Label setting more clearly.\n- Would be good to vary the fixed sequence of domains in the experiments, has this been done? What is the sequence used to generate the data shown in Tables 1-3?\n- What is the model used in section 6? Add more details for the experiments and streamline such discussion with the main text.\n\nMinor comments:\n- In eq. (1) w_hat's dependence on t_{k-1} is confusing: in (2) we minimize that function w.r.t w_hat so t_{k-1} seems to be erroneous.\n\nLIMITATIONS:\nI think the authors could say more about the limitations of the paper, as of now only the Gaussian assumption of the parameter distribution as the limiting factor is mentioned in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}