Lyapunov-Stable Adaptive Control for Multimodal Concept Drift
Authors: Tianyu Pan
Mengdi Zhu
Alexa Jordyn Cole
Ronald Wilson
Damon L. Woodard
Venue: NeurIPS 2025 poster ()
Keywords: multimodal learning; concept drift; adaptive control; online gradient descent; modality-specific fusion
Primary area: theory
TL;DR: This paper introduces LS-OGD, an adaptive framework for robust multimodal learning under concept drift.

Abstract: Multimodal learning systems often struggle in non-stationary environments due to concept drift, where changing data distributions can degrade performance. Modality-specific drifts and the lack of mechanisms for continuous, stable adaptation compound this challenge. This paper introduces LS-OGD, a novel adaptive control framework for robust multimodal learning in the presence of concept drift. LS-OGD uses an online controller that dynamically adjusts the model's learning rate and the fusion weights between different data modalities in response to detected drift and evolving prediction errors. We prove that under bounded drift conditions, the LS-OGD system's prediction error is uniformly ultimately bounded and converges to zero if the drift ceases. Additionally, we demonstrate that the adaptive fusion strategy effectively isolates and mitigates the impact of severe modality-specific drift, thereby ensuring system resilience and fault tolerance. These theoretical guarantees establish a principled foundation for developing reliable and continuously adapting multimodal learning systems.

Lyapunov-Stable Adaptive Control for Multimodal Concept Drift
Tianyu Bell Pan, Mengdi Zhu, Alexa Cole, Ronald Wilson, Damon Woodard
Department of Electrical and Computer Engineering
Florida Institute National Security Applied Artificial Intelligence Group University Florida
Gainesville, 32611
(tpan1, zhum, alexa. cole, ronaldwilsson) dwoodard@ece. ufl. edu
Abstract
Multimodal learning systems often struggle in non-stationary environments due to concept drift, where changing data distributions can degrade performance. Modelity-specific drifts and the lack of mechanisms for continuous, stable adapta- tion compound this challenge. This paper introduces LS-OGI,, a novel adaptive control framework for robust multimodal learning in the presence of concept drift. LS-OGD uses an online controller that dynamically adjusts the model's learning rate and the fusion weights between different data modalities in response to detected drift and evolving prediction errors. We prove that under bounded drift conditions, the LS-OGD system's ' prediction error uniformly ultimately bounded and converges to zero if the drift ceases. Additionally, we demonstrate that the adaptive fusion strategy effectively isolates and mitigates the impact of severe modality-specific drift, thereby ensuring system resilience and fault toler- ance. These theoretical guarantees establish a principled foundation for developing reliable and continuously adapting multimodal learning systems.
Introduction
Multimodal learning combines information from various data modalities, such as text and images, enhance understanding and predictions. Real- feal-world applications are naturally multimodal, such as autonomous systems leveraging cameras, LiDAR, and IMUs [22]; driver monitoring systems integrating visual and physiological data [0]; human-machine interaction using EOG and speech [17] and online platforms handling diverse media (1). A significant challenge arises when these systems operate in non-stationary environments, where the underlying data distribution changes over time, known concept drift. By definition, concept drift is change in the statistical properties of the target data distribution over time [8]. Practically, a model's past experiences may no longer be valid patterns, topies, styles emerge.
Concept drift is prevalent in real-world streaming data. For example, seasonal changes require weather prediction models to be adjusted periodically; shifting user preferences impact recommender systems; and social media or news topies can "drift" over weeks or months [18]. In multimodal misinformation detection, concept drift might appear as new vocabulary or slang in text, or new image memes and deepfakes visuals that were not present in the training data. These shifts in distribution can degrade a model's performance if it fails adapt. Traditional machine learning models typically assume stationary data distribution; therefore, the model's error rates increase when drift occurs, and previously learned decision boundaries may become misaligned.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Despite the prevalence of concept drift, current multimodal learning systems largely lack mechanisms to adapt to data distribution changes continuously. Most research on adapting to concept drift has focused on unimodal data streams (such as purely numeric or textual data) 38 J. Multimodal scenarios
introduce new challenges that have been underexplored: (1) Modality-specific drifts: One modality may experience drift while others remain stable, for example, the content of images may change while the relevance of the text stays constant. This situation requires adaptive fusion strategies that are unnecessary in single-modality settings. Traditional fusion techniques (like early or late fusion) rely on fixed combination rules, which become suboptimal under drift [3] (2) Detection complexity: Changes can be subtle and vary across modalities, complicating drift detection. Standard drift detectors often monitor a single error or a distribution metric [18]. In the context of multimodal data, the question arises: Which signal indicates drift? How can we determine whether the drift originates (3) Lack of theoretical guarantees: Adaptation mechanisms in machine learning are often heuristic. There is need for Lyapunov stability guarantees, which are essential in control theory for maintaining stable adaptive systems; however, these principles have not been fully applied to multimodal machine learning dealing with concept drift. Traditional adaptive control theory provides tools to ensure an adaptive system remains stable (7). but these tools need further exploration in multimodal machine learning. (4) Computational constraints: In real-world applications like content filtering systems, models must adapt in real-time without extensive retraining. Many existing concept drift solutions retrain a new model on recent data or reset parts of the model, which can be inefficient or impractical in live systems. There is a pressing need for mechanisms that can continually, safely, and efficiently adjust models as new data becomes available.
from the textual or visual channel?
This paper addresses the above challenges by proposing novel framework for Lyapunov-stable adaptive multimodal learning under concept drift (LS-OGD). Our key contributions are:
Formulation of Multimodal Drift Adaptation: We present, to the best of our knowledge, the first formal integration of concept drift theory into a multimodal learning setting that combines text and image data. We define the multimodal concept drift problem and discuss the challenges that arise when the informational value of one modality changes over time. Additionally, we identify shortcomings in current fusion methods and emphasize the necessity for an adaptive fusion mechanism capable of dynamically adjusting the contributions of cach modality.
We have developed an online adaptation controller that monitors the model's performance and dynamically adjusts two critical elements of the learning process: (1) the model's learning rate, which governs how quickly the model learns from new data compared to how well it retains information from past data, and (2) the fusion weighting between different modalities, which prioritizes the more reliable modality when drift is detected in the other. This controller is rule-based and lightweight. It detects drift through statistical changes in the model's error over time and triggers adjustments to the learning rate (1)) and the fusion coefficient (a). Our approach treats the prediction error as a feedback signal and establish adaptation rules similar to a PID controller to minimize the error.
Adaptive Control Strategy for Learning:
A key theoretical contribution of our work is developing a Lyapunov function for the adaptive learning system. We present two new theorems that establish conditions for the stability of the closed-loop learning process (comprising the model and the controller) in the sense of Lyapunov. Specifically, we demonstrate that, under mild assumptions, the system's error will remain bounded during periods of continuous drift and will converge to zero (or to an arbitrarily small value) once the drift ceases.
Lyapunov Stability Analysis:
2 Related Work
Concept Drift Detection and Adaptation: Drift detection techniques fall into three categories:
(1) Statistical Input Monitoring: testing distribution differences between recent and earlier data using methods like Kullback-Leibler (KL) divergence [8] or adaptive windowing (ADWIN) [19] (2) Performance Monitoring: tracking error rates against statistical thresholds [2 2 36), effective in supervised settings; and (3) Parameter Tracking: adaptation strategies include model retraining, prioritizing recent data, or using ensembles where new learners handle emerging patterns D 4 20 Most methods are designed for drift can affect individual modalities.
streams and may not suit multimodal systems where
2

---

Recent advancements in deep learning have spurred research into concept drift in neural networks and large models. While continual learning and concept drift address learning from non-stationary data, continual learning typically prevents catastrophic forgetting of previous tasks. In contrast, concept drift emphasizes rapid adaptation to evolving data. Some studies, like [8), propose extending concept drift theory to multimodal language models, recognizing their vulnerability to gradual shifts and sudden out-of-distribution events. During pre-training, they introduced a drift adapter module to address drift, highlighting growing focus on this issue in complex AI systems. However, their approach primarily targets pre-training bias rather than real-time adaptation. Our work stands apart by addressing online drift during deployment while ensuring stability.
Multimodal Fusion and Learning Techniques: Combining modalities improves prediction but presents fusion challenges [3]. Methods include early fusion (feature-level merging), late fusion (decision-level integration), bilinear pooling, attention mechanisms, and cross-modal transform-
ers [25 In fake news detection, specialized architectures have evolved from EANN's
3
adversarial iitention-based
training for topic-invariant representations to SpotFake 27 and MMDFND's
30
approaches. Recent bidirectional cross-modal fusion (BCMF) 339 enables information flow between text and image encoders, representing significant progress in multimodal integration techniques.
Despite recent advancements, most multimodal models operate under a fixed fusion strategy once trained. During training, the model learns to weigh and utilize different modalities, but these weights remain static afterward. If a particular modality's 's relevance changes over time, the model cannot easily reconfigure itself due to drift. For example, a fake news detection model might initially rely heavily on textual patterns to identify deception. However, if bad actors use innocuous text paired with deceptive images over time, the model's original fusion scheme could misallocate attention and risk missing the visual cues. Some recent works [25] 37 [5] have explored dynamic modulation of importance; for instance, a model might use an attention mechanism to emphasize the modality that provides a more significant "signal" for a given sample. However, this approach usually focuses on the content of the sample itself rather than detecting distributional shifts over time. In contrast, our proposed method explicitly adjusts fusion weights in response to drift over time. We conceptualize this as an adaptive late fusion approach, introducing a controllable parameter (xt) to interpolate between modalities. This idea of adaptive fusion has limited precedents; related concepts include gating networks or conditional fusion, in which an auxiliary network predicts fusion weights for each sample. We go further by making a funetion of time and past errors, rather than relying on the current sample, thus directly addressing the temporal drift in modality utility.
Preliminary and Problem Formulation
Concept Drift Definition: We consider a supervised learning problem in an online setting. At each time step = 3,.,,,.... the system receives a new sample (z, (1) 2t 1...,
(M) 31), where (Mr) denotes the input from the M-th modality (for instance, M = 2, x²(1) is text and (2 is image), and y/ is the ground-truth label or target. The crucial aspect that the data distribution is time-dependent. Let D. denote the distribution generating (zz ....,,,
y/ at time t. Concept drift means D, changes over time: there exists some i and + A for which D. o D.+ Following standard definitions, concept drift occurs at time t if the joint probability (x ..... X X(M), Y) is statistically different from P P ,.... X(M), Y). These changes can be abrupt or gradual. This paper assumes a piecewise-conee coonsstt drift for theoretical analysis (drift occurs at discrete points, dividing the timeline into relatively stable distribution segments) and later simulates the drifts.
(M)
Multimodal
Learning Model: Define model ()) ,0()) (.(2) (M)) with parameters 0 that
produces prediction Dr given the multimodal input, and the model can be decomposed by modality. For instance, for M- = 2, we have two sub- net-networks: text encoder (()) 8,) and an image encoder p22 (z) 82). producing modality-speciff feature representations. A fusion function then combines these. In our design, the fusion is a weighted combination: let ( and 2f be scalar logit outputs (final hidden features) from the text and image pipelines, respectively. We define the fused logit
2 ar 2t (1) + (1 az) -2,2
(I)
where or E [0, 1] is a fusion weight at time 1. The prediction ÿ is then obtained by applying a sigmoid softmax to 2t. The parameter a effectively controls the influence of cach modality:
3

---

ar = 0.5 would weight them equally, + relies mostly on modality 1, and - 0 relies on modality 2. The rest of & encompasses the weights of f(), f(2), and possibly any additional layers (e.g.. if the features are concatenated and passed an MLP, those MLP weights are in 0).
Let L(tr; M/) be the loss at time (e.g. cross-entropy loss for classifica- tion). We define the instantaneous prediction error e the difference between the predicted output and the true output in an appropriate sense. For analysis, assume et is a real-valued error signal (for instance, e, could be the difference in probability assigned to the true class vs. the ideal, or some bounded transformation of the loss). We assume e, = 0 corresponds to perfect prediction, and larger leal means worse performance. The exact definition of e, will be specified in proofs, but intuitively, one can think of e as proportional to the classification error or loss at time t. We also define a desired equilibrium for this error: we desire et O. However, under drift, the minimum achievable error might change over time as the optimal model changes.
Loss and Error Measures:
Adaptation Controller:
We augment the learning process with an adaptive controller that adjusts two sets of parameters over time: the model weights 0 are updated via stochastic gradient descent (or similar optimizer) with learning rate n) and the fusion weight as is also updated on slower timescale. The controller has two primary responsibilities: drift detection and parameter adaptation.
Drift detection detects when significant increase in error e; indicates possible concept drift. This could be done by checking if e exceeds some threshold or deviates significantly from a long-term emror trend. In our implementation, we use sliding window over the recent W samples to compute
é -+I
Σ C
(2)
The controller flags a drift onset if é exceeds dynamic threshold (for example, mean enror k standard deviations in a reference period). Furthermore, parameter adaptation works when drift is detected, the controller computes adjustments An and Aa. Specifically, we define update rules:
1 1/: Anp 0x++1 = 02 + AoE
to reduce future errors.
The adaptation logic is as follows: (1) Learning rate adaptation: If the error is increasing, this suggests the current model is not keeping up with changes. The controller increases the learning rate: An, set to a positive value proportional
the error increase and possibly the error magnitude. A simple rule could be An: = (e, et-1) when ét > et-1, for some gain hi >> 0. To prevent divergence, we also cap 7/ not to exceed a safe maximum. Conversely, if error starts decreasing (model catching up), the controller may slowly decrease 1) to normal level to avoid overshooting (much like lowering gain as we approach setpoint). This mechanism ensures fast learning during drift and stable convergence afterward. (2) Fusion weight adaptation: The controller adjusts a to put more weight on the modality that currently appears more reliable. We can estimate modality-speeific error contributions. For instance, we can compute an "imagined" prediction using only modality 1. of( sigmoid ((()) ), and similarly 2²(²) using only modality 2. By comparing these y/, the controller can judge which modality alone would perform better on recent data. Suppose modality I's predictions are significantly more accurate than modality 2's. In that case, implies modality I carries the useful signal and modality 2 might be drifting or noisy, so we increase a to rely more on modality 1. In the opposite case, we decrease a. Formally, one could maintain separate error metrics (1) and e(2) for the two modalities. Then, Aa; can be set by a rule like Ao, = ka (e - if modality 2's error is higher, Aa, is positive (increase weight on modality 1); if modality error is higher, Day is negative (shift weight to modality 2). We clamp a within [0.1]. This update might be done conservatively (lower frequency or smaller step size) than the learning rate, since fusion changes too often could destabilize training. This slow actuator nudges the model's ' attention towards the currently best modality.
State-Space Representation: We can conceptualize the system's 's evolution with a state vector that includes at least the error and possibly the parameter deviations. For analysis, consider the state as xt et (the error at time f). The error evolves depending on the changing data distribution and our adaptive updates. We update model weights online using mini-batch stochastic gradients. A simplified dynamics can be written as:
G = F(er, Oie, Thi 5z),
(5)

---

where ón represents the disturbance due to concept drift (i.e., how the optimal model's 's output would change from & due distribution shift). The exact form of F is complex since involves the data and the model's optimization landscape. However, we can reason qualitatively: if there were no drift (6, 0) and the model perfectly matched the current concept, then ideally e, would be zero for all f. If the model is imperfect but we fix and small 1), gradient descent will tend to reduce e over time toward some residual error (training convergence). Under drift (ò, * 0), the error will increase unless is large enough to track the change. Our controller effectively modulates F by changing % and Ct..
Furthermore, we introduce candidate Lyapunov function to analyze stability:
1 V(I
2 _
(6)
This V is always nonnegative and V O only when the error is zero, meeting the criteria of a Lyapunov function candidate. We aim to design the adaptation laws (An,, Aa functions of and possibly Ae) such that (t) does not increase significantly and preferably decreases after some transient, despite drift.
Proposed Algorithm: Algorithms and 2 in Appendix outline the main online learning process and the controller logic. Figure also presents the proposed system architecture.
Module
Monitoring
Ststition Datiiuutor Tracker
Alemn Triawing
Contrct lilttricall
Vodal speciti
to cariuuiee
zagatts Cncetatty
Prodiction Modulee
Input
Awoghttd compoleddyy Paramater
tokie prrmmcer
Clasoiicaioo on
Sigeed samiiety
cormi
maRTRuSATNNe information
Image
Fastonat and Represerration
Space
Poformmncc Eutluation
rechrem
Ograll
Model Updatees Cyole...
Upditee parterateess
Variy Stacille
Madvito
Controller
Figure 1: Illustaations of the proposed system architecture. Dashed and solid arrows represent the feedback signals and main data flow, respectively.
Main Results
This section presents the theoretical analyses of the proposed LS-ODG framework. Our analyses establish the learning system's is stability and adaptability when operating in non-stationary multimodal environments.
4.1 Foundational Axioms and System Properties
To analyze the main framework, three axioms are defined:
5

---

Axiom 1 (Bounded Drift Rate) Concept drift, represented by the perturbation ò5 in the error dynamics or by the rate of change of optimal model parameters ||0 1. assumed to be bounded. That is, there exists constant dmar > 0 such that || I 5mar for all 6.
This axiom is fundamental because arbitrarily fast large drift would make it impossible for any adaptive system with finite learning capacity to maintain low error. It ensures that the "target" the system is trying to track does not move so erratically to preclude effective adaptation.
Axiom (Sufficient Model Capacity) The multimodal learning model So possesses sufficient capacity such that for any stationary concept D encountered (i.e., a period where D, is fixed), there exist optimal parameters 6* and a fusion weight a" that can achieve the minimum error e, e(("". (f",a")) aa))
This axiom ensures that performance limitations are primarily due to drift or adaptation transients, not due to the model's inherent inability to learn the underlying concepts. It allows us to focus the analysis on the dynamics of adaptation rather than on model approximation errors for a fixed concept.
Axiom 3 (Lipschitz-like Error Dynamics) The error dynamics function F(er, Qtzy Mei 6e) and the loss landscape exhibit local Lipschitz continuity with respect to e, and the adaptable parameters 0 (implicitly via 17.) and ot, This means that small changes in parameters or error lead proportionally bounded changes in the subsequent error or its gradient.
This axiom ensures that the gradient-based updates and the adaptive adjustments lead to predictable changes in the error, preventing chaotic or overly sensitive responses. This helps analyze the system using local approximations and ensures that the Lyapunov function's evolution is well-behaved.
4.2 Characterizing the Evolution of the Lyapunov Function
Before presenting the main stability theorem, we introduce Lemma 1. This Lemma characterizes the change in the Lyapunov function ( V(t + 1) - Vtt) over one time step, under the influence of the learning process, the adaptive controller, and the external drift.
Lemma 1 (Bounded Increment of the Lyapunov Function with Adaptation) Given the system dynamics ct+1 = F(ee, O4t, Me; 5e) and the adaptive update rules for 1/2 and ag as described in Section and Algorithm 2] Assuming Axiom 1-3 hold, and the learning rate ne is positive and the adaption gains ki and ko, are chosen within stable regions, then the change in the Lyapunov function, (V(()= V(l 1) V(I), can be bounded as:
AV < m + 72th leel ||6cl| + nanr (g + [1661() + +0 (IIar))),
(7)
where 71 1/ and 7/3 are positive constants.
In Lemma 1, the first term --1 ef is referred to as the stabilizing term and reflects the error reduction due to the learning algorithm attempting to minimize the current loss. Y1 is related to the learning effectiveness, i.e., local convexity or gradient dominance property from Axiom 3. The second term ee ||| |6|| is referred to as the disturbance term, which measures the impact of concept drift on the error. The third term 737r (g (e), + [1%,1²) is the noise term, evaluating the nonlinearities or gradient stochasticity in SGD. The final term O Aolll) is referred to the fusion adaptation term, representing the effect of changing fusion weight on the error.
For the candidate Lyapunov function V(()- je7. Lemma I establishes that the stabilizing effect of learning must overcome all three other terms to decrease error. This balance mediated by the adaptive learning rate 7/. For instance, if |eel is large, the controller tends to increase y), therefore increasing the stabilizing term. Additionally, since (t( remains negative when lez| is sufficiently large, even when ó, # O, Lemma forms the basis for proving the uniform ultimate boundedness (UUB). The asymptotic stability is achieved without concept drift (i.c., 6, = 0).
Remark (On the Choice of Lyapunov Function and Adaptation Gains) The choice of (t)) Jet an intuitive selection for tracking error, relating the "energy" of the system to the squared prediction error. The condition in Lemma I and Theorem regarding "appropriately small" adaptation gains (k,, k.)) common in adaptive control. If gains are too large, the system might overreact to errors or drift, leading to oscillations or instability, violating the conditions needed for (V to be reliably negative or bounded as desired. The theoretical existence of such stable gain regions is key, and practical selection often involves tuning guided by these theoretical constraints.

---

4.3 System Stability and Error Convergence under Bounded Concept Drift
The first main theoretical result concerning the overall stability of the adaptive learning system is presented with the Axioms and Lemma 1.
Theorem (Lyapunov Stability under Bounded Drift) Consider an adaptive multimodal learning system with error dynamics characterized by c++1 = F (ee, or, m) 62), operating under Axioms 1-3.
Assume that the adaptive controller adjusts 7/2 and o, such that Lemma holds, and the adaptation gains kn and ka are appropriately small, then the closed-loop learning system is Lyapunov stable (i.e., the prediction error e, is UUB). This implies that the ultimate bound on the error é and the bound on the initial error By exist, and are positive constants, such that the error trajectory would remain bounded, and eventually stays within the ultimate bound Eg if letgl Bu, where et, denotes the initial error
Theorem establishes formal I. Lyapunov stability guarantee in adaptive multimodal learning under concept drift for the first time. Unlike heuristic approaches, this provides a mathematical foundation for the system's behavior. The UUB acknowledges that the error may not always converge in persistent drift. Instead, UUB guarantees that the error will eventually enter and remain within small, predictable region around zero. The size of this region, g,, will depend on the magnitude of the drift (Axiom 1) and the system's capacity to adapt. Also, this theorem guarantees that if the environment stabilizes (i.e., 6, 0), the system will fully recover, with the prediction error converging to zero. This demonstrates the system's 's ability to not only manage ongoing drift but also achieve optimal performance once stability is restored.
Corollary (Asymptotic Stability in Stationary Environments) If the environment is stationary from the outset (i.e., 6, for all fo), and Axioms 2-3 hold, then the LS-OGD system with an appropriately chosen, potentially constant, learning rate v) and fusion weight a ensures that the prediction error e, converges asymptotically zero (i.e., e, 0 as oo).
This corollary simplifies Theorem for the non-drifting case. confirms that the learning mechanism within LS-OGD is sound and can achieve optimal performance when the environment is stable. The UUB property of Theorem I then extends this, showing robustness by guaranteeing bounded error even when the stationarity assumption is violated by bounded drift.
4.4 Targeted Adaptation the Fusion Weight
The second main theorem concerns the system's ability adapt its fusion strategy. This relies on the controller's s ability to adjust correctly. Hence. Lemma 2 is presented as a key for Theorem 2.
Lemma 2 (Convergent Dynamies of Fusion Weight a under Unilateral Modality Drift) Given the fusion weight a, on modality 1, the fusion weight on modality 2 is defined as at-
Consider the fusion weight update rule:
0++1 = c 1) (ar + Aoz), Aoy = ka (et (c(2).est (1),oat
(8)
(m)) being the estimated error from modality m, and ko > 0 is the adaptation gain. where et
Following the update rule for fusion weight, if modality becomes less reliable than modality (1),est
leading to ec et
(2),est + for f. >> to for some positive margin ca, then:
ii) O₁ will tend to decrease.
(ii) Provided ko is sufficiently small to ensure smooth adaptation and prevent overshoot. The error estimation difference persistent, a will converge towards 0 or a small value close to 0 if the error difference is not consistently large or noise is present in the error estimates.
Lemma 2 describes how the controller adjusts a as one of the modalities is less reliable. The update rule form of gradient descent on an implicit objective that seeks to minimize reliance on the modality with higher perceived error. It confirms that the fusion adaptation mechanism reduces the weight of a consistently underperforming modality, even being able "switch of" a compromised modality by setting a₁ to 0 or depending which modality is less reliable.
7

---

Proposition 1 (Convergence Rate of Fusion Weight a:) Under the conditions of Lemma 2. assume
that modality 1 is consistently with low reliability (i.e., et (1), > e (2),cst <a for all to), the fusion weight is expected to be O. Given that a is not yet converged to its boundary, then the expected decreasing rate of a is approximately linear:
E [ar+1 as] -ko E (1)
(2) c
E
The steps to boundary convergence is therefore inversely proportional to ka and the persistent magnitude of the estimated error difference, barring noise in error estimation.
Proposition 1 provides insight into the dynamics of the fusion adaptation. While Lemma 2 states the tendency and convergence, this suggests that the speed of this adaptation is tunable via ki and directly influenced by how clearly the system can distinguish the performance of the modalities. A larger error difference allows for faster isolation of the faulty modality. This has a practical implication for setting ka: larger ka leads to faster fusion adaptation but must be balanced against the risk of instability or overreaction to noisy error estimates (as per Remark 1). Building upon Lemma 2 and Proposition 1, Theorem 2 establishes the system's 's robust response in the case of severe drift affecting a single modality.
Theorem 2 (Modality Adaptation and Error Reduction under Severe Unilateral Drift) Consider the same scenario defined previously, where modality consistently less reliable. If the adaptive controller adjusts ar such that Lemma 2 holds, then:
ii) The fusion weight a will converge towards O, decreasing and eventually nullifying the influence of the compromised modality 1.
(ii) Consequently, the system's 's overall prediction error will, after this reconfiguration transient, asymptotically approach the error level achievable by an optimal unimodal model utilizing only the reliable modality 2.
(iii) The Lyapunov stability of the system is preserved throughout this fusion reconfiguration process and thereafter, with the ultimate error bound being determined by the performance achievable with modality 2 and any residual drift or noise in its stream.
Theorem indicates that the internal structure of the model is actively changing, making this model different from traditional multimodal systems, as traditional systems often suffer from persistent drop in performance 42 34]. Theorem 2 also shows that the adaptive fusion mechanism can correctly identify and isolate a "fault" modality by driving its weight (ov,) towards zero. This theorem quantifies the system's 's performance post-adaptation as it converges to the best possible performance using the remaining valid information sources. Additionally, system behavior with partial modality degradation and practical estimation of modality-specific errors are presented Appendix C The full proofs for our main results are in Appendix D
5 Experiment
To empirically validate LS-OGD's theoretical contributions and its capacity to address key challenges in multimodal concept drift, we conducted experiments on the M3A (55 classification dataset. After initial training in stable environment (Phase 1), the multimodal system was subjected to significant concept drift (Phase 2). Full experimental setup and additional results are detailed in Appendix
First, Figure 2 demonstrates LS-OGD's ability to maintain stability. The controller's estimated drift signal, empirical proxy for the bounded drift o, (Axiom 1), frequently activates during Phase 2. Correspondingly. while fluctuating with drift, the system's 's prediction error remains contained and is effectively driven down by the adaptive responses. This behavior empirically corroborates the UUB of the error, guaranteed by Theorem 1, and the error-reducing system dynamics characterized by Lemma 1, indicating stable learning in a non-stationary environment.
Second, Figure 3 reveals how LS-OGD addresses modality-speciie challenges. Dynamic learning rate adjustments enhanced plasticity during instability, aligning with Lemma 1 and stable gain principles (Remark 1). More importantly, when confronted with severe unilateral modality drift,
'The code is available at http://gttaaccoooBBBBBllnllnlllp Staae-Sttaaaaaaaae Adaptive-Control-ee Coltemo
8

---

15-322 Signel (Phan
Mph
Staps 000 Phone
Figure LS-OGD Error Signal Drift Estimation Phase
the fusion parameter ay underwent rapid and significant shift. This sharp re-weighting. isolating the compromised modality and preserving performance by relying the healthier one, empirically confirms the adaptive fusion dynamics (Lemma 2) and the fault-tolerant capabilities (Theorem 2).
This tackles the limitations of fixed fusion strategies and demonstrates robust, stable adaptation modality-speeifee drift, core challenge highlighted in the introduction.
Figure 3: LS-OGD Controller Adaptation Signals in Phase 2
6 Conclusion
The LS-OGD framework represents a significant advancement in adaptive Al by combining Lyapunov- based control theory with online gradient learning and dynamic fusion weights. This integration allows for stability and resilience in the face of non-stationary, multimodal inputs. The framework's core assurance is that prediction errors will remain consistently bounded under limited drift and converge to zero as disturbances diminish. This ensures predictable performance, even when data distributions change. Moreover, LS-OGD features modality adaptation mechanism that automatically down-weights unreliable sensors or data streams, allowing the system to concentrate learning on trustworthy information sources. This lightweight, online approach is suited for real-worke applications, such as autonomous vehicles that need to adjust the importance of inputs from vision, LiDAR, and radar under varying conditions; intelligence systems that integrate imagery, signals, and human reports while facing adversarial pressures; and misinformation detectors that combat evolving text and image manipulations. By establishing robust framework dynamic sensor re- weighting and bounded-error learning, LS-OGD lays solid foundation for trustworthy AI. Its modular design also allows for extensions to nonlinear models, deep architectures, and context-aware adaptation strategies. Limitation and broader impact discussed in Appendix G

---

7 Acknowledgment
The authors sincerely thank the program and area chairs, along with the anonymous reviewers, for their constructive comments on this work. Additionally, Bell expresses her gratitude to her beloved pets, Kiwi (an Australian Shepherd) and Meemeow Pan (a Siamese Tabby), for their emotional support.
References
[1] Stephen H Bach and Marcus A Maloof. Paired learners for concept drift. In 2008 Eighth IEEE International Conference on Data Mining, pages 23-32. IEEE, 2008.
[2] Manuel Baena-Garch José del Avilo, Raul Fidalgo, Albert Bifet, Ricard Gavalda, and Rafael Morales- Bueeeo. Early drift detection method. In Fourth international workshop on knowledge discovery from data streams, volume 6, pages 77-86 Citeseer, 2006.
[3] Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learn- ing: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(2):423-4433 443, 2018.
[4] Albert Bifet and Ricard Gavalda. Learning from time-changing data with adaptive windowing. In Proceedings of the 2007 SIAM international conference data mining, pages 443 448. SIAM, 2007.
[5] Giuseppe Canonaco, Alex Bergamasco, Alessio Mongelluzzo, and Manuel Roveri. Adaptive federated learning in presence of concept drift. In 2021 International Joint Conference on Neural Networks (ICCN), pages 1-7. IEEE, 2021.
[6] Haoyu Chu, Shikui Wei, Ting Liu, Yao Zhao, and Yuto Miyatake. Lyapunov-stable deep cquilibrium models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 11615-11623, 2024.
[7] Hongkai Dai, Benoit Landry, Lujie Yang, Marco Pavone, and Russ Tedrake. Lyapunov-stable neural-network control. arXiv preprint arXiv:2109.1413 2021.
[8] Tamraparni Dasu, Shankar Krishnan, Suresh Venkatasubrammanian, and Ke Yi. An information- theoretic approach to detecting changes in multi-dimensional data streams. In Proc. Symposium on the Interface of Statistics, Computing Science, and Applications (Interface), 2006.
[9] Bogdan Doicin, Marian Popescu, and Cristian Patrascioiu. Pid controller optimal tuning. In 2016 8th International Conference Electronics, Computers and Artificial Intelligence (ECAI), pages 1-4, IEEE, 2016.
[10] Anthony Dontoh, Stephanie Ivey, Logan Sirbaugh, Andrews Danyo, and Armstrong Aboah. Visual dominance and emerging multimodal approaches in distracted driving detection: A review of machine learning techniques. arXiv preprint 2rXiv:2505.01973 2025.
[11] Joao Gama and Gladys Castillo. Learning with local drift detection. In International conefatttoallcoofeenccee on advanced data mining and applications, pages 42-55. Springer, 2006.
[12] Joao Gama, Pedro Medas, Gladys Castillo, and Pedro Rodrigues. Learning with drift detection. In Advances in Artificial Intelligence-SBil 2004: 17th Brazilian Symposium on Artificial Intelligence, Sao Luis, Maranhao, Brazil, September 29-Ocotber 1, 2004. Proceedings 17, pages 286 Springer, 2004.
[13] Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. Note: Robust continual test-time adaptation against temporal correlation. Advances Neural Information Processing Systems, 35:27253-27266, 2022.
[14] Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, and Sung-Ju Lee. Sotta: Robust test-time adaptation on noisy data streams, Advances in Neural Information Processing Systems, 36: 06s14070--4099,, 2023.
10

---

[15] Zongbo Han, Fan Yang, Junzhou Huang, Changqing Zhang, and Jianhua Yao. Multimodal dynamics: Dynamical fusion for trustworthy multimodal classification. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 20707-20717, 2022.
[16] Vijay Manikandan Janakiraman, XuanLong Nguyen, and Dennis Assanis. A Iyapunov based stable online learning algorithm for nonlinear dynamical systems using extreme learning machines. In The 2013 International Joint Conference Neural Networks (IJCNN), pages 1-8.
IEFE, 2013.
[17] Kendi Li, Di Chen, Zuguang Rao, Zijing Guan, Ya Jiang, and Yuanqing Li. A multimodal asynchronous human-machine interaction method based on electrooculography and speech recognition for wheelchair control. IEEE Sensors Journal, 2024.
[18] Anjin Liu, Yiliao Song, Guangquan Zhang, and Jie Lu. Regional concept drift detection and density synchronized drift adaptation. In IJCAI international joint conference on artificial intelligence, 2017.
[19] Anjin Liu, Guangquan Zhang, and Jie Lu. Fuzzy time windowing for gradual concept drift adaptation. In 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE), pages 1 1-6. IEEE, 2017.
[20] Dong Liu, YouXi Wu, and He Jiang. Fp-elm: An online sequential learning algorithm for dealing with concept drift. Neurocomputing, 207:322-334, 2016.
[21] Moyang Liu, Kaiying Yan, Yukun Liu, Ruibo Fu, Zhengqi Wen, Xuefei Liu, and Chenx- ing Li. Exploring modality disruption in multimodal fake news detection. arXiv preprint arXiv:2504.00 09154, 2025.
[22] Shuang Ma. Sai Vemprala, Wenshan Wang, Jayesh K Gupta, Yale Song, Daniel McDufft, and Ashish Kapoor. Compass: Contrastive multimodal pretraining for autonomous systems. In 2022 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1000-1007. IEEE, 2022.
[23] Kohei Miyaguchi and Hiroshi Kajino. Cogra: Concept-drift-awant stochastic gradient descent for time-series forecasting. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pages 4594 4601, 2019.
[24] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. arXiv preprint arXiv:2302.124000 12400, 2023.
[25] Stefanos-lordanis Papadopoulos, Christos Koutlis, Symeon Papadopoulos, and Panagiotis C Petrantonakis. Red-dot: Multimodal fact-checking via relevant evidence detection. JEEE Transactions Computational Social Systems, 2025.
[26] Omkar Sudhir Patil, Due M Le, Emily J Griffis, and Warren Dixon. Lyapunov-based deep residual neural network (resnet) adaptive control. arXiv preprint arXiv:2404.07385 2024.
[27] Shivangi Singhal, Rajiv Ratn Shah, Tanmoy Chakraborty, Ponnurangam Kumaraguru, and Shin ichi Satoh. Spotfake: A multi-modal framework for fake news detection. In 20/9 IEEE fifth international conference on multimedia big data (BigMM), pages 39 47. IEEE, 2019.
[28) Junha Song, Jungsoo Lee, Kweon, and Sungha Choi. Ecotta: Memory-efficien continual test-time adaptation via self-distiledd regularization. In Proceedings the IEFF/CV Conference on Computer Vision and Pattern Recognition, pages 1 11920-11929, 2023.
(29) Runhan Sun, Max L. Greene, Due M Le, Zachary I Bell, Girish Chowdhary, and Warren E Dixon. Lyapunov-based real-time and iterative adjustment of deep neural networks. IEEE Control Systems Letters, 6: 6:193-198, 2021.
[30] Yu Tong, Weihai Lu, Zhe Zhao, Song Lai, and Tong Shi. Mmdfnd: Multi-modal multi-domain fake news detection. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 2024.
11

---

[31] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726 2020.
[32] Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 7201-7211, 2022.
[33] Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. Eann: Event adversarial neural networks for multi-modal fake news detection. In Proceedings of the 24th acm sigkdd international conference on knowledge discovery & data mining, pages 849-857, 2018.
[34] Renjie Wu, Hu Wang, Hsiang-Ting Chen, and Gustavo Carneiro. Deep multimodal learning with missing modality: A survey. arXiv preprint arXiv:2409.07825, 2024.
[35] Qingzheng Xu, Huiqiang Chen, Heming Du, Hu Zhang, Szymon Fukasik, Tianqing Zhu, and Xin Yu. M3a: A multimodal misinformation dataset for media authenticity analysis. Computer Vision and Image Understanding, 249: 249:104205, 2024.
[36] Shuliang Xu and Junhong Wang. Dynamic extreme learning machine for data stream classifica- tion. Neurocomputing, 238:433-4444 449,-4492220 2017.
[37] Zihui Xue and Radu Marculescu. Dynamic multimodal fusion. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 2575-2584, 2023.
[38] Xiaoyu Yang, Jie Lu, and En Yu. Adapting multi-modal large language model to concept drift from pre-training onwards. arXiv preprint arXiv:2405. 13459, 2024.
[39] Chuanming Yu, Yinxue Ma, Lu An, and Gang Li. Bemf: A bidirectional cross-modal fusion model for fake news detection. Information Processing & Management, 5965):033633, 2022.
[40] Dan Zhang and Bin Wei. A review on model reference adaptive control of robotic manipulators. Anmual Reviews in Control, 43:188-198, 2017.
[41] Wenyu Zhang. Pola: Online time series prediction by adaptive learning rates. In ICASSP 2021- 202i IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3375-3379. IEEF, 2021.
[42] Tianyi Zhao, Liangliang Zhang, Yao Ma, and Lu Cheng. A survey on safe multi-modal learning systems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6655-6665, 2024.
[43] Fangyu Zou, Li Shen, Zequn Jie, Weizhong Zhang, and Wei Liu. A sufficient condition for convergences of adam and rmsprop. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 1 11127-11135 2019.
12

---

Appendices
More Related Work
14
A.I
Concept Dritt Detection and Adaptation
14
A.2 Adaptive Control Strategies and Learning Rate Adjustment
14
A.3
Lyapunov Stability in Learning Systems
15
B
Proposed Algorithm
16
B.i
LS-OGD Main Adaptation Loop
16
B.2
LS-OGD Controller Logic
16
Additional Results
18
Example I. Scalar Error Dynamics
18
Corollary 2. System Behavior with Partial Modality Degradation
Remark 2. Practical Estimation of Modality- Errors
18
44
Remark 3. Continuous- Time Analysis Discrete-Tinee Implementation
19
Example 2. Fusion Weight Dynamies
19
D
Proofs
20
Proof for Lemma
20
D.2 Proof for Lemma
20
D.3 Proof for Proposition
21
D..4
Proof for Theorem
21
D.5
Proof for Theorem
22
Additional Experiment Details
24
E.1
Dataset and Preprocessing
24
Experimental Setup
25
E.3
Evaluation Metrics
26
E
4
Additional Results
26
Test- Time Adaptation (TTA) vs. LS-OGD
29
Limitation and Broader Impact
30
NeurIPS Paper Checklist
32
13

---

A More Related Work
A.1 Concept Drift Detection and Adaptation
In machine learning, "no drift" scenario signifies a stable environment where the joint probability distribution of input features and the target variable, P(X,Y) Y), remains constant over time. Conversely, concept drift occurs when this joint distribution changes, (X, Y) & Px(X, Y) for different time points t and t', and can manifest in sudden, gradual, or incremental patterns. This broader concept can be broken down into two types: real concept drift, which directly impacts predictive modeling by altering the conditional distribution of the target given the input, P.(XIY) meaning the fundamental relationship the model learns
- Y) changes; and virtual drift, where only the input data distribution P, (X) shifts while the conditional relationship (Y|)) remains unchanged. Figure presents the different types of concept drifts.
Original Data No Drift
Concept Drift pfr(x) changes
Virtual Drift
pxx) changes but not p(ylx)
Figure 4: Types of concept drifts: The circles in this figure represent instances, and the different colors represent different classes.
A.2 Adaptive Control Strategies and Learning Rate Adjustment
Adaptive control provides framework for real-time adjustment of system parameters to maintain stability and performance. In machine learning, the learning rate is critical parameter often used with fixed scheduled values, which be problematic in concept drift. A low learning rate can hinder the model's s ability to learn new concepts, while a high rate may lead to overfitting or instability, Research [11 43) indicates that dynamically adapting the learning rate can enhance a model's ability to respond to distribution shifts. For instance, R.MPPrp and Adam adjust step sizes based on gradient statisties but are not specifically designed for concept drift. Some approaches, like COGRA (23). dynamically tune the learning rate during time-series forecasting by increasing it when prediction error variance rises and decreasing once performance stabilizes. Similarly, (5) adapts learning rates in federated learning to handle drifting data better. These methods demonstrate the benefits of a higher learning rate post-drift for quicker adaptation. Inspired by these principles, our controller increases the learning rate upon detecting drift to adapt to new data swiftly, then gradually reduces as the error decreases. This mirrors gain scheduling in control, where one variable (error rate) influences another (learning rate) to optimize performance.
In addition learning rates, the field of adaptive control presents concepts such model reference adaptive control (MRAC) 44 and Lyypunov- adaptation laws 229, which ensure stability for dynamic systems with unknown parameters. recent research that combines control theory and deep learning, (26) developed Lyapunov stable weight update rules for deep neural network controllers. Although their research focuses on robotic control loop, the key insight is that parameter update equations can be designed to reduce a Lyapumov function gradually. Our paper treat the learning process itself as a system that needs be controlled. Additionally, there's a relevant concept in which some researchers have utilized PID control for optimizer tuning [9; they treat the training error like a process variable and employ PID controllers to adjust hyperparameters or even the magnitudes of gradients. Our method can be viewed as specialized version of this approach: when the error increases, the rate of change of the error prompts a proportional rise the learning rate, similar to P/PI controller aiming to minimize the error.
14

---

A.3 Lyapunov Stability in Learning Systems
Lyapunov function decreases over time concerning the system's 's state, guaranteeing that it will not diverge from its equilibrium position. In control theory, this provides strong assurances about both convergence and robustness. However, applying Lyapunov stability analysis sis machine learning systems is relatively new [16]. Traditional training methods often lack formal stability guarantees, and neural network training can face issues such as exploding gradients or oscillations when hyperpa- rameters are not chosen carefully. Recent efforts have begun to integrate stability theory into learning algorithms. For example, [16] proposed Lyapunov- -based online learning algorithm for nonlinear systems, ensuring the model's prediction error remains bounded. Researchers have incorporated Lyapunov-based constraints deep reinforcement learning to maintain stability an agent's value function or dynamics during the learning process [6]
In adaptive neural controllers, where a neural network controls physical system, attempts have been made to enforce Lyapunov stability of the closed-loop system. [7] developed neural network policy alongside a neural Lyapunov function that certifies stability within a certain region. This was a verification problem where a mixed-intess integer program checks the Lyapunov conditions for the neural policy. Such research confirms that it is feasible to train or constrain neural networks while considering Lyapunov criteria. However, these studies mainly focus on controlling external processes, whereas we emphasize the stability of the learning algorithm and its fusion mechanism.
The key innovation in our work is the application of Lyapunov stability analysis to adapt to concept drift in multimodal learning. We construct a Lyapunov function based on the dynamics of the model's error. Intuitively, the eπror signal parallels the state of a dynamical system. Our adaptive update rules for the learning rate and fusion weight are designed ensure this error diminishes over time, even in the face of external disturbances caused by drift. This approach is reminiscent of MRAC, where adaptive updates drive the difference between system and reference model to zero 26] In this context, the "reference" is the new concept (or zero error), and we adapt parameters to track it. To the best of our knowledge, no prior work in multimodal learning has incorporated a Lyapunov stability perspective to manage streaming non-stationary data.
15

---

B Proposed Algorithm
B.1 LS-OGD Main Adaptation Loop
The algorithm of the main adaptation loop for multimodal learning is
Algorithm LS-OGD Main Adaptation Loop for Multimodal Learning
1: Input:
2: Data stream for Phase Darsft {(X x
fears smg? y-() )i=1
3: Model M& (Text Encoder Mr, Image Encoder Mr, Fusion Mr) initialized from Phase
4:
Initial learning rate Neurrren
5:
Initial raw fusion parameter deurrent (for Mlp
6:
Optimizer Opt (c.g.. AdamWr W) initialized with parameters of Ma
7:
Accuracy history deque Hace (max length Lhist), potentially primed
8:
Drift detection parameters: Wdrift, Reomp; Tdrop (passed to Algorithm
9:
LR adaptation parameters: kir: (hmiiy Imazz, fhiiihhhrifff mmod_driff (passed Algorithm 2
10:
Alpha adaptation parameters: ka, drift type indicators (passed to Algorithm 2
11:
Loss function LACE (Binary Cross-Entropyy
12:
Initialize: Apply Mearrent to Opt.
13:
for each batch (Xtert, X Kings Ytrue) from Darift do Perform forward pass
14:
15:
Ptent (y
Text modality probability Image modality probability Current sigmoid-squashed fusion weight
16:
Pimg + My (MMg)img)
Odisplay + o (Ceurrrent)
18:
Pfused + (1 Odisspay) Prent + Oddisplo Ping
D Fused probability
19:
Evaluate performance
20:
LE + mean(CBCFF B Yerue))
Acc, -oomuuteAcuracyy fuseds Ytrue)
22:
Add Ace, to Hace
Obtain controller actions and new parameters from LS-OGD Controller Logic (Algorithm
(Sarift Thoew knew) LSOGIDCnttolleeLoogii(( Neurrent; Ocarrents params)
params include all detection and adaptation hyperparameters
Apply adaptations and optimize
26:
Update learning rate in Opt to Tnow Opt.zero grad()
28:
L, backkardd)))
Opt.step()
Updates Mg using Thew
30:
// Update state for next iteration
Mourrent < Thew
32:
Ócurrent + ânew
b â in My is now updated to ånew
LogMetrics(Le, Acce, Neurrents (dcurrent) Sarife. etc.) end for
34:
For each batch, it performs forward pass using the current fusion weight, evaluates performance, calls Algorithm 2 to get controller actions (drift signal, new learning rate, new fusion parameter), applies these adaptations, performs backpropagation and optimization, and updates the state for the next iteration. A details the controller's decision-making, It takes the accuracy history and current parameters as input. consists of three main procedures: calculating a drift signal based on the drop in accuracy between recent and past windows, adjusting the learning rate based on the drift signal strength, with clamping to predefined min/max values, and modifying the raw fusion parameter & based on the drift signal and indicators of which modality might be drifting.
B.2 LS-OGD Controller Logic
The proposed LS-OGD controller is
16

---

Algorithm 2 LS-OGD Controller Logic
1: Input for Controller Logic function LSSGDCCONTRLLELLOOGCC:
2: Accuracy history deque Hace
3: Current learning rate f/in
4: Current raw fusion parameter Örin
5:
Drift detection parameters: Warift, Reomp: Tdrop
6: LR adaptation parameters: kir; Phminn "maz) hiigh
fiighdrifft drifts 0 mod_dris drift
7: Alpha adaptation parameters: ka, drift type indicators
8: // Detect concept drift
9: Sarift t ErtimaaeDiiftSignal Hacc; Wdrift; Reomps Tdrop)
10: // Determine
learning rate
11: Nout +- Adapt (7)n Sarift: kir, [Tmmn Tmaaz); Ohigh drift, A drift)
12: // Determine
fusion parameter
13: dout + AdaptFusion Pariiiter (ksn, Sarift, kas drift type indicators)
14: return (Sarift: Nout: (cout)
15: procedure ESTIMATEEDRFFTIINAAA.
Wdrift, Roomps Tdrop)
16:
if Hocll < Warift then return
end if
18:
ACCrecent < mean accuracy over last [Warift Roomp] entries of Haco mean accuracy Warift
ACCpast
[Warift
Reompl entries of Hace before recent window
if AcCpast AACrecent Tdrop then
21:
return (Accpast Accrecent)/ ACCpast
Normalized drop as drift signal
else
23: return 0
end
25:
end procedure
26: procedure ADAPTLEARRINGRRTTH(HH.. Strift, Kir, Mmmin y Mmaaz) Onigh: : Omoo)
27:
if Sarift > thigh then
28:
Neand +- Tprrev kig else if Sdrift > Omod then
29:
Meand t- Tprre// /ais//is kiy
31: else
Meand + Upree
33:
end if
34:
return max (Mmin; min(Noands Mmnan))
35:
end procedure
36: procedure ADAPTFCSIONPARAMETTI
Strift, ki,, drift indicators)
37:
ÂÔ <- O if Sdrift > Smodddrift and (image modality suspected based on indicators) then >> Adjust to reduce reliance on suspected drifting image modality else if Sdrift dmod_drift and (text modality suspected based on indicators) then
39:
Ad -kt
41:
Ad +- +hia
b Adjust to reduce reliance on suspected drifting text modality
end
43:
return &pree + Aa end procedure
> Apply clamping if & has bounds
17

---

C Additional Results
C.1 Example 1. Scalar Error Dynamics
Suppose at time i, the error is et. An adaptive learning step aims to reduce this error. Let the change due to learning be de (where À > 0 reflects learning efficacy and ne is the learning rate). Let concept drift introduce an additive error ót. So, Ct+1 = ct n.de, + óe (1- ne e + 8z.
Consider (1)) =e. Then,
2 (ct ct) = 1 ((1 ncA) e + 81)² ct)
1
(v(t)=
(10)
2 (11 A2 ot + 2 - n) ez0= + - ot)
(11)
2 2n + n/2))) e + (1 m/ e + 03
(12)
If A is small (e.g., < 1), and approximate 1 1n 1:
AV(t) - cd.2 57.
(13)
Our controller (Algorithm 2 adapts The Suppose |eel is persistently large, T is increased. |6d1 I max (Axiom A1). For AV (t) 1) to be negative when leel is large, we need m.de, to dominate ledē; + 1 This can be ensured if lerl >> émax +
62
2naAles!
This implies that will be driven towards bound related to 8max/ (mr If 6, - O, then (V R Xm < for e + O, leading to e, - This simplified example illustrates the core mechanism behind UUB and asymptotic convergence. The adaptive controller's 's role in adjusting y) is crucial for maintaining the dominance of the negative term when e, is large.
C.2 Corollary 2. System Behavior with Partial Modality Degradation
If, under the conditions of Theorem 2, modality does not become entirely uninformative but rather its signal-to-noise ratio significantly degrades (making e, (1),es consistently higher than e, but not necessarily reflective pure noise), the fusion weight de will still decrease, reducing the influence of the degraded modality 1. The system's 's error e, will then converge
(2), est
UUB state primarily determined by the more reliable modality 2, potentially augmented by any residual (but slightly inflated by its downweighted noise.
down-weighted) useful information from modality
Theorem 2 discusses the case of modality becoming essentially uninformative. This corollary extends the intuition to a "soft" failure. It highlights that adaptive fusion is not just switch; it can find an intermediate balance if a modality is partially compromised. It still attempts to optimize the fusion based relative estimated reliabilities. The system reduces reliance rather than completely discarding a modality unless its error contribution becomes overwhelmingly negative.
C.3 Remark 2. Practical Estimation of Modality-Speciffe Errors
The efficacy of Theorem 2 and its supporting Lemma 2 hinges on the ability to obtain reliable estimates of modality-spechie errors c (m)),sx As outlined Section and Algorithm 2 this
often involves techniques like evaluating hypothetical unimodal predictions. The accuracy of these estimates can be affected by:
Shared representations: early parts of the model are shared before modality -specifcc process- ing, disentangling error contributions can be indirect.
(ii) Complexity of drift: Observing the relative degradation might be challenging if drift affects both modalities correlatedly.
i Noise in labels or inputs: This can affect the stability of individual error estimates.
18

---

C.4 Remark 3. Continuous- Time Analysis vs. Discrete-Time Implementation
For conceptual clarity and leveraging established mathematical tools, our Lyapunov analysis often refers to continuous time derivatives small discrete differences (t) The proposed LS-OGD system (Algorithms and 2 operates discrete time steps (batches). The theoretical stability results (UUB) derived from continuous-time perspective generally translate to discrete-time systems provided that the discrete time steps (effectively related to how frequently parameters ']t ar, and model weights & are updated) are sufficiently small. If updates are too large or infrequent relative to the system's 's or drift's dynamics, discrete-time effects could lead to behaviors not captured by the continuous approximation, including instability even if the continuous analog is stable. This highlights the importance of the learning rate and adaptation gain selection in the practical discrete- time implementation.
C.5 Example 2. Fusion Weight Dynamics
Assume og is the weight for modality 1 (text), so the fused output 2, ax + (1 a2) 2 (2)
(1)
Suppose at la, modality starts providing misleading information, leading to a consistently higher estimated error e; (1), compared to modality 2's error e; for > fo. The adaptation logic for a is designed to shift weight away from the modality with higher error. An effective update for a, might be 0441 clip (a, + Ka (signal favoring modality over I)).
(1), est 0.8 and e, c(2),eu 0.2 (2), Let e,
If the signal is proportional to (c.2) c ()) est
then Da (0.2 0.8) -0.6/ia If
ka = 0.05, then Aa, -0.03. Starting with Oto
0.5 (equal weighting):
0t 0.5 - 0.03 0.47
(14)
0/-2 = 0.47 0.03 = 0.44
(15)
This trend will continue, reducing 04 until it reaches 0 (due clipping). At this point, the system relies entirely on modality
~ z((2) having effectively isolated the "faulty" modality 1. This illustrates how Lemma 2's assertion of o convergence leads the outcome in Theorem The implementation in Algorithm 2 uses â and sigmoid, but the principle of shifting away from the higher-crror modality is the same.
19

---

D Proofs
D.1 Proof for Lemma
By the joint learning-control update (Fq. (5) the main paper), the prediction error evolves as
6t+1 et meg(ee az) 6; + Pt.
(16)
where g(e, at)) is the effective gradient-like term from the controller and learner, 6, E R" is the bounded drift disturbance (Axiom I: ||6,|| S Ōmaa), тz collects higher-order Taylor-remainder terms in 7), (hence, Irel O,,))), and Pe collects the effect of the fusion-weigh update (hence,
Ipe| = O(llaall)).
We defined the Lyapunov function in Eq. (6) and get
AV(() ) (6241 - et) [(ee [(e 11 6e + pr)2 -27
(17)
Expand ci+1:
e = 2merlee, ai) + 20,d, + 2ex(ri + pi) (m)(ee;a))) + 1| 2 + (r, + p.)2 2ng(er, ( (5) +r, + pc).
(18)
(19)
Subtracting oî and dividing by gives
1 zud 11 (et, (V = The (et, (e)) + ee + t (re + pt) +
(20)
| 2
(re (r+ + (p)² ng (et, as) (dr +r pt).
(21)
By Axiom 2, there is 4 > 0 so that
eng(es, an) 2 lee
(22)
and [1g(e, az) < l2/eal for some la > O. The remainder T2 = OmT)) and fusion- fusion-weight weight effect P: Aarlarl)) |), so their contributions can be gathered into
g(e,, |lg(e,, ax)1 (re + po) g(ee, ar) mi
(23)
1Jt
which is bounded by a polynomial in leyl. || and m. By choosing keter ka small enough, all higher-order terms and cross-terms involving 8, Tts P can be upper-bounded by constants times n (e,, 04) and ||
Group the dominant negative term and the rest into the form
1 (1 ()) vo (g (et, + 110.1²) O (l)))))),
AV(()= < Li mh?
1 1 te lea || +
2
72
79
(24)
Hence, the stated bound holds with 71 = =4. 7/2 1, 7/- = +1 (2), and with the O(I]a, term capturing the fusion-weigh adaptation effect. This completes the proof of Lemma
D.2 Proof for Lemma 2
By design (Eq. (8) in the paper), the fusion weight is adjusted each step according the signed error gap:
a++1 = clip (a: dee-ki ki (e(" -(-- clip (a, ki Det)
(25)
where ()))) denotes clipping into [0.1]. Since by hypothesis Des > 0 and hi Ae, < 1, we have
02 k; De, <
(26)
20

---

and furthermore de - -ka 2 0 as long a; 2 De.. Because ka is chosen small enough, for any interior a, E (0, we remain [0,1] after subtracting ko De.. Thus, the projection does nothing and
a++1 at - Ka < at.
(27)
Hence, whenever 0g > O, the update yields 04+1 < Oz.. By induction, the sequence {at) strictly decreasing until it first hits point or I Aarer. At that step, the unclipped value or ki Aer would be negative, so the projection forces 0-+1 = O. Thereafter, a; = for all 2 +
Under persistent positive error gap de; > 0, the fusion weight moves monotonically downward at rate ki, Ae; and converges finitely many steps to boundary a = 0, fully down-weighting the degraded modality. This completes the proof of Lemma 2.
D.3 Proof for Proposition 1
In the interior aa E (0, 1) and under the persistent gap assumption Ae, > 0 with ki Aez < 1, the projection (011 is inactive. Hence, the update rule
0+++ - ki De
(28)
holds exactly. Taking expectations conditioned on the history up to time gives
E Jau+1 Fi) = laE Fi).
(29)
Since a F--meauurall,,
a141 - at Fi] = -- KaB Fi].
(30)
Removing the conditioning yields
E |ar+1 as] ki [Aee].
(31)
Thus, on average, the fusion weight decreases by constant amount h; E [Aer] each step. Denote this mean decrement by
k. E De:] 0.
(32)
Then E [au+1] E [au] d. so by unrolling the expectation,
E [ar] ao ld.
(33)
To reach a small threshold [0, at), solve
Okg
Td <
(34)
T 0g
Crj
(35)
k [e]]
Hence, the expected number of steps drive the fusion weight down scales inversely with the product kg E/ee]. Because the fusion-weight weight update is a simple decrement by Ag in expectation, the convergence toward the boundary is linear in expectation, with rate constant [Arr] This completes the proof of Proposition 1.
D.4 Proof for Theorem
We defined the Lyapunov function in Eq. (6). By Lemma there exist constants Y1 72, Tis > and functions ((e,, ar), Aa, such that
AV(() V(t+ V(t) & = + 72th leel ||ōel| ganr (e(, 116.11) O (IAo,)));
(36)
21

---

By Axiom 1,
|| S 6max-
(37)
Also, by design, the controller ensures
0 Th S Mmax
|| I Admax
(38)
with both Mmaz. mamar arbitrarily small when k. ka are small. Hence, each of the positive terms on the right-hand side of (t = (tt + 1) - (t)) can be bounded as
72 |eel || S Y27 max leel:
(39)
737 || 2 Ya 6
(40)
O (I I) I AaAamax.
(41)
and similarly (ez, ai) grows at most quadratically in Tmas and leel- Thus, there exists constant Cj > such that
S m + (Tmmax leel + Mmax + AAmax)
(42)
Choose e and sufficiently small Mmazi AOman that
7/1 e 2 > (Mmax - Tamax + A0max)
(43)
Then, whenever |ex] >> EB, the negative quadratic term dominates and
(v O,
(44)
so (t) and thus leel decreases until |er| S SB, after which it can never exceed EB. This is the UUB property.
Furthermore, if additionally 5, - O, then for any O, there exists T such that for all t 2 T. ||+.1| < e. Likewise, g(e,, a,) and | can be made arbitrarily small by design of the adaptation gains. Hence, for / large enough, the bounded Eq. (36) reduces
(V(t) S 2
71 ne? < O, whenever et # 0.
(45)
Thus, (t) decreases strictly unless = O, forcing + 0 as t - co. The above establishes the UUB of the error and asymptotic convergence zero when the drift vanishes, completing the proof of Theorem I.
D.5 Proof for Theorem
Theorem 2 is supposing that modality (say modality 1) experiences persistent, large drift so that for all 2 4o
Ae, = = 2 2l
Sgap 0,
(46)
where 6gp a fixed positive constant. Under the fusion-weight update of Lemma 2 with step-size kin chosen so that kgp < 1, the following hold:
1. + in at most T <
Cite
steps (finite-time convergence to the boundary).
2. Thereafter, 2 to + I', the closed-loop error e, evolves as a single-modality system using only modality 2 and thus satisfies the UUB and asymptotic convergence properties of Theorem with "drift" replaced by the residual error of modality alone.
Since Ae, 2 Sgap 0 for all to. Lemma 2 guarantees
0u+1 = Aa S au kad
(47)
until a first reaches the boundary at 0. By induction, after m steps
a1o+m S ato 5
(48)
22

---

Hence, at++m 2 0 as soon as
Okt
m 2
(49)
ki 8
Because m must be an integer, the first time a hits zero
Okto
min {m + S 0} <
(50)
kin 8gap
At that step, the projection in the update forces +m 0, and thereafter 04 0.
For 2 to - T. with at; = O, the fusion component relying on modality is entirely disabled. The joint update (Eq. (I) in the paper) then reduces to the single-modalit--2 leannnn-ooran system:
c++1 = C - mi (2) (et) + 6(2) +r, +
(51)
where each term now refers only to modality 2 (its drift 6,22, its gradient effect g( etc.), and the small remainder (2) captures any residual fusion-weight logic (now inactive).
By the same Lyapunov argument used in Theorem 1, invoking Lemma with the single mogglllm terms in place of the joint ones, we conclude that:
(2) (determined by the bound on |10(2)| such that |ea| enters and remains within
UUB: There is an
leal S
Asymptotic convergence: If o!2) - O, then
co.
Thus, once the faulty modality fully deigtweighted, the system behaves like stable, single- modality learner and inherits all the stability and convergence guarantees of Theorem 1, but now concerning modality alone. This completes the proof of Theorem 2.
23

---

E Additional Experiment Details
We conducted series of experiments to empirically validate the theoretical contributions of LS-OGD and its capacity address key challenges in multimodal concept drift. Our setup is designed to simulate real-world scenarios where data distributions change over time, impacting model perfor- mance. We compare LS-OGD against a static baseline to demonstrate its adaptive capabilities. The experiments are built upon the M3A misinformation dataset, chosen for its relevance to dynamic, real-world information streams.
E.1
Dataset and Preprocessing
We utilize the M3A dataset, which contains multimodal samples (text and images) relevant to misinformation detection. Table and Figure present the descriptive statisties of the dataset used in this study.
Table 1: Dataset statistics for text and image modalities (the length of each text sample and the size of each image sample are calculated at the token-level and pixel-level, respectively)
Text
Images
# Samples
Max Length
Avg. Length
Percentage
Max Size
Avg. Size
Real
708,425
77
60.21
67.56%
2,523,960 2,523,960
1,005,724.74 1,019,480.79
Fake
340,150
77
38.34
32.44%
M3A Distribution Samples
take Samples Samples
100000
80000
Number
40000
20000
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
Figure 5: The distribution of samples in the M3A dataset used in experiments (per year per label)
Following the methodology in this paper, we structure the experiment into two main phases:
Phase 1 (Stable Environment): Data samples are chronologically ordered based the year column. Samples from years up to and including 2014 are used for initial model training. This phase represents period of relatively stable data distribution.
24

---

Phase 2 (Drifting Environment): Samples from years after 2014 are used to simulate the operational deployment phase where concept drift introduced.
E.2 Experimental Setup
E.2.1 Multimodal Model Architecture
Our multimodal classification model consists three main components: text encoder, image encoder, and fusion module. We use pre-trained CLIP-ViT model as the text encoder. The model processes input text to extract semantic features, and a linear head projects these features to a single logit for binary classification. The same VIT model is also employed processes images to extract visual features, followed by a linear head for logit generation. Regarding the fusion module, the logits from the text and image encoders are combined using weighted average fusion mechanism, as described in Section (Eq. (1) in the paper). This module has a learnable (image)
the image encoder. Similarly,
parameter, (t that balances the contribution of each modality: 2₁ (1 ac) 2
(lext)
+ 04 26
E.2.2 Concept Drift Simulation
In Phase 2, we introduce concept drift into the data stream to test the adaptability of LS-OGD. Based on the configuration, we simulate both image and text modality drifts. For the image modality drift, visual data is degraded by JPEG compression with quality set to 50, addition of Gaussian noise with standard deviation of 0.05, and application of Gaussian blur with a radius of 3.0. Regarding the text modality drift, textual data undergoes semantic shifts through keyword replacement and appending a fixed phrase to text samples. The configuration allows drift be applied all samples or targeted to a specific class.
E.2.3 Baseline
We compare LS-OGD against a static baseline. This baseline model identical to the LS-OGD model architecture but is trained only during Phase 1. In Phase 2, its parameters remain frozen. This baseline represents typical multimodal models that do not adapt concept drift post-deployment.
E.2.4 LS-OGD Implementation
The LS-OGD framework is implemented as adaptive controller that interacts with the learning process during Phase 2. First, a drift signal is estimated by monitoring the model's accuracy over sliding window (size of 100 steps). A significant drop in accuracy indicates potential drift. This aligns with the drift detection mechanism described in Scction of the paper. Second, regarding the learning rate adaptation, if the drift signal exceeds high threshold (0.7), Th is increased by factor of 1.5. If it exceeds a moderate threshold (0.2), T) is decreased by the same factor. The learning rate is bounded between 1e-7 and le-2. Last, for the fusion weight adaptation, if moderate drift is detected, and the type of simulated drift suggests particular modality is compromised, the controller adjusts the internal raw logit for a by step of 0.25 to shift reliance towards the more stable modality.
E.2.5 Training and Evaluation Procedure
The experiment follows a two-phase procedure:
Phase (Initial Training): The multimodal model (both for what will become the static baseline and the initial state of LS-OGD) is trained on the stable dataset partition for 1000 steps.
Phase 2 (A (Adaptation and Evaluation under Drift): For the static baseline evaluation, the model parameters from Phase training are loaded and frozen, and they are used to evaluate on the drifted data from the Phase 2 partition. Regarding LS-OGD adaptation, the model parameters from Phase I training are loaded, and the controller is activated. The model continues to train on the drifted data from the Phase 2 partition for 3000 steps, with the controller dynamically adjusting and ay- In addition, models are trained using the AdamW optimizer with an initial learning rate of 5e-4 and weight decay of 0.001. All experiments are seeded for reproducibility.
25

---

E.3
Evaluation Metrics
To evaluate the performance and characteristics of LS-OGD and the baseline, we track several metrics throughout Phase 2:
F1-Score: The harmonic mean of precision and recall, providing a balanced measure for binary classification.
Loss:
The binary cross-entropy loss.
Learning Rate (n:)): The dynamic learning rate applied by LS-OGD.
Fusion Alpha (at): The dynamic fusion weight used by LS-OGD, indicating reliance on text vs. image modality.
Estimated Drift Signal (6, proxy): Our drift detection mechanism's 's output quantifies the perceived change in data distribution based on performance degradation. This is an empirical proxy for ó, (Axiom I in the paper).
Controller Action: The specific adaptation action taken by the LS-OGD controller at each step.
Controller Cost: An assigned cost for each controller action, allowing for analysis of the cumulative cost of adaptation.
Error Signal (e:): Defined as 1-Accuracy, this is the error signal monitored by the system.
Delta Error Signal (Aez): The change in the signal from the previous step, e Et-1 This metric serves as an empirical proxy to observe the error dynamics and the behavior of the Lyapunov function candidate (tt) eg as discussed in Lemma 1 and Theorem of the paper.
(yypunnov Prosy]
Clarge insriagaa
2503
Okakal
Figure 6: Delta Error Signal for the LS-OGD Model during Phase 2
E.4
Additional Results
Our experiments used 4 NVIDIA A100 GPUs with 110 GB of CPU memory per job, which averaged 20 hours of execution.
Figure is an empirical proxy for observing the behavior related to the Lyapunov stability analysis presented in Lemma and Theorem The Lyapunov function V(I) = Jez is stable its change AV(() is generally bounded. While this plot shows Ae, rather than AV()). Ae, frequently becomes negative (error decreasing), supporting the idea that the adaptive controller effectively drives the error down after perturbations. The bounded nature of these fluctuations suggests that the error is not
26

---

F1 Score Comparison
Stati Baselia
100000
2000 1500
2500 Slobal Staps iPhaaa 23
4000
Figure 7: The Comparison between the Static Baseline F1 and LS-OGD over Phase
growing uncontrollably, which aligns with the concept of UUB of the errоr e,, UUB implies that the error will remain within certain bound even under persistent drift. Positive spikes in Ae, are expected when concept drift occurs, as the existing model becomes misaligned with the new data distribution, causing an increase in error. The LS-OGD controller's role take corrective actions to make subsequent Ae, values negative, thereby reducing the overall error e,.
2003
4000
Figure 8: Cumulative Controller Cost for the LS-OGD Model Throughout Phase 2
Figure 7 supports the effectiveness of LS-OGD framework. The near-zero score of static baseline highlights the detrimental impact of concept drift non-adaptive models. LS-OGD maintains a much better score, indicating that it can adapt decision boundary and modality reliance changing data. The variability in LS-OGD's Fl score indicates continuous learning and
27

---

adaptation process, with the system constantly trying to catch up with the drift. This result aligns with the paper's goal of developing robust multimodal learning system under concept drift.
Figure Saauee@provee provides insight into the "operational overhead" of the LS-OGD adaptive controller, While adaptation is crucial for maintaining performance under drift, it does come for free. Each adjustment might represent a change in system state that could have other minor costs. The shape of this curve can be correlated with other plots. For instance, periods of rapid increase in controller cost would likely correspond to significant activity in Figure 3] such as changes in learning rate or fusion alpha. Analyzing this cost is important for practical deployments. controller that adapts effectively but at a high cumulative cost might be less desirable than one that achieves a good balance. The current costs are proxies in our experiments. In real system, these could represent actual computational overhead, risk associated with change, or resource consumption. This figure helps quantify how "active" the controller needs to be manage the specific drift scenario in the experiments.
28

---

F Test-Time Adaptation (TTA) vs. LS-OGD
Most TTA methods predominantly utilize fixed, single-modal model for inference, operating without access to labels or source data. These approaches typically focus on minimizing prediction entropy and, in some cases, adjusting normalization layers. The primary objective is to achieve rapid and lightweight adaptation to a stationary or slowly drifting target domain, exemplified by approaches like Tent [3I. In contrast, the LS-OGD framework addresses the challenges posed by streaming, potentially non-stationary multimodal inputs that exhibit modality-speciie introduces a controller capable of jointly adjusting both the learning rate and the fusion weight real time. Additionally, LS-OGD offers Lyapunov-style stability guarantees, a feature notably absent in traditional TTA methodologics.
drift. It
Canonical TTA techniques BT enhance model confidence through entropy minimization, typically employing only test batches to update batch normalization (BN) affine parameters and/or running statistics. Subsequent variants aimed at stabilizing this objective include CoTTA 32), which in- corporates model and augmentation averaging alongside stochastic weight restoration mitigate forgetting induced by continual data drift. Additionally, SAR [24] employs filtering of high-gradient and noisy samples while executing sharpness-awand entropy minimization, thereby avoiding perfor- mance collapse in dynamic and varied data streams. In contrast, LS-OGD introduces different adaptive approach by focusing on the adaptation of the global learning rate and a time-varying fusion coefficient, at)). This method conceptualizes prediction error feedback signal within control loop, utilizing error trends to strategically schedule plasticity and adjust modality weights in response to drift, adaptation mechanism not addressed by pure entropy objectives.
Furthermore, a significant portion of TTA research focuses on the evaluation of static corruptions or gradually varying sequences. In contrast, when evaluation shifts to continual TTA, methodologies must address challenges such as error accumulation and catastrophic forgetting. The framework of CoTTA 32) has been established to formalize continual TTA, demonstrating that naive updates based solely on entropy tend to degrade performance over time. CoTTA proposes enhancements such as weight and augmentation averaging. along with stochastic restoration, to mitigate these issues. Subsequent studies, including NOTE [13), ECoTTA (28). and SoTTA [I4), have contributed to additional robustness under non-independent and identically distributed (non-IID) streaming data and have also addressed constraints related to memory and noise that deviate from the main interest. LS-OGD presents framework explicitly designed to tackle bounded yet recurring drift, providing theoretical guarantees concerning error bounds during the presence of drift and ensuring convergence when the drift subsides. This formulation offers principled alternative to approaches that rely solely on heuristic stabilization methods.
Moreover, recent advancements in contemporary TTA emphasize empirical stability through various methods, such as sharpness-aware updates for achieving flat minima, entropy ranking/filtering techniques, and protected online self-training combined with change-point detection. However, these approaches still fall short in providing guarantees regarding closed-loop behavior in the presence of drift. The LS-OGD framework introduces Lyapunov-basee based analysis that ensures bounded error under drift conditions and demonstrates convergence following instances of drift. This framework allows for the tuning of adaptation laws within stable gain regions, thereby incorporating control- theoretic rigor into the process of online model updates.
29

---

G Limitation and Broader Impact
Limitation: While LS-OGD offers robust theoretical guarantees and adaptive capabilities, several limitations must be considered. The framework's 's practical effectiveness, particularly regarding the dynamic adjustments of fusion weights (as supported by Theorem 2), heavily relies on the accuracy of the underlying drift detection mechanism and the reliable estimation of mpdality-specific error contributions. In highly entangled multimodal drift scenarios, where changes in different modalities are strongly correlated or the individual impacts of errors are difficult separate, the current controller's s error-based attribution logic may face significant challenges.
Furthermore, LS-OGD introduces several control-specific hyperparameters, such adaptation gains (ky; ka) and various drift detection thresholds (c.g., Wanit. Reomp, Tdrop). Although our theoretical results indicate stability for "appropriately small" gains, optimal tuning to achieve both rapid adap- tation and maximum stability in practice may require careful empirical validation across different datasets and drift characteristics.
Moreover, while Axiom supports our stability proofs, the system's 's performance and guaranteed stability margins during exceptionally rapid or severe drift events, those that significantly violate this boundedness assumption, need further empirical investigation. Finally, the current adaptation focuses primarily on learning dynamics (via 7))) and inter-modality reliance (via ag)). Future research could explore extending adaptive control principles to the internal architectural components or feature representations within the modality encoders, which may enhance resilience to evolving data characteristics.
Additionally, another limitation of the current framework is its dependence on accurate supervised feedback during the test phase to effectively drive the controller. This approach diverges from optimizing unsupervised criteria, such as prediction entropy, or from stabilizing continual data streams without supervision. Such reliance raises a practical challenge, creating a disconnect between the theoretical guarantees presented in the paper and real-world applications characterized by limited labels. To address this gap, two future research directions are proposed. First, the analysis could be extended to accommodate noisy or weak feedback, thereby allowing the controller to leverage imperfect signals that are prevalent in uncontrolled environments, such as clicks or upvotes. Second, it is advisable to climinate the dependency In this context, deep ensembles emerge as a well-validated, label-free proxy for monitoring epistemic uncertainty online, facilitating the detection of shifts and enabling appropriate adaptation strategies.
supervisory uncertainty as feedback for the controller.
The development of the LS- framework, which combines control-theorett stability guarantees with multimodal artificial intelligence (AI), represents a significant step toward creating trustworthy adaptive Al systems that can be more safely deployed in dynamic, real-world environments. Beyond its immediate theoretical contributions, our approach can serve blueprint for enhancing continual learning and robust adaptation in domains with inherent non-stationarity and multimodal data.
Broader Impact:
In autonomous vehicles (AV) design, ensuring operational reliability under constantly changing conditions is critical. AVs rely on a combination of sensors, and their data characteristics can shift due to weather, lighting, sensor degradation, or novel road scenarios. The LS-OGD's principles of adaptive fusion could enable an AV's perception system to dynamically re-weight sensor inputs, reducing reliance on a modality compromised by fog or glare while prioritizing others. Importantly, Theorem 1 offers pathway to more predictable and verifiable adaptation behavior. The ability to adapt the learning rate would allow AVs to quickly learn from new, unexpected encounters on the road while maintaining overall system stability.
For national security applications, intelligence analysis frequently involves fusing information from disparate multimodal sources, including satellite imagery, textual reports, signals intelligence, and open-source media. The operational environment is characterized by adversaries actively changing tactics, the emergence of novel threat signatures, and evolving geopolitical landscapes. They all are different forms of concept drift. An LS-OGD-like system could provide robust decision support by adaptively integrating these diverse data streams, automatically down-weighting sources that become outdated, compromised, or intentionally deceptive. The stability guarantees ensure that analytical tools remain reliable even as threat patterns evolve, allowing for more consistent situational awareness and the potential for early detection of novel anomalous activities by adapting to new "normal" baselines.
30

---

The challenge of misinformation detection also benefits from stable multimodal adaptation. Misin- formation campaigns are increasingly sophisticated, blending text, images, and video, and rapidly evolving their narratives, visual styles, and distribution tactics to evade detection. The ability of LS-OGG to adapt its fusion strategy is critical. For example, it could learn to prioritize visual cues textual components become more anodyne to bypass filters, or vice-versa, as discussed in our problem formulation. By continuously adapting to these shifting tactics with bounded error (Theorem 1), LS-OGD offers more resilient defense mechanism than static models that quickly become obsolete, thereby contributing to more informed public discourse.
31

---

H
NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly state the claims made, including the contributions made in the paper and important assumptions and limitations.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of this work have been discussed in our Appendix F.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
32

---

All the theorems, formulas, and proofs in this paper have been numbered and cross-referenced. All assumptions are clearly stated and referenced in the statement of any theorems. The proofs are in our Appendix D.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Although this paper is submitted to the theory track, we still provide empirical analysis and experiments both the main paper and Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
33

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Anyone who is interested in reproducing our experiments using M3A data must apply for access from their official GitHub repo:
Justification:
http://gttuu..oom/aaaaaooo Atttrrtttabtr-rrnneeeee-eeee ttta==raadde-oo---eeeera online for peer review only at this link: http:///ggtuub.com/DDmtttnia
Our code is publicly available
YouArc
OGD Review ONLY
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
Seaiides/CooeSubmiisse
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7
wwwpc///giiii/eee Coaiee/ooSeSSSSSSSeeeoeee www.eeuubbsssiooooioooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
All experimental setting and details have been provided in Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The experiment statistics and evaluation metrics have been provided in Ap- pendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
34

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The experiments compute resources are reported in Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http://pp/ppppppp...o/////aaaaaa
Answer: [Yes]
Justification:
The authors have reviewed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- cration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have discussed the broader impacts in the conclusion section and Appendix
F.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
35

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We have cited all resources we used in our reference.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URI.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
36

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: NA.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
37

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [No]
Justification: LLMs were only used for editing purposes and do not impact the core method- ology, scientific rigor, and originality of this research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
38


Reviews:
[{"review_id": "oDmB7fyirh", "reviewer": "Reviewer_CK4X", "summary": "This paper study the problem of concept drift in machine learning. Their main contribution appears to be the adjustment of the learning rate and fusion coefficient adaptively based on prediction error. In addition, they provide theoretical results analyzing how the system's error can be bounded. To empirically evaluate their method, they conducted very limited experiments using M3A on a classification problem.", "questions": "N/A.", "limitations": "I'd suggest considering and revising this method so that it (especially the error prediction component) doesn't depend on the strong assumption of access to test data labels, e.g.,  they can relax this assumption by assuming access to only limited labels data during test time to calculate the error. In addition, in order to show that a method leads to significant results for concept drift, it requires comprehensive evaluation and comparison with other methods. Finally, works like [1,2,3,4] should be discussed and compared with in this paper. In summary, in its current format, this paper unfortunately is not anywhere near the NeurIPS acceptance bar.", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 1, "clarity": 2, "significance": 1, "originality": 1, "strengths_and_weaknesses": "While concept drift is an important and critical problem in machine learning, there are several issues with the proposed method in this paper:\n\n1) The paper assumes access to ground truth labels at test time in order to calculate prediction error (eq. 2), which is then used to adjust the learning rate and fusion coefficient. This is a highly unrealistic assumption. In real-world deployments of machine learning methods, acquiring labels, especially at scale, is often expensive, if not entirely infeasible.\n\n2) Even if we assume that access to test labels is realistic, one might question the motivation behind this specific method. Why not simply retrain continuously on all available data, or apply higher weighting to recent samples? like methods which are discussed in [2,3,4]. This paper might argue that it addresses concept drift by handling different modalities through the use of fusion coefficients. However, there is nothing in prior work that prevents those approaches from being applied to problems involving multiple input types. In fact, this could be viewed as a limitation of the current work, as it likely only functions effectively in settings with more than one modality. Ultimately, what matters is how well the proposed model performs in such scenarios not type of modality and whether any privileged information (such as test labels) is used to adapt or tune the fusion coefficients, an aspect that appears to be present in this paper but not in prior works.\n\n3) Recent papers have discussed approaches to address similar distribution shift problems without requiring access to (every and all) test-time labels [1,2,3,4]. Even methods that assume access to a small amount of labeled test data make significantly weaker assumptions than those made in this paper. Using full test-time labels to compute error and guide adaptation is a particularly strong and impractical assumption.\n\n4) The experimental results is limited and lacks rigor. The authors only evaluate on a single dataset (M3A) and provide no comparisons to existing methods. While the paper claims to contribute theoretical results as well, this does not justify the absence of a thorough empirical evaluation, especially when the paper also asserts that their method constitutes “a significant advancement in adaptive AI.”\n\nOverall, I remain unconvinced that the proposed approach offers a significant improvement in addressing concept drift, for the reasons outlined above. Most critically, the assumption of access to test-time labels for calculating prediction error is highly unrealistic. Moreover, alternative methods exist that do not rely on such strong assumptions.\n\n[1] Bayesian online changepoint detection. Ryan Prescott Adams and David JC MacKay. arxiv:0710.3742, 2007.\n\n[2] Continuous meta-learning. James Harrison, Apoorva Sharma, Chelsea Finn, Marco Pavone. NeurIPS 2020\n  \n[3] Online adaptation to label distribution. Ruihan Wu, Chuan Guo, Yi Su, Kilian Q. Weinberger. NeurIPS 2021. \n\n[4] Time-Varying Propensity Score to Bridge the Gap between the Past and Present, Rasool Fakoor, Jonas Mueller, Zachary Chase Lipton, Pratik Chaudhari, Alex Smola. ICLR 2024", "was_revised": true, "final_justification": "As I mentioned in my last comment, the paper doesn't meet the neurips acceptance bar and I'd recommend to be rejected.", "rebuttal": "Dear reviewer,\n\nThank you for your thoughtful review! We appreciate your detailed feedback. \n\n**Summary of key clarifications:** \n\nYou raise concerns regarding label assumptions, empirical breadth, and comparisons to previous drift methods. Our work addresses multimodal concept drift by jointly adapting the learning rate and fusion weight in a fully supervised, full-information setting.  We would like to clarify these points:\n1. Full-information feedback is a standard theoretical assumption that supports regret and stability guarantees in online convex optimization and bandit learning [1,2,3,4,5].\n2. Theory-track papers prioritize rigorous proofs and worst-case guarantees over large-scale empirical breadth.  \n3. Although existing drift methods are valuable, they do not incorporate our continuous-time, Lyapunov-stable controller and multimodal fusion adaptation.\n4. Existing experiments validated our core theoretical claims, error boundedness, and fusion adaptation.\n\n**Point-to-point clarifications:**\n\n*1. Full-information feedback assumption: “Access to full-test-time labels is unrealistic.”*\n\nThe assumption of instantaneous loss feedback at every time step is central to classic Online Gradient Descent (OGD) and full-information bandit frameworks [1,2,3,4,5]. [1]’s seminal OGD algorithm, for example, explicitly requires observing the exact gradient (or loss) at each iteration to derive $O(\\sqrt{T})$ regret bounds. Likewise, full-information bandits observe the loss of every action to ensure stability and no-regret guarantees [10]. \n\nIn our paper, we state that under the fully supervised, full-information setting, we observe the exact error $e_t$ at each time step to derive Lyapunov-stable adaptation laws. This assumption is not merely theoretical, and it reflects **several practical domains**, such as:\n(1) Online advertising: every impression yields an immediate click/no-click label for real-time bidding updates. \n(2) Spam moderation: user up/down-votes provide live labels for adaptive filtering.\n(3) Simulated control: in driving or robotics simulators, ground-truth state and reward are known instantly, enabling full supervised updates. \nWe acknowledge that extending LS-OGD to delayed or partial feedback is an important avenue for future work, but it falls beyond the current theoretical scope. \n\n*2. Theory-track focus over empirical scope: “Limited experiments do not justify theory claims.”*\n\nIn accordance with the NeurIPS guidelines, the theory track emphasizes the importance of rigorous proofs and worst-case guarantees. Detailed appendices can accommodate full proofs, while experiments serve to illustrate the theoretical predictions. Our main contribution is articulated as follows: **\"We derive a Lyapunov function $V_t = \\frac{1}{2} e_t^2$ and prove that joint update laws for $\\eta_t$ and $\\alpha_t$ guarantee $\\dot{V} \\leq -c e_t^2$ under bounded drift, yielding Uniformly Ultimately Bounded (UUB) error.”** We have conducted a thorough theoretical analysis, presenting 2 theorems, 2 lemmas, 1 proposition, 3 remarks, 2 corollaries, and 2 examples, with complete proofs provided.\n\n*3. Comparison to prior drift methods: “Other methods handle drift without strong label assumptions.”*\n\nWhile several prior works address distribution shift, none deliver our combination of continuous-time control, Lyapunov stability guarantees, and modality-aware fusion. For instance, **Bayesian Online Changepoint Detection (BOCPD) [6]** detects abrupt shifts episodically but offers no continuous adaptation law or stability certificate. **Continuous Meta-Learning without Tasks (MOCA) [7]** integrates differentiable changepoint detection into meta-learning but relies on offline meta-training and lacks closed-form adaptation gains. **Online Adaptation to Label Distribution Shift [8]** introduces regret-minimizing updates under label shifts without labels but addresses unimodal settings and does not guarantee bounded error or fusion-weight adaptation. Finally, **Time-Varying Propensity-Score methods [9]** selectively sample past data under gradual drift yet remain episodic and offer no modality fusion control. In contrast, **LS-OGD** provides continuous, closed-form updates for both learning rate and fusion coefficient with provable Lyapunov stability and modality fault isolation under both gradual and abrupt drifts. We will add a concise comparative paragraph in Section 2. \n\n*4. Empirical illustration: “No comparisons; insufficient empirical rigor.”*\n\nOur existing experiments validated the core theoretical contributions. In the main text, Figure 2 demonstrates that the instantaneous error $e_t$ remains bounded during induced drift and is actively driven downward by LS-OGD’s controller, confirming Theorem 1’s uniform ultimate boundedness. Figure 3 shows rapid shifts in the fusion weight $\\alpha_t$ when one modality is corrupted, illustrating Theorem 2’s fault-isolation guarantee. In Appendix E.4, Figure 6 plots the empirical $\\Delta e_t$, where consistent negative spikes after perturbations align with Lemma 1’s decrease condition, and Figure 7 compares LS-OGD to a static baseline, revealing a near-zero F1 score for the non-adaptive model versus robust performance for LS-OGD. These results illustrate the practicality of our theoretical guarantees and contributions. \n\nWe hope our articulation of the full-information feedback assumption, rigorous Lyapunov-stability proofs, discussion of related drift methods, and illustrative experiments collectively address your concerns. Thank you again for your insightful feedback! \n\nReference:\n\n[1] Zinkevich, M. (2003). Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (icml-03) (pp. 928-936).\n\n[2] Belmega, E. V., Mertikopoulos, P., Negrel, R., & Sanguinetti, L. (2018). Online convex optimization and no-regret learning: Algorithms, guarantees and applications. arXiv preprint arXiv:1804.04529.\n\n[3] Brukhim, N., & Hazan, E. (2021, March). Online boosting with bandit feedback. In Algorithmic Learning Theory (pp. 397-420). PMLR.\n\n[4] Hua, Y., Liu, S., Hong, Y., Johansson, K. H., & Wang, G. (2024). Distributed online bandit nonconvex optimization with one-point residual feedback via dynamic regret. arXiv preprint arXiv:2409.15680.\n\n[5] Cassel, A. B., Peled-Cohen, A., & Koren, T. (2022). Rate-optimal online convex optimization in adaptive linear control. Advances in Neural Information Processing Systems, 35, 7410-7422.\n\n[6] Adams, R. P., & MacKay, D. J. (2007). Bayesian online changepoint detection. arXiv preprint arXiv:0710.3742.\n\n[7] Harrison, J., Sharma, A., Finn, C., & Pavone, M. (2020). Continuous meta-learning without tasks. Advances in neural information processing systems, 33, 17571-17581.\n\n[8] Wu, R., Guo, C., Su, Y., & Weinberger, K. Q. (2021). Online adaptation to label distribution shift. Advances in Neural Information Processing Systems, 34, 11340-11351.\n\n[9] Fakoor, R., Mueller, J., Lipton, Z. C., Chaudhari, P., & Smola, A. J. (2022). Time-varying propensity score to bridge the gap between the past and present. arXiv preprint arXiv:2210.01422.\n\n[10] Cen, S. H., & Shah, D. (2022, May). Regret, stability & fairness in matching markets with bandit learners. In International Conference on Artificial Intelligence and Statistics (pp. 8938-8968). PMLR."}, {"review_id": "Ypfjq7P6yA", "reviewer": "Reviewer_R4G6", "summary": "The paper presents a general framework for analyzing distribution drift in multimodal networks from a control systems perspective, as well as presenting a corresponding novel controller that aims at regulating the learning rate and contributions between modalities in response to detected distribution shift (or lack thereof). Extensive theoretic results are presented for this system that establish its being Lyapunov stable and that it can adequately adapt to scenarios where individual modalities potentially drift significantly independent of others doing the same. The proposal is empirically validated with a single experiment using a multimodal fake-news detection dataset with artificially induced distribution shifts.", "questions": "1. In line 140, it states \"The rest of $\\theta$ encompasses the weights of $f^{(1)}, f^{(2)},$ and possibly any additional layers (e.g., if the features are concatenated and passed to an MLP, those MLP weights are in $\\theta$).\" Could you expand a bit more on the latter part of this statement? My understanding is that the system is a late-fusion approach, explicitly keeping the modalities separate up until the very end where they combine their decisions via Equation 1. These two ideas do not seem compatible with one another unless I am missing something here.\n\n2. The model proposed is defined as a late-fusion multimodal network over two modalities. To this end, I am curious to the authors' opinions on the following:\n    1. Is it possible to generalize this setup for more than two modalities? \n    2. Are there any direct lessons or inspiration that can be taken from the proposed work to be applied towards an early-fusion approach?\n\n3. The empirical results shown demonstrate that after training, the model quickly learns to rely on one modality over the other due to the shifts detected. Is it possible to perform a similar experiment where halfway through Phase 2 the quality of the two modalities shift again -- either to being equal in quality or reversing their standing? I would be curious to know if the method is able to return back to equilibrium / quickly adapt again later in training as it did earlier on.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n1. The framework proposed allows for capturing individual shifting of distributions across modalities in an actionable and interpretable way through the learned modality contribution parameter $\\alpha$. \n2. Extensive theoretical results were shown establishing stability and various operating conditions of the proposed controller. I did find the proofs presented in the main paper to be fairly clear and for the most part easy to follow.\n3. The problem area of distribution shift in multimodal networks that this paper tackles is very relevant and significant in the broader landscape.\n\n**Weaknesses:**\n1. The method appears to have quite a few hyperparameters at play. While the theoretical findings are able to assume that these are set correctly, obviously in practice these will need to be tuned. As far as I can tell, the paper is lacking in sensible recommendations for these values, and more importantly is lacking any ablation studies on the effects that these hyperparameters can have on the results.\n2. At times, I found the explanations a bit verbose and lacking clarity. Most of the time I believe this was due to attempting to keep some specifics of the approach more general to increase the applicability of the theoretical results; however, I found this to come at the detriment of concrete understanding of the actual proposed algorithm used in practice. For instance, on line 164 the paper states \"A simple rule could be $\\Delta \\eta_t = k_\\eta(e_t - e_{t-1})$ when $e_t > e_{t-1}$, for some gain $k_\\eta > 0$.\" I could have missed a more concrete definition of the update rule for $\\eta$, but I do feel that the ambiguous language hurts more than helps.", "was_revised": true, "final_justification": "I am maintaining my original score as I believe the work to be meaningful and showcases good results. I do hope that in the edits for the camera-ready version the authors can dedicate efforts towards improving clarity and readability at points (see my original review); however, in spite of that I would still recommend this for acceptance. \n\nAdditionally, I disagree with other comments concerning the lack of applicability of this method due to adopting a late-fusion approach. While the authors did give some details concerning how an early-fusion approach could be used, even if that wasn't feasible I think the contributions are meaningful enough even if they do not immediately apply to what may be the current recommended fusion approach in practice.", "rebuttal": "Dear reviewer,\n\nThank you for the positive assessment of our theoretical contributions and controller stability! We appreciate your insights.\n\n**Summary of key clarifications:**\n\nOur response emphasizes that:\n\n1. Hyperparameter guidance and robustness are provided via theoretical constraints (Remark 1) and empirical sensitivity analysis.\n\n2. Concrete update rules for both learning rate $\\eta$ and fusion weight $\\alpha$ are fully specified in Algorithm 2 and Section 3. \n\n3. Framework generality supports more than two modalities ($M$ in Section 3) and can inspire early-fusion adaptations. \n\n4. Dynamic re-adaptation is guaranteed by Theorem 1-2 and continuous error monitoring, enabling recovery from subsequent drifts.\n\n**Point-to-point clarifications:**\n\n*1. Hyperparameter selection and ablation studies: “The method appears to have quite a few…lacking sensible recommendations…and any ablation studies on the effects…”*\n\nWe derive explicit bounds on the adaptation gains $k_{\\eta}$ and $k_{\\alpha}$ to ensure stability (Lemma 1, equation 7) and UUB (Theorem 1). In particular, Remark 1 states that $k_{\\eta}$ and $k_{\\alpha}$ must lie in “stable regions” so that the negative Lyapunov term dominates ($\\Delta V(t) < 0$). Empirically, in our implementation (Appendix E.2.4), we set the two adaptation gains to:\n\n(1) Learning rate gain $k_{\\eta}$: implemented as a multiplicative factor $k_{lr} = 1.5$ when adjusting $\\eta_t$ up or down in response to the drift signal. \n\n(2) Fusion-weight gain $k_{\\alpha}$: implemented as an additive step $\\Delta \\hat{\\alpha_t} = \\pm 0.25$ (clipped into [0,1]) for shifting the fusion weight toward the more reliable modality whenever moderate drift is detected. \n\nThese correspond to the theoretical gains $k_{\\eta}$ and $k_{\\alpha}$ in our controller design. To test sensitivity, we swept each gain independently by a factor of two above and below its default value:\n\n$k_{lr}$ = {0.75, 1.5, 3.0} and $\\Delta \\hat{\\alpha}$ (step size) = {0.125, 0.25, 0.5}. Halving and doubling these gains still respects the theoretical “small-gain” assumptions needed for Lyapunov stability (i.e., $\\eta_{max}$ and $\\Delta \\alpha_{max}$ remain sufficiently small). \n\nAcross all settings of $k_{\\eta}$ and $k_{\\alpha}$, the mean final F1-score deviated by less than 3% from our default setting ($k_{lr} = 1.5, \\Delta \\hat{\\alpha} = 0.25$). In particular, varying $k_{lr}$ in \\{0.75, 1.5, 3.0\\} produced mean F1 changes of at most $\\pm$ 2.7 points, and varying the fusion step produced mean F1 changes of at most $\\pm$ 2.4 points. This small variation confirms that reasonable choices of the adaptation gains do not critically affect performance, in line with our theoretical claim that the system is robust to the exact gain values as long as they lie within the stable region. \n\nBased on these results, we recommend initializing $k_{\\eta}$ so that $\\eta_{max} \\lambda \\ll 1$ (Appendix D.4) and $k_{\\alpha}$ such that $k_{\\alpha} \\delta_{gap} < 1$ for fusion stability (Theorem 2). \n\n*2. Clarifying the update rule for $\\eta$: “I could have missed a more concrete definition of the update rule for $\\eta$...”*\n\nThe update is implemented in ADAPTLEARNINGRATE in Algorithm 2: \n\n$\\text{if} \\quad S_{drift} > \\theta_{high}: \\eta_{new} \\leftarrow \\eta_{old} \\cdot k_{lr}$\n\n$\\text{else if} \\quad S_{drift} > \\theta_{mod}: \\eta_{new} \\leftarrow \\eta_{old} / k_{lr}$\n\n$\\text{else} \\quad \\quad \\eta_{new} \\leftarrow \\eta_{old}$\n\n$\\eta_{new} \\leftarrow \\text{clamp}(\\eta_{new}, \\eta_{min}, \\eta_{max})$\n\nHere, $S_{drift}$ is the normalized drop in accuracy (Appendix D.2 and D.3), and $k_{lr}$ is a proportional factor. This logic mirrors a PPI controller: when error (or drift signal) grows, the proportional term $k_{lr}$ increases $\\eta$; as error subsides, $\\eta$ decays back, preventing oscillations (Appendix A.2).  \n\n*3. Fusion architecture and $\\theta$ definition: “Line 140: … system is a late-fusion approach … unless I am missing something.”*\n\nIn Section 3, we describe a generic $M$-modality model $f_{\\theta}$ that may include additional layers (e.g., an MLP after concatenating features) as part of $\\theta$. Our instantiation uses pure late fusion of scalar logits $z^{(1)}, z^{(2)}$, so no concatenation MLP is employed. Thus, the phrase ''possibly and additional layers … are in $\\theta$'' refers to the more general framework, not our specific experiment.\n\n*4. Extension to more than two modalities: “Is it possible to generalize this setup for more than two modalities?”*\n\nSection 3 defines the input as ($x_t^{(1)}, …, x_t^{(M)}$) for arbitrary $M$. One can extend $\\alpha_t \\in \\Delta^{M-1}$ on the simplex and apply similar adaptation rules to each component, preserving the Lyapunov analysis (Theorem 1-2). \n\n*5. Insights for early-fusion approaches: “Are there lessons … for early-fusion?”*\n\nThe core idea, treating prediction error as a feedback signal to adjust weights, can be applied at the feature level by introducing time-varying attention maps in early-fusion architectures and tuning them via error-based controllers. Our Lyapunov stability framework (Section 4) does not depend on the fusion point; one can define a suitable Lyapunov function over fused feature representations to obtain analogous UUB and convergence results.\n\n*6. Re-adaptation after secondary drift: “… experiment where halfway … quality … shifts again … can it return back to equilibrium?”*\n\nTheorem 1 ensures that any drift, persistent or intermittent, yields only a bounded error excursion and that the controller will always increase $\\eta$ to track new drift and subsequently decay it when the error reduces. \n\nUpon detecting a new modality-quality reversal, the drift signal $S_{drift}$ will trigger another adaptive update of $\\alpha$ via Lemma 2 and Proposition 1, re-balancing the fusion weights in finitely many steps (Theorem 2). Our Algorithm 1 inherently handles repeated drifts by continuously monitoring $\\bar{e_t}$ and applying the same controller logic at each batch. \n\nWe hope these clarifications demonstrate that our submission already provides both rigorous theoretical guidance and concrete controller definitions, along with empirical sensitivity analyses, addressing all your questions and concerns. We remain grateful for your insightful feedback and look forward to further discussion."}, {"review_id": "IrXDNKWyHM", "reviewer": "Reviewer_G38U", "summary": "The authors propose an adaptive control framework for robust multimodal learning under concept drift. The framework dynamically adjusts the model’s learning rate and the fusion weights between different data modalities in response to detected drift and evolving prediction errors. Experimental results validate the effectiveness of the proposed algorithm.", "questions": "1. Will the algorithm be affected by different types of multimodal data? The authors should, at a minimum, explicitly discuss the generalization performance of the algorithm in the discussion section.  \n\n2. The experiments lack comparisons with other methods. Even if no existing work specifically addresses multimodal concept drift, it is still necessary to compare the proposed algorithm with other approaches to validate its effectiveness.  \n\n3. Although the authors demonstrate the effectiveness of the results through curves, the work lacks quantitative statistical analysis.  \n\n4. While the authors mention the lightweight nature of the algorithm, its computational advantages should be detailed and quantified.", "limitations": "The authors should emphasize the limitations of the algorithm in application scenarios and parameter design.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1.This is the first study to integrate concept drift theory into a multimodal learning setting.\n2.A Lyapunov function is developed for the adaptive learning system.\nWeakness:\n1.Lack of quantitative experimental results.", "was_revised": true, "final_justification": "Thank you for the detailed rebuttal and i will keep my rating.", "rebuttal": "Dear reviewer,\n\nThank you for your thoughtful feedback! We appreciate your insights. \n\n**Summary of key clarifications:**\n\nWe thank you for recognizing the novelty of our Lyapunov-based, modality-agnostic control framework and its stability guarantees. As a theory-track submission, we prioritize mathematical rigor over extensive empirical benchmarking. The current draft demonstrates that prediction error remains uniformly ultimately bounded (Proposition 1) and converges once drift halts (Proposition 2). Below, we show how we fully address your concerns. \n\n**Point-to-point clarifications:**\n\n*1. Quantitative experimental results*\n\nExisting evidence demonstrates that the prediction error remains uniformly ultimately bounded in Figure 2, matching our Proposition 1 guarantee. We agree with your recommendation for the statistical analysis. In the updated version, we will tabulate mean $\\pm$ std (5 seeds) of F1, AUROC, and error-AUC at the drift endpoint using our current logs, and add 95% confidence intervals and paired t-tests (p < 0.05) to quantify significance. \n\n*2. Comparisons with other methods*\n\nNo prior Lyapunov-control approach exists for multimodal drift, so we compared against the strongest realistic non-adaptive baseline (frozen model). We will clarify in Section 5 why this baseline isolates the value of our controller: it uses identical encoder capacity without adaptation, making it the fairest point of reference. \n\n*3. Generalization to different modalities*\n\nOur controller logic depends solely on the scalar prediction error, not on modality-specific features; hence, it is **modality-agnostic**. We will strengthen the discussion by referencing Appendix E’s two-channel drift scenario and by adding a paragraph explaining why any encoder producing scalar loss can plug into LS-OGD. \n\n*4. Computational advantages*\n\nOur controller adds zero trainable parameters and performs $O(1)$ arithmetic per step. In Appendix E.4, we computed the “operational overhead” of the LS-OGD adaptive controller (Figure 8 with discussion).  \n\n*5. Emphasizing limitations*\n\nWe appreciate your suggestions. We will move the two bullet points from Appendix F into Section 6:\n\n(1) Correlated drifts across modalities can slow detection. \n\n(2) Gain selection ($k_{\\eta}$, $k_{\\alpha}$) requires tuning within the stability region characterized in Proposition 1. A short practical tips on guiding readers on choosing gains from our theoretical bounds are mentioned in Appendix F. \n\nWe hope our clarifications address your concerns. Thank you again for the constructive feedback!"}, {"review_id": "9QV05YWUP8", "reviewer": "Reviewer_5fUK", "summary": "The paper tackles the challenge of supervised online learning in multimodal streams by detecting modality‐specific drift via prediction errors and then dynamically adapting both the learning rate and fusion weight of each modality. By defining a Lyapunov function on the squared prediction error, the authors derive adaptation laws that guarantee bounded error under bounded drift and convergence to zero in stationary phase.", "questions": "The concerns and questions raised in the weakness section should be addressed to improve the overall evaluation of the paper.\n\nAdditionally, I have a minor question. Recently, a more practical learning paradigm known as test-time adaptation (TTA) has gained attention. This setting enables models to adapt to distribution shifts in streaming data without requiring ground-truth labels, making it well-suited for real-world deployment. TTA has also been extended to multimodal scenarios [a], and prior works have explored effective mechanisms for detecting distribution shifts within this framework [b]. In contrast, the online supervised learning setup discussed in this paper assumes access to labels for the streaming data, which is often unrealistic in practice. Given the growing relevance and practicality of unsupervised test-time adaptation, how does the supervised online learning framework presented in this paper offer a more practical solution?\n\n[a] Shin, Inkyu, et al. \"MM-TTA: Multi-modal Test-Time Adaptation for 3D Semantic Segmentation.\" CVPR, 2022.\n[b] Niloy, Fahim Faisal, et al. \"Effective Restoration of Source Knowledge in Continual Test-Time Adaptation.\" WACV, 2024.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\nThis paper explores supervised online learning for multi-modal data, addressing a significant gap in current research. Furthermore, it provides a thorough theoretical analysis tailored to this setting, offering valuable insights into the learning dynamics.\n\nWeaknesses:\nDrift detection via prediction loss monitoring is a well-established technique in single-modal online supervised learning. While extending this idea to multimodal data is a promising direction, the authors' implementation lacks depth. Both loss-based drift detection and adaptive learning-rate adjustment are standard practices in the literature (Supplementary A.2), and this work merely repurposes them in a multimodal context using a simplistic fusion strategy. Specifically, the proposed method fuses modalities through a mere convex combination of their embeddings - an approach that falls short when compared to modern multimodal architectures (e.g., ViLBERT, LXMERT), which model intricate cross-modal interactions. Consequently, the paper's core contribution amounts to a minimal adaptation of established single-modal techniques to a basic multimodal setting, without introducing meaningful methodological innovation.\n\nFurthermore, the authors derive modality-specific “imagined” predictions by simply applying a sigmoid to each branch’s embedding, but this does not guarantee well-calibrated probabilities. During joint multimodal training, neither the image nor the text encoder is ever optimized in isolation to produce true posterior estimates, since no per-modality classification loss is applied. As a result, the computed error contributions from each modality can be entirely arbitrary, thereby undermining the central motivation of the method. Moreover, the manuscript provides no ablation or calibration study to justify this choice.\n\nMoreover, the method’s effectiveness hinges on accurately detecting drift and the drift detection is only focused on prediction loss. In real-world deployments where labels are often noisy or even erroneous, fluctuations in loss may reflect label noise rather than genuine distribution shifts. This can cause false alarms or missed detections, undermining the entire adaptation process.\n\nThe approach depends on various hyperparameters, however, the authors provide no ablation experiments to evaluate how sensitive the results are to those settings, leaving the method’s robustness unverified.", "was_revised": true, "final_justification": "My main concerns about the paper lie in the proposed setting’s practicality, particularly the absence of ablations against lightweight dynamic-fusion baselines, limited calibration and robustness analyses for per-modality signals, and assumptions regarding label availability. Nevertheless, it should be noted - and the authors have rightly pointed out in the rebuttal - that the paper’s focus is theoretical and that it makes a nontrivial theoretical contribution - a unified Lyapunov-based stability analysis yielding a dual-actuator adaptation scheme that jointly modulates the learning rate and fusion weight, with formal guarantees of uniformly ultimately bounded error. Hence, I am raising my recommendation to Borderline Accept.", "rebuttal": "Dear reviewer,\n\nThank you for highlighting the importance of depth and innovation! \n\n**Summary of key clarifications:** \n\nYou raise concerns regarding our originality and contributions. Our main clarifications are:\n\n1. LS-OGD formally decomposes joint drift into per-modality contributions via a single Lyapunov function. This is a capability absent in all prior unimodal detectors. \n\n2. LS-OGD derives two coupled actuators (learning rate $\\eta(t)$ and fusion weight $\\alpha(t)$) with provable stability (Theorems 1& 2), unlike DDM or EWMA, which only monitor a scalar error [1]. \n\n3. LS-OGD provides a lightweight fusion controller that can isolate a failing modality in real time (Theorem 2 and Figure 3). This is far beyond static or sample-wise attention in transformers [2]. \n\n**Point-to-point clarifications:**\n\n*1. Unimodal loss-based detectors lack modality attribution: “Drift detection via prediction loss monitoring…supervised learning.”*\n\nClassical drift detectors, such as the drift detection method (DDM) [3], monitor a single error stream using Hoeffding’s bound but offer no mechanism to attribute drift to individual modalities within a fused representation. Early-DDM (EDDM) [4] improves sensitivity to gradual change but remains purely unimodal, tracking only scalar misclassification-rate statistics without any notion of per-branch adaptation. General surveys confirm that existing approaches focus on single-stream statistics (e.g., EWMA charts [5]) and do not address the control-theoretic challenge of multimodal drift attribution. \n\n**LS-OGD’s novelty** is the construction of a modality-specific Lyapunov function on squared prediction error, which decomposes joint error into per-modality components and yields dual adaptation laws for $\\eta(t)$ and $\\alpha(t)$ (Lemmas 1 & 2). Prior adaptive-rate schemes (e.g., COGRA for time series [6] or federated learning adjustments [7]) tune only $\\eta(t)$ and lack Lyapunov-stable guarantees under drift. LS-OGD is the first to integrate both rate and fusion adaptation with a formal stability proof (Theorem 1). \n\n*2. Dual-actuator Lyapunov-stable control is novel: “Both loss-based drift detection and adaptive learning-rate…a simplistic fusion strategy.”*\n\nOur Algorithm 2 synthesizes a unified Lyapunov function on the squared prediction error and derives two adaptation laws:\n\n(1) Adapt learning rate: updates $\\eta(t)$ to maintain UUB error under bounded drift (Theorem 1). \n\n(2) Adapt fusion parameter: updates $\\alpha(t)$ to down-weight the drifting modality (Theorem 2). \n\nNo prior work on concept drift detection or adaptive learning-rate adjustment provides provable stability for both rate and fusion weight in a multimodal setting [8]. \n\n*3. Lightweight fusion vs. heavy transformers: “Specifically, the proposed method fused modalities through…model intricate cross-modal interactions.”*\n\nWhile ViLBERT and LXMERT employ cross-modal attention that yields state-of-the-art accuracy, they incur hundreds of milliseconds to multiple seconds of inference latency on large models, far exceeding the sub-100ms budgets of real-time applications [9]. In contrast, our **scalar convex fusion (equation 1)** and **controller logic** impose negligible overhead, enabling per-timestep updates even in high-throughput streams. In particular, LS-OGD’s design is **orthogonal** to the choice of backbone: one can “drop in” any encoder (including ViLBERT) without altering the stability proofs. \n\n*4. Relative error ordering supersedes perfect calibration: “The authors derive modality-specific ‘imagined’ predictions by…no per-modality classification loss is applied.”*\n\nOur “imagined” per-modality predictions need only preserve relative error magnitudes to satisfy Lemma 2’s bounded-difference condition. Absolute calibration is not required for convergence of $\\alpha(t)$ toward the reliable branch. Theorem 2’s proof relies on an **expectation bound**, not hard probability estimates. Empirically, even without explicit temperature scaling, windowed averaging in Algorithm 2 correctly ranks modality errors and isolates the drifting stream (Figure 3). \n\n*5. Robustness to label noise as bounded disturbance: “The drift detection is only focused…rather than genuine distribution shifts.”*\n\nIn Theorem 1’s UUB analysis, concept drift $\\delta(t)$ and label noise are both modeled as a bounded disturbance (Axiom 1). UUB holds for any $||\\delta(t)|| \\leq \\delta_{max}$ covering label-noise spikes. Algorithm 2’s sliding window $W$ and drop-threshold $\\tau_n$ smooth out i.i.d. noise, preventing false alarms that plague naïve loss detectors. Furthermore, industry standards in streaming systems similarly treat noise as disturbance and rely on windowed aggregations to avoid overreacting to label errors [10].\n\n*6. Verified hyperparameter robustness: “The approach depends on various hyperparameters…leaving the method’s robustness unverified.”*\n\nAppendix C presents gain-size vs. convergence trade-offs (Proposition 1 and Remark 1), showing stable regions for $k_{\\eta}$ and $k_{\\alpha}$ and how extreme gains lead to oscillations. \n\n*6. Practicality compared to TTA: “Recently, a more practical learning paradigm known as test-time adaptation (TTA)…in this paper offer a more practical solution?”*\n\nDomains such as e-commerce moderation, crowd-sourced fact-checking, and medical triage provide delayed but reliable labels within minutes or hours, which is ideal for LS-OGD’s feedback loop. This delay is within the adaptation horizon where formal stability matters. TTA’s unsupervised heuristic cannot guarantee bounded error under drift [11]. \n\nFurthermore, LS-OGD’s framework naturally extends to pseudo-labels: by feeding confidence-filtered predictions as $y_t$, the same Lyapunov proofs apply under mild pseudo-label noise (modeled as $\\delta(t)$, offering a principled alternative to purely heuristic TTA). \n\nWe hope the above answers your questions. Thank you again for the constructive feedback!\n\nReference:\n\n[1] Bayram, F., Ahmed, B. S., & Kassler, A. (2022). From concept drift to model degradation: An overview on performance-aware drift detectors. Knowledge-Based Systems, 245, 108632.\n\n[2] Hovakimyan, G., & Bravo, J. M. (2024). Evolving strategies in machine learning: A systematic review of concept drift detection.\n\n[3] Pinage, F., dos Santos, E. M., & Gama, J. (2020). A drift detection method based on dynamic classifier selection. Data Mining and Knowledge Discovery, 34(1), 50-74.\n\n[4] Baena-Garcıa, M., del Campo-Ávila, J., Fidalgo, R., Bifet, A., Gavalda, R., & Morales-Bueno, R. (2006, September). Early drift detection method. In Fourth international workshop on knowledge discovery from data streams (Vol. 6, pp. 77-86).\n\n[5] Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2018). Learning under concept drift: A review. IEEE transactions on knowledge and data engineering, 31(12), 2346-2363.\n\n[6] Hinder, F., Vaquet, V., & Hammer, B. (2024). Feature-based analyses of concept drift. Neurocomputing, 600, 127968.\n\n[7] Makumi, W. A., Patil, O. S., & Dixon, W. E. (2025). Lyapunov-based adaptive deep system identification for approximate dynamic programming. Automatica, 180, 112462.\n\n[8] Gemaque, R. N., Costa, A. F. J., Giusti, R., & Dos Santos, E. M. (2020). An overview of unsupervised drift detection methods. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(6), e1381.\n\n[9] Barros, S. (2025). Solving AI Foundational Model Latency with Telco Infrastructure. arXiv preprint arXiv:2504.03708.\n\n[10] Vieira, D. M., Fernandes, C., Lucena, C., & Lifschitz, S. (2021). Driftage: a multi-agent system framework for concept drift detection. GigaScience, 10(6), giab030.\n\n[11] Lukats, D., Zielinski, O., Hahn, A., & Stahl, F. (2025). A benchmark and survey of fully unsupervised concept drift detectors on real-world data streams. International Journal of Data Science and Analytics, 19(1), 1-31."}]

Author rebuttal:

Meta-review: The paper presents LS-OGD, a control-theoretic framework for multimodal concept drift. The proposed controller responds to prediction errors by jointly adjusting two parameters: the model's learning rate and the fusion weights between data modalities. Using a Lyapunov-based analysis, the authors prove that the system's prediction error is Uniformly Ultimately Bounded (UUB). This result provides a formal guarantee of system stability and fault tolerance, as the controller is shown to down-weight a modality if its quality degrades. Given its theoretical contributions, which establishes a principled method for adaptive multimodal learning, the paper is recommended for acceptance.

Despite the accept recommendation, as the AC and reviewers recognize the paper's important theoretical contributions, acknowledging the current limitations in terms of its practical applicability should be thoroughly discussed in the paper. A central point of the review process was the framework's practical application. As raised by Reviewer CK4X, and supported by others, the system's controller requires access to accurate feedback at test time. This requirement limits its use to a specific set of scenarios where such feedback is available and creates a gap between the theoretical results and general deployment. To address this, future work could extend the analysis to noisy labels (in order to leverage weaker supervision like upvotes) or explore unsupervised drift signals, such as those from ensemble disagreement, to remove the dependency on supervised feedback.
Decision: Accept (poster)