RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget
Authors: Adam Piaseczny
Md Kamran Chowdhury Shisher
Shiqiang Wang
Christopher Brinton
Venue: NeurIPS 2025 poster ()
Keywords: concept drift
resource constraints
efficient updates
provable guarantees
update policy
adaptive policy
Primary area: general_machine_learning
TL;DR: To adapt ML models to concept drift under strict resource constraints, we propose a lightweight drift-plus-penalty policy that provably limits resource usage and achieves robust results.

Abstract: Machine learning (ML) algorithms deployed in real-world environments are often faced with the challenge of adapting models to concept drift, where the task data distributions are shifting over time. The problem becomes even more difficult when model performance must be maintained under adherence to strict resource constraints. Existing solutions often depend on drift-detection methods that produce high computational overhead for resource-constrained environments, and fail to provide strict guarantees on resource usage or theoretical performance assurances. To address these shortcomings, we propose RCCDA: a dynamic model update policy that optimizes ML training dynamics while ensuring compliance to predefined resource constraints, utilizing only past loss information and a tunable drift threshold. In developing our policy, we analytically characterize the evolution of model loss under concept drift with arbitrary training update decisions. Integrating these results into a Lyapunov drift-plus-penalty framework produces a lightweight greedy-optimal policy that provably limits update frequency and cost. Experimental results on four domain generalization datasets demonstrate that our policy outperforms baseline methods in inference accuracy while adhering to strict resource constraints under several schedules of concept drift, making our solution uniquely suited for real-time ML deployments.

RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget
Adam Piaseczny Purdue University apiasecz@puråue. edu
Md Kamran Chowdhury Shisher Purdue University msh mishiherrpuurde. edu
Shiqiang Wang
IBM Research shiqiang. w wwngoieee..eeeeecoooomooooooe... org
Christopher G. Brinton
Purdue University cgb@purdue edu
Abstract
Machine learning (ML) algorithms deployed in real-world environments are often faced with the challenge of adapting models concept drift, where the task data distributions shifting over time. The problem becomes when model performance must be maintained under adherence to strict resource constraints. Existing solutions often depend on drift-detection methods that pro- duce high computational overhead for resource-conse consrreind environments, and fail provide strict guarantees resource usage theoretical performance assur- ances. To address these shortcomings, we propose RCCDA: dynamic model update policy that optimizes MI. training dynamics while ensuring compliance to predefined resource constraints, utilizing only past loss information and tun- able drift threshold. In developing evolution of model loss under concept drift with arbitrary training update decisions. Integrating these results into a Lyapunov lightweight greedy-optimal policy that provably limits update frequency and cost. Experimental results on four domain generalization datasets demonstrate that our policy outperforms baseline methods in inference accuracy while adhering to strict resouree constraints under several schedules of concept drift, making our solution uniquely suited for real-time ML deployments.
more difficult
policy, analytically characterize the
sift--plus-pnnalyy framework produces a
Introduction
In recent years, deep learning (DL) models have become popular resource-constrinee constrained environments e.g., mobile devices, where computational power and memory limited. These settings typically assume static data distributions during inference, in part due to resource limitations 7 In practice, this poses major challenge in the form of concept drift, which is phenomenon where the underlying relationships in data shift over time, causing model performance to degrade 8. The computational demands of frequent updates that may be necessary overcome rapid drifts present significant challenge in resource-constraiee model updates adapt such drift while adhering resource constraints becomes critical problem.
2
settings. As a result, effectively managing
To address concept drift and mitigate model performance deterioration. researchers have proposed variety of adaptive learning methods [10 [2. These strategies often tailored to different operational contexts and environment characteristics 13
[5] 16]. While most are heuristic, validating their efficacy empirically 18). certain techniques have offered convergence guarantees to give principled insights into algorithm design, e.g., [9 20 Even so, these theoretical analyses
14
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

often rely on specific assumptions about the drift, or require prior knowledge about the environment that may not be readily available for decision decision-making.
Fundamental Challenges. The question of how to maintain strong performance in the presence of concept drift under strict resource constraints remains largely unexplored. Existing lightweight adaptation algorithms do not provably guarantee resource budgets will be met during drift adaptation. On the other hand, established convergence guarantees depend on unconstrained updates: as we will see, artificially restricting these algorithms to stay within the budget leads to suboptimal performance. Thus, the fundamental challenge lies in developing theoretically grounded scheme that guarantees adherence to strict resource constraints while optimizing model performance problem the field. To the best of our knowledge, no prior work has proposed or systematically analyzed strategies that optimally address concept drift while prioritizing rigid resource constraints. As result, in this paper, we are motivated to answer the following question:
long- standing open
How can we develop a lightweight, resource-aware policy for dynamic model updates that (i) effectively mitigates performance loss incurred by concept drift in real-time applications while (ii) theoretically guaranteeing adherence to strict resource constraints?
Contributions. Through our investigation, we develop RCCDA (Resource-Constrainee Concept Drift Adaptation): a first-of-its -f-t-of--i---dd adaptive threshold- based policy that determines the optimal time to update model parameters in the presence of concept drift, using only past loss information, and ensuring strict bounds on constraint violations. Our contributions are summarized as follows:
We provide a rigorous analysis of how concept drift impacts a deployed model's convergence over time, focusing on the evolution of loss as data distributions shift. Under mild assumptions, our bounds quantify the degradation caused by concept drift, implicitly relating model performance to the retraining policy choice. This analysis lays the groundwork for development of adaptive strategies in dynamic, resource-constraing settings.
Convergence Analysis:
We formulate and solve an optimization problem to determine the optimal times for updating a model, minimizing the loss induced by concept drift while satisfying resource constraints. Leveraging the Lyapunov drift-plus- drift-plus-penalty framework [21]. we develop RCCDA: a bhase policy that triggers updates based on inference loss data and virtual queue tracking resource expenditure. The resulting lightweight algorithm relies solely on historical loss data and a tunable drift threshold, making it efficient for real-time applications, while ensuring a greedy-optimal balance between model performance and computational overhead.
Optimal Policy Design:
Empirical Validation: We evaluate RCCDA performance through experiments on four domain generalization datasets: PACS, DigitsDG, OfficeHome, and MEMD-ABSA, simulating different concept drift schedules. Our results demonstrate that our policy consistently outperforms four baseline policies across a variety of drift settings while adhering to resource budgets. This highlights our policy's 's ability to adapt efficiently to data with time-varying distributions.
2
Related Work
Concept Drift. Many of the existing works on studying concept drift have placed an emphasis on
characterizing or detecting drift [2 23 24 25 26 For instance, windowing techniques, such as the Adaptive Windowing (ADWIN) algorithm 277. dynamically adjust the size of data window to
monitor statistical changes, flagging drift when significant deviations occur. Alternatively, observing
29 30 31 32 involves tracking metrics like accuracy or error rates, where a sudden decline may signal that the model no longer aligns with the current data distribution. Additionally, machine learning models can be trained to detect drift by identifying shifts in feature distributions relationships among variables (3) 34). Detecting drift is crucial step in drift adaptation, as it provides actionable insights into the optimal times to update or retrain the model with fresh data [99 35). However, these methods do not consider operating under limited resource budget, a problem that is addressed by the policy introduced in our paper.
performance indicators 288
Domain Generalization. Domain generalization (DG) techniques address the related problem of training models to generalize well across multiple domains, aiming to improve robustness to unseen target distributions during inference [36] 37
38 39. Classical strategies include data/feature
45). More recent works have investigated temporal domain generalization 46 47 48) to adapt to time-varying distributions,
augmentation 44 42 or leveraging domain-invarian features

---

as encountered during concept drift. In practice, however, these approaches demand significant resources, such as continuous data monitoring and frequent model updates. making them impractical for resource-constraiiee settings. Furthermore, these works have not explicitly considered adherence to resource constraints, which is one of the key motivations of our paper.
Continual Learning, As mitigating concept drift involves retraining the model at strategic intervals guided by policy, the drift-adaptation problem shares similarities with continual learning (CL), where the goal is to enable models to learn from a sequence of tasks over time [49 50). A major hurdle CL the problem of catastrophic forgetting, where adapting new tasks causes the model to lose performance on previously learned ones [51 [5]. The problem studied in this paper prioritizes a different objective than CL: optimizing model performance on the current data distribution each time step under resouree constraints, i.e., immediate adaptability versus long-term memory retention. As such, our work is applicable to settings where the concept drift does not exhibit cyclic patterns.
3 Methodology
3.1 System Model
We consider a system with an agent running an inference task using machine learning model within an environment that evolves over discrete time / E {0, 1,... - 1} == T. The environment characterized by an underlying data distribution pt that changes over time due to concept drift. Consequently, the agent's is available dataset, DD ~ Pt. varies at each time step, resulting in changing conditional distributions of the data:
Pr(ye ax)) * Pr(ye ze).
where (ze, Me) E Dr and (ze,ge) E De are feature-targer pairs sampled different times t and t'. We assume that the concept drift experienced by pt is bounded between any time steps and t + 1, which expressed by the time-narying bound 5(t) on natural-log-ban based KI.-divergence between consecutive data distributions:
(D. || D++1) S 8(t) Vt, 5(1) >
At any time the agent uses its current model, with parameters 6, E Rd, to perform the inference task. The inference performance is measured using a local loss function, f : RRd IR Pe|x|=| + R. the definition of which remains fixed throughout the entire operation. By randomly sampling an inference batch & (where |&el < (Dr)) from D,, the agent obtains an unbiased estimate of the loss, (() (s.t. Eg,wD, (f E = f(t-11, 1 Dr), and 0 = 8o). Before the agent is deployed, the agent's model (with parameters bu) pretrained to be within some Eo O proximity to the initial optimal model, i.e |10p 05 S Eo for some En > O, where 8% == arg mingeR" f(8, Do). ensuring a low initial inference loss estimate value.
Due to the concept drift experienced by the underlying distribution Pt, the optimal model parameters, 0r == arg ming / (0, Dr), also change over time, resulting in the model experiencing growing inference loss and performance degradation. This problem is illustrated in Figure| la
To address this issue, the agent performs actions aimed at keeping the loss increase to a minimum in the presence of concept drift. To achieve that, the agent follows a model update policy, where at any time t, the model parameters f₂ can be updated with the data from the most recent dataset Dr, with the number of updates constrained by the resources available. A single update formulated as:
0, = - - (f)-1.
(3)
where 17 >> is the learning rate and G. CD, Pt is a uniformly sampled training batch of data (independent of the & inference batch). The choice of policy affects the model performance over time, as illustrated in Figure lb]
Note that while the update in Equation 33) is formulated as a single gradient descent step, this framework can be easily extended to allow for multiple update rounds per time step. This is achieved by interpreting each gradient descent step as a sub-interval of larger decision time frame, allowing for more granular control over the effective update frequency.
Finally, each model update is associated with a time-varying resource cost X()) > 0. This cost is an abstract measure and can represent variety of factors, such as computational load, energy
3

---

of optimal modal path
2e((56)
Policy update path
Policy update path
0
0 (e(3.4)
0g
0
0 T1y
(e(1.2
0o
saspect loday, the queecctack
Rowardher share rpdates on the povel
02
0 22
gewermmti consifering
(E
6: di distance
from optimum
0 (e(1.2.3)
We path closer to optimum than ztz from optimum
8.ah siaancce
update,
doridi
spdatc with new disributioo
updare
(a) Optimal point and model parameter drift
(b) Effect of different policies on model performance
Figure 1: Interplay between policy design, concept drift, and model performance. (a) Optimal model point movement time due drift. Different policies result in varying total distances to the drifting optimal model. (b) Update timing influences average performance given set number of updates.
consumption, wall-clock time, or monetary cost (details available in Appendix Dj. The cost relates the effective frequency of updates to the pre-defined time-average cost budget X, which corresponds to the maximum average resources the agent can spend per time step over the entire horizon T.
3.2 Problem Formulation
To mitigate concept drift, the agent employs a model retraining policy. At any time I, the policy decides an action (t.) (0, 1}, where (t) and π (1) = O correspond to model update and no update respectively. The resulting sequence of actions over the entire horizon is denoted by the vector
(T(0),.. T (T' 1)). This sequence is an element of the action sequence space
II = 10, Utilizing a retraining policy results in the following evolution of the model parameters over time:
0: = 0; - π Vof (0:-1, Ct)-
For the sake of practicality and computational efficiency of our solution, we consider policies that causal and distribution ignorant, meaning (i) the decision maker neither evaluates nor knows the data distributions Po. .i.............................. Pr-1 and (ii) the decision maker decides on the action at time t based on the history expected losses of the past inference batches of data εor .......fr which are available through running the inference task. This is an important distinction of our setting, as any further modifications to the action space would result an increased task overhead. For example, trying to detect the concept drift requires performing operations on the dataset, which is highly resource consuming (see Appendix for more details). Using the inference loss to evaluate the model performance is common established strategy [53]
Given such setup, the goal of the agent to find policy resulting in set of actions ग E that minimizes the expected loss over the entire time horizon, subject to the average cost of model updates exceeding A:
min ZEIL
Σ E
D.)
subject T
1 T-1
E (())(() SX.
(5)
T
Optimality. Without resource constraints, the optimal solution to (5) would be to update the model every time However, finding an optimal update policy under the resource constraint presents fundamental challenge: obtaining such a policy requires knowledge of the dataset's 's entire future evolution, which is not practical. To address this, we pivot from the direct time-averaged loss objective in (5) to a new objective based on the provable upper bound on the time-averaged gradient norm (formally derived in Theorem This pivot allows us to naturally decompose the problem into a sum of per-time-step step performance penalties. In the following section, we introduce the Lyapunov drift-plus-pentt framework, which provides a mechanism to greedily minimize the upper bound on these per- ver-time-step penalties while provably upholding the long-term resource constraint.

---

4
Policy Design
Idea and Overview. The central challenge of this work is adapting to changing conditions while respecting resource constraints. This issue spans beyond machine learning, appearing in areas where responsiveness and resource management must align, and requires strategies that balance performance and efficiency across dynamic systems. Existing solutions often rely computationally expensive methods or fail to provide strict guarantees on resource usage and performance. To overcome these limitations, leverage the Lyapunov drift-plus-penalty framework (21), it allows to transform an intractable long-term optimization problem into a series of greedy, per-time-step decisions. The core intuition is dynamically incorporate the long-term constraint into each per-time-step objective. This is achieved by defining "penalty" term, which represents our immediate objective (based on minimizing the convergence bound), and a "drift" term, which measures how far the system deviates from its long-term resource budget. By optimizing bound on this combined drift-plus-penan expression each time i, the agent is compelled to make greedy, online decisions that provably balance these two competing factors. This process creates an explicit, online trade-off between short- term performance and long-term budget adherence, allowing us derive a principled, lightweight policy with provable guarantees.
By tailoring the Lyapunov framework the analyzed setting, we derive RCCDA (Algo- rithm D novel policy that dynamically de-
Algorithm 1: RCCDA
Input: model parameters fg. update cost A(t), time-average cost A, tunable weight V. learning rate 17, estimator 9
termines when to update the model based on historical performance metrics, while ad-
hering to resource constraints. The algo-
Output: Model parameters 6,VE
rithm relies on a threshold- -bade rule, trig- gering an update when a constructed expres- sion exceeds a dynamic threshold influenced by past decisions and constraints, reflecting trade-off. This ap-
Virtual Queue (()) + 0
2 History Variables H,, Hy. Hr < 0.000 for = to T do ft + E [f 1,))
the performance-resouue proach offers distinct benefits over traditional methods: it avoids complex drift detection, requires few assumptions about data behav- ior, is simple to implement, and provably guarantees adherence to resource constraints.
H, H2 U [[+)
G. + ((H,, Hg. Hr)
C(t) + a(t)Q(t) + [A(t)2 2XA(t))
VV 2 C(t) then
=(1)= +-
10
0 --- 11 (0.11,))
4.1 Policy Derivation
Hg Hy (V
Lyapunov Analysis. To develop RCCDA,
12
else
T(() +- 0
we apply the Lyapunov framework to the per- step penalty objective derived from our con-
14
0, + 0:-1
vergence bound (Theorem [. subject to the
H. +- H., U (=(())
resource constraint. The time-averaged sum of these per-step penalties defines our core objective:
Q(++1) + max A(0=(()) -AY
T
Σ 2=0
- -(0 Ex+1) + (1 (()) (()))((()))))) (ss
(6)
model related loss difference
drift loss
where for every time step t. f((e f(0++1, E(+1) is an inference loss term associated with the evolution of model parameters over time, which itself depends on the dataset evolution too, and A Afa_ := f(0r. Ex+1) f()r. is the drift induced inference loss term that relates the evolution of the data stream with the incurred loss. Note that this objective is derived from the right-hand side of our convergence bound in Theorem 5.1 which will be presented in Section 5 and serves as a proxy for minimizing the bound on time-averaged gradients, 7 ΣT= [ 2211))1
The practical objective in 6 introduces two key modifications from the theoretical bound. First, explicitly incorporates the policy decision by scaling the drift loss with Tl), isolating the drift penalty to non-update steps. Second, while the theoretical bound is defined for losses over the entire (and often inaccessible) datasets D,, our objective employs the practical losses computed on inference batches & for different times t.
5

---

Virtual Queue. To enforce adherence to resource constraints over time, we introduce a virtual queue Q(t), which evolves over time according to:
Q(t + 1) max {0, Q(t) + ((1)=(() -},
(7)
with Q(0) O. Intuitively, the virtual queue acts "debt accumulator," which tracks the violations of the resource constraint over time. When an update occurs, x(t) = 1, the queue grows in size by A(t), signaling resource expenditure; the subtraction of À reflects the permissible usage rate. As result, the queue evolves dynamically over time, signifying excessive updates when large, or suggesting room for more updates when close O. This mechanism is central to the Lyapunov framework: by seeking to keep the queue stable (i.e.. minimizing its "drift"), the resulting optimization ensures the average resource consumption aligns with À over time.
To formalize this, we first define the Lyapunov drift as AQ(() == (Q(L+ -())) which
corresponds to change in the "resource debt", The core of the method to greedily minimize a bound on the single drift-plus-penaal expression at each time I, which is a weighted sum of this drift and our performance objective:
(8) where (t) is the per-step penalty objective (the term inside the summation in Equation 6 and V > 0 is tunable parameter that weights the penalty term against the Lyapunov drift.
E[]Q(+)) + V (t),
4.2 Our Algorithm
Decision-Making, Solving the optimization problem involves greedily minimizing the bound on the
drift- ciitt-puupppaaae expression at each time step t. The result of the derivation, available in Appendix C
a theoretically greedily-pptimal threshold-basee policy at the core of RCCDA (Algorithm ! that
dictates an update (1) = 1) if and only the following condition holds:
v( ((00-1, - S(0 - (V) + +V+(()
X(0(0)))))00)))))0)00)))00)))))) + 2 [2(12 - 2λA((),
(9)
where [t) ==
G) + (1 1iai--nvVOO0 VV Ct), C++1); Ee+1)
[(,,- ((( + ()([++1)V/((--...+..)) 11+1) The intuition behind the update rule is direct orf The left-hand side, which we denote (I) represents the performance gain scaled by the trade-off parameter V. The G(t) term is a complex, non-causal expression that accounts for the loss reduction from the gradient step and the nested effects of how this action impacts the loss at time t ++ 1, given the optimal next action π (t + 1). This scaled gain is compared against the right-hand side, denoted C(t), which represents the effective resource cost. This cost term is fully known at time t and combines the cost associated with the current resource debt, Q(t) with the instantaneous cost of the update itself. The resulting rule (t) 2 C(t), dictates that the agent should update only when the scaled performance benefit is greater than the resource cost.
Practical Policy and Estimator. While the threshold in 9 provides per-time-step optimal solution, it is intractable and non-causal, making it impossible to implement in practice. The performance gain term (()) directly depends on the future data distribution De+1, the optimal future action (t + 1) (which itself creates a recursive dependency), and computations of various gradient terms that cannot be performed without violating resource constraints. There are several strategies to address this intractability, such as optimizing looser convergence bound or truncating the multi- step dependencies. In this work, however, we retain the structure of the ideal policy and instead utilize a causal approximation of this rule. We define the practical policy for RCCDA by replacing the intractable gain G()) with causal estimator function, (HH.), which utilizes only historical information Hi = fH Hgo.. Hoo available up to time t. The practical decision rule thus becomes (HH,)) > C(I). While this design introduces implicit estimation error that may affect performance, it also establishes RCCDA as a flexible framework, allowing the specific choice the estimator g to be tailored to different problems.
RCCDA Operation. Based on this practical, estimator- based policy, the RCCDA algorithm (Algo- rithm 1 operates over T = {O..... T 1}. After initializing the virtual queue Q(0) = 0 and history variables H. the algorithm proceeds at each time step / by first computing the current inference loss fe and updating the loss history Hy. It then uses its chosen estimator function ((H.) to calculate

---

the estimated scaled performance gain (H4). This gain is compared to the known resource cost threshold C(t) (the RHS of (9) make the update decision T(t). Based on this action, the model parameters 0, are either updated or kept constant, and the relevant histories are recorded. Finally, the virtual queue is updated to Q + 1) using the action taken, π (t), and its cost X(t), carrying the resource debt forward to the next time step.
5 Theoretical Analysis
In this section we conduct a theoretical analysis of the examined setting. In the analysis, we consider perfect estimator, i.e. (7H,) = (t)), which achieves zero estimation error. This allows us to derive performance bounds for the ideal policy, which serve as theoretical benchmark for any practical implementation. The analysis is structured as follows: We first introduce set of common assumptions required for the analysis. We then derive the general convergence rate bounds and conduct and demonstrate the result of stability analysis on the proposed update policy. The complete proofs of our theorems are available in the appendix.
5.1 Convergence Analysis
Assumption 1 (Smoothness). The agent's 's loss function is difserentiable and L-smooth, iee. Vlo, L1, 12 follows that f(O,, Di.) - f((r,, Pio)ll I L [|,,, - -Otall-
Assumption 2 (Bounded Variance). For all times t, the agent's 's stochastic gradient V f (t) is an unbiased estimate of the true gradient, i.e. Bi.~D. [V [e) f De), and the variance of the stochastic gradient bounded as EE,ND. [[V (x) - ff(,,, P.III 5
Assumption 3 (Bounded Concept Drift). For all times i, the magnitude of the concept drift between two data distributions Pt and p1t1 is bounded, which corresponds bound the KL-divergence measure on datasets at two consecutive times defined as Vt, (.(5)()))) > S.1. (RL(D, || De+1) S 5(1) v 8 sup (
Assumption 4 (Bounded Loss). For all times t, the magnitude of the loss f(0,, Dz) satisfies 0 f(e,. D.) B.
Theorem 5.1. If Asssumptions 1-2 hold and the learning rate is chosen such that n < time-averaged gradient satisfies
2Pa b. then the
T-1 Σ A fo(t)
Ln
T ((.,Dll & Tμ f(00, Do) -f(rr, +
+
2
Ln
min
(10)
where is the total number of time steps, Aff(t) = f(,, D++1) f(0,, D.) is the time-varying drift-induced loss change, μ (1) Lnj² and pa is the minimum probability of policy (t) being equal for all times t. Corollary 5.2. Under Assumptions 3-4, by the result in Theorem 5.1, the time-averaged gradient satisfies
T T
B
T-1 2 2=
L
VVf(e. D+11)l
1
Σ
Vo(t)
0²
(11)
T ((1
P
PT
Laj
2P min - Lm
0
There are several key steps in our convergence proofs. By incorporating the drift-induced loss term A ((), we able to quantify the effect of shifting data distributions between time steps. Utilizing Pinsker's inequality allows us to bound [f)) < between the drift induced loss and the magnitude of concept drift. Conditioning on the past dataset and model evolution allows us to model dependencies on past states, and decouple the policy action from the evolution of loss over time. Finally, by directly considering the action (() in the model update and introducing p we are able to integrate general policy information into the final bound. A detailed proof is provided in Appendix B
2 establishing an explicit connection
7

---

Discussion. Theorem 5.1 provides a general bound on the time- aveerage squared gradient norm, quantifying the impact of concept drift and the update policy on convergence. The bound's dependence on the cumulative drift, Σi= (t))), reveals an important insight: the time-averaged gradient only vanishes if the total drift grows sublinearly with T. Persistent, non-vanishing drift prevents the gradients from converging. The policy's 's role twofold: it explicitly appears in Pminn which inversely scales both the drift-dependent term (via (1) and the error term O(o²), highlighting the trade-off where more frequent updates (a higher puin lead to a tighter convergence bound, at the cost of more computation. Implicitly, the policy also governs the parameter sequence (0:) which determines the loss evolution term f(o. Do) - f0r Dr) + E ffa(t). A more effective policy will yield a more favorable loss evolution and thus a tighter overall convergence bound.
Corollary 5.2 demonstrates the relationship between convergence and magnitude of the drift. Without any further assumptions the drift, the gradients are not guaranteed converge, even when o as it would require an increasing learning rate, which is bounded by 1 < A. If Assumption holds t co, then 36 sup (ō(t) t N U {0)), and the time averaged gradients are bounded by O (V)) + +0 (o"). suggesting that in environments with persistent but bounded drift, the model
2P
achieves "stable" performance by oscillating within a region proportional to vō and a². Given additional assumptions about the concept drift, it is possible to demonstrate convergence. Suppose E. 8(t) O(T²-a) with > 0. Then, if we use 7) O(T such that 0 < B < min (a, 1), the bound converges as + 2 E [V, D++1) < O(T + O(T + O(T In practical scenarios, this corresponds to a decaying drift that eventually vanishes, allowing the algorithm to successfully converge a stationary point.
5.2 Stability Analysis
Now, we discuss how RCCDA satisfies the resource constraint of the problem (5) by following the threshold structure O9. In Theorem 5.3 provide an upper bound the constraint violation of our algorithm. This upper bound goes to zero as T + co.
Theorem 5.3. If Assumptions 1-4 hold, then the policy following Algorithm estimator; satisfies
with zero-error
T-1 Σ
A T-1 7
X2 T + 2VB
5
T-1 T
A(DE [T()) X=
X(t)
25(t)
(12)
T
The proof of Theorem 5.3 proceeds in two key steps. First, we show that the constraint violation term (the left hand-side of 12h is upper bounded by the queue length QT). for which an upper bound is also provided. Second, we establish an upper bound for the drift term AQ(t) 1/2(Q( + 1)
Q(1)²). This bound derived by comparing our policy with a stationary random policy that updates each time slot with probability min (A) detailed proof can be found in Appendix C.2
Discussion. Theorem 5.3 guarantees that RCCDA satisfies the resource constraint in 5 as 7
provided the cumulative drift ETD Vō(t) grows sublinearly in T and the update costs X(t) remain bounded, i.e., X(t) (11. This reflects a key strength of the Lyapunov drift-plus-penalty framework (21]. For finite T, the violation bound can be tightened toward zero by tuning V optimally. Additionally, under high but bounded drift rates (8(t) (()), the constraint violation decays as
O(I/VT) ensuring the policy adheres to the budget over long horizons without requiring drift to vanish.
6 Experiments
Datasets.
To represent concept drift, we utilize four widely used domain generalization datasets:
PACS [54), DigitsDG [55), and in the appendix OfficeHome (56), and MEMD-ABSA [57] The drift is simulated by introducing samples from different domains into the dataset. The model architectures utilized are detailed in Appendix H.2 Initially, we pretrain the models on data from a single source domain.
8

---

Table 1: Average validation accuracy of policies across concept drift schedules for PACS and DigitsDG datasets, mean update rate constraint: 0.1. We see that RCCDA consistently outperforms others, with the largest improvement for Burst and Spikes drift schedules.
Pollcy
PACS
Digits-DG
Drift Schedule
Burst
Step
Wave
Spikes
Burst
Step
Wave
Spikes
RCCDA (ours)
72.8 +:
72.0
67.0
4.7
73.0 = 3.6
77.6 = 6.3
74.6
= 7.0
65.0 11.2
74,3 + 7.3
Uniform
64.0
65.3 6.3
61.8 7.
60.8
1.7
11.4500
73.1
48
63.6
67.0
Periodic
4:
65.5 7.1
6
55.3
+ 4.0
69.8
72.7
: 46
63.5
68.1 6.88
Budget-Increase
59.1 13.5
9.9
9.0
53.1
16.4
65,9 =0:44 4.5
7
5.6
63.5
6.88
Budget-Thresholl Threshold
67.4 14,9
71.7 8.3
54.5
4 7.0
58.4 +
12.3
71.9
72.8
728815
57.4
62.
Drift Schedules. After the pretraining phase, the models are deployed at an agent with some fixed- size dataset, initially comprised only of source domain data. Over time, the dataset experiences concept drift, in the form of influx of data points from different domains, replacing existing data. The incoming domains, drift rate, and time of drift are determined according drift schedule. We consider 4 main drift schedules:
(i) Burs/ at pre-determined times, bursts with high drift rate happen, replacing most of the dataset with new domains. Otherwise, the dataset remains constant.
(ii) Step innt-illly the drift is O, then at some point, new domains start incoming slowly into the dataset, until the original is introduced back.
(ii) Wave - the drift is either O, or new domain data is introduced at a low rate into the dataset over some long interval periodically.
(iv) Spikes At random times, new domain data is inserted for some time with random drift rate.
At each time, the agent runs an inference task, calculating inference loss on separate, smaller holdout dataset that evolves the same way as the training dataset. The agent has access to a retraining policy that decides when to update the model, as well as the desired time-average-cos X. Each model update equivalent taking nstes SGD steps, with the gradients computed on batches sampled form the local dataset, and as such we consider constant update cost X. The agent also sets the value of Ā A. The entire framework runs for 250 time steps. The detailed experimental setup is available in Appendix B
Policies. We consider the following update policies.
| To implement the algorithm, we use a custom estimation function (?H)) = (f & (o... ((: (())-1, + (f(((()-1,)))
RCCDA: Our policy is provided in Algorithm
&(-=-,, where K, and K₂ are tunable constants.
The policy updates the model at every time slot with probability X/2. This policy is motivated from [58)
Uniform Random:
Periodic: This deterministic policy updates the model after every time interval A/,, i.e. TPerooiic 1 if = [AAA],, & e NN N (0), and 0 otherwise. The policy is modified from [59].
Budget-Increase The policy keeps track of an update budget available over time, then if it detects n E N consecutive loss increases and if there is budget available, the model is updated. Based on modified trend-detection methods in (60]
Hhugee--Trreshold The policy keeps track of update budget and window of losses Wy. Then if ft 2 (1 + <) maxxeww; Ji and there is budget available, the model is updated. Based on a modified version of [10]
Given this setup, we evaluate the classification accuracy at all times on the holdout dataset, while keep- ing track of the number of updates to monitor resource constraint satisfaction. Further experimentation details are available in the supplemenal material section.
Results and Discussion. Table [ presents the average model accuracy under mean update rate constraint O.I. We observe that RCCDA consistently outperforms the baselines across all tested configurations. Furthermore, Figure illustrates the dynamic behavior of accuracy and update rates, confirming that the robust performance is achieved while adhering to the long-term resource constraints.

---

Busget
Budger Thresholld
Burst
Step
Wove
Splkes
Accuracy (Pg)
75-
0.20
Aug Update Rate
0.10
0.05
Detegee Comportion
2.0
86-
0.-
:00
200
150
200
250
100
150
250
100
202
Figure 2: Dynamics of model accuracy and update rate for proposed and baseline policies under various concept drift schedules, with corresponding dataset composition over time. We see that the proposed policy recovers the quickest after sharp transitions while adhering to resource constraints.
Key Observations. First, compared with the baselines, RCCDA is able to effectively adapt to concept drift. Figure demonstrates that for fast drops in accuracy, the update rate increases rapidly, leading to fast performance recovery. This quick adaptation results in performance gains over the haseline polices. Notably, the Budget-Thhreshe capability, however it cannot adapt low-drift settings, not updating the model when could be beneficial. Second, the drift schedule directly affects the performance difference between the policies. For Burst, the rapid domain changes allow the model quickly adapt to new data, resulting high average accuracy. RCCDA is able to adapt quickly to these changes, which is why its performance gap is highest on Burst and Spikes. In contrast, the more gradual introduction of data the Wave and Step schedules contributes to a smaller performance gap. The low accuracy is most likely result of model capacity limitations, but the small performance difference suggest that lower drift rates, the time of the update might not be as significant as for higher drift rates.
policy also demonstrates a strongly reactive
Resource Utilization. Finally, RCCDA demonstrates its update converging towards the desired constraint of 0.1 over time, all while maintaining robust performance. This lends further practical significance Theorem 5.3] and also suggests that in order to maximize model performance, the policy maximally utilizes the available resources. In contrast, policies focused solely on resource constraints (Uniform, Periodic) fail to update when beneficial for performance, while the budget- based policies are not capable of modeling the complex drift behavior, leading to significant resource inder-utilization and lower average accuracy.
Additional Results and Ablation Studies. Additional experimental results, including more drift. schedules, different update rates, and more datasets, are available in Appendix
7 Conclusion and Limitations
In this paper, we proposed RCCDA: novel model update policy for adapting to concept drift under resource constraints. We first established convergence bounds under drift, then utilized Lyapunov analysis to arrive a policy that minimizes the bound on driit-plus-penalty term per each time slot. We then derived stability bounds, provably demonstrating adherence to resource constraints. Finally, we confirmed the effectiveness of our policy through experiments on various domain generalization datasets, with improvements in accuracy and drift recovery time relative to baselines. A potential limitation of our work is the estimation function ć introduced in the algorithm, its associated error can affect the threshold policy performance. Future work can rigorously characterize this impact and employ it to derive tighter performance bounds.
10

---

Acknowledgements
This paper was supported in part by the National Science Foundation (NSF) under grant CPS-2313108 the Office of Naval Research (ONR) under grant N000044-2-1-2305,, and the Air Force Office of Scientific Research (AFOSR) under grant FA955002441100883
References
[1] Zechun Liu, Changsheng Zhao, Forrest landola. Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernic Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, and Vikas Chandra. MobileLLM: Optimizing sub-billion parameter language models for device use cases. In Forty-first International Conference on Machine Learning, 2024. URL https://openrevieer httt//oroummdddII00X
[2] Zhihao Shu, Xiaowei Yu, Zihao Wu, Wenqi Jia, Yinchen Shi, Miao Yin, Tianming Liu, Dajiang Zhu, and Wei Niu. Real-time core-periphery guided vit with smart data layout selection grave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang. editors, Advances in Neural Information Processing Systems, volume 37, pages 95744-95763. Curran Associates, Inc., 2024. URL/htpps: httpttpp:///proceddnnggss nwwrips. cc/paper.....p..... pllls//ppper/222//iieel) adb777ccccaaa28811ccc66666666662 Conference .pdf
mobile devices. In A. Globerson, L. Mackey, D. Bel-
/ wwtppppppppppppppppeeaae.e....
[3] Mchdi Mohammadi, Ala Al-Fuqaha, Sameh Sorour, and Mohsen Guizani. Deep learning for iot big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials, 20(4): 2923-2960, 2018.
doi:
[4] Sachin Mehta and Mohammad Rastegari. Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. In International Conference on Learning Representations, 2022. URL /tttps://openrrrviee
[5] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pages 6105-61 14. PMLR, 2019.
[6] Timothy Castiglia, Yi Zhou, Shiqiang Wang, Swanand Kadhe, Nathalie Baracaldo, and Stacy Patterson. LESS-VFL: Communication--ffiiiee feature selection for vertical federated learning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learming, volume 202 of Proceedings of Machine Learning Research, pages 3757-3781. PMLR, 23-29 Jul 2023. /RRllttts://pproeeeeii
mlr wwwpppwww//eeeeepppppppp.. http://////assaaaaaa..a.... html
[7] Yuyang Deng, Mohammad Mahdi Kamani, Pouria Mahdavinia, and Mehrdad Mah- davi. Distributed personalized empirical risk minimization. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neu- ral Information Processing Systems, volume 36, pages 70812-70846. Curran Associates, Inc., 2023. URL wttp.tttttt/////oee//agooe neurips. wwwrips.cc/ppppeeeeeeee patles//pape///2////eee #ffe04496aaaa000000dd44666633333 Conere pdf
Paper- Conference
[8] Ronan Perry, Julius von Kügelgen, and Bernhard Schölkopf. Causal discovery in het- erogeneous environments under the sparse mechanism shift hypothesis. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neu- ral Information Processing Systems, volume 35, pages 10904 10917. Curran Associates, Inc., 2022. URL http:tttttts::/pceeediiii wwwpeiiinnnnnnnuuissueeeemee wwwpappr.pppppcppccpecpcec....ee wiwes/lllaaaeeee httes///ppr//222//1133 665126992ect54410055588222222222
pdf
[9] Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei Koh, and Chelsea Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. URL
[10] Albert Bifet and Ricard Gavalda. Learning from time-changing data with adaptive windowing. In Proceedings of the 2007 SIAM international conference on data mining, pages 443-448.
SIAM. 2007.
11

---

[11] Leandro L. Minku and Xin Yao. Ddd: A new ensemble approach for dealing with concept drift. IEEE Transactions on Knowledge and Data Engineering, 24(4):619-633, 2012. doi: 10.1109/KKE:2011 1.58.
[12] Ryan Elwell and Robi Polikar. Incremental learning of concept drift in nonstationary environments. IEEE Transactions on Neural Networks, 22(10):1517-153, 2011. doi: 10.1 N..11099TNN.20 1.2160459.
[13] Kai Helli, David Schnurr, Noah Hollmann, Samuel Müller, and Frank Hutter. Drift-resilient tabpfn: In-context learning temporal distribution shifts on tabular data. Advances in Neural Information Processing Systems, 37:98742 2024.
[14] Junbao Chen, Jingfeng Xue, Yong Wang. Zhenyan Liu, and Lu Huang. Classifier clustering and feature alignment for federated learning under distributed concept drift. arXiv preprint 244.:24400.1847,, 2024.
[15] YiFan Zhang, Qingsong Wen, Xue Wang, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Onenet: Enhancing time series forecasting models under concept drift by online ensembling, In Thirty-seventhh Conference on Neural Information Processing Systems, 2023. URL http::://ppenrrriee... htt.://ww.ettoorouumooo//oe/eeea
[16] Kunjal Panchal, Sunav Choudhary, Subrata Mitra, Koyel Mukherjee, Somdeb Sarkhel, Saayan Mitra, and Hui Guan. Flash: Concept drift adaptation in federated learning. In International Conference Machine Learning, pages 26931-26966 PMLR, 2023.
[17] Li Yang and Abdallah Shami. A lightweight concept drift detection and adaptation framework for iot data streams. IEEE Internet of Things Magazine, 10), 2021. doi: 10.1109/ 10TM.00M 100.0001..0 100002..
[18) Yu Sun, Ke Tang, Zexuan Zhu, and Xin Yao. Concept drift adaptation by exploiting historical knowledge. IEEE Transactions on Neural Networks and Learning Systems, 29(10):4822-488
2018. doi: 10.
[19] Ashraf Tahmasbi, Ellango Jothimurugesan. Srikanta Tirthapura, and Phillip B Gibbons. Drift- surf: Stable-state reattive-state learning under concept drift. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10064. PMLR, 18-24 Jul 2021. URL httptppsp//rrroceenngs mlr. /ttess/s//13///aaaabb1111.1..
[20] Alessio Mazzetto and Eli Upfal. An adaptive algorithm for learning with unknown distribution drift. In A. Oh, T. Naumann, A. Globerson, K. Sacnko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 10668-10887.0 Curran Associates, Inc., 2023. URL https: h/ppttpepp///rooeeeiinngg. wtwp:/eeeddininnneeeeeeeeeeeeeee neur wwt.erriippccppppppeeepppppp fatpppeprrfleseppprrrraepe .dff
[21] Michael Neely. Stochastic network optimization with application communication and queueing systems. Morgan & Claypool Publishers, 2010.
[22] Geoffrey I Webb, Roy Hyde, Hong Cao, Hai Long Nguyen, and Francois Petitjean. Characteriz- ing concept drift. Data Mining and Knowledge Discovery, 30(4):964 994, 2016.
[23] Supriya Agrahari and Anil Kumar Singh. Concept drift detection in data stream mining: A literature review. Journal of King Saud 9523-9540, 2022.
and Information Sciences, 34(10):
[24] Joao Gama, Pedro Medas, Gladys Castillo, and Pedro Rodrigues. Learning with drift detection. In Advances in Artificial Intelligence-SBl. 2004: 17th Bruzilian Symposium on Artificial Intelligence, Sao Luis, Maranhao, Brazil, September 29-Ocotber 1, 2004. Proceedings 17, pages 286-295. Springer, 2004.
[25] Denis Moreira Dos Reis, Peter Flach, Stan Matwin, and Gustavo Batista. Fast unsupervised online drift detection using incremental kolmogorov-smiroov test. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1554-1554, 2016.
12

---

[26] Pranoy Panda, Vineeth N. Balasubramanian, and Gaurav Sinha. Feature intuuur-interrretbble real concept drift detection. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy MI., 2023. URL URI.ttps: wttpttts: httttp:://openeevieew net/forumm atte//torummmdddAAAAAAAAAA 1d=pA2uDAvIpp
[27] Fabian Hinder, André Artelt, and Barbara Hammer. Towards non-parane parametric drift detection via dynamic adapting window independence drift detection (DAWIDD). In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 1 119 of Proceedings of Machine Learning Research, pages 4249 4259. PMLR, 13-18 Jul 2020. URL BtLLLttss///ppceeddiie
mlr wwwps//eee//spipeeeeee
html
[28] Maayan Harel, Shie Mannor, Ran El-Yaniv, and Koby Crammer. Concept drift detection through resampling. In International conference on machine learning, pages 1009-1017. PMLR, 2014.
[29] Albert França Josuá Costa, Régis Antnio Saraiva Albuquerque, and Eulanda Miranda dos Santos. A drift detection method based on active learning. In 2018 International Joint Conference Neural Networks (IJCNN), pages 1-8, 2018. doi: 10. 1091//NNNNN 8.8499644.
[30] Manuel Baena-Garcia, José del Campo- Ávila, Raul Fidalgo. Albert Bifet, Ricard Gavalda, and Rafael Soorales-Bueng Early drift detection method. In Fourth international workshop knowledge discovery from data streams, volume 6, pages 77-86. Citeseer, 2006.
[31] Oliver Cobb and Arnaud Van Looveren. Context-aware drift detection. In International conference on machine learning, pages 4087-4111. PMLR, 2022.
[32] Aodong Li, Alex Boyd, Padhraic Smyth, and Stephan Mandt. Detecting and adapting to irregular distribution shifts in bayesian online learning. Advances in neural information processing systems, 14:6816-6822 2021.
[33] Bruno Iran Ferreira Maciel, Silas Garrido Teixeira Carvalho Santos, and Roberto Souto Maior Barros. A lightweight concept drift detection ensemble. In 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), pages 1061-1068, 2015. doi: 10.1 109/ 1CTAL2015.151.
[34] Ahmad Abbasi, Abdul Rehman Javed, Chinmay Chakraborty, Jamel Nebhen, Wisha Zehra, and Zunera Jalil. Elstream: An ensemble learning approach for concept drift detection in dynamic social big data stream learning. IEEE Access, 9:66408-66411 2021. doi: 10.1 10./109/ACS
[35] Dariusz Brzezinski and Jerzy Stefanowski. Reacting to different types of concept drift: The accuracy updated ensemble algorithm. IEEE transactions neural networks and learning systems, 25(1):81994, 2013.
[36] Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip S. Yu. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 35()):8052-8072.
2023. doi: 10. 10.1099KKDEE.2222.7788.11
[37] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4396 4415,
2023. doi: 10. 10.1109/AAM..222.3113999
[38] Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Yu Philip. Generalizing to unseen domains: A survey on domain general- ization. IEEE transactions on knowledge and data engineering, 35(8):8052 8072, 2022.
[39] Mixue Xie, Shuang Li, Longhui Yuan, Chi Liu, and Zehui Dai. Evolving standardization for continual domain generalization over temporal drift. Advances Neural Information Processing Systems, 36:21983-2208 220022 2023.
[40] Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle
In International Conference on Learning Representations.
13

---

[41] Xiaotong Li, Yongxing Dai, Yixiao Ge, Jun Liu, Ying Shan, and LINGYU DUAN. Uncertainty modeling for out-of-distribution generalization. In International Conference on Learning Representations.
[42] Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M Hospedales. A simple feature augmentation for domain generalization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8886-8895, 2021.
[43] Krikamol Muandet, David Balduzzi, and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International conference on machine learning, pages 10-18. PMLR, 2013.
[44] Wang Lu, Jindong Wang, Haoliang Li, Yiqiang Chen, and Xing Xie. Domain-invariant feature exploration for domain generalization. Transactions Machine Learning Research, 2022. ISSN 2835-8856. URL httptttttt/:/eaeetttt
[45] Manh-Ha Bui, Toan Tran, Anh Tran, and Dinh Phung. Exploiting domain- specific features to enhance domain generalization. In M. Ranzato, A. Beygelzimer, Dauphin, P.S. Liang and J. Wortman Vaughan, editors, Advances in Neural Infor- mation Processing Systems, volume 34, pages 21189-21201. Curran Associates, Inc., URL htttps://prooeeedinng www.wedddninnnnnnnnnneeeeeeeeeee www..ps.cccpppppeererreee. wttparrpppsccppprfftleeeeeeeeeee http:ppae_fffeppaapeeeeeeee httpe//eeeeeaaee/////////e/////e 2021.
pdf
[46] Guangji Bai, Chen Ling, and Liang Zhao. Temporal domain generalization with drift-aware dy- namic neural networks. In The Eleventh International Conference on Learning Representations, 2023. URL httL:tttt//::poorreete http:/ppoor.iii...eeteototorrioa
net wwwp/ooroomiddwwoeeeee. 4n
[47] Zekun Cai, Guangji Bai, Renhe Jiang, Xuan Song, and Liang Zhao. Continuous temporal domain generalization. In The Tiihty- Annual Conference on Neural Information Processing Systems, 2024. URL
[48] Mixue Xie, Shuang Li, Longhui Yuan, Chi Liu, and Zehui Dai. Evolving standard- ization for continual domain generalization over temporal drift. In A. Oh, T. Nau- mann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neu- ral Information Processing Systems, volume 36, pages 21983 -20022. Curran Associates, Inc., 2023. URL www.e/ddiinnnooiis.c..coooaaa bacpppprrrrll__s_eeeleeeeeeeeeee fiwesflllppaeeee httles//aperr222//E11. 355a111649cd201000555e1566666663
pdr
[49] Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Puttern Analysis and Machine Intelligence, 46(8):5362-5383. 2024. doi: 1.1110//TPAMM.222.43366
[50] Ju Xu and Zhanxing Zhu. Reinforced continual learning. In S. Bengio, H. Wal- lach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL. https:://proceedinnn wwwpiinnnnnnneenreeeeeeeee neurips. httpr//pe.c/ppppee..eee///pppp/p httpe///////apee////////212 feec111cc2222f522210556666-----3
.ddff
[51] Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128-135 135, 1999.
[52] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Overcoming catastrophic forgetting in neural networks. sciences, 1 114(13):3521-3526, 2017
et al. Proooenss....rerrio Proceedings of the national academy of
[53] Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the eighth international workshop on data mining for online advertising, pages 1-9, 2014.
[54] Daniel J. Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, and Nicholas D. Lane. Flower: friendly federated learning research framework. CoRR, abs/2007. 14390, 2020. https:// .rg. htsp//bs/20001 14390
14

---

[55] Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision - ECCV 2020, pages 561-578, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58517-4
[56] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018-5027, 2017.
[57] Hongjie Cai, Nan Song, Zengzhi Wang, Qiming Xie, Qiankun Zhao, Ke Li, Siwei Wu, Shijie Liu, Heqing Ma, Jianfei Yu, al. Memd-absa: A multi-element multi-domain dataset for aspect-based sentiment analysis. Language Resources and Evaluation, pages 1-29, 2025.
[58] Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12): 2019. doi: 10.1099TKDE.208.276636.
[59] Cristiano Mesquita Garcia, Ramon Abilio, Alessandro Lameiras Koerich, Alceu de Souza Britto Jr, and Jean Paul Barddal. Concept drift adaptation text stream mining settings: systematic review. ACM Transactions on Intelligent Systems and Technology, 16(2):1-67, 2025.
[60] João Gama, Indré Žliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. survey concept drift adaptation. ACM computing surveys (CSUR), 46(4):1-37, 2014.
[61] Frank J Massey Jr. The kolmogorov-smirnov test for goodness of fit. Journal of the American statistical Association, 46(253) -87 1951.
[62] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier domain generalization, 2017. URt.htttps://arxii org/
03077
[63] Kaiming He, Xiangyu Zhang, Shaoging Ren, and Jian Sun. Deep residual learning for image recognition, 2015. URL
03385
[64] Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv: 10351, 2019.
[65] Mike Schuster and Kaisuke Nakajima. Japanese and korean voice search. In 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5149- 5152. IEEE, 2012.
15

---

Appendix
Societal Impact
17
Convergence Proofs
17
B Proof of Theorem 5.1
17
B.2 Proof of Corollary 5.2
19
Lappnov Analysis
20
Lyapunov Drift-Plus-Penan Framework
20
Stability Analysis: Proof of Theorem 5.3
22
Computational Resource Evaluation
24
D.I Resource Abstraction and Generalizability
24
D.2
Overhead Analysis of the RCCDA Policy
24
Detailed Experimental Setup
26
Datasets
26
Models
26
E.3
Pretraining Phase
28
E,4
Evaluation Phase
28
Additional Experimental Results
OfficeHome Dataset
31
MEMD-ABSA Dataset Textual Modality
31
Supplementary Concept Drift Schedules
32
Varying Update Rate
33
F5
Robustness under Uncertainty
34
F.6
Replay Buffer
34
Statistical Significance of the Results
36
Code Documentation
37
16

---

A Societal Impact
RCCDA ofters significant positive societal contributions by enabling artificial intelligence that is more sustainable, adaptable, and private. By optimizing the efficiency of model updates, our policy directly lowers the energy consumption of edge devices, reducing operational costs, increasing operation times, and promoting environmental sustainability. Additionally, this resource efficiency combined with RCCDA's lightweight nature makes it possible to deploy sophisticated, adaptive models on low-cost hardware, allowing for effective and reliable access to advanced technology even in dynamic, constrce- environments. Furthermore, by ensuring this adaptation happens directly on-device, RCCDA provides robust performance while fostering greater data privacy and autonomy for its users.
However, same adaptive capability that drives these benefits also introduces important consider- ations for responsible deployment. Since the policy adapts incoming data, it may inadvertently reinforce harmful patterns, especially if the data stream contains biases. The reliance on incoming data also creates vulnerability adversarial poisoning. where malicious actor could inject data designed to degrade model performance. Finally, continuous on-device personalization could bias the model favor type of data, resulting in the creation of insulating "echo chambers," potentially narrowing a user's exposure to different perspectives. As result, deploying this system respon- sibly requires integration of safeguards that ensure safe and fair operation, such as bias detection mechanisms, data verification protocols, or regular performance audits.
B Convergence Proofs
B.1
Proof of Theorem 5.1
First, we that
0,41-0 - - L) Sc+1),
(13)
where Ç++1 is a random batch sampled uniformly from the dataset D at time t ++11 1. For simplicity, in the following analyses, we use simplified gradient notation V f(,, Se+1) (ef G+11).
The loss difference over two time steps is comprised of two parts:
(10+1,Di41)) [(,, D.) = [f(0 ((0, (e+1) + [f(0,, D++1) - f(,,, D.) (14)
model-induced loss
Afo(t)
By definition, the second term is the drift-induced loss Afo(t) = f(0,, D(+1) f(0,, Dz).
Next, by Assumption and Equation 13 the first term (10++1,D++1l) - [0,, De+1) ) bounded as:
L ((04),,D+1) 5(0,, Dz+1) S (011)) -0)) + 110 -0112. (15)
Define the conditional expectation E,+1 [.] := Bz(++1) [. (t))), HoD(t (t + 1)], where Ho(t) a variable containing model parameters 0, 01,... .,. Or across time horizon {O...., ..), and HD((!)
stores the evolution of the dataset Do, D Dr41 across {O...., 1). As such, given the history
17

---

variables Ho),), Hplt + 1), we obtain:
Ex+1 [f(00+1, Dz+1) ((+,,)
(4)
(Vf(0,, Di41,, 8(+1 - -0r) + 10 -bel?
L
Es+1
2
(6)
V F((,, Dct1), Ex41 101+1 - 0.1 ) 1e+ |110+1--00 0cll
L
2 But1 (1' + ( ((1.
Vi(0,, D2+1) Be (t 1)7 f(0, Set1)l (c
V VS(0,, P++1), 7 [ 1)] 11 V [V
2
Ex+1 (T + 1)2 Be+1 (+.((0,,++1)1
2
(
7/441[I(( 1)]
Vf(0,, D++1), E+41 (V (
Ly Est1 (0..
Beti [i[( 1)]
(V//(,, (V), De+1), VS(O,, D(+1) ((+,))
L
Be++
[IVS((L
2
B141 (a(t + 1)]
Lm
E++1
[IV((0+
2
where:
(a) follows from Equation 15
(b) is true due to the fact that [(,,, D:41) (Ho(t), Hp(L + m))-measurable.
(c) follows from Equation
13
(d) is valid because the policy's action T(t 1) is independent of the training data batch Se+11 conditioned on the past history.
(c) is true because [x+I[T(t +1)] 1E+11 + 1) since r(t) {0, 1}.
(f) follows from Assumption 2.
Next, define Pmin such that as the minimum probability of the policy being equal to 1 at any time I, given the evolution of model parameters and datasets up to times and ++ 1 respectively. Additionally, it is true that [x(T(() < 1 VL. Combining this fact with the derived result, it follows that:
P min E. (t)) VI NU (0} for a given policy म, which is interpreted
Laj² Eet1 [ (1,)
E44 (54,,, D(41) [(,,, D(+1)] S Pm (V, D1 + 2
(16)
By applying the law of total expectation to Equation (6). follows that:
Elf(0 f(,, D.-1)] <-- - P-1)) Ln² E (VV
Ln7²
=mppr min ff( 11
E
(Vf(0..C(+1) f((,, ((+1) + ((.,)) D(+1)]1
2
L1j²
Pm min
E V D 11+1)11 +
E
[V (+1)
= P(+1))))
2
② P minn [ 1112
Ln
Ln²
0² +
E
D:41)l1
2
Lui
P
V (0,,
2
2
(17)
2
min
where (g) follows from Assumption 2 on the bounded variance of stochastic gradients.
18

---

By adding E (s411. f(0,, D.)l B [[f(((t on both sides of Equation 17 the expression becomes:
Laf p
La - 2 (l))(())
E (.
(18)
Now note that all expectations are of values calculated for the entire dataset at every time, so is true that: (1+(1 Dt+1)] ;(0++1, (i, D.)] (0,, D.). E. [[A[(( Afx((, and
V D++1)ll = P
Utilizing this fact and rearranging the terms, the bound on the gradient becomes:
(0,, D--)l1
Lng (S(Oc: D,)
p min? 7
2
ff/(t)
Ln d².
(19)
pit Pmiun 7 - Lnd² 2
2PTh
Ln
min
Finally, using / (Pmin La") ) by telescoping over and taking a time average, the expression
becomes:
T
T-1
VV (141)) D1+1)11
[(00, Do) - 1(0T, Dr) +
Σ
Alo(1)
T
nT
1=
Ln
(20)
2P
Lm
min
proving Theorem 5.1
B.2 Proof of Corollary 5.2
We can relate the drift-induced loss term with the dynamic bound on the concept drift as follows:
ftt(t) f(e,, D1+1) f(0,, D:)
(c) EunD [f(0, w)] - IENDD [f(O,.w)
Σ Pe (w)) - E (w:))))((0),)))
(c)
<
Σ
PD (wi) -PD, (wi)) B
(
PD.
P 111 B
(c)
< B 2Dxi. (Dr D(+1)
5
2 28(1),
(21)
where:
(a) follows from Assumption the unbiased estimate of the true gradient.
(b) uses the definition of expectation over discrete probability space. The sample space {w:}
is the of all possible batches that can be formed from the union of all datasets UT- D.-
Since this union is finite, the set {wi] is also finite. Pp, (wx) is the probability of sampling batch w; from the dataset Dr, where PD, (w:) 0 if wi cannot be formed from D. This probability distribution Pp, varies over time due to concept drift.
(c) is true by Assumption 4 on the boundedness of the loss, and the fact that Vo E RR S |x]- (d) follows from the definition of the I.1 loss, where PD, is the probability distribution vector, containing probabilities of sampling batches w, Vi from the dataset Dr.
19

---

(c) is true by the Pinsker's Inequality.
(f) follows from Assumption on the boundedness of the concept drift at different times t.
Now, by Assumption 4 on the boundedness of the loss:
f(0u Do) [(Or,Do) f(0, Du) B.
(22)
By applying the results of Equations bound becomes:
and 22
Theorem 5.1 and utilizing the definition of p. the
B
Ln
IV,( De+1)ll
1 /2
8(1)
0².
(23)
O
T
n
Lnp
0
2P
In
This completes the proof of Corollary
5.2
5.2
C Lyapunov Analysis
C.1 Lyapunov Drift-Plus-Perte Framework
The convergence result depends on two key components bounding the loss gradients over time: the telescoped loss f(o, Du) f(Or, Dr) and the drift-induced loss (t) By undoing the telescoping step, and utilizing the inference losses (which are an unbiased estimate of the true loses that use the entire dataset D, for computation), we can define the effective policy, policy that solves the following problem:
min T
Σ [0e, ((--)))))00----)-)))+)
x11) Alae().
model evolution loss under drift
drift loss
subject to T E (t))r(t) S
(24)
Note that the sum utilized in this optimization problem differs from the one in Theorem 5.1 as we
use the inference losses, and we introduced a (1 rt))) multiplier of &f.. The inference losses are used as that's what the agent can access at different time steps 1. The (1 (()) was introduced for two reasons:
It emphasizes the penalty associated with no update under strong drift, and rewards the updates better.
If the model is updated, (t) effectively corresponds to theoretical loss increase that would have happened had the model parameters not been updated. By introducing (1 (t)), we exclude this theoretical increase from optimization, placing emphasis only on observed loss values.
While this modification results in different policy compared to one derived from the unmodified penalty, & ET- f(0,, D.) f(0++1, D++1) + A fa)... it is important to recognize that applying the chittppus-penally framework to either problem formulation would not solve the long-term minimization directly. The Lyapunov method, by design, converts long-term stochastic optimization problem into series of deterministic, per-time-slot minimizations. This greedy, "one-shot" decision inherently an approximation of the true optimal policy that would minimize the sum over the entire horizon T.
Given that the per-slot minimization already a well-motivated heuristie, we justified in designing the per-slot penalty function to be a more effective proxy for our control goals. Our modification, which explicitly isolates the drift-loss penalty the (t)) 0 ("no-update") action, simply provides a more targeted and intuitive objective for the greedy controller to optimize at each step.
With the modified penalty, the optimization problem aim to solve is:
T
min
f(Or, Ex) -(0, ( + 1) (0,s Se+1), E++1) + (1 (1)) A Af8.e())
T
0=0 0
0
20

---

subject to T E (
(25)
Finding the optimal solution that minimizes entire sum is hard due to the evolving nature of the environment. A truly optimal policy would require knowledge of the future, as given full information about the evolution of the dataset over the entire time horizon, the optimal points to retrain will depend on all t. This requirement makes the problem intractable to solve directly. As such, we utilize the Lyapunov analysis framework to derive an online policy that does not require the knowledge of the entire environment evolution. We begin by defining the virtual queue as:
Q 1) max (0, Q(t) + X(t)x(t) x3,
(26)
from which it follows that the Lyapunov drift is:
(2(0)(1)- x)
1 _ + Q(t))) 2
Q(0) (((0 -
AQ(() The drift-plus-pentt term is bounded as:
(27)
2
[(QQ(t) (000)Q00 [(+) V (()) E Q(t) (A())(()) 2) (A(1)(()) - -2)² Q(t)
((O).E)) f(0,- - a
+ +(1 (1- (L-T(t))Afos(G)
Q00 (A())(())))))))))))))))))))))() - X)+ (A(t)m(t)-))) -2)² Q(1)) + +V((00,,5.
2
V *(1+ (O))) + V(1 (0,, ((L)))--((((((00000 (28)
where (t) is the penalty term at any time f, which corresponds to the inference loss increase due the evolution of the environment (0,,, Ex) [0, T(t ((,, (++1); Ex+1) (,++)) combined with the irift-induced inference loss increase when not updating the model (1 - (fa)
Next, is standard in Lyapunov analysis, solve the problem a per-time-slot basis. The problem to solve at each time t becomes (note that the expectation is dropped, because for a given value of (f), all values in it are deterministic):
min Q(1) (A(1)((t)-) (A(t)T(t) - x) =E(0,1)
V.(
Vf(0: T + f(0,, (441), + (1 m(t))VAfa((t))
Then, if T(t) = 0, we have 0, = - and the immediate bound becomes:
-iQ(1)+ X
VS
V = (co
Otherwise, x() 1, we have 0, 0.-1 V/V (0.-1, (t) and the immediate bound becomes:
X(t)((t) xQ(() + (A(t) - 2)2
+ -VV(--1--V/(ee
-V 1/ (0)-,,))))) 7(I 100v//0--~100000000 (44)))
By comparing the two resulting costs, the policy should update if the cost associated with not updating is greater than or equal to the cost of updating:
X²
-iQ(t) + + V V
2
2X(1Q(()) - AQ(1) + (A(t) - x12²
+ V
2
1 -1(( (t -0
21

---

By rearranging, we get
V (((0)-1,E++1) -00000/0-0-00000000000000----310
+ - - m - (et
Vf(0;-1 a(t 1
2 ((()Q(()) + [ 22x(t)]
For the simplicity notations, we define
(t) : - -1
f( -
Further, by rearranging, we arrive at the update rule proposed (
V (((0)-1. ((0-1100V((--.)))...)) + V (()Q(t) -
This completes the analysis.
Note that (t) corresponds the future impact on the loss the current decision, xt)). The choice of π (t) directly affects the virtual queue length Q + 1), altering the state observed by the policy at time + 1 and influencing the next decision (t + 1). This then changes the expected value of the future model, E[O++]]. As result, (+(() couples the immediate, greedy decision at time t with its expected consequence on the loss at time t + 1. It is intractable to compute online system, it depends on future values of environmental variables. The same is true for the f( - f( 7) V f(, (t). Et) term (where while computing the gradient at every time step t is feasible, it would violate the resource constraint). As result, while both terms crucial for the theoretical derivation, in our practical implementation of this policy we use an approximation for
practical
((00-1) f(0 1 f10--11177((0 + J+))..
C.2 Stability Analysis: Proof of Theorem 5.3
Let x* denote our proposed policy. At each time slot t, the policy was chosen to minimize
Q()) (X())((t)- - 2) (1())(()) - x) VS(0,, & f(0e+1, (1 (())-(())Afsse))
with probabiiiyy min i 1)
Next, consider policy Uniform such that TUniform (t)
Then, for all i, the following holds
E
Q(t) (A(t)=((t) - A) + (A())= - -x))
+ - -1- (1)VV\
(29)
<E Q(t) (A(t)mUnifomml (t) -
(X(t)TUnifoom) (t) (x) +V((0..)) - (o.t3)))-((((00 V
+ (1 Uniioor (1)VASS(1)
Next note that:
[Q(t) U())/Unif (t) [(t)] (Q(t))(t) IT Uniform (t) Q())) E (Q(t)Ā Q(t)
Q(()A(()TE) ( Uniform (1) [[(1) XQ(1)
= Q(1)A(() X(e) XQ(t)
22

---

0.
(30)
Then, using the same principle as above, as well as the fact that E[x(t)] = [t))))),, it follows that:
E
A(m)Fomm (1) (1)
2) Q(1)
2 (A(1) - X)
Using the above, Assumptions 3-4, and introducing E.- [+] and Exvaten [1] the expectation over the drift-plus-penalty given Q() for policies π and TUniform: respectively, the inequality becomes:
Ex Q(1) (A(1)= - 2) + (1)))=((1-))) x)²
Ā (A(t) 2) - VE (f E:)1 + VE,.- (((1))
2
Viniinno [f(0), Ex]l - V Suation [((1)+
VETTIaT [1)) V Vartiim min
X x(T) (
VE [t) + VE,, ITT (t)Aff,e(t)
Σ 1 (A(t) - X) - 2 + VB (5(t) + 3VB 2
(A(0) - X) - 5V + VBV 25(1).
(31)
By taking the total expectation, and by definition of the Lyapunov drift:
E[AQ(t) S x (2(t) X) + 5VB + VB 28t)t(t).
Next, we have
T-1 1 Q(T)) - 200) -210)
Σ AQ()
2
E [Q(T) - [Q(0)]]
22
2
T-1
Σ E B(QQ(t) S 1
Σ A(t) - T2)
+ + VB
20())
2
t=
+E (E() < ([()))
EIQ(O) +A
Σ X(1 -T
TÀ
2V
57 Σ 225(1)
and assuming Q(0) 0, we arrive at an intermediate bound on the expected queue length at time T:
T-1
E [Q(()) <
Σ x(t) TÀ
+ 2VB
5T +
/ /8((t)
(32)
t=
Then, ((1)=(1) - - Q(U((+1) Q(1) implies
Σ (0)] 1 } EIOMT
T-1
X2
5
T-1
A(1)
+
2VB
Σ
25(())
T
T
T
T
t=0
So, the final inequality becomes:
(()) - À
Σ
T-1
X(t)
X2
+ 2VB
5
1
T-1
25(t)
(33)
T t
T
T
T
T
m=
1=
This completes the proof Theorem 5.3
23

---

D Computational Resource Evaluation
In this section, we provide detailed evaluation of the computational resources associated with the operation of RCCDA. We first discuss the abstract nature of "resources" assumed in our framework demonstrate its generalization. We then conduct an overhead analysis quantify the additional resource consumption of RCCDA itself compared to the evaluated baseline policies and discuss its efficiency relative traditional drift detection methods.
D.1 Resource Abstraction and Generalizability
Throughout this paper, the concept "resource" treated abstractly. This is a deliberate design choice to ensure that our framework is broadly applicable to a wide range of real-world, resource- constrained scenarios. resource can represent any quantifiable commodity that consumed during model update and is subject to limited budget. This could be computational cycles, FLOPs, power consumption, data usage, or wall-clock time. At any time step f, the agent can perform model update, which incurs general, time-van vareiia cost, X(t), associated with the considered resource type. The agent's ss operational constraint is total resource budget over the entire time horizon, which is enforced via the time-average resource constraint, X, used bound resource usage in Equation (5). This is equivalent to ensuring the total amount of consumed resources, Σ A(t)r(t), does not exceed the total available resource budget, TÀ.
To demonstrate that this approach is generalizable across different resource types, we conducted a new empirical analysis to quantify the average resource usage per update for several distinct metrics. In our setup, we utilized the same experimental settings as in our main experiments on the PACS dataset, details available in Appendix [E] averaging the results resource usage for single model update across the following metrics: computation (GFLOPs), data processed (GB), memory-lime usage (GB-s), energy consumption (Joules), and wall-clock time (s).
250 updates. We measured the
Table 2
reports the empirically measured average cost per update X(t), and the corresponding total available resource budget 7' for various metrics.
Table 2: Empirical analysis of resource consumption for different resource types for the main experiment on the PACS dataset.
Resource Metric Type (Unit) Constant Average Update Cost
Total Available Budget
Computation (GFLOPs)
11438.0 + 0.0
285,951.0 2.92975
Data Processed (GB)
0.11719 + 0.0
Memory-Time (GB-s)
20.11 + 0.3
502.75 75
Energy (Joules)
626.67 = 24.35
15.666.75
Wall-Clock Time (s)
2.174 + 0.037
54.35
Note that RCCDA performs identically for all the resource metrics in Table 2 as the experiments were configured to use the same ratio X(2) (with A(t) set to a constant value A), effectively defining a target update frequency constraint. Since the decision logic in RCCDA is driven by this ratio and the observed model loss, its behavior remains consistent regardless of how the "resource" is defined. This result therefore confirms that our framework is generalizable to different resource metrics, demonstrating that RCCDA is a versatile solution for resource-cone constrained adaptation.
D.2 Overhead Analysis of the RCCDA Policy
critical aspect of designing a model update policy for resource-constraine environments is ensuring the decision- decision-making process itself does not introduce significant computational overhead. Many existing approaches to mitigating concept drift rely on explicit drift detection mechanisms, which makes them computationally intensive and ill-suited for real-time decision making.
To demonstrate the necessity for lightweight policy that does not rely on explicit drift detection, we conducted empirical analysis comparing RCCDA against two established drift detection methods: the Adaptive Windowing (ADWIN) (7 algorithm and the Kolmogorov-Smimng (KS) 61 test. Traditional drift detection methods like these typically require constant monitoring of
24

---

the statistical properties of the entire incoming data stream (KS-Test) or sequentially processing eтror rates (ADWIN). In contrast. RCCDA relies solely on the model's inference loss, which is lightweight scalar value, often computed or estimated as part of the model's primary task evaluation. The reliance on a pre-existing simple signal makes our approach inherently efficient.
To quantify this efficiency, we measured the decision-making wall-clock time taken by RCCDA, and compared it to a wall-clock time taken by ADWIN and KS-Test to finish performing drift detection operations, which is required step if using these algorithms for decision making. We assume all necessary inputs (such as the inference loss for RCCDA, data windows for the others) are already available through running the initial model evaluation. The experiment was conducted using the PACS dataset with the setup detailed Appendix E averaging results over 4 domains and 3 different seeds.
The results in Table 3 demonstrate the computational efficiency of our approach. RCCDA's decision time negligible, measuring in the sub-microsecond range. In contrast, ADWIN is over three orders of magnitude slower. While small in isolation, especially compared to the model inference time, this latency would accumulate over long operational horizon, increasing both response delay and cumulative resource consumption. The KS-Test proves computationally prohibitive, exceeding RCCDA's decision time by over eight orders of magnitude due to the requirement of performing statistical tests on the entire data window. These vast time disparities underscore a conclusion that policies reliant explicit drift detection introduce a temporal overhead that is unsustainable in many resource-constrained scenarios, validating our lightweight, loss-driven approach.
Table 3: Comparison of additional decision decision-making time for RCCDA versus established drift detection methods.
Algorithm Additional Decision Making Time (ms) Relative Slowdown (vs. RCCDA)
RCCDA
0.000744 = 0.000142
ADWIN
1. 1.845618 + 0.469732
2480 169, 350, 134
KS-Test
125996.5 9605.7
While this comparison demonstrates that the traditional drift detection methods add a significant computational overhead from monitoring data streams or performance metrics, ADWIN and the KS-Test are not policy algorithms by design, and therefore are not designed to operate under strict resource budget. As such, more direct and informative benchmark for comes from comparing RCCDA against the baseline methods from the main experiments, which are explicitly designed for operation under resource constraints. These policies, such as Uniform or Periodic updates, are inherently lightweight and serve as an excellent benchmark for verifying that RCCDA does not introduce prohibitive computational and resource cost relative to simple heuristics. We therefore evaluated the total memory footprint, decision time, and thcoretical complexity for each policy to provide a comprehensive overhead analysis. The experiment dataset, with a window size of w = 40 used by Budget-Increas and Budget-Threshod policies. The results summarized in Table 4 As shown, RCCDA's resource consumption is comparable the baseline policies on all evaluated metrics.
overhead
conducted the PACS
Table 4: Overhead comparison of RCCDA against baseline policies. All policies exhibit minimal overhead, with RCCDA remaining competitive while providing robust adaptation.
Policy
Total Memory (bytes)
Wall-Clock Time (us)
Complexity
RCCDA
276
1.94
(1)
Uniform Random
72
1.79
Periodic
108
0.99
Budget-Increase
212 + 32w
1.74
(w)
Budget-Threshold Threshold
212 + 32w
1.68
(ww) (1)1) optimized)
This two-fold resource analysis demonstrates that RCCDA both vastly superior to traditional drift detectors and is competitively efficient against the lightweight baselines, confirming that our policy successfully integrates an intelligent, adaptive mecision mechanism without incurring prohibitive operational cost.
25

---

E Detailed Experimental Setup
In this section, describe the experimental setup in detail. Our main experiment consists of two major phases, In the pretraining phase (described in E.3), we train the models on selected source domains from the datasets until convergence. the subsequent evaluation phase (described [E.4). the agent deploys the pretrained model in an environment that simulates concept drift by introducing new domains, and uses retraining policy mitigate that drift.
E.1 Datasets
To simulate concept drift our experiments, we used common domain generalization (DG) datasets:
PACS consists of images from domains: Photo, Art Painting, Cartoon, and Sketch, each with the 7 common categories: dog, elephant, giraffe, guitar, horse, house, and person. The default size of the images is 224 x 224. We transform all images into 128 x 128.
PACS 62]:
DigitsDG [55]: DigitsDG encompasses images of handwritten digits to 9, sourced from different domains: SVHN, SYN, MNIST, and MNIST-M. The image sizes are 32 x 32.
OfficeHome [56]: OfficeHome consists of images from four domains: Art, Clipart, Product, and Kkeal-Wordd, each featuring 65 object categories commonly encountered in office and home environments, including chairs, desks, and household items. The image sizes are 224 x 224.
MEMD-ABSA [57]: MEMD-ABSA is multi-domain textual dataset for Aspect-Based Sentiment Analysis (ABSA). consists of approximately 20,000 sentences from 5 distinct domains: Books, Clothing, Hotel, Laptop and Restaurant. Each sentence annotated with one or more sentiment quadruples: aspect, category, opinion, and sentiment. The sentiment is then classified as positive, negative, or neutral.
E.2 Models
Below, we specify the models employed for each dataset:
PACS. For the PACS dataset, skip connections. The model has 11,177,223 parameters, and its architecture is detailed below:
PACSCNN, custom convolutional neural network (CNN) with
Layer
Layer Description
Convolutional layer: input channels, 64 output channels, kernel size 3, stride 1, padding
2
Batch normalization layer
ReLU activation
Residual block: 64 input channels, 64 output channels, stride
Residual block: 64 input channels, 64 output channels, stride
Residual block: 64 input channels, 128 output channels, stride 2
Residual block: 128 input channels, 128 output channels, stride
Residual block: 128 input channels, 256 output channels, stride 2
Residual block: 256 input channels, 256 output channels, stride
10
Residual block: 256 input channels, 512 output channels, stride 2
Residual block: 512 input channels, 512 output channels, stride
12
Adaptive average pooling layer: reduces spatial dimensions to x
Flatten operation
14
Dropout layer: p = 0.5
Linear layer: 512 input features to 7 output classes
Each residual block has the following structure:
26

---

Layer
Layer Description
Convolutional layer: input channels to output channels, stride passed as argument,
kernel size 3, padding
2
Batch normalization layer
ReLU activation
Convolutional layer: output channels to output channels, kernel size 3, stride 1, padding 1
Batch normalization layer
Skip connection: identity mapping if input channels equal output channels
and stride=1; otherwise, a 1xl convolutional projection with the given stride followed by batch normalization ReLU activation
DigitsDG. For the Di dataset, DigitsDGCNN, CNN model with 1.907,146 parameters, and the following architecture:
Layer
Layer Description
Convolutional layer: 3 input channels, 64 output channels, kernel size 3, stride padding
Batch normalization layer
ReLU activation
Convolutional layer: 64 input channels, 64 output channels,
kernel size stride 2. padding
Convolutional layer: 64 input channels, 128 output channels, kernel size 3, stride padding
Batch normalization layer
ReLU activation
Convolutional layer: 128 input channels, 128 output channels,
kernel size 1, stride 2, padding
Convolutional layer: 128 input channels, 256 output channels, kernel size 3, stride padding
10
Batch normalization layer
ReLU activation
12
Convolutional layer: 256 input channels, 256 output channels,
kernel size 1, stride 2, padding
13
Convolutional layer: 256 input channels, 512 output channels, kernel size 3, stride padding
14
Batch normalization layer
15
ReLU activation
16
Convolutional layer: 512 input channels, 512 output channels,
kernel size 1, stride 2, padding
17
Adaptive average pooling layer: reduces spatial dimensions to 1
18
Flatten operation
19
Dropout layer: p = 0.5
20
Linear layer: 512 input features to 10 output classes
OfficeHome. For the OfficeHome dataset, we use OfficeHomeNet, a model based on pretrained ResNet-18 architecture (63. To adapt the model for the dataset's complexity, we replace the original fully connected layer with custom classification head. We fine-tune the model by training only the final convolutional block (layer4) and our custom head, keeping all earlier layers frozen. This results in model with 8,541,761 trainable parameters out of 11.224,54 total. The custom head consists of:
Layer
Layer Description
Linear layer: 512 input features to 256 output features
ReLU activation
3
Dropout layer: p = 0.5
Linear layer: 256 input features to 65 output classes
MEMD-ABSA. For the MEMD-ABSA dataset, fine-tune model based the TinyBert
architecture (64), called TinFBenFForSettmment The model comprises 4,386,307 parameters, and has the following structure:
27

---

Layer
Layer Description
Embeddings Layer: word embeddings (30522 vocab size, 128 dim),
position embeddings (512 max sequence length, 128 dim),
token type embeddings (2 types. 128 dim), layer norm, dropout layer (p = 0.1) Encoder layer transformer encoder block
3
Encoder layer 2: transformer encoder block
Pooler: linear layer (128 to 128), Tanh activation
Dropout layer: p 0.1
Linear layer: 128 input features to 3 output classes
Each transformer encoder block is comprised of 2-head self-attention mechanism with hidden size of 128 and GELU activation functions. We tokenize input text using pretrained BERT tokenizer that employs the WordPiece tokenization 65]. All sequences are subsequently padded or truncated to a fixed length of 128 tokens.
E.3 Pretraining Phase
The pretraining phase begins by partitioning the entire dataset into training and holdout sets. We utilize the 80:20 split and uniform random sampling for the image datasets, and adopt the predefined training and validation (holdout) splits for the text dataset. We then filter both the training and holdout sets to contain only samples from a single designated source domain. The resulting single-domal training set is denoted by D.
new agent with a custom model then instantiated and the model is trained on D. Upon initialization of the loss criterion and optimizer, the training loop begins. Within each iteration, the agent employs the update steps method to refine the model parameters. Unlike traditional epoch-based training, this method samples batches, each of size [c]. from the training set for Rsteps steps. For each batch, it computes the gradients and updates the model with the specified optimizer and learning rate 1). This process is repeated for total of I updates.
After each training iteration, the agent evaluates the revised model on the single-domaie holdout set. For the image datasets, to prevent overfitting, we employ an early stopping mechanism that terminates the training process if the holdout set validation accuracy exceeds predefined threshold acCuresh For the text dataset, we train for the full 7' iterations. Upon completion, save the parameters of the final model for the subsequent evaluation phase, which corresponds to the last updated model for the image datasets and the best model for the text dataset.
All hyperparameters are detailed in Table Their values were determined through combination of grid search and random search.
Table 5: Hyperparameter settings for the pretraining phase across PACS, DigitsDG, OfficeHome, and MEMD-ABSA datasets.
Hyperparameter
PACS
DigitsDG
OfficeHome
MEMD-ABSA
Learning Rate (1))
0.005
0.05
0.05
5e-5
Batch Size ((E)
128
128
256
32
of Steps/Iter (Nateps)
10
10
20
60
Number Iterations (T)
200
200
100
10
Max Sequence Length
N/A
N/A
N/A
128
Loss
Cross Entropy SGD
Cross Entropy SGD
Cross Entropy SGD
Cross Entropy
Optimizer
AdamW
Momentum
0.9
0.9
0.9
N/A
aCCthresh
75%
75%
90%
N/A
E,4
Evaluation Phase
Dataset and General Setup. The initial data partitioning and single-domain filtering for the evaluation phase are identical to the pretraining phase. From these domain-specific splits, we construct training set, with a fixed size [Deal, and corresponding holdout set of size 0.25 x DDeal.
28

---

The size [Peel is chosen to ensure that any single domain contains sufficient samples to fully populate these sets after the initial partitioning.
We then initialize the agent and run a concept drift simulation for T = 250 time steps. The drift is simulated by modeling dynamic environment in which the data distributions change over time. At each time step t, the agent's 's model performance (e.g., loss and accuracy) is measured exclusively on the holdout set. This performance information guides the agent's update policy that determines whether to retrain the model. If an update is triggered, the agent retrains its model on the training set using the update steps method with nspps steps, batch size of [<], and learning rate 1). The policy operates under a budget constraint A, which, given a constant update cost X, results in an upper bound on the effective update rate equal to X- All evaluation phase hyperparameters are summarized in Table
Table 6: Update hyperparameter settings for the evaluation phase across all datasets.
Hyperparameter
PACS
DigitsDG
OfficeHome
MEMD-ABSA
Training Set Size ([Dseil)
1024
1024
2048
1024
Learning Rate (1)
0.01
0.05
0.05
5e-5
Batch Size ((E))
128
128
256
32
of Steps/Update (Mstees)
5
5
60
Max Sequence Length
N/A
N/A
N/A
128
Loss
Cross Entropy SGD
Cross Entropy SGD
Cross Entropy SGD
Cross Entropy AdamW
Optimizer
Drift Schedules. To simulate concept drift, employ drift scheduler. The scheduler alters the domain composition of the training and holdout sets by replacing existing samples with data from new target domains. For any given time t, this process is governed by drift schedule, which defines the specific pattern, timing, and rate of this change, including periods of no drift where the rate is zero.
We implement total of seven schedules: four are presented in the main paper, with three additional schedules introduced in Appendix F.3 All schedules use a replace strategy, where a fraction of the existing data is substituted with samples from the target domains. This ensures the dataset size remains constant throughout the simulation.
The specific configurations are detailed below using the PACS dataset as an example. The setups for DigitsDG, OfficeHome, and MEMD-ABE ABMD-ABSA are analogous, using their respective domains.
Burst: Introduces new domains in periodic deterministic bursts. After initial delay of 45 time steps, the first burst starts at = 45 (introducing Photo domain data) and the second starts at t 165 (introducing Cartoon). Each burst lasts for 3 time steps with drift rate of 0.4, effectively replacing the entire dataset.
Step: Introduces domains in discrete steps with increasing drift rates. After an initial period of no drift, the drift rate is set to 0.004 at / 60 (introducing Sketch), changing to 0.006 at / = 120 (introducing Cartoon), and finally changing to 0.008 at t = 180 (introducing Art Painting).
Wave: Consists of repeating cycles. After an initial delay of 50 time steps, a wave begins where the domains are introduced at rate of 0.032 for 30 time steps. This is followed by 70- (0-time-step period no drift. This 100-step cycle (30 drift and 70 no drift) then repeats. Domains cycle the order: Photo, Cartoon, Sketch.
Spikes: Extends the Burst schedule with randomized characteristics. The spike interval is uniformly random between 90 and 130 time steps. Similarly, the initial delay is between 30 and 60, the duration is between 3 and 6 time steps, and the rate is between 0.3 and 0.6. Domains cycle in the order: Photo, Cartoon, Sketch.
The drift occurs a constant rate of 0.016. The target domain cycles every 50 time steps in the order: Sketch, Photo, Cartoon, Art Painting.
Constant:
Decaying Spikes: Modifies the Burst schedule with increasing intervals between spikes. After an initial delay of 20 time steps, the first spike occurs. The interval to the next spike starts at 30 and
29

---

increases by 10 after each subsequent spike. Each spike lasts for 3 time steps and has drift rate of 0.35. Domains cycle in the order: Sketch, Photo, Cartoon.
Scasonal Flux: Simulates eyclic drift. After initial delay of 10 time steps, the schedule cycles between the Photo and Sketch domains. A full cycle (e.g.. from Photo to Sketch and back) takes 150 time steps. The drift rate varies sinusoidally between 0.001 and 0.016.
Update Policies We implement the proposed RCCDA policy and four baseline strategies, detailed below.
Uniform (Policy 1): Retrains randomly with a fixed probability A π.
Periodic (Policy 2): Retrains at fixed intervals X² =
Budget-Ineree (Policy 3): Retrains when the loss increases for three consecutive time steps at the end of sliding window of size = 40, provided sufficient budget available. The budget accumulates at a rate of = क per time step.
Budget-Threshen (Policy 4): Retrains when the current loss exceeds the maximum loss a window of size w 40 by threshold e = 0.1, provided a sufficient budget is available. The budget accumulates rate of Σ मे per time step.
RCCDA (Policy 5): The proposed policy with estimation defined as (?..) = (f minee (o....tt) f(,,, +.)) ((00:::,))-((00..----000 The update occurs if: (f m f(O,, {.)) + (Ka(f((... {.) f(0)-1, 2 Q(t) + 0.5 R/ We set V = 10 constant across all experiments, and tune the Kp, Ka values for different settings and resource
constraints. The virtual queue updated as Q(t + 1) max { 0, Q(t) x(t) - } where
(l) {0, 1) is the update decision at time L.
In Tables 7 and we summarize the values of Kp, Ka used by RCCDA across the conducted experiments.
Table 7: Optimal (Kp, Ka) parameters for RCCDA across various datasets and drift schedules. The update rate was set to 0.1 for the image datasets and 0.01 for MEMD-ABSA.
Drift Schedule
PACS
DigitsDG
OfficeHome
MEMD-ABSA
Burst
(1.0, 0.1,
(2.5,
(1.0,
(0.1,2.5
Step
(1.0, 0.1)
(1.0,0.5)
0.5
N/A
Wave
(0.5,
(1.0,0.5)
(1.0,
N/A
Spikes
(0.75.0.5)
(..0,0.1)
(1.0.055)
N/A
Constant
(0.2,
(0.3,0.25
N/A
N/A
Decaying Spikes
(..5
(0.5,0.1)
N/A
N/A
Seasonal Flux
(0.3,0.25)
(0.5,0.1)
N/A
N/A
Table 8: (Kp, K2) parameters used by RCCDA across different update rates for the Burst drift schedule, Evaluated on the PACS and DigitsDG datasets
Update Rate X
PACS
DigitsDG
0.02
(0.06, 0.05)
(0.1,
0.03
(0.13, 0.05)
(0.28, 0.05)
0.05
(0.5, 0.1)
(0.55, 0.1)
0.07
0.17,0.1)
(.2)
0.15
1.0)
(..0,
0.20
(7.0,2.5) 2.5)
(7.0,
0.25
(10.0,4.0)
(14.0.6.0)
0.30
5.5)
(20.0.8.0)
Evaluation Loop With this setup, we execute the evaluation loop for 250. At each time step 4, the loop executes the following operations:
30

---

1. Apply Drift:
The drift scheduler determines the current drift rate and target domains. Drift is then applied to the agent's 's training and holdout datasets.
2. Evaluate Performance: The agent's model performance (current loss and accuracy) is computed on the holdout set.
3. Policy Decision: The agent's s policy determines whether to retrain based on current and historical performance metrics.
4. Retrain if Decided: If retraining is triggered, the agent updates its model parameters for Rsteps
steps using the training set.
Results All results are averaged over multiple random seeds (10 for experiments in the main paper and 3 for those in the Appendix). In Figure consider single starting domain for each of the drift schedules for the PACS dataset. For the numerical results in Table [ we further average the seed-averaged results across different starting domains. Specifically, we average the results for PACS over starting domains: Photo, Art Painting, Cartoon, Sketch; for DigitsDG over starting domains: MNIST, MNIST-M, and SVHN; for OfficeHome over starting domains: RealWorld, Product, Art; and for MEMD- over starting domains: Books, Clothing, Hotel, Laptop and Restaurant. In all cases, the reported accuracy is the average performance on the holdout set over the entire operational period T.
F Additional Experimental Results
OfficeHome Dataset
In this section, we provide additional results on the OfficeHome [56] dataset, a domain generalization benchmark consisting of four domains: Art, Clipart, Product, and Real-World.
The results are reported in 9 They are consistent with our findings on the PACS and DigitsDG datasets (Sec. []) with RCCDA achieving significant performance improvements for the Burst schedule, moderate improvements for the Wave and Spikes schedules. On the Step schedule, it achieves the highest accuracy, performing competitively with the other baselines. This further demonstrates the effectiveness of our policy under resource constraints.
Table 9: Average validation accuracy (%) of policies across concept drift schedules for the OfficeHome dataset. The mean update rate constraint is X 0.1. The best-performing method for each configuration shown in bold.
Policy
OfficeHome
Drift Schedule
Burst
Step
Wave
Spikes
RCCDA (ours)
86.9 + 2.8
90.0 + 1.8
87.5 = 2.9
89.9 + 2.9
Uniform
81.9 + 3.3
88.9 + 2.5
85.0 3.0
85.8 + 5.2
Periodic
82.8 + 3.5
89.6 2.2
86.3 = 3.0
88.3 + 4.8
Budget-Increase
70.7 1 4.7
89.3 2.2
85.4
79.3 1 3.7
Budget-Threshon
77.7 1: 5.2
85.4 1: 2.6
79.2 - 7.5
87.7 1 2.4
F.2 MEMD-ABSA Dataset Textual Modality
In this section, we provide additional results on the MEMD-ABSA dataset [57]. multi-domain benchmark for textual Aspect- Based Sentiment Analysis (ABSA) with reviews from five domains: Books, Clothing, Hotel, Laptop and Restaurant. This experiment evaluates method on new, textual, modality, and explores its performance in a setting with less severe concept drift.
Our theoretical analysis is general and not limited to specific data modality. Consequently, our policy is provably modality-invariint, with only its algorithmic constants being task- -es--deenddet. Operationally, however, the efficacy of RCCDA strongly relies on the magnitude of performance degradation detected during distributional shift. On the MEMD-ABSA dataset, this degradation is modest when transferring between domains, especially compared to the vision tasks, as we demonstrate Table TO While the top-performing model for any given domain is the one trained specifically on it, the performance drop from using models trained on other domains is not significant,
31

---

with those models still achieving competitive classification accuracy. This strong generalization suggests that, for the architecture used, the textual domains are not sufficiently distinct generate concept drift as severe as that observed in our vision-based experiments.
Table 10:
Cross-domain test accuracy (%) for TinyBERT on the HEMD-ABSA dataset. Each column shows the performance of models pretrained on different source domains and evaluated on the specified target domain.
Pretrained on
Evaluated on
Books
Clothing
Hotel
Laptop
Restaurant
Books
81.7
1.7
80.3 0.6
88.2 + 1.8
73.4 1.6
73.6 + 2.9
Clothing
75.0 = 0.3
0.6
88.8 + 1.2
74.8 = 1.6
74.5 - 0.2
Hotel
72.0 1 1.1
0.9
95.8 1 0.4
66.1 = 1.8
72.6 1 1.5
Laptop
70.7 + 1.5
40.9
84.2
+ 3.3
80.1 = 0.2
73.8 1 1.5
Restaurant
78.1 + 1.4
81.3 4: 0.5
91.4 +
1.4
75.6 1.0
82.2
+ 0.3
Given this small inter-domain drift, we focused our evaluation on the Burst drift schedule, as demonstrated the most pronounced performance differences among the policies. The results, marized in Table [I] confirm that RCCDA still achieves the highest average accuracy, with a robust 1.4% improvement over the next-best performing baseline (Periodic). However, the performance gap less pronounced than the vision-based tasks, which is expected given the smaller effective drift magnitude. Notably, some baselines maintained high accuracy with near-zero effective update rate (e.g. 0,001). indicating that in this low-drift setting, the model's 's inherent generalization ability was more influential than the update strategy. This result highlights that while RCCDA robust across different modalities, the henefits of an intelligent update policy are most significant when the concept drift is severe.
Table 11: Average validation accuracy (%) of evaluated policies for the Burst drift schedule on the MEMD-ABSA dataset. The mean update rate constraint is A = 0.01. The hest-performing method is shown in bold.
Policy
Average Validation Accuracy (%)
RCCDA (Ours)
81.4 + 2.9
Uniform
79.8 + 3.5
Periodic
80.0 + 3,2
Budget-Incree Increase
79.2 + 3.6
Budget--Threshe
79.2 + 4.2
F.3 Supplementary Concept Drift Schedules
In this section, we provide supplementary results three additional drift schedules, designed to further evaluate the robustness of our proposed method, RCCDA. These schedules simulate diverse and challenging temporal distribution shifts:
Constant steady drift where the target domain changes at a constant rate. The domains cycle every set number of steps.
(vi) Decaying Spikes A series of sudden drift events, modifying the Burst schedule. After an initial delay, spikes occur increasing intervals, becoming less frequent.
(vii) Seasonal Flux A cyclical drift that simulates seasonal variations. The schedule alternates between two domains following a sinusoidal pattern with a constant period, amplitude, and offset.
Table 12 reports the average validation accuracy on the PACS and DigitsDG datasets for these new schedules. The results are consistent with main paper, demonstrating that RCCDA consistently outperforms or matches all baseline methods across these new drift configurations.
our findings for drift schedules (i)-(iv) presented in the
32

---

Table 12: Average validation aceuracy (%) of evaluated policies the PACS and DigitsDG datasets for three additional concept drift schedules. The mean update rate constraint A/A 0.1. The best-performing method for each configuration is shown in bold.
Policy
PACS
Digits-DG
Constant
Decaying Spikes
Scasonal Flux
Constant
Decaying Spikes
Scasonal Flux
RCCDA (ours)
54.1 + 1.5
59,7 + 3,3
63.4
2.8
58.1
i 944
54,5 1 15.4
63.7 1: 9.6
Uniform
$1.3
1.3
48.0 + 5.9
55.1
6.0
56.2
+ 7.2
48.0
+ 12.7
61.9 + 8.2
Periodic
48.4
3.5
51.3 1 4,4
57.9
5.8
56.8
1 82
49.6
14.0
62.0 1 8,3
Budget-Inne Increase
Budget-Increase
2.4
42.1
= 6.1
53.3
4.3
9.3
45.1
16.8
61.4 = 9,9
Budget-Threstee Threshold
3.7
44.6 10.9
58.2
14.5
52.3
49.0
14.8
61.5 + 11.5
F.4
Varying Update Rate
Dataset: PACS
Schedule: Burst
Average (96
75-
70
70
55-
55
50-
50-
Average Update Rate
Update Constraint
ROCDA
Uniform
Periodic
Budget-incraase
Budget--hreshholt
Dataset: DigitsDG Schedule: Burst
Average Accuracy (96)
75-
85
85
0.00
0.05
0.10
0.15
0.70
0.25
0.30
3.35
0.05
0.10
0.15
0.20
0.25
0.10
Average Update
Update Rate Constraint AMA
ROCDA
Unitorm
Budget-increase
Budget-Threshold
Figure 3: Average validation accuracy of policies across different update rates for the burst drift schedule, across PACS and DigitsDG datasets. The left column is the accuracy per real update rate, and the right column is the accuracy per constrained update rate X
In this section, we analyze the performance of RCCDA and baseline policies under various update rate constraints $. The results for the Burst drift schedule om the PACS and DigitsDG datasets are available in Figure 3 This study yields several key insights into the behavior of our proposed policy.
First, the results confirm that RCCDA effectively adheres to the specified resource constraints. As demonstrated by comparing the left (effective update rate) and right (constrained update rate) columns of Figure 3, the average update rate achieved by our policy closely matches the predefined constraint X- This indicates that our policy utilizes allocated budget more effectively, making update decisions that lead to better model performance.
33

---

Notably, the Budget- Thrres-l policy was able to marginally surpass RCCDA in accuracy for effective update rates between 0.10 and 0.15 on the DigitsDG dataset. However, this observation highlights a critical flaw in the baseline policy design: a lack of precise resource utilization control. To achieve the effective update rate in this narrow range, the Budget-Threshet significantly higher allowed resource constraint. Essentially, it only achieved this rate because its update rule failed to spend the allocated (higher) budget, meaning it was operating under much looser resource constraint.
policy required a
In contrast, RCCDA's performance achieved while strictly adhering the specified time-average resource constraint. As such, the baseline's 's slight advantage is result of its uncontrolled behavior; it adhered to higher constraint to achieve the same effective update rate. RCCDA's performance, while slightly lower, is a more accurate reflection of its ability to maximize accuracy under a true, enforced resource constraint.
F.5 Robustness under Uncertainty
Our framework assumes that the exact model inference loss, ft. can be accessed by our policy at any time step t. While this is a standard assumption in supervised drift adaptation literature, practical scenarios, especially ones involving small or partially labeled datasets, may only provide high-variance, noisy estimate of the true loss. To evaluate the robustness of RCCDA under such conditions, we conducted an additional experiment quantifying the impact of a noisy loss estimate on the policy's performance.
The experiment was performed on the PACS dataset using the Burst drift schedule, with all hy- perparameters and the environment identical to our main experiments (see Appendix [E]. The key modification was the introduction of synthetic noise to the loss signal accessed by the policy, Specifi- cally, at each time step t, the policy received a noisy loss estimate, festimate defined as:
pestimate = ft + (0, (ka x f1)
(34)
where the additive noise is sampled from zero-mean normal distribution N, with its standard deviation proportional the true loss f, via the uncertainty coefficient kyg We evaluated the policy's performance across several values of ku. For each degree of uncertainty, we re- re-tuned the policy's (Kp, Ka) parameters to ensure the effective update rate remained close to the target of A 0.1.
Table 13: Performance of RCCDA on the PACS dataset given (Burst schedule) under varying degrees of loss uncertainty.
Degree of Uncertainty (kw,)
Validation Accuracy (%)
Effective Update Rate
(Kp, Ka)
0.0 (certain)
79.4 = 5.2
0.093 + 0.030
(1.5, 0.3)
0.05
79.3 =
0,099 : 0.011
(0.7, 0.3)
0.1
79.2 = 6.8
0.103 + 0.009
(0.6, 0.25)
0.2
78.0 = 7.0
0.103 0.007
(0.4, 0.25)
0.3
77.4 = 8.0
0.104 + 0.004
(0.3, 0.25)
The results, available in Table 1 demonstrate that RCCDA is highly robust to noisy loss signals. While the inference performance gradually degrades as the uncertainty coefficient k. increases, the drop is modest compared to the uncertainty increase. Even with significant uncertainty (k: = 0.3), the validation accuracy decreases by only 2.0% relative drop of 2.5%) compared to the certain case. At the same time, the standard deviation of the validation accuracy increases with higher uncertainty an expected outcome given the noise introduces greater variability into the framework's behavior. Importantly, RCCDA successfully maintained an average effective update rate near the target X 0.1 across all uncertainty conditions, demonstrating its ability to effectively manage the resource budget resilient to loss signal perturbations. This analysis validates that RCCDA is a practical solution for real-world scenarios where the loss signal may be imperfect.
F.6 Replay Buffer
As established by our theoretical analysis, RCCDA designed to optimize an immediate, per-time- step bound on convergence. This design prioritizes rapid, resource-cfficient adaptation, making the
34

---

policy highly effective in environments characterized by new, rarely-recurring concept drifts. As a consequence of this focus, the policy is concentrated on adapting the model to the most recent data distributions, which can lead to the gradual forgetting of past data representations. While this objective is distinct from the explicit knowledge-retention goals of continual learning (see Section 2 it can pose practical challenge in real-world scenarios with recurring drifts, such as those with seasonal or cyclical environmental changes, making recurring drifts a key consideration.
Table 14: Performance comparison of RCCDA with and without replay buffer recurring drift schedule. Aceuracy (%) reported for each domain specific time steps.
Time Step
Domain
No Buffer (100% New)
With Buffer (80% New, 20% Old)
50
photo
30,60 + 9.58
30.60 + 9.58
art painting
21.35 1 2.31
21.35 1: 2.31
80.08 +: 11.16
80.08 11.16
sketch
26.43 +:: 3.99
26.43 3.99
150
photo
78.26 + 3.26
71.74 + 4.06
art painting
25.00 + 1.99
27.60 + 1.92
cartoon
20.31 + 3.04
63.93 + 2.71
sketch
18.49 + 0.80
16.67 + 6.63
250
photo art painting
50.78 + 2.76
57.29 + 4.90
63.28 + 3.31
57.03 + 3.62
cartoon
36.33 1 3.38
60.16 1 3.76
sketch
22.01 2.89
21.88
350
photo art painting
39.71 + 4.61
58.07 0.49
34.64 1 3.81
43.75 3.31
cartoon
70.31 + 3.68
79.82 + 5.94
sketch
30.60 + 2.89
31.90 +: 1.92
450
photo
12.76 +: 3.41
48.44 +: 7.10
art painting
15.10 + 4.47
38.93 + 6.33
28.39 + 16.44
70.18 + 6.20
sketch
58.07 2.08
57.94 +
550
photo art painting
79.43 + 0.80
79.04 + 3.10
29.43 2.71
41.41 1 4.82
cartoon
14.45 1 6.37
65.10 1 7.91
sketch
18.49 1 0.80
44.66 1 10.80
650
photo
52.08 1 2.17
$3.52 10.73
art painting
76.17 1: 10.13
58.59 1 10.46
38.28 +: 2.73
62.50 1 6.99
sketch
29.30 4.07
46.88 6.27
750
photo art painting
42.84 + 6.95
57.16 + 6.10
37.37 2.31
52.21 + 6.20
cartoon
73.83 + 5.37
75.26 + 8.60
sketch
30.86 + 6.51
47.53 + 5.12
850
photo art painting
11.72 + 3.08
55.08 + 3.08
12.24 +: 1.44
39.58 + 1.76
cartoon
15.89 11 5.09
65.62 1 4.98
sketch 64.58 7.79
62.50 12.77
950
photo
80.86 4.47
85.29 1. 6.22
art painting cartoon
36.72 1 1.94
49.35
23.57 + 4.31
61.33 + 9.13
sketch
18.62 + 0.97
57.03 + 7.94
To enhance long-term knowledge retention and adapt RCCDA for such environments, the framework can be naturally extended with replay buffer mechanism. The buffer stores small, representative subset of data from previously encountered distributions. When the policy triggers a model update, the training batch is then composed of majority of new data alongside minority of data sampled from this buffer. By systematically re-exposing the model to past concepts, this approach mitigates the effects of catastrophic forgetting and preserves knowledge across recurring domains.
35

---

To evaluate this extension, we conducted preliminary experiment using a recurring burst schedule on the PACS dataset, where the active domain was cycled every 100 time steps (starting with Cartoon, then rotating Photo, Art Painting, Cartoon, Sketch in order). We compare two configurations: baseline "No Buffer" agent, where the training set shifts entirely to the new domain at each drift event, and a "Buffer" agent. For the latter, at each drift event, the training set is recomposed such that 80% of the data is from the new target domain, while the remaining 20% forms a replay buffer. This buffer is populated by sampling uniformly from all previously-seen majority domains. Note that we utilized fixed buffer size and a simple sampling strategy, as this experiment represents a preliminary investigation. A detailed exploration of buffer management presents compelling direction for future work.
The results, presented in Table 14, show the model's 's accuracy an each domain at key time steps just before the next drift occurs. These findings reveal a clear trade-off between immediate adaptation and long-term knowledge retention. The "No Buffer" agent achieves a higher peak accuracy when a new target domain is introduced for the first time, such as for Photo at 150 (78.26% vs 71.74%) or Art Painting at 250 28% vs 57. 57.03%). as it dedicates all training resources to the new concept. In contrast, while the "Buffer" agent has lower accuracy when a new domain is introduced, it mitigates catastrophic forgetting by maintaining significantly higher accuracy on previously seen domains. For example, at t = 150, it achieves accuracy of 63.93% on the previous Cartoon domain, compared to just 20.31% achieved by the "No Buffer" agent. In addition. this retained knowledge proves advantageous when certain domains reappear, as evident with Cartoon at = 350, where the "Buffer" agent achieves 79.82% validation accuracy, significantly higher than the 70.31% achieved by the "No Buffer" agent. This highlights a fundamental trade-off between maximizing immediate performance on novel drifts and ensuring robust generalization across recurring domains, and offers an interesting direction for future research.
F.7 Statistical Significance of the Results
As mentioned in Appendix E. the main result presented in Table [ are averaged not only across multiple random seeds but also across different starting domains for each dataset. While this provides high-level summary, it can also introduce high reported standard deviation, as the policy performance over time will depend on the specific sequence of domains encountered. This variance can obscure the statistical significance of performance differences between policies.
To provide more statistically robust view, we included a deconstructed analysis of the results. T5 breaks down the performance of all policies on the PACS dataset for the Burst drift schedule, showing the average validation accuracy for each starting domain individually. The results are averaged over 10 random seeds.
Table 15: Deconstructed validation accuracy (%) on the PACS dataset for the Burst schedule, by starting domain.
Policy
Starting Domain
Photo
Cartoon
Art Painting
Sketch
RCCDA (ours)
80.6
7.4
85,8 2.1
73.8 : 5.3
71.7 1.6
Uniform
70.0 + 1.8
67.6 6.3
51.7 : 6,8
Periodic
77.1 + 4.3
70.9 + 1.7
69.7 5.5
57.4 + 1.8
Boudget-Increae
72.0 5.8
61.3
5.6
66.5 = 1.4
36.7 + 5.6
Thdget-TTreshold
79.4
6.4
79.3
2.1
67.8 3.4
43.0
5.0
This per-domain analysis provides further insight into the aggregated results. The high standard deviations reported in the main result can be attributed to high performance variations across the different source domains. The updated standard deviations are lower in most cases compared to the original result, with a few exceptions. Additionally, RCCDA again consistently outperforms all baseline policies in each source domain, with statistically significant improvement achieved for all domains except Photo. Notably. while original results suggested a potential performance overlap between RCCDA and the Tudget-Threshold policy, this breakdown reveals that RCCDA achieves a higher mean accuracy in all cases, with the improvement being statistically significant in all domains except for Photo. As such, this analysis explains the high variance in the main results and confirms the statistical significance of our method's improvements.
36

---

G Code Documentation
In this section, we provide specific details on the code and hardware used for our experiments.
Code Description We wrote our code in Python 3.12.3, and utilized the following key imported modules:
PyTorch (2.3.0)
torchvision (0. 18.0)
numpy (1.26.4)
flwr_datasets 54
PIL (from Pillow 10.3.0)
pandas (2.2.3)
matplotlib (3.9.2)
seaborn 13.1).2)
transformers (via huggingface-hute 0.26.3; for BenfFonSeqeeneeCaassiaattto and BertTok- enizer in MEMD-ABSA)
standard libraries such os, argparse, json, time, random, etc.
We implemented a custom package to streamline code operations. The package code is and includes source code for
available under Lederated_Learning Beerning_Base_Tiie our core classes and methods, such as BaseNeuralNetwork with the update steps method, DriftAgent, class DomainDrift, or the custom datahandlers and datasets. The code in concept_drift.. wwt.ppppaaaaappteeeeeeeeeaaaaaee scheduling, update policies, and the policy evaluation code. We included the source code in supplemental materials. Further details are available in our implementation.
uses these methods to implement pre-training, drift
Executing Code These instructions assume Unix-like system (Linux/macOSS); for Windows, use adjusted paths. The code can be executed through the following steps:
1. Assuming conda is installed, navigate to the directory with the code and run the commands:
cog_fl conda create -f cog fl 11m_ yml
conda activate cog cog_fl_1lm_es
2. Navigate to directory Leearing BaootistoToltttttoeeeeet and run the package installation command:
python pip install
3. To pretrain model navigate to dataset-specific directory, and run:
python3 pretrain_model saterasannnndedsssdaatttttt...
<dataset>. --options
where <dataset> is the name of the corresponding dataset (PACS, DigitsDG, OfficeHome, or MEMD-ABSA), and options specify various parameters of the pretraining, the details are available in the code.
(torchvision. d with dowmload=True); no manual download or root_dir needed.
PACS loads via fl ffl_toolkkittPAASeeeaaaaaaa
For DigitsDG, OfficeHome, and MEMD-ABSA, download the dataset from source and update root_dir in code.
4. To evaluate a model, navigate a dataset-specific directory, and run
python3 evaluate_pollcc_caattaa
-y --options
where options specify various parameters of the evaluation process, the policy evaluated, the setting used, and the drift schedule.
Uses the models from the pretraining step.
37

---

For DigitsDG, OffeceHome, and HEMD-ABB., datasets, root directory with the downloaded dataset has to be specified in order for the code work.
Our entire codebase is available at:
Hardware. For each experiment, we utilized 14 Intel Xeon Platinum 8480+ CPU cores and 1 NVIDIA H100 GPU. CPU memory was allocated proportionally the core request, with approximately GB per core, yielding about 126 GB of CPU memory per job. The H100 GPU provided 80 GB of dedicated memory. The GPU executed all seeds concurrently for given experimental configuration. The upper time limit for pretraining was set to 2 days, and for each evaluation job was set to 6 hours. In practice, the pretraining took at most hours per configuration. Notably, the evaluation execution time varied by dataset, with the DigitsDG dataset taking 5 minutes per configuration on average, PACS evaluations averaging minutes (up to 20 minutes), and OfficeHome evaluations averaging around 30 minutes per configuration.
38

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA]. .
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable "[No] ", is perfectly acceptable answer "[No] provided a proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased binary way, we acknowledge that the true amswer is often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The abstract clearly states the contributions regarding the proposed policy. These claims supported by the theoretical analysis in the experimental results in 6 as well as the proofs in Appendix and B.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
39

---

Justification: We discuss the infeasibility of implementing the exact policy, then propose using estimation techniques. We discuss different techniques in Appendix We point out that our assumption about loss availability is realistic one. Finally, in our conclusion we mention how using the estimator limits the theoretical effectiveness of the proposed policy.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: We introduce two theories and a corollary. The assumptions and theoretical
results are clearly stated in section 5 The proofs and derivations are available in the Appendix A and B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
40

---

Answer: [Yes]
Justification: We detail our experimental setup in section 6 The exact hyperparameters, models, and settings are available in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, it may be that access to the model is limited in some way (e.g.. registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The code used run the experiments is provided the supplemental material. The datasets utilized are discussed in and are widely popular and casily accessible datasets.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc)
public/guidest Saaitddss/CddeSSubsssssooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https! http:///ddeeeeeeeeeeeiiseeeeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
41

---

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Available in the supplemental material section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Figure 2
we display standard deviation around the plotted lines each of the policies. We also add standard deviation values to the results reported in Table 1.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The entire experimental setup is included in supplemental material section.
42

---

Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't Midn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The paper and code included are fully anatomized to the best of our knowledge.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: To the best our knowledge, deriving a model update policy under drift does not have direct societal impact. The policy derived is applicable general resource- constrained settings where deep learning model is deployed and experiences
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g.. disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
43

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: To the best our knowledge, derivation of an optimal retrain policy in environments under drift poses no such risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite all experiments we conduct experiments in 6 In supplemental materials we further credit the authors of modules used, authors of used models (excluding custom ones), dprovide lincense and terms of use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, ppaperswithoode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The new assets, including the written code, simulation results, and the corre- sponding documentation, are included in the supplemental material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
44

---

The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMSs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
LLMs were not used as part of the core methods in this research.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non non-standard components.
Please refer to our LLM policy http:///eerrrpppoco/oo/oaerrr/// for what should or should be described.
45


Reviews:
[{"review_id": "NchQzpRAbS", "reviewer": "Reviewer_iUQT", "summary": "This paper presents an adaptive threshold-based policy that determines the optimal time\nto update the parameters of a predictive model in the presence of concept drift, using only past loss information, and ensuring strict bounds on constraint violations. The proposed policy triggers updates based on accumulated loss and a virtual queue tracking resource expenditure.\nThe paper also presents a convergence analysis on the relationship between model accuracy and computational cost by estimating bounds on the model accuracy depending on the update policy and the concept drift.\nFinally, the paper shows empirically that the proposed solution is able to outperform common baselines for concept drift on three datasets for domain generalization.", "questions": "- Is there a reason to limit the evaluation to vision? Other modalities (e.g. audio or text) could also be evaluated.\n\n- What kind of recognition model is used? The theoretical analysis is independent of the model. However, the empirical evaluation should be tested on different models.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "\\+ The proposed algorithm has some guarantees on the quality of the solution while keeping a constrained resource consumption.\n\n\\+ The paper is well presented even though the figures could be more clear and there a some typos in the text.\n\n\\- It is not clear what is the recognition model used for the experiments. Different models with different capacity should be evaluated. Also, the proposed approach could be applied to different domains than vision. Is there a specific reason to limit experiment to visual recognition?\n\n\\- The empirical evaluation is limited to three datasets. To better verify the quality of the proposed solution, authors could consider other datasets with multiple domains such as TerraIncognita and DomainNet.", "was_revised": true, "final_justification": "The authors answered all my questions and clarified my doubts with additional experiments with new datasets and new domains. I confirm my positive evaluation of the paper.", "rebuttal": "We thank the reviewer for the constructive feedback and the opportunity to clarify our experimental design and further validate the generalizability of our approach. We address the points regarding model diversity and evaluation modalities below.\n\n## Different capacity models\nWe acknowledge the reviewer's point about the importance of evaluating our method on models with different capacities. We would like to clarify that we had incorporated this in our experiments by using **three distinct neural network architectures** across the three vision datasets, each with varying complexity:\n- For the PACS dataset, we used a custom deep convolutional neural network with residual connections to handle the significant domain shift between different image styles. The model had 11,177,223 parameters in total, all trainable. \n- For the DigitsDG dataset, we utilized a more standard, shallower convolutional neural network suitable for the simpler task of digit recognition. The model had 1,907,146 parameters in total, all trainable. \n- For the OfficeHome dataset, we adapted a pretrained ResNet-18 model to handle the higher complexity of more classes and larger images. During both pretraining and evaluation, we froze the initial layers and exclusively fine-tuned the final convolutional block (layer4) and our custom classification head. The model had 11,324,545 parameters in total, of which 8,541,761 were trainable. \n\nTesting across different model sizes and architectures allowed us to confirm that **RCCDA's performance is robust across varying model capacities**. This aligns with our theoretical analysis, which is model-agnostic as long as the loss function satisfies the stated standard assumptions.  The complete architectural specifications for each model, along with all training and evaluation hyperparameters, are provided in the \"Supplemental Material: Detailed Experimental Setup, Documentation\" for full transparency and reproducibility.\n\n## Different modalities\nOur theoretical analysis is domain-agnostic, i.e., the provided bounds are applicable to all domains. We utilized the **vision domain in our experiments as it allowed us to model the desired environment in a straightforward way** and demonstrate the efficacy of RCCDA. The domain generalization datasets readily emulate concept drift, as pictures in different domains will have different underlying distributions. Furthermore, the vision models pre-trained on only a single domain performed poorly on new domains, which demonstrated the need for model updates, and allowed us to test our proposed algorithm in settings it would likely be deployed. \n\nWhen it comes to utilizing **other modalities, the experimental setup becomes more challenging**, and simulating concept drift becomes less straightforward. To demonstrate this, we added an experiment on the MEMD-ABSA dataset [R1] - a multidomain dataset developed for textual Aspect-Based Sentiment Analysis (ABSA) task. The dataset contains reviews partitioned into five distinct domains: books, clothing, hotel, laptop, and restaurant. Similar to our main experiments, we first pre-trained a neural network - the TinyBERT [R2] model - on each domain independently. Then, each pre-trained model was deployed in an environment simulating the 'burst' drift schedule, where the domain of the incoming text data changes abruptly at specific intervals, simulating sudden shifts in the underlying distributions. However, the difference in the domains was not substantial enough to be able to simulate a meaningful concept drift well. We illustrate this in the table below, demonstrating the testing accuracy of a model pre-trained on one domain when evaluated on all other domains:\n\n|Pretrained \\Tested | Books | Clothing | Hotel | Laptop | Restaurant |\n|-|-|-|-|-|-|\n|Books|81.7% ± 1.7%|80.3% ± 0.6%|88.2% ± 1.8%|73.4% ± 1.6%|73.6% ± 2.9%|\n|Clothing|75.0% ± 0.3%|84.8% ± 0.6%|88.8% ± 1.2%|74.8% ± 1.6%|74.5% ± 0.2%|\n|Hotel|72.0% ± 1.1%|77.5% ± 0.9%|95.8% ± 0.4%|66.1% ± 1.8%|72.6% ± 1.5%|\n|Laptop|70.7% ± 1.5%|80.9% ± 1.2%|84.2% ± 3.3%|80.1% ± 0.2%|73.8% ± 1.5%|\n|Restaurant|78.1% ± 1.4%|81.3% ± 0.5%|91.4% ± 1.4%|75.6% ± 1.0%|82.2% ± 0.3%|\n\nAs demonstrated by these results, while the model performance drops when evaluated on unseen domains, the pre-trained model still remains fairly competitive achieving relatively high classification accuracy. This is in contrast to the vision-based tasks where different domains caused a substantial performance drop, as can be seen in Figure 2 in the main paper, where the validation accuracy drops from around 90% to around 35% the moment a new domain is introduced to the dataset. In other words, for certain tasks, the **drift associated with domain changes is small**. As a result, when we evaluated RCCDA and the baselines in the ABSA setting, the performance improvement was less significant compared to other datasets. The table summarizing the results for the \"burst\" drift schedule is available below, with an effective update rate $\\frac{\\bar \\lambda}{\\lambda}=0.01$.\n\n|Policy | Average Validation Accuracy (%) |\n|-|-|\n|RCCDA (Ours) | **81.4 ± 2.9** |\n|Uniform| 79.8 ± 3.5| \n|Periodic|80.0 ± 3.2|\n|Budget-Increase|79.2 ± 3.6|\n|Budget-Threshold|79.2 ± 4.2|\n\nOf note here is the fact that some of the baselines achieved the accuracy of approximately 80% with almost no updates - having effective average update rate of 0.001. This demonstrates that due to the small magnitude of the drift, the performance boost achieved by our method is not as significant as the performance gain achieved by model's generalization ability. \n\n\n## Additional dataset\nTo further validate our proposed solution, we conducted an additional experiment on the DomainNet dataset. Due to time constraints during the rebuttal period, we restricted the experiments to only 4 domains of the DomainNet dataset: real, clipart, painting, and sketch. We pre-trained a modified version of the ResNet-50 model on each of these domains, and deployed the pre-trained model in an environment experiencing concept drift simulated by the \"burst\" drift schedule, abruptly injecting new domain data at specific intervals. The effective update rate $\\frac{\\bar \\lambda}{\\lambda}$ was set to $0.1$, just as in the main paper experimental setup. After tuning the hyperparameters and evaluating the performance of the baselines, we arrived at the results summarized in the table below:\n\n|Policy | Average Validation Accuracy (%) |\n|-|-|\n|RCCDA (Ours) | **68.9 ± 2.1** |\n|Uniform| 66.6 ± 3.4| \n|Periodic|67.8 ± 4.3|\n|Budget-Increase|63.7 ± 4.0|\n|Budget-Threshold|65.6 ± 2.2|\n\nCompared to the other policies, when averaged over all seeds and domains, our policy once again achieved highest average validation accuracy, further corroborating the findings from PACS, DigitsDG, and OfficeHome from the main paper. \n\nWe sincerely thank the reviewer for their careful and thorough review of our work. We will gladly address any additional questions or concerns. \n\nReferences:\n\n[R1] Cai, Hongjie, et al. \"Memd-absa: A multi-element multi-domain dataset for aspect-based sentiment analysis.\" Language Resources and Evaluation (2025): 1-29.\n\n[R2] Jiao, Xiaoqi, et al. \"Tinybert: Distilling bert for natural language understanding.\" arXiv preprint arXiv:1909.10351 (2019)."}, {"review_id": "LQ5ArpuBOz", "reviewer": "Reviewer_bCey", "summary": "In this paper the authors introduce RCCDA, an adaptive model update policy designed to address concept drift in machine learning under strict resource constraints. The policy leverages Lyapunov analysis to minimize a drift-plus-penalty term, ensuring stability and adherence to resource constraints. The authors showed theoretical convergence bounds. They also demonstrated the policy's effectiveness through experiments on domain generalization datasets. RCCDA demonstrates superior accuracy and drift recovery compared to baseline policies, while efficiently utilizing resources.", "questions": "How does RCCDA handle scenarios where the estimation function for gradient magnitude is inaccurate or noisy?\n\nCan RCCDA be extended to handle multi-modal concept drift or non-stationary environments?\n\nWhat are the computational overheads of maintaining the virtual queue?\n\nIt would be good to Include some benchmarks against state-of-the-art adaptive policies to highlight comparative advantages.", "limitations": "The authors have explicitly discussed the limitations of their work, which is commendable. However they have not discussed any societal impact. It would have been good if they had included a discussion on broader implications (e.g., fairness in resource allocation or unintended consequences in deployment) in the paper.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper provides a robust theoretical foundation for the RCCDA policy. Experimental results seem to be reproducible, with detailed hyperparameters and settings provided. The results also seemed to be quite significant.\n\nMoreover the paper is well-structured, with clear explanations of the theoretical framework and experimental setup. Limitations are explicitly discussed, which is laudable.\n\nThe focus on concept drift in resource-constrained environments is a real world problem which needs enough attention.\n\n RCCDA introduces a Lyapunov drift-plus-penalty framework, ensuring stability and adherence to resource constraints. It dynamically adjusts update timing based on historical loss data, outperforming static or heuristic policies. These contributions seem to be unique and novel.\n\n\nThe paper is really well written. My only concern is its assumptions about loss availability which may not be feasible in real world scenarios.", "was_revised": true, "final_justification": "The authors have answered all my questions and have agreed to add the societal impact of their work to the main paper. I think this is a good paper and can be accepted.", "rebuttal": "We thank the reviewer for their thorough and positive assessment. We are grateful that they found our paper's theoretical foundation robust, the results significant, and the contributions novel. Below we address the raised concern and answer the posed questions.\n\n## Loss availability\nOur assumption is a standard one used in **supervised drift detection** methods. Establish methods like ADWIN and DDM, which operate on the error rate, also require ground truth labels in practice. We can here point to scenarios where the labeled data for loss calculation becomes available right away or with a slight delay, e.g., in tasks such as text prediction (predicted and ground truth text are both available in real time), content moderation (labels are provided through feedback), and online behavior tracking (real-time user responses).\n\nThe alternative is **unsupervised drift detection**, which monitors feature distributions and therefore does not require labels. However, these methods are computationally intensive. To see this, we have conducted a new set of experiments comparing the added computational cost of RCCDA with the cost of running (supervised) drift detection using ADWIN, as well as detecting drift using the (unsupervised) Kolmogorov–Smirnov (K-S) test, which analyzes the incoming data stream to detect drift. The experiments were conducted on the PACS dataset with a size of 1024 samples using a single Nvidia H100 GPU. We measured the wall clock-time taken by each method to finish detecting drift/making a decision after the initial model evaluation. The results, averaged over 4 starting domains and 3 seeds each are as follows:\n\n|Algorithm|Additional Decision Making Time (ms)| Relative Slowdown (vs. RCCDA) |\n|-|-|-|\n|RCCDA|0.000744± 0.000142| 1x|\n|ADWIN|1.845618 ± 0.469732|~2480|\n|KS-Test|125996.5 ± 9605.7| ~169,350,134|\n\nThe decision time of approximately two minutes for the K-S test would be prohibitive for real-time monitoring applications, making such methods impractical for the evaluated setting. Additionally, these results demonstrate that given the same assumptions, by using just the loss value for the policy decision, we can achieve significant speedup compared to detecting the drift using error rate calculation, which is the method utilized by ADWIN. We will include these results and this discussion in the final version of the paper. \n\n## Response to questions\n**How does RCCDA handle inaccurate/noisy gradient estimation**: Our policy is designed around the Lyapunov optimization framework's virtual queue, which acts as a long-term controller. This ensures that **regardless of the noise in the gradient estimation term, the long-term time-average resource constraint will be satisfied**, as established by our stability analysis in Theorem 5.3. However, noisy estimates might lead to suboptimal retraining decisions at certain time steps; depending on how noisy the estimate is, the inference performance of the agent's model might deteriorate over time. Note that in our experiments, we utilize an estimation function of $\\hat{g}(\\cdots) = K_{d} (f_t-f_{t-1})$, which is a simple and lightweight approach to estimating the gradient magnitude with which we are able to achieve robust performance across the variety of tested datasets and drift schedules.\n\n**Can RCCDA be extended to multi-modal drift**: As we utilize only loss and gradient magnitude in decision making, **the theoretical analysis is generalizable to a multi-modal setting**, supporting different modalities separately as well as potentially multi-modal tasks. Due to computational constraints, we only considered single-modal concept drift in the paper. When it comes to the evaluated environments, simulating the concept drift through domain changes gives us a convenient mechanism for evaluating non-stationary environments. This is also encapsulated in the theoretical analysis by the drift-induced loss term $\\Delta f_{\\delta}(t)$ in Theorem 5.1 and then the drift bound $\\delta(t)$ in Corollary 5.2.\n\n**Computational overhead of maintaining the virtual queue**: Maintaining the virtual queue adds **negligible overhead**. The queue itself keeps track of only a single floating point number. The update at each step involves one addition, one subtraction, and one $\\max$ operation, resulting in $\\mathcal O(1)$ time complexity in both time and memory, making the added overhead minimal from a resource perspective. \n\n**Benchmarks against SOTA adaptive policies**: In the earlier response we have included a new comparison of times required to execute operations required to make a policy decision. In our main paper, we further included comparisons with baseline policies that guarantee adherence to strict resource constraints but are not designed with a learning objective in mind. Other retraining policies do not explicitly guarantee they will provide the desired long-term time-averaged resource utilization, and often don't even consider operating under limited resource budget [10, 11, 12, 22, 27, 28]. As such, to the best of our knowledge, the evaluated baselines, as well as the added computational complexity comparison, are a reasonable set of benchmarks to compare against. Through our experimental section we demonstrate that our policy is capable of outperforming these benchmarks in terms of recovery time from drifts under a fixed resource budget. \n\n## Societal impact discussion\nThis is an excellent point. RCCDA has a variety of broader positive impacts. The reduced computational load on devices coupled with maintaining overall high performance results in reduced computational load of devices, which then reduces energy consumption and lowers operational costs for individuals or organizations running models on the edge. Furthermore, RCCDA's lightweight nature makes it feasible to run on low-cost low-power hardware, allowing advanced AI capabilities to be deployed in a wider range of settings. Furthermore, by enabling on-device models to adapt to concept drifts, RCCDA allows for better personalization over time while preserving computational resources.\n\nAt the same time, we acknowledge that there might be some negative impacts associated with RCCDA as well. For example, if the incoming data streams become biased, our policy might update the model to fit the new biased data, potentially reinforcing negative sentiments. In-line with this, we did not consider adversarial settings where some malicious entity might inject the agent's data stream with malicious data, causing our policy to decide to update the model, degrading its performance. We will add the above discussion to the final version of the paper. \n\nOnce again, we would like to thank you for your time and effort invested in reviewing our paper. If you have any further additional questions or concerns please let us know, we will gladly address them."}, {"review_id": "Vjm2vaFMvB", "reviewer": "Reviewer_3pwb", "summary": "This paper addresses the challenge of adapting machine learning models to concept drift in resource-constrained environments. The authors propose techniques that dynamically select an appropriate retraining policy.", "questions": "Why are the bounds stated for \\theta_t, D_{t+1} while the objective in equation (5) is defined for \\theta_t, D_t? \n\nWhat is the relationship between the proposed bounds and those in the single-task setting? As I understand, in the single-task case (T = 1), generalization bounds are typically on the order of 1/ \\sqrt{n}, where n is the number of samples. The connection between the proposed bounds and the number of samples per dataset is unclear and would benefit from further explanation.\n\nWhat is the relationship between the proposed bounds and existing state-of-the-art bounds?", "limitations": "The paper discuss some limitations. I also believe that the assumptions made in the paper represent a limitation of the proposed approach.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper addresses the problem of learning under concept drift.\n\nThe paper provides theoretical guarantees in settings with concept drift.\n\nWeaknesses:\n\nAre the assumptions made in the paper standard? It seems like a strong assumption to require some form of prior knowledge and access to a loss estimator.\n\nDo [10] and [26] refer to the same work?\n\nThe proposed problem is particularly interesting in scenarios where the dataset is small. If the number of samples per dataset is large, one could simply learn using only the data from that sample set, making the concept drift less critical. However, in the small-sample regime, is it still reasonable to assume access to an unbiased estimate of the loss?", "was_revised": true, "final_justification": "The authors have addressed all of my questions and comments. In addition, the paper provides theoretical guarantees in settings with concept drift, which are particularly challenging.", "rebuttal": "We thank the reviewer for their insightful feedback. We appreciate that the reviewer recognized the value of providing theoretical guarantees for learning under concept drift. Below we further address the raised concerns and questions. \n\n## Assumptions in the paper. \nWe would like to clarify a few points on \"**prior knowledge**\" with respect to our assumptions. The L-smoothness, bounded variance, and bounded loss assumptions are **standard assumptions** common in most theoretical analyses of stochastic optimization algorithms. Furthermore, the assumption about bounded KL-divergence between two time steps reflects the fact that the data distribution will not change arbitrarily quickly in practice. We note that even though we make this assumption, we only utilize it to provide convergence and time-average constraint violation bounds; RCCDA does not use any prior knowledge of concept drift throughout its operation.\n\nWhen it comes to the assumption of the estimate of the loss, our method uses the model's inference loss, calculated when ground truth labels are available. This is a **foundational assumption** for many of the most-cited methods in the concept drift literature. For instance, the widely used drift detectors we mentioned in the related works section, like ADWIN and EDDM operate by monitoring the model's error rate, which explicitly requires access to ground truth labels. While we do not explicitly try to detect or categorize the drift in our paper, assuming loss availability aligns with the standard requirements for supervised drift adaptation. Additionally, this assumption corresponds to a practical setup for many real-world systems like text or ad-click prediction, trend-tracking, or content moderation in which real-time feedback is available.\n\n## Duplicate citation ([10] and [26])\nWe are grateful to the reviewer for pointing out this issue. The mentioned references are indeed the same work, and we will correct this citation error of the final version. \n\n## Assumption on access to an unbiased estimate of the loss for small datasets\nThis assumption stems directly from being able to access the model's inference loss at any time $t$, and as mentioned beforehand, it is a **standard assumption in supervised drift adaptation literature**. Regarding its feasibility, consider that for a larger dataset, small parts of it may be labeled, resulting in a good estimate of the inference loss. If $\\mathcal D_t$ is small, the empirical loss can result in a high-variance estimate of the true loss. However, even though the variance might increase, even for small datasets, as long as it is possible to sample a batch $\\xi_t$ representative of the distribution at time $t$, the empirical loss should not include any bias for that dataset.\n\nTo address the issue with variance in the estimate of the loss, we conducted an **additional experiment evaluating how adding error to the loss estimate affects the agent model's performance**. The experiment was conducted on the PACS dataset, using the \"burst\" drift schedule. In the experiment, once the evaluation loss was calculated, a scaled value sampled from a normal distribution with mean 0, and varying degrees of variance was added to the original loss value. Mathematically, the uncertain loss is modeled as $f_t^{\\text{estimate}} = f_t + \\mathcal N(0, (k_u \\cdot f_t)^2)$, where $k_u$ is the degree of uncertainty. The results of this experiment are available in the table below:\n\n|Degree of uncertainty $k_u$ | Average Validation Accuracy (%) | Average Update Rate $\\frac{\\bar\\lambda}{\\lambda}$ | ($K_p$, $K_d$) Hyperparameters| \n|-|-|-|-|\n|0 (certain result) |79.4 ± 5.2| 0.093 ± 0.030| (1.5, 0.3) |\n|0.05 |79.3 ± 6.1|0.099 ± 0.011 | (0.7, 0.3) |\n|0.1 |79.2 ± 6.8|0.103 ± 0.009| (0.6, 0.25) |\n|0.2 |78.0 ± 7.0|0.103 ± 0.007| (0.4, 0.25) |\n|0.3 |77.4 ± 8.0| 0.104 ± 0.004| (0.3, 0.25) |\n\nNaturally, with growing estimation uncertainty, we observe some performance degradation at the same update rates. Nonetheless, overall, RCCDA demonstrates a high degree of robustness to loss estimation error. The largest performance drop in validation accuracy is only about 2.5% relative to the case when the exact value of the loss is known, achieved when the loss uncertainty becomes fairly high, with an error variance equal to 30% of the loss at any given time $t$. We also observe an increase in the standard deviation of the average validation accuracy as the uncertainty grows, reflecting higher variability in model performance due to the added estimation noise. Despite that, the new result demonstrates that the proposed algorithm performs robustly even when the loss estimate is noisy. This is particularly relevant for small-dataset regimes, where inherent sampling variance would have a similar effect on the estimate of the loss.\n\nWe will add this experiment and associated discussion to the revised manuscript.\n\n## Response to questions\n**Why are the bounds stated for $\\theta_t, D_{t+1}$ while the objective in equation (5) is defined for $\\theta_t, D_t$**: This is a direct result of the **proof technique used for online optimization**, available in appendix A. The objective is to minimize the cumulative loss over time under some resource constraint. To analyze this, we derive convergence bounds by examining the system's evolution from one discrete time step $t$ to the next. The final convergence bound arises from conducting that analysis and utilizing various techniques to provide a bound on the gradients. The Lyapunov optimization objective comes from attempting to minimize the derived convergence bound.\n\n**Single-task setting bounds relationship**: Standard generalization bounds, typically on the order of $\\mathcal O(1/\\sqrt{n})$, measure how well a model trained on $n$ samples from a static distribution will perform on new data from that **same distribution**. In contrast, our work provides a convergence bound on the expected squared magnitude of the gradient for a dynamic setting with concept drift. This bound addresses a fundamentally different challenge of characterizing the model's performance over a time horizon $T$ as the **underlying data distribution $p_t$ evolves**. Because we consider non-stationary setting, where the distribution of the data changes over time, deriving a traditional generalization bound is highly challenging, as the standard assumption of a fixed data distribution becomes inapplicable to the considered scenario.\n\n**Relationship to existing state-of-the-art bounds**: As stated in the paper, to the best of our knowledge, **no prior work provides convergence and stability guarantees for model adaptation under concept drift subject to a strict, time-averaged resource constraint**. Existing works on concept drift primarily focus on either detecting drift or adapting to it assuming unconstrained updates. The bounds provided in these works do not account for a restrictive resource budget that may prevent the model from updating at the theoretically optimal moments.\n\nThank you for your insights and effort in reviewing our paper! We will add all of these clarifications to the manuscript. If there are other questions we could answer or issues we could address, please let us know."}, {"review_id": "ffGIZqYcM8", "reviewer": "Reviewer_EHwV", "summary": "To tackle the high computational overhead inherent in existing concept drift adaptation models, the authors propose a model update policy. The approach specifically applies Lyapunov optimization to minimize the model’s time-averaged expected loss. Theoretical analysis establishes the algorithm’s asymptotic convergence and stability guarantees.", "questions": "1. My primary concern is whether the scenario setup is reasonable. Specifically, is there evidence to demonstrate that the computational cost of existing methods for detecting concept drift is unacceptable? The authors need to provide concrete evidence quantifying the computational overhead of existing approaches under specific hardware conditions, and show that this overhead is indeed unacceptable. Otherwise, the paper appears to be more of an exploration under self-defined conditions.  \n\n2. How are certain hyperparameters in the algorithm determined? For example, in the threshold-based policy, does \\(\\lambda\\) vary depending on the dataset? I would recommend the authors provide additional details on these aspects to ensure the reproducibility of the algorithm and to verify its generalization capability.  \n\n3. The experiments lack validation of the paper's core motivation. The authors should supplement the study with specific results, such as computational parameters and time consumption, to substantiate their claims.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strength\n1.Utilizing Lyapunov Optimization is interesting.\n2.The paper is well organized\n\nWeakness\n1.The application scenario require greater specificity.​", "was_revised": true, "final_justification": "Thank you for the response, i will keep my rating.", "rebuttal": "We thank the reviewer for valuable feedback and for recognizing the novelty of applying Lyapunov optimization. We believe that our clarifications and new experimental results below address each of the raised concerns.\n\n## Computational cost of drift detection\nWe agree that our paper's core motivation would be strengthened by providing quantitative evidence demonstrating that our derived policy has much lower computational overhead than the existing methods. To address this, during the rebuttal period, **we have conducted a new set of experiments comparing the added computational cost of RCCDA with established drift detection methods** - ADWIN and Kolmogorov–Smirnov (K-S) Test. To detect drift, ADWIN uses sequential data processing and error rate calculation, while K-S test analyzes the statistical properties of incoming data streams. We conducted the new experiment on the PACS dataset with a size of 1024 samples and utilized a single Nvidia H100 GPU. We measured the clock-time taken by each method to finish detecting drift/making a decision after the initial model evaluation. The loss, as well as the ground truth labels and the predicted outputs required by ADWIN, were available to each method at the start of their operation. The results, averaged over 4 domains and 3 seeds each, are as follows:\n\n|Algorithm|Additional Decision Making Time (ms)| Relative Slowdown (vs. RCCDA) |\n|-|-|-|\n|RCCDA|0.000744± 0.000142| 1x|\n|ADWIN|1.845618 ± 0.469732|~2480|\n|KS-Test|125996.5 ± 9605.7| ~169,350,134|\n\nThese results provide concrete evidence that the overhead of existing methods may be prohibitive in many practical scenarios. For distribution-based methods like the K-S test, an overhead of approximately 126 seconds on average per monitoring step would be unacceptable for any application requiring real-time monitoring. On the other hand, for the error-based methods represented by ADWIN, the ~2480 slowdown compared to our method represents an unnecessary computational burden added on top of the computation associated with model operation and retraining under strict resource constraints. If compounded over long operation time, it could lead to much faster resource depletion than RCCDA. Additionally, while on the evaluated hardware, a 1.68ms average time is not significant, our work is motivated by resource-constrained scenarios, including on-device ML (e.g. vehicle camera monitoring with limited battery, on-device text prediction, etc) where the computational increase might be exemplified by slower hardware, resulting in the computation time required by ADWIN becoming a challenge. Furthermore, in high-performance real-time scenarios, where every millisecond of compute time is critical and can have severe effects on the operation time and performance of such devices, a lightweight, near-zero overhead policy like RCCDA is a major improvement over the comparable baselines. We will include these results along with the above discussion in the final manuscript of the paper.\n\n## How are hyperparameters determined\nThe parameter $\\lambda(t)$ represents the time-varying resource cost of performing a model update. This parameter is **system-defined**, reflecting the real-world cost of an update in a given application, such as energy consumption, wall clock-time, financial cost of a GPU instance, etc. In our experiments, for evaluation simplicity, we set it to a constant value and divide it by $\\bar \\lambda$, the desired time-average cost. As a result, $\\frac{\\bar \\lambda}{\\lambda}$ effectively becomes the desired time-average update frequency of the agent's model. In the main experiment, we consider $\\frac{\\bar \\lambda}{\\lambda}= 0.1$, simulating a realistic resource-constrained setting, where the model can only be updated once every 10 evaluation steps. Furthermore, in the supplemental material section, we evaluate different desired update frequencies on a single drift schedule to formalize the effect of policy given different degrees of resource availability. Other tunable hyperparameters related to the decision process and the network updates were **determined experimentally via grid search**.\n\nAll utilized hyperparameter values are available in the \"Supplemental Material: Detailed Experimental Setup, Documentation\" section 1.3 and 1.4. These sections demonstrate that the proposed policy can be tuned to generalize across a variety of datasets and dynamic environments. Additionally, we included the code used to conduct our experiments in the supplemental materials, including the different settings testing a variety of hyperparameter values, which improves the reproducibility of the proposed algorithm and our experimental results.\n\n## Validation of paper's core motivation\nWe thank the reviewer for this important comment. The **additional experiment comparing RCCDA with the existing methods demonstrates the lightweight nature of the proposed algorithm**. On the other hand, the effectiveness of our proposed policy is supported by the results in Table 1 and Figure 2, demonstrating that compared to similar policies that have to adhere to resource constraints, our policy achieves robust performance, outperforming the existing baselines on a variety of drift schedules and across the tested datasets. Finally, Figure 2 demonstrates that the update rate of our frequency always tends to the desired set update frequency, validating Theorem's 5.3 claim, and demonstrating adherence to resource constraints. Thorough these experimental results and our theoretical analysis, **we validate the core paper's motivation** stated in our research question: \"Can we develop a lightweight, resource-aware policy for dynamic model updates that (i) effectively mitigates performance loss incurred by concept drift in real-time applications while (ii) theoretically guaranteeing adherence to strict resource constraints?\" In the final version of the manuscript, we will include the new experiment comparing the computational overhead with the established drift-detection algorithms as well.\n\nOnce again, thank you for your effort in reviewing our paper and providing important comments. If there are additional questions we could answer or issues we could answer we would be grateful for an opportunity for further discussion."}, {"review_id": "5NUvmPMI0E", "reviewer": "Reviewer_8AhD", "summary": "The work proposes a concept drift adaptation method which is temporally resource-aware. It works on the principle of deciding whether to update the model with the current batch based on the expected loss change, which in turn is determined by running inference on the current batch. The theoretical analysis derives convergence and stability bounds, showing the viability of the proposed approach.", "questions": "**Questions**\n\na. Based on Line 45, can the authors support the claim of their policy being lightweight? Is it in terms of time?\n\nb. What happens when there's no drift? Does the algorithm perform at least as well as standard gradient descent with non-adaptive or adaptive optimizer? \n\nc. Similar to the above question, according to Algorithm 1; if there are no drifts, would there be no updates happening to the model? (Referring to lines 165, 166 \"dynamically determines when to update the model based on historical performance metrics\")\n\nd. What are the practical scenarios where we can see these drifts for a single client or a user?\n\ne. How realistic is the assumption of concept drift being bounded by the KL divergence between the past and the current distributions?\n\nf. Can the authors please justify line 144: \"For example, trying to detect the concept drift requires performing operations on the dataset, which is highly resource consuming\"? And by \"resource\", do the authors mean runtime to reach convergence?\n\ng. Why doesn't the \"model related loss difference\" part of Equation 6 have the same dataset? Why do we need both past and current datasets?\n\nh. What is the storage overhead of storing both past and current data for Equation 6?\n\ni. What is the memory overhead of persisting all the past gradients according to Algorithm 1?\n___\n**Minor Comments**\n\na. Till page 4, the definition of \"resources\" was not clearly given. Maybe the authors can explain from the beginning what they mean by resources, so the readers can align their expectations properly.\n\nb. Avoid framing the research question (line 45) in can/does form, as the answer to such questions can be given in yes/no, which does not give further insights. A research question starting with what/why/how/under which conditions is generally preferred.\n\nc. Line 153 is introducing some symbols before defining them.\n\nd. Constants of Algorithm 1 \"Inputs\" should also be defined explicitly.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThe manuscript is well-written. The proposed method is novel and explained thoroughly with rationale behind each step.\n____\n\n**Weaknesses**\n\na. The motivation behind using \"time\" as the only resource is unclear. In my opinion, peak memory consumption or computation cost (per-iteration and total) are also good candidates for measuring the resource usage. Furthermore, if the authors are targeting time as their main resource, then Figure 2 needs to be in wall-clock time instead of number of steps for a fairer comparison. \n\nb. Computation and memory overheads of the proposed approach versus its baselines are not given, making it hard to gauge the feasibility and adaptability of the approach. E.g., the virtual queue is tracking all the past gradients, that must cause a linear increase in memory per step. Furthermore, it's unclear if the inference time before each training iteration is included in Figure 2. A more explicit breakdown of (a) inference time before each training iteration, and (b) training time itself would be interesting to see.\n\nc. While the related work section is well-written, line 88 stating \"these approaches demand ... continuous data monitoring and frequent model updates, making them impractical for resource-constrained settings\" seems similar to what the proposed approach is also doing. At each iteration, the historical gradients and losses (as well as current losses) are used to determine whether to update the model. Isn't that also continuous monitoring? And doesn't that add a temporal overhead?\n\nd. Since the work only focuses on optimizing for the current data distribution, when the older data distributions reemerges, the algorithm would have to do more work (for retraining) to recover the previously trained weights. It's unclear under what scenarios or use cases we would only have to care about the current distribution. In this regard, continuous learning methods seem more flexible. \n\ne. The results of Table 1 do not look statistically significant for the proposed approach, RCCDA. As an example, for PACS dataset and Burst drift case, Budget-Threshold has average accuracy of 67.4% with the deviation of +/- 14.9%, making it as high as 82.3, while RCCDA can only go as high as 77.3%. Similar observations can be made for other datasets as well.", "was_revised": true, "final_justification": "While they support the claim that RCCDA is adaptable to resource-constrained settings, I remain somewhat skeptical about the practicality of the proposed use case. Changes in lighting, seasons, or weather can often be categorized as recurring drifts, which would require RCCDA to perform additional work due to its tendency to \"forget\" past distributions. Given these considerations, I believe a borderline accept is an appropriate evaluation.", "rebuttal": "We thank the reviewer for the insightful comments and constructive feedback. We address the specific weakness and questions below and hope that our clarifications and planned revisions will resolve the concerns raised.\n\n## Time as the only resource\nOur **theoretical framework is designed to be general and is not limited to \"time\" as a resource**. In the evaluated scenario, a model update is a discrete event represented by a general time-varying function, $\\lambda(t)$. This function is an abstraction that can represent various types of resources, including peak memory consumption or computation cost. Similarly, the resource constraint $\\bar \\lambda$ is an average cost constraint over this general function $\\lambda(t)$. For our experiments, we chose $\\lambda(t)$ to be a constant-update computation cost, which makes the resource constraint equivalent to limiting the frequency of updates, effectively framing time as the only resource. We did this as it allows us to compare performance differences between policies as a result of their update timing decisions. Furthermore, such setup avoids the complexities of precisely simulating hardware-dependent metrics, which can vary significantly based on the underlying system. As such, Figure 2 uses \"Time step\" as the x-axis because it represents the discrete decision to spend resources at a given time in our problem formulation, making it a fair comparison between the evaluated policies.\n\n## Computation and memory overheads of RCCDA\nThe **main computational and memory bottlenecks in RCCDA are associated with the gradient estimation function** $\\hat g(\\cdot)$ and the history buffers $\\mathcal H$. The choice of the estimation function is left to the agent, allowing for flexibility given memory and time constraints. In our experimental section, we utilized gradient estimation function $\\hat{g}(\\cdots) = K_{d} (f_t-f_{t-1})$, as per line 315 in the manuscript. This estimation function only requires storing the previous loss value, resulting in **minimal increase in memory overhead**. All other parts of our proposed algorithm are either computed as part of the operation of the system, such as $f_t$ (independent of the decision process), or take minimal computation time. Additionally, the **virtual queue** keeps track of a single scalar value and only requires a single update at any time $t$, keeping the **added memory and computational overhead to a minimum**. Overall, compared to the introduced baselines, our algorithm  performs a few more constant-time operations on average, but these add negligible computational and memory cost. \n\nRegarding the inference time, in our framework, we model the agent as being deployed in the evaluated environment. As such, the agent runs the evaluation task at every time $t$, modeling continuous operation, and making the inference time independent of the decision process. Similarly, when it comes to the training time, that time is captured implicitly in the abstract model update cost $\\lambda(t)$.\n\n## Continual data monitoring\nThank you for pointing out this opportunity to improve the presentation. To clarify, what distinguishes RCCDA from the existing baselines is the cost associated with the monitoring operation. Traditional TDG or drift detection methods often require monitoring the data distribution properties, or sequentially evaluating all datapoints. In contrast, **RCCDA monitors only the model's inference loss**. The loss is a lightweight, scalar value that can be computed by the evaluation process of the model's primary task. Its computation is parallelizable and is optimized through a variety of techniques. Reliance on loss makes our approach lightweight, and as discussed before, results in minimal added temporal overhead.\n\nTo illustrate this point, during the rebuttal, **we conducted additional experiment that compares the operation time of RCCDA with two established drift detection methods**: ADWIN, which monitors the model error rate, and Kolmogorov–Smirnov (K-S) test, which monitors the incoming data stream. We measured the clock-time taken by each method to finish detecting drift/making a decision after the initial model evaluation. We utilized the PACS dataset and a single Nvidia H100 GPU, and averaged the results over 4 different domains and 3 different seeds.  The results are:\n\n|Algorithm|Additional Decision Making Time (ms)| Relative Slowdown (vs. RCCDA) |\n|-|-|-|\n|RCCDA|0.000744± 0.000142| 1x|\n|ADWIN|1.845618 ± 0.469732|~2480|\n|KS-Test|125996.5 ± 9605.7| ~169,350,134|\n\nClearly, **the temporal overhead added by our method is orders of magnitude smaller** compared to the established drift detection methods.\n\n## Older data distributions reemerging\nAccounting for reemergence of older data distributions is an important consideration in Continual Learning, where the goal is long-term knowledge retention. **RCCDA, by contrast, is designed for scenarios where adapting to the most recent data distribution is the primary objective**. The goal of our work is immediate resource-efficient adaptation to a drifting distribution of a single task. This is an altogether different problem formulation from continual learning, with a different set of objective, and results in natural a trade-offs between the model's in-time performance and generalizability. \n\n## Results statistical significance\nThe high standard deviations of the results stem from averaging the result across different starting domains, which have highly varying performances. To illustrate this point, we can further break down the average validation accuracy and its standard deviation for all policies across the different starting domains. The result shown below is for the burst drift schedule on the PACS dataset for the update rate of $0.1$ (and using 10 seeds instead of 3):\n\n||Photo|Cartoon|Art Painting|Sketch|\n|-|-|-|-|-|\n|RCCDA|80.6%±7.4%|85.8%±2.1%|73.8%±5.3%|71.7%±1.6%|\n|Uniform|78.0%±5.0%|70.0%±1.8%|67.6%±6.3%|51.7%±6.8%|\n|Periodic|77.1%±4.3%|70.9%±1.7%|69.7%±5.5%|57.4%±1.8%|\n|Budget-Increase|72.0%±5.8%|61.3%±5.6%|66.5%±1.4%|36.7%±5.6%|\n|Budget-Threshold|79.4%±6.4%|79.3%±2.1%|67.8%±3.4%|43.0%±5.0%|\n\nWhen deconstructed in this way, our policy consistently outperforms the baseline methods, demonstrating that the high standard deviation stems from averaging over the different starting domains. This result is also true for other drift schedules and datasets, which we will include in the final version of the manuscript.\n\n## Response to questions\na. **Lightweight claim**: As demonstrated in our earlier response, the added computation overheads of RCCDA are minimal, supporting the claim of our policy being lightweight.\n\nb & c. **Behavior with no drift**: Under no drift, the loss $f_t$ remains stable, with in the term $(f_t - \\min_i f_i)$ approaching zero. As such, the likelihood of an update becomes small, diminishing the value of $Q(t)$ over time, leading to a stop in model updates, thus conserving resources. This is a key feature of RCCDA: the algorithm improves over standard gradient descent methods by not diminishing resources on unnecessary updates in low-drift environments.\n\nd. **Practical scenarios for drift**: Drifts are common in single user scenarios like adapting to a user's changing interests in recommender systems, mobile devices adapting to a changing environments (lighting, seasons, weather, etc.), or real-time systems like ad prediction. \n\ne. **Realism of bounded KL-divergence**: This is a standard theoretical assumption in the analysis of concept drift. It formalizes the notion that the data distribution will not change arbitrarily quickly between two consecutive time steps in practice.\n\nf. **Clarification of line 144**: As seen in the new results above, drift detection operations that require data monitoring like the K-S test take significantly longer than RCCDA. On the other hand, ADWIN, by sequentially comparing model outputs with targets for drift detection, is order of magnitudes slower than the proposed RCCDA algorithm. By \"resource\" in this context we mean the number of operations required to determine whether a drift happened or to make a policy decision. \n\ng. **Different datasets in Eq 6**: Equation 6 represents the objective of minimizing the time-averaged loss increase, which is then used in Lyapunov optimization to derive our update policy. Line 631 in the manuscript clarifies that the model difference arises from unrolling the telescoped loss $\\mathbb E[f(\\theta_0, \\mathcal D_0)] - \\mathbb E[f(\\theta_T, \\mathcal D_T)]$. The result is then combined with the drift-induced loss to define the Lyapunov optimization objective. The loss terms themselves come from the derived convergence bound, and the Lyapunov optimization goal is to find an approximate solution minimizing it. \n\nh. **Storage overhead of Eq 6**: Equation 6 is part of the problem formulation but not the algorithm. The final practical implementation (Algorithm 1 block) does not require storing past and current full datasets. At any time $t$, the agent only needs to store the current dataset $\\mathcal D_t$ reflecting the current data distribution.\n\ni. **Memory overhead of storing past gradients**: As mentioned earlier, the memory overhead associated with storing the historical data of the gradients, loss values, and policy decisions depends on the gradient estimator function. In the practical algorithm implementation for our experiments, that overhead was minimal and reduced to storing just one additional scalar. The history buffers are meant to allow the agent to decide on how accurately they want to estimate the gradient magnitude versus how much computation and time they want to devote to that improving estimation accuracy.\n\nWe will address the minor comments in the final version as well.\n\nOnce again, thank you for your detailed comments! We will add all of these responses to the final version. Please let us know if there are any other questions we can answer."}]

Author rebuttal:

Meta-review: This work studies the problem of adapting model training when the data distribution is drifting over time under while constraining the number of times the model can be updated. The authors show that the proposed algorithm, RCCDA, will satisfy some notion of first-order optimality for the time-averaged gradients, that depends on the amount of concept drift, while approximately satisfying the resource constraints. The theoretical guarantees are accompanied by an empirical evaluation.

All reviewers agree that this is an important problem with real world applications, that the paper is well-written and that the theoretical findings are somewhat interesting. The main concerns of reviewers were about the practicality of the algorithm and the somewhat limited empirical evaluation. The authors address these two points by running extra experiments and a careful discussion on the computational complexity, together with a table that verifies the efficiency of the proposed algorithm.

Overall the paper seems to provide an interesting and novel approach for solving the concept drift problem that comes with theoretical guarantees and a good empirical evaluation.
Decision: Accept (poster)