Title: RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget
Venue: NeurIPS 2025
Accepted: yes
Keywords: concept drift

resource constraints

efficient updates

provable guarantees

update policy

adaptive policy
Abstract: Machine learning (ML) algorithms deployed in real-world environments are often faced with the challenge of adapting models to concept drift, where the task data distributions are shifting over time. The problem becomes even more difficult when model performance must be maintained under adherence to strict resource constraints. Existing solutions often depend on drift-detection methods that produce high computational overhead for resource-constrained environments, and fail to provide strict guarantees on resource usage or theoretical performance assurances. To address these shortcomings, we propose RCCDA: a dynamic model update policy that optimizes ML training dynamics while ensuring compliance to predefined resource constraints, utilizing only past loss information and a tunable drift threshold. In developing our policy, we analytically characterize the evolution of model loss under concept drift with arbitrary training update decisions. Integrating these results into a Lyapunov drift-plus-penalty framework produces a lightweight greedy-optimal policy that provably limits update frequency and cost. Experimental results on four domain generalization datasets demonstrate that our policy outperforms baseline methods in inference accuracy while adhering to strict resource constraints under several schedules of concept drift, making our solution uniquely suited for real-time ML deployments.

RCCDA: Adaptive Model Updates in the Presence of Concept Drift under a Constrained Resource Budget

Abstract

Machine learning (ML) algorithms deployed in real-world environments are often faced with the challenge of adapting models to concept drift, where the task data distributions are shifting over time. The problem becomes even more difficult when model performance must be maintained under adherence to strict resource constraints. Existing solutions often depend on drift-detection methods that produce high computational overhead for resource-constrained environments, and fail to provide strict guarantees on resource usage or theoretical performance assurances. To address these shortcomings, we propose RCCDA: a dynamic model update policy that optimizes ML training dynamics while ensuring compliance to predefined resource constraints, utilizing only past loss information and a tunable drift threshold. In developing our policy, we analytically characterize the evolution of model loss under concept drift with arbitrary training update decisions. Integrating these results into a Lyapunov drift-plus-penalty framework produces a lightweight greedy-optimal policy that provably limits update frequency and cost. Experimental results on four domain generalization datasets demonstrate that our policy outperforms baseline methods in inference accuracy while adhering to strict resource constraints under several schedules of concept drift, making our solution uniquely suited for real-time ML deployments.

1 Introduction

In recent years, deep learning (DL) models have become popular in resource-constrained environments [1, 2, 3, 4], e.g., on mobile devices, where computational power and memory are limited. These settings typically assume static data distributions during inference, in part due to resource limitations [5, 6, 7]. In practice, this poses a major challenge in the form of concept drift, which is a phenomenon where the underlying relationships in data shift over time, causing model performance to degrade [8, 9]. The computational demands of frequent updates that may be necessary to overcome rapid drifts present a significant challenge in resource-constrained settings. As a result, effectively managing model updates to adapt to such drift while adhering to resource constraints becomes a critical problem.

To address concept drift and mitigate model performance deterioration, researchers have proposed a variety of adaptive learning methods [10, 11, 12]. These strategies are often tailored to different operational contexts and environment characteristics [13, 14, 15, 16]. While most are heuristic, validating their efficacy empirically [17, 18], certain techniques have offered convergence guarantees to give principled insights into algorithm design, e.g., [19, 20]. Even so, these theoretical analyses

often rely on specific assumptions about the drift, or require prior knowledge about the environment that may not be readily available for decision-making.

Fundamental Challenges. The question of how to maintain strong performance in the presence of concept drift under strict resource constraints remains largely unexplored. Existing lightweight adaptation algorithms do not provably guarantee resource budgets will be met during drift adaptation. On the other hand, established convergence guarantees depend on unconstrained updates: as we will see, artificially restricting these algorithms to stay within the budget leads to suboptimal performance. Thus, the fundamental challenge lies in developing a theoretically grounded scheme that guarantees adherence to strict resource constraints while optimizing model performance - a long-standing open problem in the field. To the best of our knowledge, no prior work has proposed or systematically analyzed strategies that optimally address concept drift while prioritizing rigid resource constraints. As a result, in this paper, we are motivated to answer the following question:

How can we develop a lightweight, resource-aware policy for dynamic model updates that (i) effectively mitigates performance loss incurred by concept drift in real-time applications while (ii) theoretically guaranteeing adherence to strict resource constraints?

Contributions. Through our investigation, we develop RCCDA (Resource-Constrained Concept Drift Adaptation): a first-of-its-kind adaptive threshold-based policy that determines the optimal time to update model parameters in the presence of concept drift, using only past loss information, and ensuring strict bounds on constraint violations. Our contributions are summarized as follows:

Convergence Analysis : We provide a rigorous analysis of how concept drift impacts a deployed model's convergence over time, focusing on the evolution of loss as data distributions shift. Under mild assumptions, our bounds quantify the degradation caused by concept drift, implicitly relating model performance to the retraining policy choice. This analysis lays the groundwork for development of adaptive strategies in dynamic, resource-constrained settings.

Optimal Policy Design : We formulate and solve an optimization problem to determine the optimal times for updating a model, minimizing the loss induced by concept drift while satisfying resource constraints. Leveraging the Lyapunov drift-plus-penalty framework [21], we develop RCCDA: a threshold-based policy that triggers updates based on inference loss data and a virtual queue tracking resource expenditure. The resulting lightweight algorithm relies solely on historical loss data and a tunable drift threshold, making it efficient for real-time applications, while ensuring a greedy-optimal balance between model performance and computational overhead.

Empirical Validation : We evaluate RCCDA performance through experiments on four domain generalization datasets: PACS, DigitsDG, OfficeHome, and MEMD-ABSA, simulating different concept drift schedules. Our results demonstrate that our policy consistently outperforms four baseline policies across a variety of drift settings while adhering to resource budgets. This highlights our policy's ability to adapt efficiently to data with time-varying distributions.

2 Related Work

Concept Drift. Many of the existing works on studying concept drift have placed an emphasis on characterizing or detecting drift [22, 23, 24, 25, 26]. For instance, windowing techniques, such as the Adaptive Windowing (ADWIN) algorithm [27], dynamically adjust the size of a data window to monitor statistical changes, flagging drift when significant deviations occur. Alternatively, observing performance indicators [28, 29, 30, 31, 32] involves tracking metrics like accuracy or error rates, where a sudden decline may signal that the model no longer aligns with the current data distribution. Additionally, machine learning models can be trained to detect drift by identifying shifts in feature distributions or relationships among variables [33, 34]. Detecting drift is a crucial step in drift adaptation, as it provides actionable insights into the optimal times to update or retrain the model with fresh data [19, 35]. However, these methods do not consider operating under limited resource budget, a problem that is addressed by the policy introduced in our paper.

Domain Generalization. Domain generalization (DG) techniques address the related problem of training models to generalize well across multiple domains, aiming to improve robustness to unseen target distributions during inference [36, 37, 38, 39]. Classical strategies include data/feature augmentation [40, 41, 42] or leveraging domain-invariant features [43, 44, 45]. More recent works have investigated temporal domain generalization [46, 47, 48] to adapt to time-varying distributions,

as encountered during concept drift. In practice, however, these approaches demand significant resources, such as continuous data monitoring and frequent model updates, making them impractical for resource-constrained settings. Furthermore, these works have not explicitly considered adherence to resource constraints, which is one of the key motivations of our paper.

Continual Learning. As mitigating concept drift involves retraining the model at strategic intervals guided by a policy, the drift-adaptation problem shares similarities with continual learning (CL), where the goal is to enable models to learn from a sequence of tasks over time [49, 50]. A major hurdle in CL is the problem of catastrophic forgetting, where adapting to new tasks causes the model to lose performance on previously learned ones [51, 52]. The problem studied in this paper prioritizes a different objective than CL: optimizing model performance on the current data distribution at each time step under resource constraints, i.e., immediate adaptability versus long-term memory retention. As such, our work is applicable to settings where the concept drift does not exhibit cyclic patterns.

3 Methodology

3.1 System Model

We consider a system with an agent running an inference task using a machine learning model within an environment that evolves over discrete time t ∈ { 0 , 1 , . . . , T -1 } := T . The environment is characterized by an underlying data distribution p t that changes over time due to concept drift. Consequently, the agent's available dataset, D t ∼ p t , varies at each time step, resulting in changing conditional distributions of the data:

̸

where ( x t , y t ) ∈ D t and ( x t ′ , y t ′ ) ∈ D t ′ are feature-target pairs sampled at different times t and t ′ . We assume that the concept drift experienced by p t is bounded between any time steps t and t +1 , which is expressed by the time-varying bound δ ( t ) on natural-log-based KL-divergence between consecutive data distributions:

At any time t , the agent uses its current model, with parameters θ t ∈ R d , to perform the inference task. The inference performance is measured using a local loss function, f : R d × R |D t |×| x | → R , the definition of which remains fixed throughout the entire operation. By randomly sampling an inference batch ξ t (where | ξ t | ≤ |D t | ) from D t , the agent obtains an unbiased estimate of the loss, f ( θ t -1 , ξ t ) (s.t. E ξ t ∼D t [ f ( θ t -1 , ξ t )] = f ( θ t -1 , D t ) , and θ -1 = θ 0 ). Before the agent is deployed, the agent's model (with parameters θ 0 ) is pretrained to be within some ε 0 > 0 proximity to the initial optimal model, i.e. ∥ θ 0 -θ ∗ 0 ∥ ≤ ε 0 for some ε 0 > 0 , where θ ∗ 0 := arg min θ ∈ R d f ( θ, D 0 ) , ensuring a low initial inference loss estimate value.

Due to the concept drift experienced by the underlying distribution p t , the optimal model parameters, θ ∗ t := arg min θ f ( θ, D t ) , also change over time, resulting in the model experiencing a growing inference loss and performance degradation. This problem is illustrated in Figure 1a.

To address this issue, the agent performs actions aimed at keeping the loss increase to a minimum in the presence of concept drift. To achieve that, the agent follows a model update policy, where at any time t , the model parameters θ t can be updated with the data from the most recent dataset D t , with the number of updates constrained by the resources available. A single update is formulated as:

where η > 0 is the learning rate and ζ t ⊆ D t ∼ p t is a uniformly sampled training batch of data (independent of the ξ t inference batch). The choice of policy affects the model performance over time, as illustrated in Figure 1b.

Note that while the update in Equation (3) is formulated as a single gradient descent step, this framework can be easily extended to allow for multiple update rounds per time step. This is achieved by interpreting each gradient descent step as a sub-interval of a larger decision time frame, allowing for more granular control over the effective update frequency.

Finally, each model update is associated with a time-varying resource cost λ ( t ) > 0 . This cost is an abstract measure and can represent a variety of factors, such as computational load, energy

0optimalmodelpath

----+

Policy1updatepath

Policy2updatepath

π2

π2

Data Type

日

0π1

tE{3,4}

Data Type

ConceptDrift

t

tE{1,2}

00

=

In sports news

The federal

Researchers

0

π2

tE{4,5,6}

quarterback...

today,the

government is

considering...

share updates on

the novel ...

0l distance

from optimum

tE{1,2,3}

Tpathcloserto

02distance

--:no update,resource

--:may decide to

Ce

: update with new

optimumthanπ2

from optimum

constrained

update

distribution

consumption, wall-clock time, or monetary cost (details available in Appendix D). The cost relates the effective frequency of updates to the pre-defined time-average cost budget ¯ λ , which corresponds to the maximum average resources the agent can spend per time step over the entire horizon T .

3.2 Problem Formulation

To mitigate concept drift, the agent employs a model retraining policy. At any time t , the policy decides on an action π ( t ) ∈ { 0 , 1 } , where π ( t ) = 1 and π ( t ) = 0 correspond to a model update and no update respectively. The resulting sequence of actions over the entire horizon is denoted by the vector π = ( π (0) , . . . , π ( T -1)) . This sequence is an element of the action sequence space Π = { 0 , 1 } T . Utilizing a retraining policy results in the following evolution of the model parameters over time:

For the sake of practicality and computational efficiency of our solution, we consider policies that are causal and distribution ignorant, meaning (i) the decision maker neither evaluates nor knows the data distributions p 0 , p 1 , . . . p T -1 and (ii) the decision maker decides on the action at time t based on the history of expected losses of the past inference batches of data ξ 0 , ξ 1 , . . . , ξ t , which are available through running the inference task. This is an important distinction of our setting, as any further modifications to the action space would result in an increased task overhead. For example, trying to detect the concept drift requires performing operations on the dataset, which is highly resource consuming (see Appendix D for more details). Using the inference loss to evaluate the model performance is a common established strategy [53].

Given such a setup, the goal of the agent is to find a policy resulting in a set of actions π ∈ Π that minimizes the expected loss over the entire time horizon, subject to the average cost of model updates not exceeding ¯ λ :

Optimality. Without resource constraints, the optimal solution to (5) would be to update the model at every time t . However, finding an optimal update policy under the resource constraint presents a fundamental challenge: obtaining such a policy requires knowledge of the dataset's entire future evolution, which is not practical. To address this, we pivot from the direct time-averaged loss objective in (5) to a new objective based on the provable upper bound on the time-averaged gradient norm (formally derived in Theorem 5.1). This pivot allows us to naturally decompose the problem into a sum of per-time-step performance penalties. In the following section, we introduce the Lyapunov drift-plus-penalty framework, which provides a mechanism to greedily minimize the upper bound on these per-time-step penalties while provably upholding the long-term resource constraint.

4 Policy Design

Idea and Overview. The central challenge of this work is adapting to changing conditions while respecting resource constraints. This issue spans beyond machine learning, appearing in areas where responsiveness and resource management must align, and requires strategies that balance performance and efficiency across dynamic systems. Existing solutions often rely on computationally expensive methods or fail to provide strict guarantees on resource usage and performance. To overcome these limitations, we leverage the Lyapunov drift-plus-penalty framework [21], as it allows us to transform an intractable long-term optimization problem into a series of greedy, per-time-step decisions. The core intuition is to dynamically incorporate the long-term constraint into each per-time-step objective. This is achieved by defining a 'penalty' term, which represents our immediate objective (based on minimizing the convergence bound), and a 'drift' term, which measures how far the system deviates from its long-term resource budget. By optimizing a bound on this combined drift-plus-penalty expression at each time t , the agent is compelled to make greedy, online decisions that provably balance these two competing factors. This process creates an explicit, online trade-off between shortterm performance and long-term budget adherence, allowing us to derive a principled, lightweight policy with provable guarantees.

By tailoring the Lyapunov framework to the analyzed setting, we derive RCCDA (Algorithm 1), a novel policy that dynamically determines when to update the model based on historical performance metrics, while adhering to resource constraints. The algorithm relies on a threshold-based rule, triggering an update when a constructed expression exceeds a dynamic threshold influenced by past decisions and constraints, reflecting the performance-resource trade-off. This approach offers distinct benefits over traditional methods: it avoids complex drift detection, requires few assumptions about data behavior, is simple to implement, and provably guarantees adherence to resource constraints.

4.1 Policy Derivation

Lyapunov Analysis. To develop RCCDA, we apply the Lyapunov framework to the perstep penalty objective derived from our convergence bound (Theorem 5.1), subject to the resource constraint. The time-averaged sum of these per-step penalties defines our core objective:

Algorithm 1: RCCDA

where for every time step t , f ( θ t , ξ t ) -f ( θ t +1 , ξ t +1 ) is an inference loss term associated with the evolution of model parameters over time, which itself depends on the dataset evolution too, and ∆ f δ,ξ ( t ) := f ( θ t , ξ t +1 ) -f ( θ t , ξ t ) is the drift induced inference loss term that relates the evolution of the data stream with the incurred loss. Note that this objective is derived from the right-hand side of our convergence bound in Theorem 5.1, which will be presented in Section 5, and serves as a proxy for minimizing the bound on time-averaged gradients, 1 T ∑ T -1 t =0 ∥∇ f ( θ t , D t +1 ) ∥ 2 .

The practical objective in (6) introduces two key modifications from the theoretical bound. First, it explicitly incorporates the policy decision by scaling the drift loss with 1 -π ( t ) , isolating the drift penalty to non-update steps. Second, while the theoretical bound is defined for losses over the entire (and often inaccessible) datasets D t , our objective employs the practical losses computed on inference batches ξ t for different times t .

Virtual Queue. To enforce adherence to resource constraints over time, we introduce a virtual queue Q ( t ) , which evolves over time according to:

with Q (0) = 0 . Intuitively, the virtual queue acts as a 'debt accumulator,' which tracks the violations of the resource constraint over time. When an update occurs, π ( t ) = 1 , the queue grows in size by λ ( t ) , signaling resource expenditure; the subtraction of ¯ λ reflects the permissible usage rate. As a result, the queue evolves dynamically over time, signifying excessive updates when large, or suggesting room for more updates when close to 0 . This mechanism is central to the Lyapunov framework: by seeking to keep the queue stable (i.e., minimizing its 'drift'), the resulting optimization ensures the average resource consumption aligns with ¯ λ over time.

To formalize this, we first define the Lyapunov drift as ∆ Q ( t ) := 1 2 ( Q ( t +1) 2 -Q ( t ) 2 ) , which corresponds to change in the 'resource debt'. The core of the method is to greedily minimize a bound on the single drift-plus-penalty expression at each time t , which is a weighted sum of this drift and our performance objective:

where Ψ( t ) is the per-step penalty objective (the term inside the summation in Equation 6), and V > 0 is a tunable parameter that weights the penalty term against the Lyapunov drift.

4.2 Our Algorithm

Decision-Making. Solving the optimization problem involves greedily minimizing the bound on the drift-plus-penalty expression at each time step t . The result of the derivation, available in Appendix C, is a theoretically greedily-optimal threshold-based policy at the core of RCCDA (Algorithm 1) that dictates an update ( π ( t ) = 1 ) if and only the following condition holds:

where J π ( t ) := f ( θ t -1 -η ∇ f ( θ t -1 , ζ t ) -π ( t + 1) η ∇ f ( θ t -1 -η ∇ f ( θ t -1 , ζ t ) , ζ t +1 ) , ξ t +1 ) -f ( θ t -1 -π ( t +1) η ∇ f ( θ t -1 , ζ t +1 ) , ξ t +1 ) . The intuition behind the update rule is a direct trade-off. The left-hand side, which we denote V G ( t ) , represents the performance gain scaled by the trade-off parameter V . The G ( t ) term is a complex, non-causal expression that accounts for the loss reduction from the gradient step and the nested effects of how this action impacts the loss at time t +1 , given the optimal next action π ( t +1) . This scaled gain is compared against the right-hand side, denoted C ( t ) , which represents the effective resource cost. This cost term is fully known at time t and combines the cost associated with the current resource debt, λ ( t ) Q ( t ) , with the instantaneous cost of the update itself. The resulting rule V G ( t ) ≥ C ( t ) , dictates that the agent should update only when the scaled performance benefit is greater than the resource cost.

Practical Policy and Estimator. While the threshold in (9) provides a per-time-step optimal solution, it is intractable and non-causal, making it impossible to implement in practice. The performance gain term G ( t ) directly depends on the future data distribution D t +1 , the optimal future action π ( t +1) (which itself creates a recursive dependency), and computations of various gradient terms that cannot be performed without violating resource constraints. There are several strategies to address this intractability, such as optimizing a looser convergence bound or truncating the multistep dependencies. In this work, however, we retain the structure of the ideal policy and instead utilize a causal approximation of this rule. We define the practical policy for RCCDA by replacing the intractable gain G ( t ) with a causal estimator function, ˆ G ( H t ) , which utilizes only historical information H t = {H f 0: t , H g 0: t , H π 0: t } available up to time t . The practical decision rule thus becomes V ˆ G ( H t ) ≥ C ( t ) . While this design introduces an implicit estimation error that may affect performance, it also establishes RCCDA as a flexible framework, allowing the specific choice of the estimator ˆ G to be tailored to different problems.

RCCDA Operation. Based on this practical, estimator-based policy, the RCCDA algorithm (Algorithm 1) operates over T = { 0 , . . . , T -1 } . After initializing the virtual queue Q (0) = 0 and history variables H , the algorithm proceeds at each time step t by first computing the current inference loss f t and updating the loss history H f . It then uses its chosen estimator function ˆ G ( H t ) to calculate

the estimated scaled performance gain V ˆ G ( H t ) . This gain is compared to the known resource cost threshold C ( t ) (the RHS of (9)) to make the update decision π ( t ) . Based on this action, the model parameters θ t are either updated or kept constant, and the relevant histories are recorded. Finally, the virtual queue is updated to Q ( t +1) using the action taken, π ( t ) , and its cost λ ( t ) , carrying the resource debt forward to the next time step.

5 Theoretical Analysis

In this section we conduct a theoretical analysis of the examined setting. In the analysis, we consider a perfect estimator, i.e. ˆ G ( H t ) = G ( t ) , which achieves zero estimation error. This allows us to derive performance bounds for the ideal policy, which serve as a theoretical benchmark for any practical implementation. The analysis is structured as follows: We first introduce a set of common assumptions required for the analysis. We then derive the general convergence rate bounds and conduct and demonstrate the result of a stability analysis on the proposed update policy. The complete proofs of our theorems are available in the appendix.

5.1 Convergence Analysis

Assumption 1 (Smoothness). The agent's loss function is differentiable and L -smooth, i.e. ∀ t 0 , t 1 , t 2 it follows that ∥∇ f ( θ t 1 , D t 0 ) -∇ f ( θ t 2 , D t 0 ) ∥ ≤ L ∥ θ t 1 -θ t 2 ∥ .

Assumption 2 (Bounded Variance). For all times t , the agent's stochastic gradient ∇ f ( θ t , ζ t ) is an unbiased estimate of the true gradient, i.e. E ζ t ∼D t [ ∇ f ( θ t , ζ t )] = ∇ f ( θ t , D t ) , and the variance of the stochastic gradient is bounded as E ζ t ∼D t [ ∥∇ f ( θ t , ζ t ) -∇ f ( θ t , D t ) ∥ 2 ] ≤ σ 2 .

Assumption 3 (Bounded Concept Drift). For all times t , the magnitude of the concept drift between two data distributions p t and p t +1 is bounded, which corresponds to a bound on the KL-divergence measure on datasets at two consecutive times defined as ∀ t, ∃ δ ( t ) > 0 s.t. DKL ( D t || D t +1 ) ≤ δ ( t ) ≤ δ = sup { δ (0) , . . . , δ ( T ) } .

Assumption 4 (Bounded Loss). For all times t , the magnitude of the loss f ( θ t , D t ) satisfies 0 ≤ f ( θ t , D t ) ≤ B .

Theorem 5.1. If Assumptions 1-2 hold and the learning rate is chosen such that η < 2 P π min L , then the time-averaged gradient satisfies

where T is the total number of time steps, ∆ f δ ( t ) = f ( θ t , D t +1 ) -f ( θ t , D t ) is the time-varying drift-induced loss change, µ = ( ηP π min -Lη 2 2 ) , and P π min is the minimum probability of policy π ( t ) being equal to 1 for all times t .

Corollary 5.2. Under Assumptions 3-4, by the result in Theorem 5.1 , the time-averaged gradient satisfies

There are several key steps in our convergence proofs. By incorporating the drift-induced loss term ∆ f δ ( t ) , we are able to quantify the effect of shifting data distributions between time steps. Utilizing Pinsker's inequality allows us to bound E [∆ f δ ( t )] ≤ B √ 2 δ ( t ) , establishing an explicit connection between the drift induced loss and the magnitude of concept drift. Conditioning on the past dataset and model evolution allows us to model dependencies on past states, and decouple the policy action from the evolution of loss over time. Finally, by directly considering the action π ( t ) in the model update and introducing P π min , we are able to integrate general policy information into the final bound. A detailed proof is provided in Appendix B.

Discussion. Theorem 5.1 provides a general bound on the time-averaged squared gradient norm, quantifying the impact of concept drift and the update policy on convergence. The bound's dependence on the cumulative drift, ∑ T -1 t =0 ∆ f δ ( t ) , reveals an important insight: the time-averaged gradient only vanishes if the total drift grows sublinearly with T . Persistent, non-vanishing drift prevents the gradients from converging. The policy's role is twofold: it explicitly appears in P π min , which inversely scales both the drift-dependent term (via µ ) and the error term O ( σ 2 ) , highlighting the trade-off where more frequent updates (a higher P π min ) lead to a tighter convergence bound, at the cost of more computation. Implicitly, the policy also governs the parameter sequence ( θ t ) T t =0 , which determines the loss evolution term f ( θ 0 , D 0 ) -f ( θ T , D T ) + ∑ T -1 t =0 ∆ f δ ( t ) . A more effective policy will yield a more favorable loss evolution and thus a tighter overall convergence bound.

Corollary 5.2 demonstrates the relationship between convergence and magnitude of the drift. Without any further assumptions on the drift, the gradients are not guaranteed to converge, even when σ = 0 , as it would require an increasing learning rate, which is bounded by η < 2 P π min L . If Assumption 3 holds as t →∞ , then ∃ δ = sup { δ ( t ) | t ∈ N ∪ { 0 }} , and the time averaged gradients are bounded by O ( √ δ ) + O ( σ 2 ) , suggesting that in environments with persistent but bounded drift, the model achieves a 'stable' performance by oscillating within a region proportional to √ δ and σ 2 . Given additional assumptions about the concept drift, it is possible to demonstrate convergence. Suppose ∑ T -1 t =0 δ ( t ) ∼ O ( T 1 -α ) with α > 0 . Then, if we use η = O ( T -β ) such that 0 < β < min( α, 1) , the bound converges as 1 T ∑ T -1 t =0 E [ ∥∇ f ( θ t , D t +1 ) ∥ 2 ] ≤ O ( T β -1 ) + O ( T β -α ) + O ( T -β ) . In practical scenarios, this corresponds to a decaying drift that eventually vanishes, allowing the algorithm to successfully converge to a stationary point.

5.2 Stability Analysis

Now, we discuss how RCCDA 1 satisfies the resource constraint of the problem (5) by following the threshold structure (9). In Theorem 5.3, we provide an upper bound on the constraint violation of our algorithm. This upper bound goes to zero as T →∞ .

Theorem 5.3. If Assumptions 1-4 hold, then the policy following Algorithm 1, with zero-error estimator, satisfies

The proof of Theorem 5.3 proceeds in two key steps. First, we show that the constraint violation term (the left hand-side of (12)) is upper bounded by the queue length Q ( T ) , for which an upper bound is also provided. Second, we establish an upper bound for the drift term ∆ Q ( t ) = 1 / 2( Q ( t +1) 2 -Q ( t ) 2 ) . This bound is derived by comparing our policy with a stationary random policy that updates at each time slot with probability min ( ¯ λ/λ ( t ) , 1 ) . A detailed proof can be found in Appendix C.2.

Discussion. Theorem 5.3 guarantees that RCCDA satisfies the resource constraint in (5) as T → ∞ , provided the cumulative drift ∑ T -1 t =0 √ δ ( t ) grows sublinearly in T and the update costs λ ( t ) remain bounded, i.e., λ ( t ) ∼ O (1) . This reflects a key strength of the Lyapunov drift-plus-penalty framework [21]. For finite T , the violation bound can be tightened toward zero by tuning V optimally. Additionally, under high but bounded drift rates ( δ ( t ) ∼ O (1) ), the constraint violation decays as O (1 / √ T ) , ensuring the policy adheres to the budget over long horizons without requiring drift to vanish.

6 Experiments

Datasets. To represent concept drift, we utilize four widely used domain generalization datasets: PACS [54], DigitsDG [55], and in the appendix OfficeHome [56], and MEMD-ABSA [57]. The drift is simulated by introducing samples from different domains into the dataset. The model architectures utilized are detailed in Appendix E.2. Initially, we pretrain the models on data from a single source domain.

Table 1: Average validation accuracy of policies across concept drift schedules for PACS and DigitsDG datasets, mean update rate constraint: ¯ λ λ = 0 . 1 . We see that RCCDA consistently outperforms others, with the largest improvement for Burst and Spikes drift schedules.

Drift Schedules. After the pretraining phase, the models are deployed at an agent with some fixedsize dataset, initially comprised only of source domain data. Over time, the dataset experiences concept drift, in the form of influx of data points from different domains, replacing existing data. The incoming domains, drift rate, and time of drift are determined according to a drift schedule. We consider 4 main drift schedules:

(i) Burst - at pre-determined times, bursts with high drift rate happen, replacing most of the dataset with new domains. Otherwise, the dataset remains constant.

(ii) Step - initially the drift is 0, then at some point, new domains start incoming slowly into the dataset, until the original is introduced back.

(iii) Wave - the drift is either 0, or new domain data is introduced at a low rate into the dataset over some long interval periodically.

(iv) Spikes - At random times, new domain data is inserted for some time with a random drift rate.

At each time, the agent runs an inference task, calculating inference loss on a separate, smaller holdout dataset that evolves in the same way as the training dataset. The agent has access to a retraining policy that decides when to update the model, as well as the desired time-average-cost ¯ λ . Each model update is equivalent to taking n steps SGD steps, with the gradients computed on batches sampled form the local dataset, and as such we consider a constant update cost λ . The agent also sets the value of ¯ λ < λ . The entire framework runs for 250 time steps. The detailed experimental setup is available in Appendix E.

Policies. We consider the following update policies.

RCCDA: Our policy is provided in Algorithm 1. To implement the algorithm, we use a custom estimation function ˆ G ( H t ) = K p ( f ( θ t -1 , ξ t ) -min i ∈{ 0 ,...,t } f ( θ i -1 , ξ i )) + K d ( f ( θ t -1 , ξ t ) -f ( θ t -1 , ξ t -1 )) , where K p and K d are tunable constants.

Uniform Random: The policy updates the model at every time slot with probability ¯ λ/λ . This policy is motivated from [58].

Periodic: This deterministic policy updates the model after every time interval λ/ ¯ λ , i.e. π Periodic = 1 if t = k ⌈ λ/ ¯ λ ⌉ , k ∈ N ∪ { 0 } , and 0 otherwise. The policy is modified from [59].

Budget-Increase: The policy keeps track of an update budget available over time, then if it detects n ∈ N consecutive loss increases and if there is budget available, the model is updated. Based on modified trend-detection methods in [60].

Budget-Threshold: The policy keeps track of an update budget and a window of losses W f . Then if f t ≥ (1 + ϵ ) max f i ∈W f f i and if there is budget available, the model is updated. Based on a modified version of [10].

Given this setup, we evaluate the classification accuracy at all times on the holdout dataset, while keeping track of the number of updates to monitor resource constraint satisfaction. Further experimentation details are available in the supplemenal material section.

Results and Discussion. Table 1 presents the average model accuracy under mean update rate constraint of 0 . 1 . We observe that RCCDA consistently outperforms the baselines across all tested configurations. Furthermore, Figure 2 illustrates the dynamic behavior of accuracy and update rates, confirming that the robust performance is achieved while adhering to the long-term resource constraints.

Periodic

Uniform

Budget-lncrease

Budget-Threshold

Burst

Step

Wave

Spikes

100

Accuracy (%)

万025

h

0

0.20

0.15-

0.10

0.05

0.00

98888

Dataset Composition

00

0

50

100

150

200

2500

50

100

150

200

2500

50

100

150

200

2500

50

100

150

200

250

Time Step

Time Step

Time Step

Time Step

Photo

Sketch

ArtPainting

Cartoon

Key Observations. First, compared with the baselines, RCCDA is able to more effectively adapt to concept drift. Figure 2 demonstrates that for fast drops in accuracy, the update rate increases rapidly, leading to fast performance recovery. This quick adaptation results in performance gains over the baseline polices. Notably, the Budget-Threshold policy also demonstrates a strongly reactive capability, however it cannot adapt to low-drift settings, not updating the model when it could be beneficial. Second, the drift schedule directly affects the performance difference between the policies. For Burst, the rapid domain changes allow the model to quickly adapt to new data, resulting in high average accuracy. RCCDA is able to adapt quickly to these changes, which is why its performance gap is highest on Burst and Spikes. In contrast, the more gradual introduction of data in the Wave and Step schedules contributes to a smaller performance gap. The low accuracy is most likely a result of model capacity limitations, but the small performance difference suggest that at lower drift rates, the time of the update might not be as significant as for higher drift rates.

Resource Utilization. Finally, RCCDA demonstrates its update rate converging towards the desired constraint of 0 . 1 over time, all while maintaining robust performance. This lends further practical significance to Theorem 5.3, and also suggests that in order to maximize model performance, the policy maximally utilizes the available resources. In contrast, policies focused solely on resource constraints (Uniform, Periodic) fail to update when beneficial for performance, while the budgetbased policies are not capable of modeling the complex drift behavior, leading to significant resource under-utilization and lower average accuracy.

Additional Results and Ablation Studies. Additional experimental results, including more drift schedules, different update rates, and more datasets, are available in Appendix F.

7 Conclusion and Limitations

In this paper, we proposed RCCDA: a novel model update policy for adapting to concept drift under resource constraints. We first established convergence bounds under drift, then utilized Lyapunov analysis to arrive at a policy that minimizes the bound on drift-plus-penalty term per each time slot. We then derived stability bounds, provably demonstrating adherence to resource constraints. Finally, we confirmed the effectiveness of our policy through experiments on various domain generalization datasets, with improvements in accuracy and drift recovery time relative to baselines. A potential limitation of our work is the estimation function ˆ G introduced in the algorithm, as its associated error can affect the threshold policy performance. Future work can rigorously characterize this impact and employ it to derive tighter performance bounds.

References

Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, and Vikas Chandra. MobileLLM: Optimizing sub-billion parameter language models for ondevice use cases. In Forty-first International Conference on Machine Learning , 2024. URL https://openreview.net/forum?id=EIGbXbxcUQ .

Zhihao Shu, Xiaowei Yu, Zihao Wu, Wenqi Jia, Yinchen Shi, Miao Yin, Tianming Liu, Dajiang Zhu, and Wei Niu. Real-time core-periphery guided vit with smart data layout selection on mobile devices. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , volume 37, pages 95744-95763. Curran Associates, Inc., 2024. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/ adb77ecc8ba1c2d3135c86a46b8f2496-Paper-Conference.pdf .

Mehdi Mohammadi, Ala Al-Fuqaha, Sameh Sorour, and Mohsen Guizani. Deep learning for iot big data and streaming analytics: A survey. IEEE Communications Surveys & Tutorials , 20(4): 2923-2960, 2018. doi: 10.1109/COMST.2018.2844341.

Sachin Mehta and Mohammad Rastegari. Mobilevit: Light-weight, general-purpose, and mobile-friendly vision transformer. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id=vh-0sUt8HlG .

Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning , pages 6105-6114. PMLR, 2019.

Timothy Castiglia, Yi Zhou, Shiqiang Wang, Swanand Kadhe, Nathalie Baracaldo, and Stacy Patterson. LESS-VFL: Communication-efficient feature selection for vertical federated learning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 3757-3781. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/castiglia23a.html .

Yuyang Deng, Mohammad Mahdi Kamani, Pouria Mahdavinia, and Mehrdad Mahdavi. Distributed personalized empirical risk minimization. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 70812-70846. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ dfee09496a5a8b0b01d9d4c589758832-Paper-Conference.pdf .

Ronan Perry, Julius von K¨ ugelgen, and Bernhard Sch¨ olkopf. Causal discovery in heterogeneous environments under the sparse mechanism shift hypothesis. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 10904-10917. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ 46a126492ea6fb87410e55a58df2e189-Paper-Conference.pdf .

Huaxiu Yao, Caroline Choi, Bochuan Cao, Yoonho Lee, Pang Wei Koh, and Chelsea Finn. Wild-time: A benchmark of in-the-wild distribution shift over time. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2022. URL https://openreview.net/forum?id=F9ENmZABB0 .

Albert Bifet and Ricard Gavalda. Learning from time-changing data with adaptive windowing. In Proceedings of the 2007 SIAM international conference on data mining , pages 443-448. SIAM, 2007.

Leandro L. Minku and Xin Yao. Ddd: A new ensemble approach for dealing with concept drift. IEEE Transactions on Knowledge and Data Engineering , 24(4):619-633, 2012. doi: 10.1109/TKDE.2011.58.

Ryan Elwell and Robi Polikar. Incremental learning of concept drift in nonstationary environments. IEEE Transactions on Neural Networks , 22(10):1517-1531, 2011. doi: 10.1109/TNN.2011.2160459.

Kai Helli, David Schnurr, Noah Hollmann, Samuel M¨ uller, and Frank Hutter. Drift-resilient tabpfn: In-context learning temporal distribution shifts on tabular data. Advances in Neural Information Processing Systems , 37:98742-98781, 2024.

Junbao Chen, Jingfeng Xue, Yong Wang, Zhenyan Liu, and Lu Huang. Classifier clustering and feature alignment for federated learning under distributed concept drift. arXiv preprint arXiv:2410.18478 , 2024.

YiFan Zhang, Qingsong Wen, Xue Wang, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. In Thirty-seventh Conference on Neural Information Processing Systems , 2023. URL https://openreview.net/forum?id=Q25wMXsaeZ .

Kunjal Panchal, Sunav Choudhary, Subrata Mitra, Koyel Mukherjee, Somdeb Sarkhel, Saayan Mitra, and Hui Guan. Flash: Concept drift adaptation in federated learning. In International Conference on Machine Learning , pages 26931-26962. PMLR, 2023.

Li Yang and Abdallah Shami. A lightweight concept drift detection and adaptation framework for iot data streams. IEEE Internet of Things Magazine , 4(2):96-101, 2021. doi: 10.1109/ IOTM.0001.2100012.

Yu Sun, Ke Tang, Zexuan Zhu, and Xin Yao. Concept drift adaptation by exploiting historical knowledge. IEEE Transactions on Neural Networks and Learning Systems , 29(10):4822-4832, 2018. doi: 10.1109/TNNLS.2017.2775225.

Ashraf Tahmasbi, Ellango Jothimurugesan, Srikanta Tirthapura, and Phillip B Gibbons. Driftsurf: Stable-state / reactive-state learning under concept drift. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 10054-10064. PMLR, 18-24 Jul 2021. URL https://proceedings.mlr.press/v139/tahmasbi21a.html .

Alessio Mazzetto and Eli Upfal. An adaptive algorithm for learning with unknown distribution drift. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 10068-10087. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/ 2023/file/1fe6f635fe265292aba3987b5123ae3d-Paper-Conference.pdf .

Michael Neely. Stochastic network optimization with application to communication and queueing systems . Morgan & Claypool Publishers, 2010.

Geoffrey I Webb, Roy Hyde, Hong Cao, Hai Long Nguyen, and Francois Petitjean. Characterizing concept drift. Data Mining and Knowledge Discovery , 30(4):964-994, 2016.

Supriya Agrahari and Anil Kumar Singh. Concept drift detection in data stream mining: A literature review. Journal of King Saud University-Computer and Information Sciences , 34(10): 9523-9540, 2022.

Joao Gama, Pedro Medas, Gladys Castillo, and Pedro Rodrigues. Learning with drift detection. In Advances in Artificial Intelligence-SBIA 2004: 17th Brazilian Symposium on Artificial Intelligence, Sao Luis, Maranhao, Brazil, September 29-Ocotber 1, 2004. Proceedings 17 , pages 286-295. Springer, 2004.

Denis Moreira Dos Reis, Peter Flach, Stan Matwin, and Gustavo Batista. Fast unsupervised online drift detection using incremental kolmogorov-smirnov test. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , pages 1545-1554, 2016.

Pranoy Panda, Vineeth N. Balasubramanian, and Gaurav Sinha. Feature-interpretable real concept drift detection. In ICLR 2023 Workshop on Pitfalls of limited data and computation for Trustworthy ML , 2023. URL https://openreview.net/forum?id=pA2uDAvTpp .

Fabian Hinder, Andr´ e Artelt, and Barbara Hammer. Towards non-parametric drift detection via dynamic adapting window independence drift detection (DAWIDD). In Hal Daum´ e III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 4249-4259. PMLR, 13-18 Jul 2020. URL https://proceedings.mlr.press/v119/hinder20a.html .

Maayan Harel, Shie Mannor, Ran El-Yaniv, and Koby Crammer. Concept drift detection through resampling. In International conference on machine learning , pages 1009-1017. PMLR, 2014.

Albert Franc ¸a Josu´ a Costa, R´ egis Antnio Saraiva Albuquerque, and Eulanda Miranda dos Santos. A drift detection method based on active learning. In 2018 International Joint Conference on Neural Networks (IJCNN) , pages 1-8, 2018. doi: 10.1109/IJCNN.2018.8489364.

Manuel Baena-Garcıa, Jos´ e del Campo- ´ Avila, Raul Fidalgo, Albert Bifet, Ricard Gavalda, and Rafael Morales-Bueno. Early drift detection method. In Fourth international workshop on knowledge discovery from data streams , volume 6, pages 77-86. Citeseer, 2006.

Oliver Cobb and Arnaud Van Looveren. Context-aware drift detection. In International conference on machine learning , pages 4087-4111. PMLR, 2022.

Aodong Li, Alex Boyd, Padhraic Smyth, and Stephan Mandt. Detecting and adapting to irregular distribution shifts in bayesian online learning. Advances in neural information processing systems , 34:6816-6828, 2021.

Bruno Iran Ferreira Maciel, Silas Garrido Teixeira Carvalho Santos, and Roberto Souto Maior Barros. A lightweight concept drift detection ensemble. In 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI) , pages 1061-1068, 2015. doi: 10.1109/ ICTAI.2015.151.

Ahmad Abbasi, Abdul Rehman Javed, Chinmay Chakraborty, Jamel Nebhen, Wisha Zehra, and Zunera Jalil. Elstream: An ensemble learning approach for concept drift detection in dynamic social big data stream learning. IEEE Access , 9:66408-66419, 2021. doi: 10.1109/ACCESS. 2021.3076264.

Dariusz Brzezinski and Jerzy Stefanowski. Reacting to different types of concept drift: The accuracy updated ensemble algorithm. IEEE transactions on neural networks and learning systems , 25(1):81-94, 2013.

Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip S. Yu. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering , 35(8):8052-8072, 2023. doi: 10.1109/TKDE.2022.3178128.

Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence , 45(4):4396-4415, 2023. doi: 10.1109/TPAMI.2022.3195549.

Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. Generalizing to unseen domains: A survey on domain generalization. IEEE transactions on knowledge and data engineering , 35(8):8052-8072, 2022.

Mixue Xie, Shuang Li, Longhui Yuan, Chi Liu, and Zehui Dai. Evolving standardization for continual domain generalization over temporal drift. Advances in Neural Information Processing Systems , 36:21983-22002, 2023.

Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. In International Conference on Learning Representations .

Xiaotong Li, Yongxing Dai, Yixiao Ge, Jun Liu, Ying Shan, and LINGYU DUAN. Uncertainty modeling for out-of-distribution generalization. In International Conference on Learning Representations .

Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M Hospedales. A simple feature augmentation for domain generalization. In Proceedings of the IEEE/CVF international conference on computer vision , pages 8886-8895, 2021.

Krikamol Muandet, David Balduzzi, and Bernhard Sch¨ olkopf. Domain generalization via invariant feature representation. In International conference on machine learning , pages 10-18. PMLR, 2013.

Wang Lu, Jindong Wang, Haoliang Li, Yiqiang Chen, and Xing Xie. Domain-invariant feature exploration for domain generalization. Transactions on Machine Learning Research , 2022. ISSN 2835-8856. URL https://openreview.net/forum?id=0xENE7HiYm .

Manh-Ha Bui, Toan Tran, Anh Tran, and Dinh Phung. Exploiting domainspecific features to enhance domain generalization. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems , volume 34, pages 21189-21201. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ b0f2ad44d26e1a6f244201fe0fd864d1-Paper.pdf .

Guangji Bai, Chen Ling, and Liang Zhao. Temporal domain generalization with drift-aware dynamic neural networks. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=sWOsRj4nT1n .

Zekun Cai, Guangji Bai, Renhe Jiang, Xuan Song, and Liang Zhao. Continuous temporal domain generalization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems , 2024. URL https://openreview.net/forum?id=G24fOpC3JE .

Mixue Xie, Shuang Li, Longhui Yuan, Chi Liu, and Zehui Dai. Evolving standardization for continual domain generalization over temporal drift. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 21983-22002. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ 459a911eb49cd2e0192055ee156d04e5-Paper-Conference.pdf .

Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(8):5362-5383, 2024. doi: 10.1109/TPAMI.2024.3367329.

Ju Xu and Zhanxing Zhu. Reinforced continual learning. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/ cee631121c2ec9232f3a2f028ad5c89b-Paper.pdf .

Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences , 3(4):128-135, 1999.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017.

Xinran He, Junfeng Pan, Ou Jin, Tianbing Xu, Bo Liu, Tao Xu, Yanxin Shi, Antoine Atallah, Ralf Herbrich, Stuart Bowers, et al. Practical lessons from predicting clicks on ads at facebook. In Proceedings of the eighth international workshop on data mining for online advertising , pages 1-9, 2014.

Daniel J. Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, and Nicholas D. Lane. Flower: A friendly federated learning research framework. CoRR , abs/2007.14390, 2020. URL https://arxiv.org/abs/2007.14390 .

Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Learning to generate novel domains for domain generalization. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm, editors, Computer Vision - ECCV 2020 , pages 561-578, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58517-4.

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5018-5027, 2017.

Hongjie Cai, Nan Song, Zengzhi Wang, Qiming Xie, Qiankun Zhao, Ke Li, Siwei Wu, Shijie Liu, Heqing Ma, Jianfei Yu, et al. Memd-absa: A multi-element multi-domain dataset for aspect-based sentiment analysis. Language Resources and Evaluation , pages 1-29, 2025.

Jie Lu, Anjin Liu, Fan Dong, Feng Gu, Jo˜ ao Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering , 31(12): 2346-2363, 2019. doi: 10.1109/TKDE.2018.2876857.

Cristiano Mesquita Garcia, Ramon Abilio, Alessandro Lameiras Koerich, Alceu de Souza Britto Jr, and Jean Paul Barddal. Concept drift adaptation in text stream mining settings: a systematic review. ACM Transactions on Intelligent Systems and Technology , 16(2):1-67, 2025.

Jo˜ ao Gama, Indr˙ e ˇ Zliobait˙ e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM computing surveys (CSUR) , 46(4):1-37, 2014.

Frank J Massey Jr. The kolmogorov-smirnov test for goodness of fit. Journal of the American statistical Association , 46(253):68-78, 1951.

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier domain generalization, 2017. URL https://arxiv.org/abs/1710.03077 .

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. URL https://arxiv.org/abs/1512.03385 .

Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351 , 2019.

Mike Schuster and Kaisuke Nakajima. Japanese and korean voice search. In 2012 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages 51495152. IEEE, 2012.

Appendix

A Societal Impact

RCCDA offers significant positive societal contributions by enabling artificial intelligence that is more sustainable, adaptable, and private. By optimizing the efficiency of model updates, our policy directly lowers the energy consumption of edge devices, reducing operational costs, increasing operation times, and promoting environmental sustainability. Additionally, this resource efficiency combined with RCCDA's lightweight nature makes it possible to deploy sophisticated, adaptive models on low-cost hardware, allowing for effective and reliable access to advanced technology even in dynamic, resource-constrained environments. Furthermore, by ensuring this adaptation happens directly on-device, RCCDA provides robust performance while fostering greater data privacy and autonomy for its users.

However, the same adaptive capability that drives these benefits also introduces important considerations for responsible deployment. Since the policy adapts to incoming data, it may inadvertently reinforce harmful patterns, especially if the data stream contains biases. The reliance on incoming data also creates a vulnerability to adversarial poisoning, where a malicious actor could inject data designed to degrade model performance. Finally, continuous on-device personalization could bias the model to favor one type of data, resulting in the creation of insulating 'echo chambers,' potentially narrowing a user's exposure to different perspectives. As a result, deploying this system responsibly requires integration of safeguards that ensure safe and fair operation, such as bias detection mechanisms, data verification protocols, or regular performance audits.

B Convergence Proofs

B.1 Proof of Theorem 5.1

First, we note that

where ζ t +1 is a random batch sampled uniformly from the dataset D t +1 at time t +1 . For simplicity, in the following analyses, we use a simplified gradient notation ∇ f ( θ t , ζ t +1 ) := ∇ θ f ( θ t , ζ t +1 ) .

The loss difference over two time steps is comprised of two parts:

is bounded as:

By definition, the second term is the drift-induced loss ∆ f δ ( t ) := f ( θ t , D t +1 ) -f ( θ t , D t ) . Next, by Assumption 1 and Equation 13, the first term f ( θ t +1 , D t +1 ) -f ( θ t , D t +1 )

Define the conditional expectation E t +1 [ · ] := E π ( t +1) [ · |H θ ( t ) , H D ( t +1)] , where H θ ( t ) is a variable containing model parameters θ 0 , θ 1 , . . . , θ t across a time horizon { 0 , . . . , t } , and H D ( t ) stores the evolution of the dataset D 0 , D 1 , . . . , D t +1 across { 0 , . . . , t +1 } . As such, given the history

variables H θ ( t ) , H D ( t +1) , we obtain:

where:

(a) follows from Equation 15.

(b) is true due to the fact that ∇ f ( θ t , D t +1 ) is ( H θ ( t ) , H D ( t +1)) -measurable.

(c) follows from Equation 13.

(d) is valid because the policy's action π ( t +1) is independent of the training data batch ζ t +1 conditioned on the past history.

(e) is true because E t +1 [ π ( t +1)] = E t +1 [ π ( t +1) 2 ] since π ( t ) ∈ { 0 , 1 } .

(f) follows from Assumption 2 .

Next, define P π min such that P π min ≤ E t [ π ( t )] ∀ t ∈ N ∪{ 0 } for a given policy π , which is interpreted as the minimum probability of the policy π being equal to 1 at any time t , given the evolution of model parameters and datasets up to times t and t + 1 respectively. Additionally, it is true that E t [ π ( t )] ≤ 1 ∀ t . Combining this fact with the derived result, it follows that:

By applying the law of total expectation to Equation (16), it follows that:

where (g) follows from Assumption 2 on the bounded variance of stochastic gradients.

By adding E [ f ( θ t , D t +1 ) -f ( θ t , D t )] = E [∆ f δ ( t )] on both sides of Equation (17), the expression becomes:

Now note that all expectations are of values calculated for the entire dataset at every time, so it is true that: E [ f ( θ t +1 , D t +1 )] = f ( θ t +1 , D t +1 ) , E [ f ( θ t , D t )] = f ( θ t , D t ) , E [∆ f δ ( t )] = ∆ f δ ( t ) , and E ∥∇ f ( θ t , D t +1 ) ∥ 2 = ∥∇ f ( θ t , D t +1 ) ∥ 2 .

Utilizing this fact and rearranging the terms, the bound on the gradient becomes:

Finally, using µ := ( P π min η -Lη 2 2 ) , by telescoping over and taking a time average, the expression becomes:

proving Theorem 5.1 .

B.2 Proof of Corollary 5.2

We can relate the drift-induced loss term with the dynamic bound on the concept drift as follows:

where:

(a) follows from Assumption 2 on the unbiased estimate of the true gradient.

(b) uses the definition of expectation over a discrete probability space. The sample space { ω i } is the set of all possible batches that can be formed from the union of all datasets ⋃ T -1 t =0 D t . Since this union is finite, the set { ω i } is also finite. P D t ( ω i ) is the probability of sampling batch ω i from the dataset D t , where P D t ( ω i ) = 0 if ω i cannot be formed from D t . This probability distribution P D t varies over time due to concept drift.

(c) is true by Assumption 4 on the boundedness of the loss, and the fact that ∀ x ∈ R , x ≤ | x | .

(d) follows from the definition of the L1 loss, where P D t is the probability distribution vector, containing probabilities of sampling batches ω i ∀ i from the dataset D t .

(e) is true by the Pinsker's Inequality.

(f) follows from Assumption 3 on the boundedness of the concept drift at different times t .

Now, by Assumption 4 on the boundedness of the loss:

By applying the results of Equations 21 and 22 to Theorem 5.1 and utilizing the definition of µ , the bound becomes:

This completes the proof of Corollary 5.2 .

C Lyapunov Analysis

C.1 Lyapunov Drift-Plus-Penalty Framework

The convergence result depends on two key components bounding the loss gradients over time: the telescoped loss f ( θ 0 , D 0 ) -f ( θ T , D T ) and the drift-induced loss ∆ f δ ( t ) . By undoing the telescoping step, and utilizing the inference losses (which are an unbiased estimate of the true loses that use the entire dataset D t for computation), we can define the effective policy, as a policy that solves the following problem:

Note that the sum utilized in this optimization problem differs from the one in Theorem 5.1, as we use the inference losses, and we introduced a (1 -π ( t )) multiplier of ∆ f δ,ξ ( t ) . The inference losses are used as that's what the agent can access at different time steps t . The (1 -π ( t )) was introduced for two reasons:

It emphasizes the penalty associated with no update under strong drift, and rewards the updates better.

If the model is updated, ∆ f δ ( t ) effectively corresponds to a theoretical loss increase that would have happened had the model parameters not been updated. By introducing (1 -π ( t )) , we exclude this theoretical increase from optimization, placing emphasis only on observed loss values.

While this modification results in a different policy compared to one derived from the unmodified penalty, 1 T ∑ T -1 t =0 f ( θ t , D t ) -f ( θ t +1 , D t +1 ) + ∆ f δ ( t ) , it is important to recognize that applying the drift-plus-penalty framework to either problem formulation would not solve the long-term minimization directly. The Lyapunov method, by design, converts a long-term stochastic optimization problem into a series of deterministic, per-time-slot minimizations. This greedy, 'one-shot' decision is inherently an approximation of the true optimal policy that would minimize the sum over the entire horizon T .

Given that the per-slot minimization is already a well-motivated heuristic, we are justified in designing the per-slot penalty function to be a more effective proxy for our control goals. Our modification, which explicitly isolates the drift-loss penalty to the π ( t ) = 0 ('no-update') action, simply provides a more targeted and intuitive objective for the greedy controller to optimize at each step.

With the modified penalty, the optimization problem we aim to solve is:

Finding the optimal solution that minimizes entire sum is hard due to the evolving nature of the environment. A truly optimal policy would require knowledge of the future, as given full information about the evolution of the dataset over the entire time horizon, the optimal points to retrain will depend on all t . This requirement makes the problem intractable to solve directly. As such, we utilize the Lyapunov analysis framework to derive an online policy that does not require the knowledge of the entire environment evolution. We begin by defining the virtual queue as:

from which it follows that the Lyapunov drift is:

The drift-plus-penalty term is bounded as:

where Ψ( t ) is the penalty term at any time t , which corresponds to the inference loss increase due to the evolution of the environment f ( θ t , ξ t ) -f ( θ t -π ( t +1) η ∇ f ( θ t , ζ t +1 ) , ξ t +1 ) combined with the drift-induced inference loss increase when not updating the model (1 -π ( t ))∆ f δ,ξ ( t ) .

Next, as is standard in Lyapunov analysis, we solve the problem on a per-time-slot basis. The problem to solve at each time t becomes (note that the expectation is dropped, because for a given value of π ( t ) , all values in it are deterministic):

Then, if π ( t ) = 0 , we have θ t = θ t -1 and the immediate bound becomes:

Otherwise, if π ( t ) = 1 , we have θ t = θ t -1 -η ∇ θ f ( θ t -1 , ζ t ) and the immediate bound becomes:

By comparing the two resulting costs, the policy should update if the cost associated with not updating is greater than or equal to the cost of updating:

By rearranging, we get

For the simplicity of notations, we define

Further, by rearranging, we arrive at the update rule proposed in (9):

This completes the analysis.

Note that J π ( t ) corresponds to the future impact on the loss of the current decision, π ( t ) . The choice of π ( t ) directly affects the virtual queue length Q ( t +1) , altering the state observed by the policy at time t + 1 and influencing the next decision π ( t + 1) . This then changes the expected value of the future model, E [ θ t +1 ] . As a result, J π ( t ) couples the immediate, greedy decision at time t with its expected consequence on the loss at time t +1 . It is intractable to compute in a practical online system, as it depends on future values of environmental variables. The same is true for the f ( θ t -1 , ξ t +1 ) -f ( θ t -1 -η ∇ f ( θ t -1 , ζ t ) , ξ t ) term (where while computing the gradient at every time step t is feasible, it would violate the resource constraint). As a result, while both terms are crucial for the theoretical derivation, in our practical implementation of this policy we use an approximation for

C.2 Stability Analysis: Proof of Theorem 5.3

Let π ∗ denote our proposed policy. At each time slot t , the policy was chosen to minimize

Next, consider a policy π Uniform such that π Uniform ( t ) = 1 with probability min ( ¯ λ λ ( t ) , 1 )

Then, for all t , the following holds

Next note that:

Next, we have

and assuming Q (0) = 0 , we arrive at an intermediate bound on the expected queue length at time T :

Then, λ ( t ) π ( t ) -¯ λ ≤ Q ( t +1) -Q ( t ) implies

So, the final inequality becomes:

This completes the proof Theorem 5.3.

Then, using the same principle as above, as well as the fact that E [ π ( t )] = E [ π ( t ) 2 ] , it follows that:

Using the above, Assumptions 3-4, and introducing E π ∗ [ · ] and E π Uniform [ · ] as the expectation over the drift-plus-penalty given Q ( t ) for policies π ∗ and π Uniform, respectively, the inequality becomes:

By taking the total expectation, and by definition of the Lyapunov drift:

D Computational Resource Evaluation

In this section, we provide a detailed evaluation of the computational resources associated with the operation of RCCDA. We first discuss the abstract nature of 'resources' assumed in our framework to demonstrate its generalization. We then conduct an overhead analysis to quantify the additional resource consumption of RCCDA itself compared to the evaluated baseline policies and discuss its efficiency relative to traditional drift detection methods.

D.1 Resource Abstraction and Generalizability

Throughout this paper, the concept a 'resource' is treated abstractly. This is a deliberate design choice to ensure that our framework is broadly applicable to a wide range of real-world, resourceconstrained scenarios. A resource can represent any quantifiable commodity that is consumed during a model update and is subject to a limited budget. This could be computational cycles, FLOPs, power consumption, data usage, or wall-clock time. At any time step t , the agent can perform a model update, which incurs a general, time-varying cost, λ ( t ) , associated with the considered resource type. The agent's operational constraint is a total resource budget over the entire time horizon, which is enforced via the time-average resource constraint, ¯ λ , used to bound resource usage in Equation (5). This is equivalent to ensuring the total amount of consumed resources, ∑ T t =0 λ ( t ) π ( t ) , does not exceed the total available resource budget, T ¯ λ .

To demonstrate that this approach is generalizable across different resource types, we conducted a new empirical analysis to quantify the average resource usage per update for several distinct metrics. In our setup, we utilized the same experimental settings as in our main experiments on the PACS dataset, details available in Appendix E, averaging the results over 250 updates. We measured the resource usage for a single model update across the following metrics: computation (GFLOPs), data processed (GB), memory-time usage (GB-s), energy consumption (Joules), and wall-clock time (s).

Table 2 reports the empirically measured average cost per update λ ( t ) , and the corresponding total available resource budget T ¯ λ for various metrics.

Note that RCCDA performs identically for all the resource metrics in Table 2, as the experiments were configured to use the same ratio ¯ λ λ ( t ) (with λ ( t ) set to a constant value λ ), effectively defining a target update frequency constraint. Since the decision logic in RCCDA is driven by this ratio and the observed model loss, its behavior remains consistent regardless of how the 'resource' is defined. This result therefore confirms that our framework is generalizable to different resource metrics, demonstrating that RCCDA is a versatile solution for resource-constrained adaptation.

D.2 Overhead Analysis of the RCCDA Policy

Acritical aspect of designing a model update policy for resource-constrained environments is ensuring the decision-making process itself does not introduce significant computational overhead. Many existing approaches to mitigating concept drift rely on explicit drift detection mechanisms, which makes them computationally intensive and ill-suited for real-time decision making.

To demonstrate the necessity for a lightweight policy that does not rely on explicit drift detection, we conducted an empirical analysis comparing RCCDA against two established drift detection methods: the Adaptive Windowing (ADWIN) [27] algorithm and the Kolmogorov-Smirnov (KS) [61] test. Traditional drift detection methods like these typically require constant monitoring of

the statistical properties of the entire incoming data stream (KS-Test) or sequentially processing error rates (ADWIN). In contrast, RCCDA relies solely on the model's inference loss, which is a lightweight scalar value, often computed or estimated as part of the model's primary task evaluation. The reliance on a pre-existing simple signal makes our approach inherently efficient.

To quantify this efficiency, we measured the decision-making wall-clock time taken by RCCDA, and compared it to a wall-clock time taken by ADWIN and KS-Test to finish performing drift detection operations, which is a required step if using these algorithms for decision making. We assume all necessary inputs (such as the inference loss for RCCDA, data windows for the others) are already available through running the initial model evaluation. The experiment was conducted using the PACS dataset with the setup detailed in Appendix E, averaging results over 4 domains and 3 different seeds.

The results in Table 3 demonstrate the computational efficiency of our approach. RCCDA's decision time is negligible, measuring in the sub-microsecond range. In contrast, ADWIN is over three orders of magnitude slower. While small in isolation, especially compared to the model inference time, this latency would accumulate over a long operational horizon, increasing both response delay and cumulative resource consumption. The KS-Test proves computationally prohibitive, exceeding RCCDA's decision time by over eight orders of magnitude due to the requirement of performing statistical tests on the entire data window. These vast time disparities underscore a conclusion that policies reliant on explicit drift detection introduce a temporal overhead that is unsustainable in many resource-constrained scenarios, validating our lightweight, loss-driven approach.

While this comparison demonstrates that the traditional drift detection methods add a significant computational overhead from monitoring data streams or performance metrics, ADWIN and the KS-Test are not policy algorithms by design, and therefore are not designed to operate under a strict resource budget. As such, a more direct and informative benchmark for policy-specific overhead comes from comparing RCCDA against the baseline methods from the main experiments, which are explicitly designed for operation under resource constraints. These policies, such as Uniform or Periodic updates, are inherently lightweight and serve as an excellent benchmark for verifying that RCCDA does not introduce a prohibitive computational and resource cost relative to simple heuristics. We therefore evaluated the total memory footprint, decision time, and theoretical complexity for each policy to provide a comprehensive overhead analysis. The experiment was conducted on the PACS dataset, with a window size of w = 40 used by Budget-Increase and Budget-Threshold policies. The results are summarized in Table 4. As shown, RCCDA's resource consumption is comparable to the baseline policies on all evaluated metrics.

This two-fold resource analysis demonstrates that RCCDA is both vastly superior to traditional drift detectors and is competitively efficient against the lightweight baselines, confirming that our policy successfully integrates an intelligent, adaptive decision-making mechanism without incurring a prohibitive operational cost.

E Detailed Experimental Setup

In this section, we describe the experimental setup in detail. Our main experiment consists of two major phases. In the pretraining phase (described in E.3), we train the models on selected source domains from the datasets until convergence. In the subsequent evaluation phase (described in E.4), the agent deploys the pretrained model in an environment that simulates concept drift by introducing new domains, and uses a retraining policy to mitigate that drift.

E.1 Datasets

To simulate concept drift in our experiments, we used 4 common domain generalization (DG) datasets:

PACS [62]: PACS consists of images from 4 domains: Photo, Art Painting, Cartoon, and Sketch, each with the 7 common categories: dog, elephant, giraffe, guitar, horse, house, and person. The default size of the images is 224 × 224 . We transform all images into 128 × 128 .

DigitsDG [55]: DigitsDG encompasses images of handwritten digits 0 to 9, sourced from 4 different domains: SVHN, SYN, MNIST, and MNIST-M. The image sizes are 32 × 32 .

OfficeHome [56]: OfficeHome consists of images from four domains: Art, Clipart, Product, and Real-World, each featuring 65 object categories commonly encountered in office and home environments, including chairs, desks, and household items. The image sizes are 224 × 224 .

MEMD-ABSA [57]: MEMD-ABSA is a multi-domain textual dataset for Aspect-Based Sentiment Analysis (ABSA). It consists of approximately 20,000 sentences from 5 distinct domains: Books, Clothing, Hotel, Laptop and Restaurant. Each sentence annotated with one or more sentiment quadruples: aspect, category, opinion, and sentiment. The sentiment is then classified as positive, negative, or neutral.

E.2 Models

Below, we specify the models employed for each dataset:

Each residual block has the following structure:

OfficeHome . For the OfficeHome dataset, we use OfficeHomeNet, a model based on a pretrained ResNet-18 architecture [63]. To adapt the model for the dataset's complexity, we replace the original fully connected layer with a custom classification head. We fine-tune the model by training only the final convolutional block (layer4) and our custom head, keeping all earlier layers frozen. This results in a model with 8,541,761 trainable parameters out of 11,324,545 total. The custom head consists of:

MEMD-ABSA . For the MEMD-ABSA dataset, we fine-tune a model based on the TinyBert architecture[64], called TinyBertForSentiment. The model comprises 4,386,307 parameters, and has the following structure:

Each transformer encoder block is comprised of a 2-head self-attention mechanism with a hidden size of 128 and GELU activation functions. We tokenize input text using a pretrained BERT tokenizer that employs the WordPiece tokenization[65]. All sequences are subsequently padded or truncated to a fixed length of 128 tokens.

E.3 Pretraining Phase

The pretraining phase begins by partitioning the entire dataset into training and holdout sets. We utilize the 80:20 split and uniform random sampling for the image datasets, and adopt the predefined training and validation (holdout) splits for the text dataset. We then filter both the training and holdout sets to contain only samples from a single designated source domain. The resulting single-domain training set is denoted by D .

A new agent with a custom model is then instantiated and the model is trained on D . Upon initialization of the loss criterion and optimizer, the training loop begins. Within each iteration, the agent employs the update steps method to refine the model parameters. Unlike traditional epoch-based training, this method samples batches, each of size | ξ | , from the training set for n steps steps. For each batch, it computes the gradients and updates the model with the specified optimizer and learning rate η . This process is repeated for a total of T updates.

After each training iteration, the agent evaluates the revised model on the single-domain holdout set. For the image datasets, to prevent overfitting, we employ an early stopping mechanism that terminates the training process if the holdout set validation accuracy exceeds a predefined threshold accthresh. For the text dataset, we train for the full T iterations. Upon completion, we save the parameters of the final model for the subsequent evaluation phase, which corresponds to the last updated model for the image datasets and the best model for the text dataset.

All hyperparameters are detailed in Table 5. Their values were determined through a combination of grid search and random search.

E.4 Evaluation Phase

Dataset and General Setup . The initial data partitioning and single-domain filtering for the evaluation phase are identical to the pretraining phase. From these domain-specific splits, we construct a training set, with a fixed size |D set | , and a corresponding holdout set of size 0 . 25 ×|D set | .

The size |D set | is chosen to ensure that any single domain contains sufficient samples to fully populate these sets after the initial partitioning.

We then initialize the agent and run a concept drift simulation for T = 250 time steps. The drift is simulated by modeling a dynamic environment in which the data distributions change over time. At each time step t , the agent's model performance (e.g., loss and accuracy) is measured exclusively on the holdout set. This performance information guides the agent's update policy that determines whether to retrain the model. If an update is triggered, the agent retrains its model on the training set using the update steps method with n steps steps, a batch size of | ξ | , and a learning rate η . The policy operates under a budget constraint ¯ λ , which, given a constant update cost λ , results in an upper bound on the effective update rate equal to ¯ λ λ . All evaluation phase hyperparameters are summarized in Table 6.

Drift Schedules. To simulate concept drift, we employ a drift scheduler . The scheduler alters the domain composition of the training and holdout sets by replacing existing samples with data from new target domains. For any given time t , this process is governed by a drift schedule , which defines the specific pattern, timing, and rate of this change, including periods of no drift where the rate is zero.

We implement a total of seven schedules: four are presented in the main paper, with three additional schedules introduced in Appendix F.3. All schedules use a replace strategy, where a fraction of the existing data is substituted with samples from the target domains. This ensures the dataset size remains constant throughout the simulation.

The specific configurations are detailed below using the PACS dataset as an example. The setups for DigitsDG, OfficeHome, and MEMD-ABSA are analogous, using their respective domains.

Burst : Introduces new domains in periodic deterministic bursts. After an initial delay of 45 time steps, the first burst starts at t = 45 (introducing Photo domain data) and the second starts at t = 165 (introducing Cartoon). Each burst lasts for 3 time steps with a drift rate of 0.4, effectively replacing the entire dataset.

Step : Introduces domains in discrete steps with increasing drift rates. After an initial period of no drift, the drift rate is set to 0 . 004 at t = 60 (introducing Sketch), changing to 0 . 006 at t = 120 (introducing Cartoon), and finally changing to 0 . 008 at t = 180 (introducing Art Painting).

Wave : Consists of repeating cycles. After an initial delay of 50 time steps, a wave begins where the domains are introduced at a rate of 0 . 032 for 30 time steps. This is followed by a 70 -time-step period of no drift. This 100 -step cycle ( 30 drift and 70 no drift) then repeats. Domains cycle in the order: Photo, Cartoon, Sketch.

Spikes : Extends the Burst schedule with randomized characteristics. The spike interval is uniformly random between 90 and 130 time steps. Similarly, the initial delay is between 30 and 60 , the duration is between 3 and 6 time steps, and the rate is between 0 . 3 and 0 . 6 . Domains cycle in the order: Photo, Cartoon, Sketch.

Constant : The drift occurs at a constant rate of 0 . 016 . The target domain cycles every 50 time steps in the order: Sketch, Photo, Cartoon, Art Painting.

Decaying Spikes : Modifies the Burst schedule with increasing intervals between spikes. After an initial delay of 20 time steps, the first spike occurs. The interval to the next spike starts at 30 and

increases by 10 after each subsequent spike. Each spike lasts for 3 time steps and has a drift rate of 0 . 35 . Domains cycle in the order: Sketch, Photo, Cartoon.

Seasonal Flux : Simulates cyclic drift. After an initial delay of 10 time steps, the schedule cycles between the Photo and Sketch domains. A full cycle (e.g., from Photo to Sketch and back) takes 150 time steps. The drift rate varies sinusoidally between 0 . 001 and 0 . 016 .

Update Policies We implement the proposed RCCDA policy and four baseline strategies, detailed below.

Uniform (Policy 1): Retrains randomly with a fixed probability ¯ λ λ = ¯ π .

Periodic (Policy 2): Retrains at fixed intervals λ ¯ λ = 1 ¯ π

Budget-Increase (Policy 3): Retrains when the loss increases for three consecutive time steps at the end of a sliding window of size w = 40 , provided a sufficient budget is available. The budget accumulates at a rate of ¯ λ λ = ¯ π per time step.

Budget-Threshold (Policy 4): Retrains when the current loss exceeds the maximum loss over a window of size w = 40 by a threshold ϵ = 0 . 1 , provided a sufficient budget is available. The budget accumulates at a rate of ¯ λ λ = ¯ π per time step.

RCCDA (Policy 5): The proposed policy with estimation defined as ˆ G ( H t ) = K p ( f ( θ t -1 , ξ t ) -min i ∈{ 0 ,...,t } f ( θ i , ξ i ))+ K d ( f ( θ i , ξ i ) -f ( θ t -1 , ξ t -1 )) . The update occurs if: V K p ( f ( θ t -1 , ξ t ) -min i ∈{ 0 ,...,t } f ( θ i , ξ i )) + V K d ( f ( θ i , ξ i ) -f ( θ t -1 , ξ t -1 )) ≥ Q ( t ) + 0 . 5 -¯ λ λ . We set V = 10 constant across all experiments, and tune the K p , K d values for different settings and resource constraints. The virtual queue is updated as Q ( t + 1) = max { 0 , Q ( t ) + π ( t ) -¯ λ λ } , where π ( t ) ∈ { 0 , 1 } is the update decision at time t .

In Tables 7 and 8 we summarize the values of K p , K d used by RCCDA across the conducted experiments.

Evaluation Loop With this setup, we execute the evaluation loop for T = 250 . At each time step t , the loop executes the following operations:

Apply Drift : The drift scheduler determines the current drift rate and target domains. Drift is then applied to the agent's training and holdout datasets.

Evaluate Performance : The agent's model performance (current loss and accuracy) is computed on the holdout set.

Policy Decision : The agent's policy determines whether to retrain based on current and historical performance metrics.

Retrain if Decided : If retraining is triggered, the agent updates its model parameters for n steps steps using the training set.

Results All results are averaged over multiple random seeds (10 for experiments in the main paper and 3 for those in the Appendix). In Figure 2, we consider a single starting domain for each of the drift schedules for the PACS dataset. For the numerical results in Table 1, we further average the seed-averaged results across different starting domains. Specifically, we average the results for PACS over starting domains: Photo, Art Painting, Cartoon, Sketch; for DigitsDG over starting domains: MNIST, MNIST-M, and SVHN; for OfficeHome over starting domains: RealWorld, Product, Art; and for MEMD-ABSA over starting domains: Books, Clothing, Hotel, Laptop and Restaurant. In all cases, the reported accuracy is the average performance on the holdout set over the entire operational period T .

F Additional Experimental Results

F.1 OfficeHome Dataset

In this section, we provide additional results on the OfficeHome [56] dataset, a domain generalization benchmark consisting of four domains: Art, Clipart, Product, and Real-World.

The results are reported in Table 9. They are consistent with our findings on the PACS and DigitsDG datasets (Sec. 6), with RCCDA achieving significant performance improvements for the Burst schedule, moderate improvements for the Wave and Spikes schedules. On the Step schedule, it achieves the highest accuracy, performing competitively with the other baselines. This further demonstrates the effectiveness of our policy under resource constraints.

Table 9: Average validation accuracy (%) of policies across concept drift schedules for the OfficeHome dataset. The mean update rate constraint is ¯ λ λ = 0 . 1 . The best-performing method for each configuration is shown in bold .

F.2 MEMD-ABSA Dataset - Textual Modality

In this section, we provide additional results on the MEMD-ABSA dataset [57], a multi-domain benchmark for textual Aspect-Based Sentiment Analysis (ABSA) with reviews from five domains: Books, Clothing, Hotel, Laptop and Restaurant. This experiment evaluates our method on a new, textual, modality, and explores its performance in a setting with less severe concept drift.

Our theoretical analysis is general and not limited to a specific data modality. Consequently, our policy is provably modality-invariant, with only its algorithmic constants being task-dependent. Operationally, however, the efficacy of RCCDA strongly relies on the magnitude of performance degradation detected during a distributional shift. On the MEMD-ABSA dataset, this degradation is modest when transferring between domains, especially compared to the vision tasks, as we demonstrate in Table 10. While the top-performing model for any given domain is the one trained specifically on it, the performance drop from using models trained on other domains is not significant,

with those models still achieving competitive classification accuracy. This strong generalization suggests that, for the architecture used, the textual domains are not sufficiently distinct to generate a concept drift as severe as that observed in our vision-based experiments.

Given this small inter-domain drift, we focused our evaluation on the Burst drift schedule, as it demonstrated the most pronounced performance differences among the policies. The results, summarized in Table 11, confirm that RCCDA still achieves the highest average accuracy, with a robust 1 . 4% improvement over the next-best performing baseline (Periodic). However, the performance gap is less pronounced than in the vision-based tasks, which is expected given the smaller effective drift magnitude. Notably, some baselines maintained high accuracy with a near-zero effective update rate (e.g. 0.001), indicating that in this low-drift setting, the model's inherent generalization ability was more influential than the update strategy. This result highlights that while RCCDA is robust across different modalities, the benefits of an intelligent update policy are most significant when the concept drift is severe.

F.3 Supplementary Concept Drift Schedules

In this section, we provide supplementary results on three additional drift schedules, designed to further evaluate the robustness of our proposed method, RCCDA. These schedules simulate diverse and challenging temporal distribution shifts:

(v) Constant - A steady drift where the target domain changes at a constant rate. The domains cycle every set number of steps.

(vi) Decaying Spikes - A series of sudden drift events, modifying the Burst schedule. After an initial delay, spikes occur at increasing intervals, becoming less frequent.

(vii) Seasonal Flux - A cyclical drift that simulates seasonal variations. The schedule alternates between two domains following a sinusoidal pattern with a constant period, amplitude, and offset.

Table 12 reports the average validation accuracy on the PACS and DigitsDG datasets for these new schedules. The results are consistent with our findings for drift schedules (i)-(iv) presented in the main paper, demonstrating that RCCDA consistently outperforms or matches all baseline methods across these new drift configurations.

F.4 Varying Update Rate

Dataset: PACS

Schedule: Burst

85

85

Average Accuracy (%)

80

80-

75

75 -

10

70

65

65

60

55

55 -

50

50

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.05

0.10

0.15

0.20

0.25

0.30

AverageUpdateRate

Update Rate Constraint /A

Uniform

Periodic

Budget-Increase

Budget-Threshold

Dataset:DigitsDG

Schedule: Burst

(%)

75

75

Average Accuracy

70

70

65

65

60

09

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.05

0.10

0.15

0.20

0.25

0.30

Average Update Rate

Update Rate Constraint A/A

Uniform

Periodic

Budget-lncrease

Budget-Threshold

Dataset: PACS

In this section, we analyze the performance of RCCDA and baseline policies under various update rate constraints ¯ λ λ . The results for the Burst drift schedule om the PACS and DigitsDG datasets are available in Figure 3. This study yields several key insights into the behavior of our proposed policy.

First, the results confirm that RCCDA effectively adheres to the specified resource constraints. As demonstrated by comparing the left (effective update rate) and right (constrained update rate) columns of Figure 3, the average update rate achieved by our policy closely matches the predefined constraint ¯ λ λ . This indicates that our policy utilizes its allocated budget more effectively, making update decisions that lead to better model performance.

Notably, the Budget-Threshold policy was able to marginally surpass RCCDA in accuracy for effective update rates between 0.10 and 0.15 on the DigitsDG dataset. However, this observation highlights a critical flaw in the baseline policy design: a lack of precise resource utilization control. To achieve the effective update rate in this narrow range, the Budget-Threshold policy required a significantly higher allowed resource constraint. Essentially, it only achieved this rate because its update rule failed to spend the allocated (higher) budget, meaning it was operating under a much looser resource constraint.

In contrast, RCCDA's performance is achieved while strictly adhering to the specified time-average resource constraint. As such, the baseline's slight advantage is a result of its uncontrolled behavior; it adhered to a higher ¯ λ λ constraint to achieve the same effective update rate. RCCDA's performance, while slightly lower, is a more accurate reflection of its ability to maximize accuracy under a true, enforced resource constraint.

F.5 Robustness under Uncertainty

Our framework assumes that the exact model inference loss, f t , can be accessed by our policy at any time step t . While this is a standard assumption in supervised drift adaptation literature, practical scenarios, especially ones involving small or partially labeled datasets, may only provide a high-variance, noisy estimate of the true loss. To evaluate the robustness of RCCDA under such conditions, we conducted an additional experiment quantifying the impact of a noisy loss estimate on the policy's performance.

The experiment was performed on the PACS dataset using the Burst drift schedule, with all hyperparameters and the environment identical to our main experiments (see Appendix E). The key modification was the introduction of synthetic noise to the loss signal accessed by the policy. Specifically, at each time step t , the policy received a noisy loss estimate, f estimate t , defined as:

where the additive noise is sampled from a zero-mean normal distribution N , with its standard deviation proportional to the true loss f t via the uncertainty coefficient k u . We evaluated the policy's performance across several values of k u . For each degree of uncertainty, we re-tuned the policy's ( K p , K d ) parameters to ensure the effective update rate remained close to the target of ¯ λ λ = 0 . 1 .

The results, available in Table 13, demonstrate that RCCDA is highly robust to noisy loss signals. While the inference performance gradually degrades as the uncertainty coefficient k u increases, the drop is modest compared to the uncertainty increase. Even with significant uncertainty ( k u = 0 . 3 ), the validation accuracy decreases by only 2 . 0% (a relative drop of ∼ 2 . 5% ) compared to the certain case. At the same time, the standard deviation of the validation accuracy increases with higher uncertainty - an expected outcome given the noise introduces greater variability into the framework's behavior. Importantly, RCCDA successfully maintained an average effective update rate near the target ¯ λ λ = 0 . 1 across all uncertainty conditions, demonstrating its ability to effectively manage the resource budget is resilient to loss signal perturbations. This analysis validates that RCCDA is a practical solution for real-world scenarios where the loss signal may be imperfect.

F.6 Replay Buffer

As established by our theoretical analysis, RCCDA is designed to optimize an immediate, per-timestep bound on convergence. This design prioritizes rapid, resource-efficient adaptation, making the

policy highly effective in environments characterized by new, rarely-recurring concept drifts. As a consequence of this focus, the policy is concentrated on adapting the model to the most recent data distributions, which can lead to the gradual forgetting of past data representations. While this objective is distinct from the explicit knowledge-retention goals of continual learning (see Section 2), it can pose a practical challenge in real-world scenarios with recurring drifts, such as those with seasonal or cyclical environmental changes, making recurring drifts a key consideration.

To enhance long-term knowledge retention and adapt RCCDA for such environments, the framework can be naturally extended with a replay buffer mechanism. The buffer stores a small, representative subset of data from previously encountered distributions. When the policy triggers a model update, the training batch is then composed of a majority of new data alongside a minority of data sampled from this buffer. By systematically re-exposing the model to past concepts, this approach mitigates the effects of catastrophic forgetting and preserves knowledge across recurring domains.

To evaluate this extension, we conducted a preliminary experiment using a recurring burst schedule on the PACS dataset, where the active domain was cycled every 100 time steps (starting with Cartoon, then rotating Photo, Art Painting, Cartoon, Sketch in order). We compare two configurations: a baseline 'No Buffer' agent, where the training set shifts entirely to the new domain at each drift event, and a 'Buffer' agent. For the latter, at each drift event, the training set is recomposed such that 80% of the data is from the new target domain, while the remaining 20% forms a replay buffer. This buffer is populated by sampling uniformly from all previously-seen majority domains. Note that we utilized a fixed buffer size and a simple sampling strategy, as this experiment represents a preliminary investigation. A detailed exploration of buffer management presents a compelling direction for future work.

The results, presented in Table 14, show the model's accuracy an each domain at key time steps just before the next drift occurs. These findings reveal a clear trade-off between immediate adaptation and long-term knowledge retention. The 'No Buffer' agent achieves a higher peak accuracy when a new target domain is introduced for the first time, such as for Photo at t = 150 ( 78 . 26% vs 71 . 74% ) or Art Painting at t = 250 ( 63 . 28% vs 57 . 03% ), as it dedicates all training resources to the new concept. In contrast, while the 'Buffer' agent has a lower accuracy when a new domain is introduced, it mitigates catastrophic forgetting by maintaining a significantly higher accuracy on previously seen domains. For example, at t = 150 , it achieves accuracy of 63 . 93% on the previous Cartoon domain, compared to just 20 . 31% achieved by the 'No Buffer' agent. In addition, this retained knowledge proves advantageous when certain domains reappear, as evident with Cartoon at t = 350 , where the 'Buffer' agent achieves 79 . 82% validation accuracy, significantly higher than the 70 . 31% achieved by the 'No Buffer' agent. This highlights a fundamental trade-off between maximizing immediate performance on novel drifts and ensuring robust generalization across recurring domains, and offers an interesting direction for future research.

F.7 Statistical Significance of the Results

As mentioned in Appendix E, the main result presented in Table 1 are averaged not only across multiple random seeds but also across different starting domains for each dataset. While this provides a high-level summary, it can also introduce high reported standard deviation, as the policy performance over time will depend on the specific sequence of domains encountered. This variance can obscure the statistical significance of performance differences between policies.

To provide a more statistically robust view, we included a deconstructed analysis of the results. Table 15 breaks down the performance of all policies on the PACS dataset for the Burst drift schedule, showing the average validation accuracy for each starting domain individually. The results are averaged over 10 random seeds.

This per-domain analysis provides further insight into the aggregated results. The high standard deviations reported in the main result can be attributed to high performance variations across the different source domains. The updated standard deviations are lower in most cases compared to the original result, with a few exceptions. Additionally, RCCDA again consistently outperforms all baseline policies in each source domain, with statistically significant improvement achieved for all domains except Photo. Notably, while original results suggested a potential performance overlap between RCCDA and the Budget-Threshold policy, this breakdown reveals that RCCDA achieves a higher mean accuracy in all cases, with the improvement being statistically significant in all domains except for Photo. As such, this analysis explains the high variance in the main results and confirms the statistical significance of our method's improvements.

G Code Documentation

In this section, we provide specific details on the code and hardware used for our experiments.

Code Description We wrote our code in Python 3.12.3, and utilized the following key imported modules:

PyTorch (2.3.0)

torchvision (0.18.0)

numpy (1.26.4)

flwr_datasets [54]

PIL (from Pillow 10.3.0)

pandas (2.2.3)

matplotlib (3.9.2)

seaborn (0.13.2)

transformers (via huggingface-hub 0.26.3; for BertForSequenceClassification and BertTokenizer in MEMD-ABSA)

standard libraries such as: os, argparse, json, time, random, etc.

We implemented a custom package to streamline code operations. The package code is available under Federated_Learning_Base_Toolkit_torch , and includes source code for our core classes and methods, such as BaseNeuralNetwork with the update steps method, DriftAgent , class DomainDrift , or the custom datahandlers and datasets. The code in concept_drift_optimal_adaptation uses these methods to implement pre-training, drift scheduling, update policies, and the policy evaluation code. We included the source code in the supplemental materials. Further details are available in our implementation.

Executing Code These instructions assume a Unix-like system (Linux/macOS); for Windows, use adjusted paths. The code can be executed through the following steps:

Assuming conda is installed, navigate to the directory with the code and run the commands:

Navigate to directory Federated_Learning_Base_Toolkit_torch , and run the package installation command:

To pretrain a model navigate to a dataset-specific directory, and run:

where <dataset> is the name of the corresponding dataset (PACS, DigitsDG, OfficeHome, or MEMD-ABSA), and options specify various parameters of the pretraining, the details are available in the code.

PACS loads via fl_toolkit/PACSDataHandler ( torchvision.datasets.PACS with download=True ); no manual download or root_dir needed.

For DigitsDG, OfficeHome, and MEMD-ABSA, download the dataset from source and update root_dir in code.

To evaluate a model, navigate to a dataset-specific directory, and run

where options specify various parameters of the evaluation process, the policy evaluated, the setting used, and the drift schedule.

Uses the models from the pretraining step.

For DigitsDG, OfficeHome, and MEMD-ABSA datasets, a root directory with the downloaded dataset has to be specified in order for the code to work.

Our entire codebase is available at:

https://github.com/Adampi210/RCCDA resource constrained concept drift adaptation code .

Hardware . For each experiment, we utilized 14 Intel Xeon Platinum 8480+ CPU cores and 1 NVIDIA H100 GPU. CPU memory was allocated proportionally to the core request, with approximately 9 GB per core, yielding about 126 GB of CPU memory per job. The H100 GPU provided 80 GB of dedicated memory. The GPU executed all seeds concurrently for a given experimental configuration. The upper time limit for pretraining was set to 2 days, and for each evaluation job was set to 6 hours. In practice, the pretraining took at most 2 hours per configuration. Notably, the evaluation execution time varied by dataset, with the DigitsDG dataset taking 5 minutes per configuration on average, PACS evaluations averaging 15 minutes (up to 20 minutes), and OfficeHome evaluations averaging around 30 minutes per configuration.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While '[Yes] ' is generally preferable to '[No] ', it is perfectly acceptable to answer '[No] ' provided a proper justification is given (e.g., 'error bars are not reported because it would be too computationally expensive' or 'we were unable to find the license for the dataset we used'). In general, answering '[No] ' or '[NA] ' is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS Paper Checklist' ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The abstract clearly states the contributions regarding the proposed policy. These claims are supported by the theoretical analysis in 5, the experimental results in 6, as well as the proofs in Appendix A and B.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss the infeasibility of implementing the exact policy, then propose using estimation techniques. We discuss different techniques in Appendix C. We point out that our assumption about loss availability is a realistic one. Finally, in our conclusion we mention how using the estimator limits the theoretical effectiveness of the proposed policy.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate 'Limitations' section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We introduce two theories and a corollary. The assumptions and theoretical results are clearly stated in section 5. The proofs and derivations are available in the Appendix A and B.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We detail our experimental setup in section 6. The exact hyperparameters, models, and settings are available in the supplemental material.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: The code used to run the experiments is provided in the supplemental material. The datasets utilized are discussed in 6, and are widely popular and easily accessible datasets.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Available in the supplemental material section.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: In Figure 2, we display standard deviation around the plotted lines for each of the policies. We also add standard deviation values to the results reported in Table 1.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer 'Yes' if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The entire experimental setup is included in supplemental material section.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The paper and code included are fully anatomized to the best of our knowledge.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: To the best of our knowledge, deriving a model update policy under drift does not have a direct societal impact. The policy derived is applicable to general resourceconstrained settings where a deep learning model is deployed and experiences

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: To the best of our knowledge, derivation of an optimal retrain policy in environments under drift poses no such risk.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We cite all experiments we conduct our experiments on in 6. In supplemental materials we further credit the authors of modules used, authors of used models (excluding custom ones), an dprovide lincense and terms of use.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: The new assets, including the written code, simulation results, and the corresponding documentation, are included in the supplemental material.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: LLMs were not used as part of the core methods in this research.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work proposes a concept drift adaptation method which is temporally resource-aware. It works on the principle of deciding whether to update the model with the current batch based on the expected loss change, which in turn is determined by running inference on the current batch. The theoretical analysis derives convergence and stability bounds, showing the viability of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nThe manuscript is well-written. The proposed method is novel and explained thoroughly with rationale behind each step.\n____\n\n**Weaknesses**\n\na. The motivation behind using \"time\" as the only resource is unclear. In my opinion, peak memory consumption or computation cost (per-iteration and total) are also good candidates for measuring the resource usage. Furthermore, if the authors are targeting time as their main resource, then Figure 2 needs to be in wall-clock time instead of number of steps for a fairer comparison. \n\nb. Computation and memory overheads of the proposed approach versus its baselines are not given, making it hard to gauge the feasibility and adaptability of the approach. E.g., the virtual queue is tracking all the past gradients, that must cause a linear increase in memory per step. Furthermore, it's unclear if the inference time before each training iteration is included in Figure 2. A more explicit breakdown of (a) inference time before each training iteration, and (b) training time itself would be interesting to see.\n\nc. While the related work section is well-written, line 88 stating \"these approaches demand ... continuous data monitoring and frequent model updates, making them impractical for resource-constrained settings\" seems similar to what the proposed approach is also doing. At each iteration, the historical gradients and losses (as well as current losses) are used to determine whether to update the model. Isn't that also continuous monitoring? And doesn't that add a temporal overhead?\n\nd. Since the work only focuses on optimizing for the current data distribution, when the older data distributions reemerges, the algorithm would have to do more work (for retraining) to recover the previously trained weights. It's unclear under what scenarios or use cases we would only have to care about the current distribution. In this regard, continuous learning methods seem more flexible. \n\ne. The results of Table 1 do not look statistically significant for the proposed approach, RCCDA. As an example, for PACS dataset and Burst drift case, Budget-Threshold has average accuracy of 67.4% with the deviation of +/- 14.9%, making it as high as 82.3, while RCCDA can only go as high as 77.3%. Similar observations can be made for other datasets as well.\n\nQUESTIONS:\n**Questions**\n\na. Based on Line 45, can the authors support the claim of their policy being lightweight? Is it in terms of time?\n\nb. What happens when there's no drift? Does the algorithm perform at least as well as standard gradient descent with non-adaptive or adaptive optimizer? \n\nc. Similar to the above question, according to Algorithm 1; if there are no drifts, would there be no updates happening to the model? (Referring to lines 165, 166 \"dynamically determines when to update the model based on historical performance metrics\")\n\nd. What are the practical scenarios where we can see these drifts for a single client or a user?\n\ne. How realistic is the assumption of concept drift being bounded by the KL divergence between the past and the current distributions?\n\nf. Can the authors please justify line 144: \"For example, trying to detect the concept drift requires performing operations on the dataset, which is highly resource consuming\"? And by \"resource\", do the authors mean runtime to reach convergence?\n\ng. Why doesn't the \"model related loss difference\" part of Equation 6 have the same dataset? Why do we need both past and current datasets?\n\nh. What is the storage overhead of storing both past and current data for Equation 6?\n\ni. What is the memory overhead of persisting all the past gradients according to Algorithm 1?\n___\n**Minor Comments**\n\na. Till page 4, the definition of \"resources\" was not clearly given. Maybe the authors can explain from the beginning what they mean by resources, so the readers can align their expectations properly.\n\nb. Avoid framing the research question (line 45) in can/does form, as the answer to such questions can be given in yes/no, which does not give further insights. A research question starting with what/why/how/under which conditions is generally preferred.\n\nc. Line 153 is introducing some symbols before defining them.\n\nd. Constants of Algorithm 1 \"Inputs\" should also be defined explicitly.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-07"}

comments: SUMMARY:
To tackle the high computational overhead inherent in existing concept drift adaptation models, the authors propose a model update policy. The approach specifically applies Lyapunov optimization to minimize the model’s time-averaged expected loss. Theoretical analysis establishes the algorithm’s asymptotic convergence and stability guarantees.

STRENGTHS AND WEAKNESSES:
Strength
1.Utilizing Lyapunov Optimization is interesting.
2.The paper is well organized

Weakness
1.The application scenario require greater specificity.​

QUESTIONS:
1. My primary concern is whether the scenario setup is reasonable. Specifically, is there evidence to demonstrate that the computational cost of existing methods for detecting concept drift is unacceptable? The authors need to provide concrete evidence quantifying the computational overhead of existing approaches under specific hardware conditions, and show that this overhead is indeed unacceptable. Otherwise, the paper appears to be more of an exploration under self-defined conditions.  

2. How are certain hyperparameters in the algorithm determined? For example, in the threshold-based policy, does \(\lambda\) vary depending on the dataset? I would recommend the authors provide additional details on these aspects to ensure the reproducibility of the algorithm and to verify its generalization capability.  

3. The experiments lack validation of the paper's core motivation. The authors should supplement the study with specific results, such as computational parameters and time consumption, to substantiate their claims.

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-01

comments: SUMMARY:
This paper addresses the challenge of adapting machine learning models to concept drift in resource-constrained environments. The authors propose techniques that dynamically select an appropriate retraining policy.

STRENGTHS AND WEAKNESSES:
Strengths:

The paper addresses the problem of learning under concept drift.

The paper provides theoretical guarantees in settings with concept drift.

Weaknesses:

Are the assumptions made in the paper standard? It seems like a strong assumption to require some form of prior knowledge and access to a loss estimator.

Do [10] and [26] refer to the same work?

The proposed problem is particularly interesting in scenarios where the dataset is small. If the number of samples per dataset is large, one could simply learn using only the data from that sample set, making the concept drift less critical. However, in the small-sample regime, is it still reasonable to assume access to an unbiased estimate of the loss?

QUESTIONS:
Why are the bounds stated for \theta_t, D_{t+1} while the objective in equation (5) is defined for \theta_t, D_t? 

What is the relationship between the proposed bounds and those in the single-task setting? As I understand, in the single-task case (T = 1), generalization bounds are typically on the order of 1/ \sqrt{n}, where n is the number of samples. The connection between the proposed bounds and the number of samples per dataset is unclear and would benefit from further explanation.

What is the relationship between the proposed bounds and existing state-of-the-art bounds?

LIMITATIONS:
The paper discuss some limitations. I also believe that the assumptions made in the paper represent a limitation of the proposed approach.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper the authors introduce RCCDA, an adaptive model update policy designed to address concept drift in machine learning under strict resource constraints. The policy leverages Lyapunov analysis to minimize a drift-plus-penalty term, ensuring stability and adherence to resource constraints. The authors showed theoretical convergence bounds. They also demonstrated the policy's effectiveness through experiments on domain generalization datasets. RCCDA demonstrates superior accuracy and drift recovery compared to baseline policies, while efficiently utilizing resources.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper provides a robust theoretical foundation for the RCCDA policy. Experimental results seem to be reproducible, with detailed hyperparameters and settings provided. The results also seemed to be quite significant.\n\nMoreover the paper is well-structured, with clear explanations of the theoretical framework and experimental setup. Limitations are explicitly discussed, which is laudable.\n\nThe focus on concept drift in resource-constrained environments is a real world problem which needs enough attention.\n\n RCCDA introduces a Lyapunov drift-plus-penalty framework, ensuring stability and adherence to resource constraints. It dynamically adjusts update timing based on historical loss data, outperforming static or heuristic policies. These contributions seem to be unique and novel.\n\n\nThe paper is really well written. My only concern is its assumptions about loss availability which may not be feasible in real world scenarios.\n\nQUESTIONS:\nHow does RCCDA handle scenarios where the estimation function for gradient magnitude is inaccurate or noisy?\n\nCan RCCDA be extended to handle multi-modal concept drift or non-stationary environments?\n\nWhat are the computational overheads of maintaining the virtual queue?\n\nIt would be good to Include some benchmarks against state-of-the-art adaptive policies to highlight comparative advantages.\n\nLIMITATIONS:\nThe authors have explicitly discussed the limitations of their work, which is commendable. However they have not discussed any societal impact. It would have been good if they had included a discussion on broader implications (e.g., fairness in resource allocation or unintended consequences in deployment) in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an adaptive threshold-based policy that determines the optimal time\nto update the parameters of a predictive model in the presence of concept drift, using only past loss information, and ensuring strict bounds on constraint violations. The proposed policy triggers updates based on accumulated loss and a virtual queue tracking resource expenditure.\nThe paper also presents a convergence analysis on the relationship between model accuracy and computational cost by estimating bounds on the model accuracy depending on the update policy and the concept drift.\nFinally, the paper shows empirically that the proposed solution is able to outperform common baselines for concept drift on three datasets for domain generalization.\n\nSTRENGTHS AND WEAKNESSES:\n\\+ The proposed algorithm has some guarantees on the quality of the solution while keeping a constrained resource consumption.\n\n\\+ The paper is well presented even though the figures could be more clear and there a some typos in the text.\n\n\\- It is not clear what is the recognition model used for the experiments. Different models with different capacity should be evaluated. Also, the proposed approach could be applied to different domains than vision. Is there a specific reason to limit experiment to visual recognition?\n\n\\- The empirical evaluation is limited to three datasets. To better verify the quality of the proposed solution, authors could consider other datasets with multiple domains such as TerraIncognita and DomainNet.\n\nQUESTIONS:\n- Is there a reason to limit the evaluation to vision? Other modalities (e.g. audio or text) could also be evaluated.\n\n- What kind of recognition model is used? The theoretical analysis is independent of the model. However, the empirical evaluation should be tested on different models.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}