When to Retrain after Drift: A Data-Only Test of Post-Drift Data Size Sufficiency
Authors: Ren Fujiwara
Yasuko Matsubara
Yasushi Sakurai
Venue: ICLR 2026 Poster (2026)
Keywords: Concept drift
Stream learning
Data sufficiency
Time series
Primary area: learning on time series and dynamical systems

Abstract: Sudden concept drift makes previously trained predictors unreliable, yet deciding when to retrain and what post-drift data size is sufficient is rarely addressed. We propose CALIPER —a detector- and model-agnostic, data-only test that estimates the post-drift data size required for stable retraining. CALIPER exploits state dependence in streams generated by dynamical systems: we run a single-pass weighted local regression over the post-drift window and track a one-step proxy error as a function of a locality parameter $\theta$. When an effective sample size gate is satisfied, a monotonically non-increasing trend in this error with increasing a locality parameter indicates that the data size is sufficiently informative for retraining.
We also provide a theoretical analysis of our method, and we show that the algorithm has a low per-update time and memory. Across datasets from four heterogeneous domains, three learner families, and two detectors, CALIPER consistently matches or exceeds the best fixed data size for retraining while incurring negligible overhead and often outperforming incremental updates. CALIPER closes the gap between drift detection and data-sufficient adaptation in streaming learning.

WHEN TO RETRAIN AFTER DRIFT: A DATA-ONLY TEST OF POST-DRIFT DATA SIZE SUFFICIENCY
Ren Fujiwara', Yasuko Matsubara', Yasushi Sakurai',
SANKEN, The University of Osaka, Japan
(r- fujiwr88, yasuko, yasushi @sannenn osaka u. ac.
ABSTRACT
Sudden concept drift makes previously trained predictors unreliable, yet deciding when to retrain and what post-drift data size is sufficient is rarely addressed. We propose CALIPER -a detector- and model-agnostic, data-only test that estimates the post-drift data size required for stable retraining. CALIPER exploits state dependence in streams generated by dynamical systems: we run single-pass weighted local regression over the post-drift window and track one-step proxy error as function of a locality parameter 0. When an effective sample size gate is satisfied, monotonically non-increasing trend in this error with increasing a locality parameter indicates that the data size is sufficiently informative for retrain- ing. We also provide a theoretical analysis of our method, and we show that the algorithm has low per-update time and memory. Across datasets from four hetero- geneous domains, three learner families, and two detectors, CALIPER consistently matches exceeds the best fixed data size for retraining while incurring negligible overhead and often outperforming incremental updates. CALIPER closes the gap between drift detection and data-sufficient adaptation in streaming learning.
INTRODUCTION
Despite the ubiquity of data streams, building reliable time-series predictors in non von-stationary environments remains challenging. A substantial body of work shows that maintaining performance
hinges rapid adaptation to concept drift (Gama et al. 2004 Barna-García et al.) 2006 Bifet and
Gavaldà 2007 Prias-Blanco et al., 2014 Sebastião and Fernandes] 2017 Raab et al.) 2020 Pham
et al. 2023 Kawabata et al. 2023] [Zhang et al., 2023 Higashiguchi et al. 2025 Chihara et al. 2025
Matsubara and Sakurai 2025] Zhao and Shen 2025 Verma et al.) 2025). However, most practical
gains are achieved under incremental drift, where the data distribution changes gradually. In contrast, real-world streams often undergo abrupt shifts that invalidate previously learned models (Hare and Mantua 2000, Folke al. 20044 Matsubara and Sakurai 2016). When such sudden drift occurs, a pragmatic and effective remedy is to retrain the predictor on newly arrived post-drift data rather than salvaging the pre-drift model (Gama et al., 22144] Lu et al.) 2017). We focus this sudden drift regime and study how to provide stable retraining marniniinnnaamlly.. how to determine the post-drift data size needed restore accuracy safely.
Under such sudden drift, window-based strategies are widely used in streaming settings. ADaptive WINdowing (ADWIN) (Bifet and Gavalda (007) overcomes the limitations of fixed windows by dynamically splitting a sliding window into two subwindows and provides provable bounds on false positives and false negatives, which why it is widely adopted. Dolmogorov-Smimoo WINdowing (KSWIN) (Raab et al., (020) applies two-sample Kolmogorov-Sminoo test over empirical cumulative distribution functions (CDFs) to capture distributional changes beyond mean shifts, including variance and shape. However, detection alone does not tell what post-drift data size is needed to retrain a model that will generalize. Updating too early risks overfitting to transient noise; waiting too long prolongs downtime, keeps a stale pre-drift model in production for an extended period, and degrades predictive accuracy. These trade-offs call for a principled way to decide when the drit- data size has become sufficient to retrain safely.
We therefore deliberately focus on a different question than classical drift detection: given that a drift alarm has already been raised, how many post-drift samples needed to safely retrain the

---

Previous studies
CALIPER
One step ahead transition
How much data required adapt concept
Data point each ONeighbothood
When drift occurs
State dependence
No State dependence
(Enough data)
x(1 1) = f(x()))
0
1000
2000
3002
1000
5000
1000)
2000
3000
4000
5000
(a) Previous studies (e.g., ADWIN) CALIPER
(b) State dependence
A new data point arrives
State dependence
Each data point has neighboring points
Prediction lirror
of WLR
50-
and
Monotonic non-increusing
-50
0.1 1.0 2.0 4.0 8.0 16.0
Locality parameter: 0
(c) Visual illustration of CALIPER
KRR by CALIPER
MLP
Transformer
Prediction Error (Ground Truth)
Prediction Error
(Ground Truth)
Data catimatod
Prediction Error (Ground Truth)
by CALIIEER
by
6.
200
400
200
400
Data for retraining
Data size for retraining
Data retraining
(d) Preview the Results
Figure 1: Overview of CALIPER. (a) Unlike window-based detectors (e.g., ADWIN/KSWIN)
that only indicate if/when drift occurs, CALIPER estimates how much post-drift data are needed for stable retraining. (b) State dependence: for dynamical systems (Xx+1 fx:)), nearby states exhibit similar one-step transitions; thus data sufficiency reduces to testing whether the post-drift window exhibits adequate state dependence. (c) Pipeline: locality parameter / reweights nearby samples in weighted local regression; when the proxy error is monotonically non-increasing as increases and the neighborhood is sufficiently populated, retraining is triggered. (d) Results: star markers denote CALIPER's estimated data sizes that yield low post-drift errors across heterogeneous learners (Kernel Ridge Regression (KRR), MLP, Transformer). CALIPER selects the optimal post-drift data size-i.e.. the point at which retraining would be withou any retraining.
downstream model? Window-based detectors such ADWIN and KSWIN decide whether and when a drift occurs, but they are silent about the post-drift data sufficiency problem. In practice, this gap is critical: choosing post-drift window that oscillations, whereas waiting for an overly long window prolongs the use of a stale pre-drift model. Moreover, repeated probe-and and-train approaches that retrain complex predictors (e.g., deep neural networks (DNNs)) to gauge readiness are computationally prohibitive in streaming scenarios. This leads to our central question: Can we estimate the post-drift data size requirement directly from the stream, without actually retraining the model?
short leads to unstable retraining and
To answer this, we propose CALIPER, Cumulative Assessment of Locality Indicator for Post-drifi Estimation of Retraining-size. Assuming the data are generated by a (possibly nonlinear) dynam- ical system, CALIPER exploits state dependence to infer a sufficient post-drift data size without retraining. Concretely, it partitions the drit- window into reference set and test points, and tracks a self-supervised proxy prediction error (e.g., one-step-ahead error) from a weighted local regression whose weights governed by locality parameter 0. We apply exponentially decaying
2

---

kernel parameterized by 8 to distances in feature space between test points and reference samples. A monotonically non-increasing trend in proxy error with increasing 8 indicates adequate state dependence and triggers retraining decision. The procedure is single-pass and computationally efficient, leveraging dynamical structure rather than model-specific internals. Finally, our analysis links CALIPER'S trigger formal notion of state dependence: under stylized dynamical model, passing the monotone-localij test implies stronger state dependence. We also provide an interpre- tation, via data-dependent generalization bounds, suggesting that stronger state dependence can be favorable for stable retraining.
unlike window-based detectors such as ADWIN/KSWIN that merely signal whether/when a change occurs, CALIPER estimates how much post-drift data are sufficient for stable retraining. The key insight state dependence: if the data stream follows dynamical law (x(++1) = (()(,,), nearby states exhibit similar one-step transitions, so deciding sufficiency reduces to testing whether the post-drift window displays adequate local consistency. Operationally, CALIPER probes this via locality parameter 0 that upweights nearby samples in a weighted local regression; a monotonically non-increasing proxy prediction error as 8 increases trrrehee with sufficiently populated neighborhood- certifies data-side sufficiency proxy (ESS- + monotone locality curve) and triggers retraining. As previewed in panel (d), the star-marked data sizes selected by CALIPER yield few post-drift test errors across heterogeneous learners (Kernel Ridge Regression (KRR), MIP, Transformer). In contrast, excessively large data sizes worsen the error by delaying the update and prolonging the use of a stale model. Crucially, CALIPER produces these estimates without observing at the post-drift test segment or relying on model-specific internals, closing the gap between drift detection and data-sufficient adaptation.
Overview of CALIPER. Fig. | summarizes CALIPER:
Our key contributions can be summarized as follows:
Problem & Method. We formalize post-drift data sufficiency- eefiiianncy-etimatnnnng the minimum window size needed to safely retrain after a sudden drift, given an external drift alarm. In contrast to classical drift detectors, which only decide whether and when change occurred, our focus is on how much post-drift data are required for stable adaptation. We propose CALIPER, detector- and model-agnostic, data-only procedure that selects the earliest window that passes the effective sample size (ESS) gate and monotone locality test over single-pass weighted local regression.
Effective and Efficient. We show that CALIPER can determine whether the data exhibits state dependence. We provide an interpretation, via data-dependent generalization bounds, suggesting that stronger state dependence can be favorable for stable retraining under standard regularity conditions. The algorithm is streaming- frriinllg:rinny: it runs in a single pass and keeps per-update time and memory costs low by solving small weighted regressions under a fixed locality schedule.
Empirical validation. Across four datasets (MoCap, TEP, Automobile, Dysts), three model families (KRR, MLP, Transformer), and two detectors (ADWIN, KSWIN), CALIPER matches or exceeds the best fixed data size retraining without per-dataset tuning, improves post-drift error and recovery, and outperforms incremental updates with a negligible over- head.
2 PROPOSED METHOD: CUMULATIVE ASSESSMENT LOCALITY INDICATOR FOR POST-DRIFT ESTIMATION OF RETRAINING-SIZE (CALIPER)
2.1 PROBLEM DEFINITION
We consider a multivariate data stream (x(t) E h22 monitored by a drift detector. When a drift is detected at time t., we focus on the post-drift portion of the stream. Definition (Post-drift window and data size). For any t 2 +,,,, the drst-drift window is the set of observed samples
X, = (x(la), x(I++1), .... x(1)),
and its data size (window length) is the cardinality
T4e
X,| t 1.
3

---

Data Stream
drift detected
(i) Window normalization and split
New data point
Normalized data
Y
Post-drift window
X,
Xn
Time
(ii) ESS check ESS x 1))1)
(iii) Weighted local regression
(iv) Test & trigger
Learn WLRg by (X₃; Yg, at each
Predict
Enough data
Yo (LRg(xg)
Update
((e.0) = lly, Sell
Eg
Monotonic non-inereasing
ESS w,) Σ expl Omax
Eg.: Eq-188
10000
WA exp
E. (Ed)g0
Locality parameter.
Figure 2: CALIPER, a model-agnostic framework for dynamically estimating the data size required for retraining after sudden concept drift in data stream: (i) Window normalization and split: after a drift alarm, the post-drift segment is normalized and partitioned into reference pairs (X,, Yx) and query (g). (ii) ESS check: kernel weights we exp(-0 x r) at the largest define an effective neighborhood; proceed only if ESS 2 C' x (d+1), (iii) Weighted local regression: for each 9 on fixed grid, solve the weighted normal equations to obtain Yo and compute proxy prediction error. (iv) Test and trigger: a monotonic non-increase of the error increases, sustained for consecutive updates, indicates (a proxy for) sufficient local regularity/state dependence and triggers retraining. The rightmost panel illustrates the monotone trend of error versus 8.
The downstream predictor is treated as black box-we we do not access its internals. Intuitively, as data arrive, we wish to decide the minimum post-drift data size that allows a full stable retraining, avoiding triggers that
early (overfitinn/osciilatten Problem (Post-Driff Data Sufficiency). Given the post-drij stream after an alarm at ts. find the smallest v² N such that retraining the downstream predictor Xt,+n" is stable, using only the observed post-drift window (no access to model internals or post-drift test labels). Ideally,
too late (prolonged degradation).
n min{ n 2 1: Pr(Lgen (Uxx (Ix.,, S
2-8
where fx is the predictor retrained X. and Egen denotes post-drift generalization loss. Because Lgon is unobservable online, the problem reduces to designing a model-agnostic, data-side stopping criterion R(X+)= {0, 1} and selecting
n² min 1 R( R(X,, 1
This definition is intentionally idealized: in a streaming setting the post-drift generalization loss Lgon not observable online, and thus n²* cannot be computed directly. Instead, we must rely data-side surrogate that can be evaluated on the post-drift window alone. Concretely, our goal is to design aagnstig stopping rule R(X,) {0, 1} such that the smallest time with R(X,) = 1 closely approximates the ideal n² in Problem 1. The subsequent sections construct such a criterion based on state dependence and effective sample size and establish conditions under which it reliably predicts sufficient drit-drift data for retraining.
The next subsection introduces our method: concrete stopping criterion R(-) and an efficient online algorithm with which to estimate it from the stream and trigger stable retraining botth detector- and model-agnostic, and requiring no post-drift labels.
2.2 CALIPER: CUMULATIVE ASSESSMENT LOCALITY INDICATOR FOR POST-DRIFT ESTIMATION OF RETRAINING DATA SIZE
We specify (i) a concrete data-side criterion R(-) for stable retraining and (ii) an efficient online algorithm that sequentially estimates R(-) from the stream, resolving the early/late trigger trade-ofi off in a data-driven manner. In Appendix A we provide Algorithm which is an overview of our algorithm.
4

---

2.2.1 STOPPING CRITERION
We define a -agnostcc data-side stopping criterion R(X;) E {0. on the post-drift window X, (x(tx)..... x()) The criterion fires when (a) the current neighborhood is sufficiently populated and (b) increasing locality consistently improves one-step predictability. indicating state dependence and local regularity. When R(X;)) 1, we trigger stable retraining on X,.
2.2.2 ONLINE ESTIMATION ALGORITHM (CALIPER)
At each 2 t, + 1 we execute the following four steps. All operations Steps (ii) and (iii are performed for each value of the locality parameter 0 in fixed grid è = (fo...., 0 J. yielding sequence localized predictors and prediction errors indexed by 0.
(i) Window Normalization and Split. We normalize the post-drift window to obtain Z E R xd This normalization applied within the current post-drift window and used only to stabilize distances for the locality kernel; CALIPER never compares distances across different windows, so the underlying drift dynamics are not masked by this rescaling. Normalize the post-drift window to obtain Z R Define
X = Z[1:(n1-2) 2)}, Y = Z (n. 1)];
(g) = (z(n;-1). z(n()).
Thus (Xp, nn) provides n4 reference pairs (z(s), -(s--(())))))))) 1)). and (Xy. Nq) )) is the current query pair. (it) ESS Check. Fix short locality grid = {O..... mmax) with mmax giving the tightest locality. Let the raw distances be rf | X₂ || and define D mean( ).). Using the scaled distances r, = r/ /D (these T3 are sample-to-ou to-ruery distances, distinct from the effective radius
efft (0; 7)), set kernel weights w. (0) = exp( -0 rz). Compute the effective sample size (ESS) at the
tightest locality:
Cx Wsa (;(0max)) E. Wi (0max))
ESS
Proceed only if ESS(0max) 2 C (d+1). Because the kernel weights are wc((0) exp(-tr;) with ri 2 the effective sample size ESS(() is monotonically non-increasing in 6: larger 0 concentrates more weight on fewer neighbors. As a consequence, (S is the smallest ESS value on the grid. Checking the gate only at fmax therefore guarantees that (.S(())) 2 C(d+1) 1) for all 0g 6max-
(iii) Weighted Local Regression (WRL). We fit lightweight weighted local regression model around the current query point, using kernel weights w;( to emphasize nearby samples. Augment references with a bias: Xang each 0 E O, form
[x, 1] R ( with d+1, 1, and Xang [x, 1]. For
We diag(w;(()) Ao = X WoXaug
Bg = X ang WoYn.
solve the small system 3g = A
(iv) Test Trigger. Compute the query prediction and one-step proxy error
x Xnn Bp.
(t.0)) lly, ýell-
Accumulate the proxy error in the original units,
Ea.0) = Eu-11,8 + C(L,0)-
A smaller 0 yields broader (more global) averaging, whereas larger 0 focuses nearer neighbors; under state dependence, increasing 9 should reduce error until neighborhoods become too sparse an effect controlled by the ESS gate. Finally, the ordered grid + {0x), test monotonicity:
E(00 2 E(t.241))
Vk.
the test holds, set R(X.) = and trigger retraining on the current post-drift window X,; otherwise, continue streaming.
5

---

2.3 THEORETICAL ANALYSIS FOR CALIPER
We provide formal guarantees explaining why the CALIPER introduced in this work is useful for estimating the amount of data needed for retraining after sudden concept drift. In particular, we link CALIPER's trigger- monotonitty of localized one-step prediction enror under sufficient effective sample size (ESS)-to rigorous notion state dependence, and we provide an interpretation for why stronger state dependence can correlate with more stable retraining on appropriate local region. We begin by formalizing the setting and introducing the key quantities used in our analysis.
Setting. We consider d-dimensional time series (s(())=20 C IR generated by
s(t 1) f(s(t)) + Ee.
t= 0,1,
where f IR + Rd is locally L-Lipschitz and { is a zero-mean noise process with sub-Gaussin coordinates and covariance bounded by x² Ia. When needed for concentration, we assume f-mixing with summable coefficients.
Locality parameterization (# vs. effective radius). CALIPER is implemented using the locality pa- rameter / through the kernel (O) exp(-0 ||s s|)) (after window-wise normalization/scalngg) For the theoretical analysis, is sometimes convenient to index locality by an equivalent radius. Fix a threshold (0, 1) and define the effective radius
yfff (8;T) : log(1/r) 0
(0 > 0),
so that exp(-Ar) I r holds for all r 2 y.fff (0; Thus increasing 0 corresponds tighter locality (smaller effective radius). Accordingly, the implementation grid e = f0& k=1 induces a radius grid {rk [rkIk=1 via reff (0p:T] (with 0 + 0 interpreted as the global limit). In what follows, we use Og and the corresponding rx interchangeably.
1K
Definition 2 (State dependence). For s E Rd and radius r > 0, and for fixed constant define
as c) == Pr(lIs,, - ||s - sl]
(2)
where s'4 f(s) + and 8 f(s) E. Intuitively, neighborhood that typically remains neighborhood (up factor e) after one step exhibits state dependence. Given a compact set B Ted. we say a window S = {s(t))ez exhibits state dependence on B at scale if
inf r;) 2 for some a E (0, SEB
Proposition (CALIPER-triggeree windows exhibit stronger state dependence). Fix a compact set B C Rd and a constant
2 L as in equation 2 Let + = {0x]K JK be CALIPER's locality grid ordered as 0 < 81 <... < 0K = maxx and let } be the induced effective-radius grid defined above by Tk j E {1,.... ...,,,K) K} and set r := f-j
(f(::) for fixed T (0, 1), that r
TK Tmin- Fix index
Consider two data windows X {x(t)] and ( extracted from the same process equation] 7 Suppose X passes CALIPER's monotone locality test with a positive margin across Q and meets the ESS gate at the tightest locality 0max (equivalently, at min)), while X. fails the same monotone test (also with a positive margin) and meets the ESS gate mmax. Then, for any E (0, 1), there exist constants a E (0, 1) and A > (depending only on f. c, the noise envelope a², the grid θ (equivalently (rk)), the test margins, and the ESS threshold) such that, with probubility at least 1 - 8,
inf o(x,r; r)), 2 a sup 0(x,;;)) S T, XEB
a-a2A.
xEB
In particular, X exhibits state dependence B at scale r in the sense of equation whereas X is uniformly less state dependent by a nontrivial margin.
Proof sketch Proposition 1. Let θ (0 be CALIPER's locality grid (equivalently, the induced dfficcive-radius grid (rx}) and write Eg(S) (empirical) and (x) (population) for the localized one-step error at locality 0g (equivalently, radius ri) window S. Because the ESS

---

gate holds at the tightest locality mmax for both X and X, sub-Gaussian concentration under B-mixing gives the uniform deviation
Ew(S) = OV log(K/)) ESS(O S)
sup &
Ex (S) Ex (w(S),
On X. the monotone test passes with a positive margin: there exists 7x such that Ex+1 (X) I Ex (X) - tx for all k, hence 1x+1 S Eg (X) - (Tx 2s (X)). Thus shrinking the radius strictly decreases the population localized error. If many pairs with ||x' - violated ||x; x+ II cr, such a decrease could not persist (the deterministic part is absorbed by c 2 L), which forces infxen a(x, ric) 2 a. Conversely, on X the test fails with a positive margin: there exist k* and Tx > 0 such that (X) 2 Ex. (X) + T hence (X) 2 Ex- (X) + (Tx 2aw (X)
). which is incompatible with most neighbor pairs remaining cr-close after one step; therefore SUP ((x, S < a. Taking A == a E > 0 yields equation 3
Remark (State dependence and learnability of retraining). The role of state dependence in retraining can be understood through data-dependent generalization bounds. In particular, results such as (Wei and Ma. (209)) bound the test-train gap for MLP predictors by a term that depends on empirical quantities measured on the training window (e.g., hidden-layer norms and interlayer Jacobian norms). Abstracting these into single nonnegative complexity term C(S), one write (up to logarithmic factors)
(hs Er(h,) x C(S) n 1/2
Heuristically, when a window is more state dependent on (B, r), radius-r neighbors tend to remain neighbors (after one step) more frequently, so accurate one-step fitting on B can be achieved with less local variation. This tends to reduce empirical Jacobians/norms and hence the data-dependent term C(S), ) making the bound equation righttr.. Consequently, CALIPER-triggen indicate stronger state dependence via Proposition] ! are expected to be more favorable for stable retraining on B.
windows (which
Discussion of assumptions and scope. The dynamical-systems setting in (I) and the local Lipschitz and f-mmxing conditions are used only for our analysis of CALIPER's trigger, not as requirements of the algorithm itself. In practice, the observed state x(f) may include short history of the stream (e.g.. via delay embedding), so a first-order Markov model can hold for this augmented state even when the original process depends on (x(1 - k)..... x()).. Our distance-based neighborhoods operate on normalized features and are guarded by the (SS(0max)) 2 C'd + 1) gate, so CALIPER naturally asks for larger windows in higher dimensions; when the intrinsic dimension is very large, combining CALIPER with standard dimensionality reduction is sensible. Overall, we treat these as regularity assumptions for the theory, while the algorithm itself applies more broadly, illustrated by our chaotic and noisy benchmarks.
3 EXPERIMENTAL RESULTS
We evaluate CALIPER through experiments designed to answer three questions: (Q1) Effective- ness-how accurately does CALIPER estimate the data required to retrain model after a detected drift? (Q2) Scalability- wwhat is the computational overhead of CALIPER as data increase? (Q3) Adaptation under sudden drift, does retraining with CALIPER outperform incremental updates?
Datasets. We use four datasets from differentt domains: (a) MoCap, sequences from the CMU Motion Capture D (b) TEP, the Tennessee Eastman Process a benchmark discrete-time simulation of a chemical plant (Downs and Vogel 1993); (c) Automobile, five synchronized vehicle sensors (accelerometer, speed, Ga, Gy, G.) across multiple driving courses; and (d) Dysts, time series from the DYSTS library (Gilpin) 2021) covering chaotic systems with known dynamical properties. Experimental settings are detailed in Appendix
Experimental Setup. The experimental framework employed multiple algorithms and drift detectors. The base learners included kernel ridge regression (KRR), MLP, and Transformer. We also used ADWIN (Bifet and Gavalda 2007) and KSWIN (Raab et al. 2020 for drift detection. Performance
http://mocap.o cs. wtt.pp.cccccccccccooomom cmv.edu/

---

KSWIN + Fix data size
KSWIN CALIPER ADWIN + Fix data size ADWIN + CALIPER
KRR
MLP
Transformer
KRR
MLP
Transformer
MA
220
1:00
Thep sibe For nurminigg
Dart for nermiiingg (a-1) MSE
Thea sino Barniiing
Dho nummiiggg
Tars doe for servining (a-ii) MAE
Tots eiee for wrssiing
(a) MoCap dataset
KRR
MLP
Trans former
KRR
MLP
Transformer
235
MAR
6:30
300
E35
0/40
1065
2000
1000
2000
1200 Duta retuining
siae
cotnininn
(b-i) MSF
(b) TEP dataset
(b-it) MAR
KRR.R
MLP
Trans former
KRR
MLP
Transformer
110
0.00
1.8
Irce
I066
700
Iico
1006
200
0200
1000
3000
100
3039
octraiiiity
Deasiee cetriinngg
octraiiing
octeining
Data sice reasiniza
Duta sizz retuatinn
(c-1) MSE
(c) Automobile dataset Transforme
(c-li) MAE
KRR
MLP
KRR
MLP
Transformer
0.00
6.00
0.50
0.40
0:50
6-0
2000
1006
2000
1000
2200
(d-i) MSE
(d) Dysts dataset
(d-ii) MAE
Figure 3: Performance of CALIPER on four datasets (MoCap, TEP, Automobile, Dysts) and three model families (KRR, MIP, Transformer). We compare fixed data sizes (128/512/2049) blue) with CALIPER (orange, "CALIPER"). Each panel reports MSE (left, "Li"") and MAE E (right, "-if") function of the retraining data size after each drift detected by ADWIN (circles) or KSWIN (squares). CALIPER matches or exceeds the best fixed data size without per-dataset tuning; notably, the data size selected by CALIPER typically aligns with the dataset-specific optimal fixed data size. See Tables and for full results in Appendix
metrics included prediction mean squared error (MSE) and mean absolute error (MAE). Each dataset was subjected multiple independent runs with random seeds, and all algorithms within each run shared the same random seed initialization. Our experimental settings are detailed in Appendix F
(Q1) EFFECTIVENESS
Fig. 3 compares fixed data sizes (128/512/2048) blue) with CALIPER ("CALIPER", orange) across four datasets (MoCap, TEP, Automobile, Dysts), three model families (KRR, MLP, Transformer), two drift detectors (ADWIN/KSWIN; circles/squares), and two metrics (MSE "-i", MAE "-ii"). Each panel plots error as a function of the retraining data size used after each detected drift; for CALIPER, the x-value is the average data size consumed per retraining. Across datasets, detectors, and architectures, CALIPER sits near the best fixed point, matching or exceeding the prediction accuracy of the strongest fixed data size without per dataset tuning. On MoCap, it consistently attains the panel-wise minimum; on TEP, Automobile, and Dysts, it remains competitive with the best fixed choice. Overall, selecting the data size materially impacts accuracy, and CALIPER provides near-optimal choices across conditions. Even when a fixed data size numerically wins, that merely shows ex post that it happened to be optimal not that we can know the stream- -ime choice to be optimal at the time. Moreover, the fixed size with the best prediction accuracy varies widely. The fixed size with the best prediction accuracy varies widely. For example, the MIP model achieves its highest accuracy on TEP with data size of 512, whereas the same setting yields the lowest performance on MoCap. These results underscore the brittleness a priori data size selection. In contrast, CALIPER selects data sizes data-dependently near the optimum and typically achieves best- or second-bes accuracy, thereby preserving the method's s practical value. For a tree-baser learner, CALIPER exhibits the same qualitative behavior, with estimated window sizes remaining close to empirically optimal choices. These tree-based results, together with full numerical results averaged over horizons 1, 15, and 30 (Tables and and hyperparameter-seesiiiiiii study of CALIPER's
8

---

KSWIN+CALIPER KI KSWIN+FiX(512) KSWIN-FiX(249) ADWIN=FiX(522)
ADWIN+CALIPER ADWIN+Fix(13)
ADWIN-FiX(245)
Wall clock time (sec)
10²
10
10
10
10
10
10
10
10
0
2500
5000
Average
10
2500
5000
Average
Time points
Time points (ii) Sequence #2
(i) Sequence #1
Dysts dataset
Figure 4: Per time step and average wall clock time on Dysts sequences #1 and #2 for AD- WIN/KSWIN with CALIPER and fixed-size buffers (128/512/2048); curves are flat with low means, and occasional spikes reflect retraining rather than CALIPER.
Table Adaptation after drift. CALIPER CALIPER-trigered retraining vs. Incremental updating. MSE/MAE averaged over (1,15,30) with the past sequence length is 30; detectors ADWIN/KSWIN: models KRR/MLP/Transforme. Lower better; best bold, second underlined. Tables Appendix for the full results.
S and in the
MoCap
TEP
Automobile
Dysts
Model
Detector
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MILP
ADWIN
7.106
1.387
1.790
0.704
2.090
0.892
0.432
0.460
KSWIN
7.449
1.436
1.760
0.699
2.081
0.889
0.467
0.468
(Incremental)
412.6
8.462
3.699
0.879
2.406
0.894
71.75
1.524
Transformer
ADWIN
10.96
1.744
1.818
0.698
1.948
0.883
0.549
0.530
KSWIN
11.35
1.810
1.808
0.690
2.070
0.881
0.501
0.505
(Incremental)
22.08
2.654
2.376
0.767
1.866
0.773
0.582
0,585
locality grid 0 and ESS threshold for the MLP base learner (showing that selected window sizes and post-drift errors are stable over wide range of settings), are all reported in Appendix G
(Q2) SCALABILITY
We measure wall clock time on the Dysts dataset using two sequences (#1, #2) and eight detec- tor strastra combinations (ADWIN/KSWIN paired with either CALIPER or fixed data size of 128/512/2048), with MLP as the base learner. Fig. reports per time step wall clock time (log- seconds) along with the corresponding averages. Across all combinations, the wall clock time curves essentially flat and the mean cost per step is small; occasional spikes align with model retraining and are not caused by CALIPER. Overall, this indicates that CALIPER adds negligible overhead relative to the base learner, the detector, and the fixed-length baseline, consistent with our of lightweight local regression. Additional results appear Appendix
(Q3) ADAPTATION
Table compares CALIPER with an incremental-uplee update baseline (online SGD; the same architecture, no explicit retraining). Results are averaged over horizons (1, 10) with past sequence length of 30. Overall, CALIPER matches or surpasses incremental updates across datasets and models. For MIP, CALIPER yields large gains on MoCap and Dysts (e.g., MoCap: MSE 7.106 vs. 412.6; Dysts: 0.432 vs. 71.75), indicating that purely local incremental steps can unstable under drift. For Transformer, CALIPER is competitive
all datasets and superior TEP and Dysts (e.g., Dysts: MSE 0.501 with KSWIN vs. 0.582 incremental); on Automobile, incremental performs slightly better (MSE 1.866 vs. 1.948-2.077 but the gap is modest. Taken together, these results suggest that (a) selecting an appropriate retraining data size at each drift materially improves accuracy and stability, and (b) pure incremental updates often insufficient in sudden drift.

---

4 RELATED WORK
4.1 CONCEPT DRIFT
In dynamically changing and non-stationary environments, the data distribution can change over time, yielding the phenomenon of concept drift. Concept drift is phenomenon in which the statistical properties of a region of interest change over time in an arbitrary manner (Lu et al. 2014). was first proposed by (Schlimmer and Granger, 1986), who pointed out that noise data can become non-noise information at different times. Such changes could be caused by changes in hidden variables (Liu et al., (017)) that cannot be measured directly. Strategies for updating existing training models in response to the drift caused by such system changes can be categorized into two main groups: retraining and model adjustment. Each of these aims to address different types of drift. Here, we primarily focus on the retraining analysis aspect and summarize the model adjustment based approaches in Appendix B Window-based detectors (e.g., ADWIN, KSWIN) are standard in streaming settings, where they compare recent and historical windows to flag drift (Bifet and Gavaldà 2007; Raab et al.) 2022); see the introduction for a brief survey. Crucially, these detectors respond to drift but not how much post-drift data are required for reliable retraining gap our work addresses with post-drift data sufficiency criterion.
4.2 ANALYSIS OF NONLINEAR DYNAMICS
Inspired by Wold's theorem Wold 1938), time series data X; can be formally decomposed as X; = Ei= byct- + 7)- Here, The represents the deterministic component, and e, represents the stochastic component as stationary process input to the linear filter by. In many fasks-especially time-series forecasting- the deterministic part crucial: long-horizon trends capture low-frequency evolution. This deterministic structure is often modeled with flexible function classes, including
programmatically discovered structures (Champion et al.) 2019 Zheng et al., 2019 Bertsimas and Gurnee 2023 Fujiwara et al., 2025) and analyzed for description and prediction. A central phenomenon in nonlinear dynamics is state dependence (Sugihara 1994 Ye et al.] 2013), which measures how similarly futures unfold when present states are similar, operationally, a series exhibits state dependence if forecasts from a model trained locally around a point x surpass those from global model trained all data. This property underpins short-term prediction for nonlinear series, exemplified by S-Map (Hsieh et al., 2005) Perretti et al.) 2013 Deyle et al.] [2016] Ushio al. 2018), which computes distances between target state and historical library and performs restance-weighted regression for forecasting. While such analyses rely on mild assumptions, a key advantage is their independence from a posited generative model. Building on this insight, our work repurposes state dependence to assess data sufficiency. Beyond offering a model-agnostic criterion that estimates the minimum data requirement without requiring knowledge of the parametric system, it also provides practical estimation algorithm that enables deployment in real-world settings.
5 CONCLUSION
In this paper, we introduced CALIPER, detector- and model-agnostic, data-only framework that returns the earliest post-drift window that satisfies a data-side stopping rule, used as a proxy for retraining readiness. Rather than probing or stress-testing downstream model, CALIPER enforces simple, verifiable stopping rule on the stream itself: it checks that local neighborhoods are sufficiently populated (via an effective sample size gate) and that one-step predictability improves monotonically as neighborhoods become more local, all computed in a single pass using low-overhead weighted local regressions. Our theory shows that this trigger is not merely heuristic: under a stylized dynamical model, passing it implies stronger state dependence; under standard regularity assumptions, this can be favorable for learnability on a suitable local region- providiin a principled basis for post-drift sample sizing and for deciding when enough data has accumulated to retrain safely. Empirically, across four datasets, three learner families, and two drift detectors, CALIPER matches or surpasses the best fixed-size retraining without per-dataset tuning, reduces post-drift error, and consistently outperforms incremental updates at negligible computational and memory cost. In practice, CALIPER cleanly separates the when from the how of adaptation, enabling plug aadg--nd--laa deployment in streaming systems with heterogencous models and scarce labels, and making retraining decisions transparent, auditable, and robust to detector choice.
10

---

ACKNOWLEDGMENTS
This work supported by JSPS KAKENHI Grant-in-Aid for Scientific Research Number JP24KJ1618, JST CREST JPMICR23M3, JST START JPMIST2553, JST CREST JPMJCR20C6, 1ST Program JPMJKP25Y6, JST COI-NEXT JPMJPF2009, JST COI-NEXT JPMJPF2115, the Future Social Value Co-Creation Project Osaka University.
ETIICS STATEMENT
We adhere to the ICLR Code of Ethics. Some experiments use closed automobile dataset that may contain individual driving records provided under data-use agreement. No raw personally identifiable information is shared in this paper the supplementary materials; access is restricted to authorized researchers, and analysis is conducted on de-identified records following the provider's privacy policies. The dataset cannot be redistributed. We disclose no conflicts of interest.
REFERENCES
Manuel Baena-García, José Campo-Ávila, Raúl Fidalgo-Merino Albert Bifet, Ricard Gavald, and Rafael Morales-Bueno Early drift detection method. In 4th international workshop on knowledge discovery from data streams, volume 6, pages 77-86, 2006.
Dimitris Bertsimas and Wes Gurnee. Learning sparse nonlinear dynamics via mixed-integer timization. Nonlinear Dynamics, 111(7):655--6644 2023. ISSN 0924-090X. doi: 10.1007/ s1 1077-002-00878--9
Albert Bifet and Ricard Gavaldà. Learning from time-changing data with adaptive windowing. Proceedings of the 2007 SIAM International Conference on Data Mining, pages 443-445 2007. doi: 10. 0121137/1.9781611999
Kathleen Champion, Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton. Data-driven discovery of coordinates and governing equations. Proceedings of the National Academy Sciences, 116
(45):22445-22455 2019. ISSN 0027-8424. doi: 10. 20.11773pnas.
Naoki Chihara, Yasuko Matsubara, Ren Fujiwara, and Yasushi Sakurai. Modeling time-evolving causality over data streams. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1. pages 153 164, 2025.
doi: 10. (0.1114536906224.7799253
Ethan R. Deyle, Robert M. May, Stephan B. Munch, and George Sugihara. Tracking and forecasting ecosystem interactions in real time. Proceedings of the Royal Society B: Biological Sciences, 283 (1822):20152258, 2016. ISSN 0962-8452. doi: 10. 10.1098/ssb.220552222
J.J. Downs and E.F. Vogel. widnt-wide industrial process control problem. Computers & Chemical Engineering, 17(3)2245-255, 1993. ISSN 0098-1354 doi: https://oi..rr//0.1116/0099 1354(93) 80018-I. Industrial challenge problems in process control.
Carl Folke, Steve Carpenter, Brian Walker, Marten Scheffer, Thomas Elmqvist, Lance Gunderson, and C.S. Holling. REGIME SHIFTS, RESILIENCE, AND BIODIVERSITY IN ECOSYSTEM MANAGEMENT. Annual Review of Ecology, Evolution, and Systematics, 35:557-581, 2004.
Isvani Frias-Blance Blanco, José del Campo-Ávila, Gonzalo Ramos-Jimenez, Rafael Morales- Buenoo, Agustin Ortiz-Diaz, and Yailé Caballero-Man Mota. Online and non- -arrmmrmtt drift detection methods based on hoeffding's bounds. JEEE Transactions on Knowledge and Data Engineering, 27(3): 100-82,, 2014.
Ren Fujiwara, Yasuko Matsubara, and Yasushi Sakurai. Modeling latent non-lineat dynamical system time series. Proceedings of the AAAI Conference Artificial Intelligence, 39(1 I): 11663-11671, 2025. ISSN 2159-5399. doi: 10. 20.1160/aaai..v333 v39il 1.33269.
João Gama, Pedro Medas, Gladys Castillo, and Pedro Pereira Rodrigues. Learning with drift detection. In Advances in Artificial Intelligence SBIA 2004, volume 3171 of Lecture Notes in Computer Science, pages 2004.
11

---

João Gama, Indrè Zliobaité, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation. ACM Computing Surveys (CSUR), 46(4):1-37, 2014.
William Gilpin. Chaos as an interpretable benchmark for forecasting and data-driven modelling. Advances in Neural Information Processing Systems, 34, 2021.
Steven R Hare and Nathan J Mantua. Empirical evidence for north pacific regime shifts in 1977 and 1989. Progress in Oceanography, 47:103-145, 2000.
Shingo Higashiguchi, Yasuko Matsubara, Koki Kawabata, Taichi Murayama, and Yasushi Sakurai. D-tracker: Modeling interest diffusion in social activity tensor data streams. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.I. KDD 2025,
Toronto, ON, Canada, August 3-7, 2025, pages 460 160-471.44 ACM, 2025.
Chih-hao Hsieh, Sarah M Glaser, Andrew Lucas, and George Sugihara. Distinguishing random environmental fluctuations from ecological catastrophes for the north pacific ocean. Nature, 435 (7040):336-340, 2005.
Koki Kawabata, Yasuko Matsubara, and Yasushi Sakurai. Modeling dynamic interactions over tensor streams. In Proceedings of the ACM Web Conference 2023, pages 1793-1803, 2023.
Anjin Liu, Yiliao Song, Guangquan Zhang, and Jie Lu. Regional concept drift detection and density synchronized drift adaptation. Proceedings of the Twenty- Sixth International Joint Conference on Artificial Intelligence, pages 2280-22800 220002266,222 2017. doi: 10. 10.24965/iccai22117/111
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. The Seventh International Conference on Learning Representations, 2019.
Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, and Guangquan Zhang. Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering. 31(12):236-2363 2017.
Ning Lu, Guangquan Zhang, and Jie Lu. Concept drift detection via competence models. Artificial Intelligence, 209:11-25 2014. ISSN 0004-3702. doi: 10.1016/j.artint.200 201.:a.r.t...1......00000
Yasuko Matsubara and Yasushi Sakurai. Regime shifts in streams real-time forecasting of co- evolving time sequences. In Proceedings of the 22th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1045-1054, 2016.
Yasuko Matsubara and Yasushi Sakurai. MicroAdapt: Self-evolutionary dynamic modeling algo- rithms for time-evolving data streams. In Proceedings of the 3 1st ACM SIGKDD Conference Knowledge Discovery and Data Mining V.2. KDD '25, page 2125,,,,, New York, NY, USA, 2025, Association for Computing Machinery. ISBN 9798400714542. doi: 10.1145/3711896. 3737048. URL htttts::////// wogg/oorg/////
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research,
12:2825 22:2825-2830. 2011.
Charles Perretti, Stephan B. Munch, and George Sugihara. Model-free forecasting outperforms the correct mechanistic model for simulated and experimental data. Proceedings of the National Academy of Sciences, 110(13):52535222 5257, 2013. doi: 10. 10.1003/pnas... http3//paa..1167661111 12160761 10.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven C H Hoi. Learning fast and slow for online time series forecasting. In The Eleventh International Conference Learning Representations, 2023.
Christoph Raab, Moritz Heusinger, and Michhel Schleif. Reactive soft prototype computing for concept drift streams. Neurocomputing. 416:340-351 2020. ISSN 0925-2312. doi: 10.1016/j. neucom.2019.11.111
Jeffrey C. Schlimmer and Richard H. Granger. Incremental learning from noisy data. Machine Learning, 1(3):317-354, 1986. ISSN 0885-6125. doi: 10. 1007//011000 16895.
12

---

Raquel Sebastião and José Maria Fernandes. Supporting the page-hinkley test with empirical mode decomposition for change detection. In Foundations of Intelligent Systems 23rd International Symposium, volume 10352 of Lecture Notes in Computer Science, pages 2017.
Springer,
George Sugihara. Nonlinear forecasting for the classification of natural time series. Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences, 348 (1688):477-495, 1994.
Masayuki Ushio, Chih-hao Hsieh, Reiji Masuda, Ethan R Deyle, Hao Ye, Chun- Chang, George Sugihara, and Michio Kondoh. Fluctuating interaction network and vary-ng stability of natural fish community. Nature, 554(7692):360-33 2018. ISSN 0028-0836 doi: 10.1038/ nature25504.
Nilesh Verma, Albert Bifet, Bernhard Pfahringer, and Maroua Bahri. Bayesian stream tuner: Dynamic hyperparameter optimization for real-time data streams. In Proceedings of the 31tt ACM SIGKDD
Conference on Knowledge Discovery and Data Mining V.2, KDD '25, page 2871-2882, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400714542. doi: URL.ttts:://doo http://oooorr//.././/// org/10. 4744537111 373
10.1145/371 011553771186637336335
Colin Wei and Tengyu Ma. Data-dependent sample complexity of deep neural networks via lipschitz augmentation. Advances in Neural Information Processing Systems, volume 32, 2019.
Herman Wold. A study in the analysis of stationary time series. PhD thesis, 1938.
Hao Ye, Richard J. Beamish, Sarah M. Glaser, Sue C. H. Grant, Chih-hao Hsich, Laura J. Richards, Jon T. Schnute, and George Sugihara. Equation-free mechanistic ecosystem forecasting using empirical dynamic modeling. Proceedings of the National Academy of Sciences, 112(13):E156- E1576, 2015. ISSN 0027-8424. doi: 10. (0.10733pnas.14) htt.///aas..177661111111 141706111.
Yi-Fan Zhang, Qingsong Wen, Xue Wang. Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, and Tieniu Tan. OneNet: Enhancing time series forecasting models under concept drift by online ensembling. In Advances in Neural Information Processing Systems 36, volume 36, pages 99949,69980,, 2023.
Lifan Zhao and Yanyan Shen. Proactive model adaptation against concept drift for online time series forecasting. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.I, pages 2020-2031, 2025. doi: 10.1 10.1145/36906224370222
Peng Zheng, Travis Askham, Steven Brunton, Nathan Kutz, and Aleksandr Aravkin. unified framework for sparse relaxed regularized regression: SR3. IEEE Access, 7:1404-1423, 2019. ISSN 2169-3536. doi: 10.
Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the AAAI Conference on Artificial Intelligence, 35(12):11 10211100-11111 106-11115, 2021.
13

---

TECHNICAL APPENDICES AND SUPPLEMENTARY MATERIAL
A ALGORITHM OVERVIEW
In this section, provide the details of the CALIPER optimization algorithm proposed in section Algorithm |
provides an overview of CALIPER.
Algorithm 1 CALIPER Et
Input: Post-drift window: X, = (x(t,), x(t) and previous prediction error each F
Output: Retrain flag and prediction error at each A:
I:
2: /* (i) Window Normalization and Split. %/
3:
normalized X, X Z:1: nu 2/; Y, Z[2 ne 11; x, Z Z' 11: N4 Z(nu];
5:
6: /* (ii) ESS Check, */
7: for all x; E X, do
8: raw ||x;- Xpl
10: r= = FAW /mean(r
11: for all x; E XA. do
exp(pp00 - ri) 13: end for
12:
14: (Σ "w////E w < * (d 1) then
15: return false, Et-1
16: end if
17:
(iii) Weighted Local Regression
19: for 0 E 0, mman] do
20: w exp(-0 rr)
21:
Learn weighted local regression WLRs using (X., Ya) and w.
22: E(2.6) = (x,)
23: F((.)) = 1.0) ((L.)
24: end for
25:
26: /* (iv) Test & Trigger. */
27: E, = (b fe-o
10
28: if E, is monotonically non-increasing for & then
29: return true,
30: else
31:
return false, E,
32: end if
B ADDITIONAL RELATED WORKS
Instead of retraining the entire model, there ways develop models that adaptively learn from changing data. This approach is more efficient than retraining if the drift occurs only in localized regions. This approach can be easily implemented by stochastic gradient descent, and various methods have been proposed in recent years. In addition to standard fine-tuning techniques, recent studies (Pham al.) 2023] Zhang al. [023 Zhao and Shen (2025) have proposed more sophisticated model adaptation techniques. These focus on ways to effectively adapt to recent data by using predictive feedback (e.g., errors and gradients) on new training samples. Among them, FSNet (Pham et al. (023) monitors the gradients in previous fine-tunings and translates them into parameter adjustments to adapt the new predictive model to the current training sample, and dynamically adjusts the ensemble weights and the model's parameter weights according to the prediction error. OneNet (Zhang et al.) 2023) is an online ensemble network that generates ensemble weights to combine prediction models and dynamically adjusts ensemble weights and model parameter weights according prediction PRoCEED (Zhao and Shen 22055) is a framework that estimates the drift between recent training samples and the current test sample. It then proactively adjusts the model parameters before receiving
14

---

ground-truti trunth future values. This approach effectively bridges the temporal gap caused by forecast horizon delays.
C LIMITATION AND FUTURE WORK
Our theoretical analysis adopts a stylized but standard one-step state-space model,
s(t + 1) f(s(t) + &
with locally Lipschitz f and Go--Gusssian noise, to formalize state dependence and to provide an interpretation relating CALIPER's i trigger (a monotone locality curve plus an ESS condition) to learnability on a suitable local region. These assumptions should be interpreted as regularity conditions for the analysis, not as hard requirements for applying the algorithm. In practice. CALIPER operates on arbitrary feature vectors, and the state can include short history of the stream (c.g.. delay embedding) or representation learned by an upstream encoder. Nevertheless, our current guarantees do not explicitly cover fully non-Markovian dynamics or settings with strong latent or exogenous drivers beyond what is captured by the chosen representation. Extending the formal results to explicit latent-variable and history-dependent models is important direction for future work.
second limitation concerns high-dimensional geometry. CALIPER relies on distance-based local neighborhoods to form localized fits, and naive nearest-neighhor weighting can be unreliable in very high dimensions: neighborhoods become sparse, effective sample sizes can collapse, and the localized regression can degrade unless the post-drift window is sufficiently large. CALIPER mitigates this via (i) feature/distance normalization within the current post-drift window and (ii) the ESS gate as a conservative safeguard (checked at the tightest locality), which naturally forces larger windows as dimensionality increases. However, this does not eliminate the fundamental dependence on having meaningful metric in the working representation space. For extremely high-dimensional raw inputs (e.g., image streams), compact representations or adaptive metrics are likely necessary; characterizing when such representations yield reliable locality and ESS behavior remains future work.
Finally, CALIPER should be interpreted as detecting regime in which a broad class of reasonably expressive learners can retrain stably, rather than identifying a single universal optimum window, Establishing tighter theoretical link between the trigger and model-specific convergence is an interesting direction for future work.
D THE USE OF LARGE LANGUAGE MODELS (LLMS)
We use LLMs aid or polish writing. Specifically, we used LLMs for assistance when writing papers, for example to check spelling and to make grammar suggestions.
E PROOF OF PROPOSITION I
Proof of Proposition 7 We work under equation 1 and equation 2 with a fixed compact B C Rdd scale 0. and L. Let θ {0x) be CALIPER's 's locality grid, ordered so that 0 <
< 0x- = 6max. Fix J E (0, 1) and define the induced effective-radiug grid by
Tk = ell (0)::) log(1/r) 0x
h= 1,
so that increasing 0₂ corresponds to tighter locality (smaller TR). We write (s) and Ex for the empirical and population localized one-step errors at locality 0g (equivalently, radius rk) on window S.
Step 1 (Uniform concentration). By sub- Gaussian coordinates, 3-mixing with summable coefficients, and the ESS gate at the tightest locality 6max, there exists, for any E (0, 1) and with probability at least 1 - 5, uniform deviation bound
EW (S) o(
log(K/0) (K/8) ( maxx 8)
sup 1
(S) Es( (S),
SE (X.X).
15

---

Step 2 (Triggered window X: lower bound at). On X, the empirical monotone test passes across the grid with positive margin; hence there exists Tx > O such that
(x) < - TX
(k = 1, K
K....,-III,
By Step 1,
1 S FAx (X) - (Tx 2Ew (X)
K IK...... 1-1,
with probability at least 1 - 6, and the gate at 0mxx ensures TX 22w((X) > 0. inf a r;c) were too small, then a nonnegligible fraction of radius-r neighbor pairs would satisty |x+ 1x+-x+11 | cr.
which-affee ubsorbing the deterministic Lipschitz part through 2 L-would prevent E (X) Eg (X) from holding uniformly as the radius shrinks, a contradiction. Therefore there exists (0, 1) such that
inf (x, r; a on X xEB
with probability at least 8.
Step 3 (Non-triggered window X: upper bound on a and gap). On X, the empirical monotone test fails with a positive margin, there exist k* and Tx > 0 such that
(x)+
Ex (X) + TX-
By Step 1,
(X)+ 2 Eg (X) + (Tx 22w (X)),
with probability least 1 (and we ensure Tx 2 (X) > via the gate at mmax)) If s aXx,;; r) were close to one, shrinking the radius would not increase the population lo- calized error, contradicting the display. Hence there exists a E (0, 1) such that
sup a ric) a on X
x
with probability at least 1-5. Setting == a > yields equation and completes the proof.
F EXPERIMENTAL SETTINGS
Code. Our datasets and source code are available at: ltttps://gitaab..cm///eeetiiyi/p
CALIPER
Hyperparameters of CALIPER. In CALIPER, is selected from [0, 0.0, 2.0, 4.0, 8.0, 16.0] and set C Then, it is determined whether the error non-increases monotonically with respect to 8.
Computing infrastructure. The configuration includes 2 Xeon Gold 6444Y 3.6GHz CPU, 12 * 64GB DDR4 RAM (768GB), and NVIDIA RTX A6000 48GB GPU, which is sufficient for all the baselines.
Dysts dataset details. These data are obtained from dysts database (Gilpin 2021), which provides data, equations, and dynamical properties for chaotic systems exhibiting strange attractors and coming from disparate scientific fields. In our experiments, we consider 12 hyperchaotic systems representing ordinary differential equations (ODEs) with polynomial nonlinearities. We used 10 systems for training and tested them with the remaining systems. Each system is included in the test data at least twice. other words, we conducted our experiments with 12 different synthetic data We use 1500 lengths of data for the warm-up phase and 6000 lengths of data that generated for the online learning period. The time step between each sample 0.05, and the initial conditions are randomly generated according to Gilppn 2)21).
(#1-# #12).
Implementation Details. In the MoCap, Automobile, and Dysts datasets, we split the data into warm-up and online-training phases with a 20:80 ratio. We used the TEP dataset, where the normal operating interval was employed as the warm-up up phase, and the abnormal interval was treated as training phase. Following (Zhou al.) (221),
the online-training
minimize the mean squared error (MSE) using the AdamW optimizer (Loshchilov and Hutter 2019); the learning rates for both the MLP and Transformer were selected from {10-3,5 10 10 10-1 10-5) based
16

---

KSWIN+CALIPER
KSWIN+Fix
KSWIN+Fix(512) KSWIN+FiX(2048)
ADWIN+CALIPER ADWIN+FiX(28)
ADW1N+FIX(522)
ADW1N+FiX(248)
Wall clock time (sec) Wall clock time (sec) Wall clock time (sec)
10
10
10
10
10
10
10
10
10
MoCap
10
TEP
10
Automobile
10
Dysts
(i) KRR
10
10
10
10
1
10-
10
10
10
MoCap
10
TEP
10
Automobile
10
Dysts
(ii) MLP
10
10
10
10
1
10
10
10
10
MoCap
10
TEP
10 (iii) Transformer
Automobile
10
Dysts
Figure 5: The average per time step wall-clock time (bar chart) for MoCap, TEP, Automobile, and Dysts under ADWINKSWIN with CALIPER fixed data size (128/512/2048) using KRR/MLP/rrnssormer; CALIPER matches fixed data size baselines.
validation performance within the warm-up phase. The MLP uses a hidden dimension of 32, and the Transformer hyperparameters and other training settings follow the benchmark implementation! For KRR, we implemented scikit-learn's 3 Ke rne Ri áge (Pedregosa et al. (011) with an RBF kernel and search over y (None, 10-3 3 1010-- 10-1, 1.0] and {10 10 1,1.0 10.0), selecting the combination based on the validation performance within the warm-up phase. For drift detection, we used ADWIN with a confidence value of 0.002 and KSWIN with a significance level of 0.05.
G ADDITIONAL EXPERIMENTAL RESULTS
G.1 ADDITIONAL ANALYSES
Hyperparameter sensitivity of CALIPER. Figure 6 shows the sensitivity of CALIPER its two main hyperparameters: the locality parameter 9max and the ESS multiplier C.. For each dataset and each drift detector (ADWIN, KSWIN), we vary E (2, 4) and 0 {12, 14, 16, 18, 20) while keeping all other settings fixed, and plot the resulting test MAE. The y-axis of Fig. G the same absolute MAE scale as our main results. On three the four datasets, the curves are almost flat for both hyperparameters, indicating that CALIPER is highly robust in these regimes. On the more challenging MoCap dataset, changing C still has limited effect, whereas large values of 0max (around 18 (8200) lead to performance degradation. This behaviour agrees with the interpretation of 8max as a locality scale: if the window becomes too wide, the WLR probe no longer reflects local state dependence and the retraining trigger deteriorates. Around the default configuration (0max 16, C' = 3) used in all other experiments, the MAE remains close to the optimum on all datasets.
https: www.ttthhhhhh..go www.hhwww......ccooooooooe.eoeee com/thum /Tii me- Series-
17

---

1.5
1.5
MAE
MAE
MAE
1.0
MAE
1.0
0,5
0.5
12.5
15.
17.5
20.0
0.0
0.0
12.5
15.0
17.5
20.0
(a-1) ADWIN
(aIII)KSWIN
(a) MoCap dataset
0.6
0.6
0.6
MAE
MAE
0.4
0.-
02
0.2
0.0
15.0
0.0
15.0
17.5
20.0
6nox
9mxx
(b-i) ADWIN
(b-i))KSWIN
(b) TEP dataset
1.0
1.0
1.0
MAN
MAE
0.0
12.5
15.0
17.5
0.0
12.
17.5
20.0
Onax
(c-i) ADWIN
(c-i))SSIIN
(c) Automobile dataset
0.4
0.4
0.4
MAR
0.2
0.2
0.2
0.2
0.0
12.5
15.0
17.5
20.0
0.0
0.0
12.5
15.0
17.5
20.0
(d-i) ADWIN
(d-ii)KSWIN
(d) Dysts dataset
Figure 6: Hyperparameter sensitivity of CALIPER. Test MAE as function of the ESS multiplier C E 3, 3,44 4} (left panels) and the locality parameter 6 E {12, 14, 16, 18, 20} (right panels) on the four datasets and for both ADWIN and KSWIN. The y-axis is on the same absolute MAE scale as in the main results. The curves are nearly flat on three the datasets, and only large values of 0 max the MoCap dataset cause noticeable degradation, consistent with the role of max as locality parameter.
Full Experimental Results for Prediction. Tables and show full numerical results (averaged over horizons (1,15,30)). As discussed Section 3 our method achieves performance that comparable to or even substantially better than fixed-length retraining strategies, despite not having access to the final predictive accuracy in advance. A noteworthy observation is that the optimal fixed data size window varies across datasets. For instance, smaller data sizes are more advantageous for the MoCap and TEP datasets, whereas larger data sizes tend to perform better for the others. This highlights the difficulty of relying on fixed data size retraining strategy and underscores the effectiveness of our approach.
Tree-based learner (ExtraTreees. We also include an additional experiment using Extremely Randomized Trees (ExtraTreess regressor as a tree-based base learner. We keep the same detectors (ADWIN and KSWIN), datasets, and CALIPER hyperparameters in the main experiments. For each dataset and detector, we sweep several fixed post-drift window sizes and compare them to CALIPER. Across all settings, CALIPER's R estimated window sizes remain close to the empirically optimal fixed choice, and its post-drift errors match or improve upon the best fixed window. These results confirm that CALIPER behaves similarly for tree-based models as for KRR, MLP, and
18

---

Transformers. Representative curves and full numerical values (averaged over horizons 1, 15, and 30) are reported in Tables 2 and 31
Additional Scalability Results. Figure shows datasct-level average wall clock time per time step (bar chart) across four datasets (MoCap, TEP, Automobile, Dysts), two detectors (ADWIN/KSWIN), and two strategies (CALIPER
fixed data size of 128/512/2044 under three base learners (KRR, MLP, Transformer). Across all settings, CALIPER is on par with the fixed data size baselines, indicating negligible additional overhead; remaining differences largely explained by the base learner and detector rather than the strategy.
19

---

Table 2: Full results with ADWIN. For each dataset and model family, we report MSE/MAE at prediction horizons (1, 15, 30) and their average. We compare CALIPER (ours) with fixed retraining windows of 128, 512, and 2048 steps ("Fix (+)") and with an online Incremental baseline (no explicit window retraining). The past sequence length is 30. Datasets span four heterogeneous domains (TEP, MoCap, Automobile, Dysts). Lower is better; best per column in bold (second best underlined, if applicable).
Dataset
Model
CALIPER
Fix (128)
Fix (512)
Fix (20-8)
Incremental
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
KRR
6.952
1.146
7.607
1.339
12.369
1.719
8.074
1.228
16.003
2.201
15
7.691
1.480
8.934
1.561
15.165
2.230
10,010
1,802
19.552
2.538
30
9,431
1.716
9.879
1,650
17.725
2.395
19.373
2.565
18.946
2.524
Avg
8.025
1.447
8.807
1.516
15.086
2.114
12.486
1,865
18.167
2.421
McCap
ExtraTrees
7.822
1.455
15.295
2.025
11.100
1.804
9.541
16666
35.622
3.337
15
9.846
1.701
17.791
2.198
13.351
2,030
10.599
1 1886
44.145
3.777
30
11.127
1,825
15.500
2,082
13.479
2.028
11.610
1,910
33.087
3.298
Avg
9.598
1.660
16.196
12.643
1.954
1.781
37.618
MLP
1
3.435
0.966
6. 169
1.361
5.548
1.409
2.199
0.919
357.793
7.636
15
7.534
1.469
9.392
1.705
24.581
2.570
9.661
1,928
578.383
9,466
30
10.349
1.725
14.122
1,959
41.531
3,368
21.282
2,845
301.643
8.285
Avg
7.106
1.387
9.894
1.675
23.887
2.449
11.047
1,897
412.606
8.462
Transformer
9.458
1.571
14.566
2.013
17.245
2.324
10,606
1,779
19.661
2.508
15
10.134
1.681
17.627
2.185
20.548
2.581
12.353
1,988
22.874
2.708
30
13.273
1,981
19.514
2.361
24.971
2.922
13,308
2,093
23,700
2.748
Avg
10.955
1.744
17.236
2.186
20.921
2,609
12.109
1.953
22,078
2.654
KRR
1.943
0.690
1.696
0.669
1,952
0.669
2.638
0.749
1.671
0.691
15
2,055
0,751
16661
0.707
2.134
0.716
2.752
0.789
1.794
0.708
2.166
0.760
1.977
0.232
2,275
0.746
2,863
0.819
1.874
0.718
Avg
2.055
0.727
18445
0.702
2,120
0.710
2.751
0.786
1,780
0.706
Extra Trees
1.674
0.637
1.383
0.638
1,820
0.667
2.340
0.734
1.314
0.621
15
1.827
0.685
1 1.682
0.697
2,174
0.731
2.749
0.800
1.572
0.663
1.951
0.721
0,728
2.411
0.770
3.007
0.840
1.716
Avg
1.817
0.681
1.649
0.688
2.135
0.722
2,699
0.792
1.534
0.654
MLP
1.651
0.663
1,905
0,746
1.478
0.642
1,910
0.706
3.136
0.829
15
1.739
0.702
2.051
0.265
1.684
0.698
2.044
0.760
4.180
0.914
30
1.979
0.746
2.161
0.792
2.247
0.769
2.837
0,849
3.781
0.894
Avg
1.790
0.704
2.039
0.768
1,803
0.703
2.264
0.772
3.699
0.879
Transformer
1.722
0.679
1,723
0,709
1.793
0.699
2.229
0.759
2.128
0.705
15
1.763
0.680
2.066
0.727
2,061
0.719
2.301
0.763
2,421
0.781
30
1.968
0.735
2,078
0,753
2.319
0.772
2,660
0.826
2.579
0.814
Avg
0.698
0.730
2.058
0.730
2.397
0.783
2.376
0.767
KRR
1,750
0.763
1.895
0,814
1.667
0.726
1.777
0.765
1.853
0.798
15
1.999
0.857
2.014
0.859
1,939
0.825
2.002
0.855
1,978
0.832
30
2,098
0,888
2.076
0.870
2,100
0.867
2.101
0.888
1.992
0.836
Avg
0.836
1,995
0.848
1.902
0.806
1,960
0.836
0.822
Ausomobile
ExtrTrees
1.692
0.749
2.141
0.811
1.616
0,716
1.770
0.760
1.944
0.775
15
1,990
0.840
2.239
0.858
1,956
0
2,036
0.844
2.274
0.843
30
2.173
0.878
2.315
0.882
2.169
0.858
2.181
0,880
2.352
0.849
Avg
1.952
0.823
2.232
0.850
1.914
0.795
1.996
0.828
2.190
0.822
MLP
1.866
0.835
2.074
0.881
1.717
0.782
1.934
0.846
2.282
0.850
15
2.154
0.910
2.170
0,918
2.314
0.916
2.119
0.901
2.526
0.918
30
2.252
0.933
2.275
0.928
2.578
0.961
2.192
0.922
2.409
0.913
Avg.
2,090
0.892
2.1
0,909
2:203
0.886
2.082
0.889
2,406
0.894
Transformer
2.027
0.873
2.198
0.895
2,006
0.842
2.075
0.880
1,558
0.688
15
1.977
0,862
2.21
0.899
2.091
0.863
1.984
0.860
1.862
0.778
2.325
2.257
0.901
2.153
0.904
2.179
Avg
2,069
0.883
2.245
0.905
2,118
0.870
2.071
0.881
1.867
0.773
KRR
0.187
0.300
0.518
0.556
0.210
0.332
0.530
15
0.474
0.528
0.626
0.623
0.418
0.490
0.452
0.524
0.599
0.611
0.614
0.662
0.553
0.623
0.608
Avg
0.425
0.481
0.602
0.608
0.381
0.458
0.394
0.465
0.579
0.601
Dysts
ExtraTrees
0.226
0.325
0.549
0.541
0.231
0.321
0.200
0.323
0.536
0.549
15
0.421
0.477
0.684
0.622
0.372
0.435
0.432
0.500
0.653
0.613
30
0.566
0.573
0.754
0.662
0.495
0.519
0.584
0.602
0.684
0.631
Avg
0.404
0.458
0.662
0.608
0.366
0.425
0.406
0.475
0.624
0.597
MLP
0.191
0.293
0.499
0.543
0.195
0.299
0.143
0.261
81.408
1.403
15
0.466
0.491
0.622
0.611
0.483
0.478
0.556
0.537
76.533
1.647
30
0.640
0.597
0.664
0.638
0.677
0.583
0,318
0.665
57.319
1.522
Avg
0.460
0.595
0.597
0.452
0.453
0.506
0.488
71.753
1.524
Transformer
0.432
0.458
0.713
0.657
0.391
0.437
0.370
0.435
0.521
0.549
0.526
0.724
0.476
0,533
0.536
0.586
0.586
30
0.666
0.607
0.740
0.675
0.569
0.554
0.717
0.647
0.640
0.622
Avg
0.549
0.531
0.726
0.664
0.474
0.489
0.540
0.539
0.582
0.585
20

---

Table 3: Full results with KSWIN. For each dataset and model family, we report MSE/MAE at prediction horizons (1, 15, 30) and their average. We compare CALIPER (ours) with fixed retraining windows of 128, 512, and 2048 steps ("Fix (+)") and with an online Incremental baseline (no explicit window retraining). The past sequence length is 30. Datasets span four heterogeneous domains (TEP, MoCap, Automobile, Dysts). Lower is better; best per column in bold (second best underlined, if applicable).
Dataset
Model
CALIPER
Fix (128)
Fix (512)
Fix (20-8)
Incremental
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAB
KRR
5.979
1,060
7.726
1.312
10.824
1.592
8.052
1,215
16.003
2.201
15
7.690
1.428
9.842
1.593
12.718
1.978
9.483
1.726
19.552
2.538
30
9,471
1.656
10.855
1,759
14,109
2.154
16.875
2.434
18.946
2.524
Avg
7.713
1.381
9:474
1.554
12.550
1.908
11.470
1.792
18.167
2.421
McCap
ExtraTrees
7.348
1,374
13.249
1.867
10.291
1.750
9,456
1.599
35.622
3.337
15
9.425
1,597
19.238
2.274
13.309
2.018
10.488
1.781
44.145
3.777
30
9.425
1.670
17.741
2.276
14.126
2.060
11.418
L.69
33.087
3.298
Avg
8.732
1.547
16.743
2.139
12.575
1.942
1.750
37.618
MLP
1
3.601
0.995
6.394
1.329
6.46.
1.431
2,389
0.949
357.793
7.636
15
7.292
1.466
9.534
1.72.5
32.887
2.864
9,939
1.956
578.383
9.466
30
11.455
1.848
11.218
18955
50. 112
3.699
22.134
2.921
301.643
8.285
Avg
7.449
1,436
9.049
1.650
29.821
2.665
11.478
1,942
412.606
8.462
Transformer
8.391
1.550
15.355
2.047
15.005
2,210
10,552
1,776
19.661
2.508
15
10.852
1,789
18.536
2.229
17,840
2.407
12.468
1,980
22.874
2.708
30
14,808
2,093
20.485
2.438
20.827
2.634
13.402
2,096
23,700
2.748
Avg
11.350
1.811
18.125
2.238
17,891
2.417
12.141
1.951
22,078
2.654
KRR
1.922
0.687
1.812
0.689
1,951
0.665
2.638
0.749
1.671
0.691
15
2,035
0.728
1.933
0,717
2.145
0.711
2.752
0.789
1.794
0.708
2.147
0.758
2.095
0.232
2,290
0.740
2,863
0.819
1.874
0.718
Avg
2.035
0.724
19960
0,715
2.129
0.705
2.751
0.786
1,780
0.706
Extra Trees
1.625
0.637
1.395
0.636
1,784
0.654
2.345
0.735
1.314
0.621
15
1.826
0.686
1 1.657
0.680
2,158
0.716
2.763
0.801
1.572
0.663
1.951
0,701
2.377
0.751
3.014
0.842
1.716
Avg
1.817
0.681
1.628
0.672
2,106
0.707
2.708
0.792
1.534
0.654
MLP
1.611
0.657
2.041
0,774
0.628
1,910
0.706
3.136
0.829
15
1.723
0.698
2.090
0.778
1.681
0.688
2.044
0.760
4.180
0.914
30
1.947
0.743
2.253
0.810
2.257
0.760
2.837
0,849
3.781
0.894
Avg
1.760
0.699
2.128
0.787
1.793
0.692
2.264
0.772
3.699
0.879
Transformer
1.712
0.671
1.802
0.704
1.798
0.698
2.229
0.759
2.128
0.705
15
1.752
0.671
2.029
0.217
2,084
0.724
2.301
0.763
2.421
0.781
30
1.959
0.727
2,092
0,742
2:325
0.773
2,660
0.826
2.579
0.814
Avg
0.690
1,974
0.72.1
2.069
0.732
2.397
0.783
2.376
0.767
KRR
1.778
0.765
1.864
0,801
1.678
0.728
1.778
0.765
1.853
0.798
15
2,002
0.855
1.982
0.837
1,934
0.824
2.002
0.855
1,978
0.832
30
2,100
0,888
2.026
0.851
2,099
0.863
2,100
0.888
1.992
0.836
Avg
1,960
0.836
1.957
0.829
1.904
0.805
1,960
0.836
0.822
Ausomobile
ExtrTrees
1,770
0.759
2.074
0.793
1.626
0.713
1.777
0.760
1.944
0.775
15
2,039
0.846
2.306
0.860
1,944
0.804
2,038
2.274
30
2.180
0.880
2,364
0.883
2.173
0.855
2.182
0,880
2.352
0.849
Avg
1.996
0.828
2.248
0.845
1.914
0.791
1.999
0.828
2.190
0.822
MLP
1.934
0.845
2.014
0.862
1.685
0.771
1.935
0.846
2.282
0.850
15
2.118
0.501
2,096
0.896
2.423
0.914
2.119
0.901
2.526
0.918
30
2.191
0.922
2.215
0.916
2.690
0.962
2.191
0.922
2.409
0.913
Avg
2,081
0.889
2,109
0.891
2.266
0.882
2.081
0.889
2,406
0.894
Transformer
2,073
0.879
2:202
0.880
1,990
0.832
2.073
0.879
1,558
0.688
15
1.985
0,860
2. 166
0.877
2.059
0.858
1.985
0.860
1.862
0.778
2.153
0.905
2.245
0.896
2.214
0.893
0.905
2.179
Avg
2,070
0.881
2.204
0.884
2,088
0.863
2.070
0.881
1.867
0.773
KRR
0.197
0.311
0.564
0.585
0.217
0.334
0.245
0.530
15
0.449
0.514
0.656
0.640
0.417
0.491
0.439
0.516
0.599
0.611
0.560
0.585
0.670
0.508
0.552
0.614
Avg
0.402
0.470
0.630
0.625
0.381
0.459
0.382
0.458
0.579
0.601
Dysts
ExtraTrees
0.227
0.320
0.587
0.562
0.217
0.312
0.196
0.319
0.536
0.549
15
0.397
0.456
0.723
0.641
0.351
0.423
0.423
0.495
0.653
0.613
30
0.520
0.543
0.757
0.668
0.474
0.507
0.570
0.594
0.684
0.631
Avg
0
0.440
0.689
0.624
0.347
0.414
0.396
0.469
0.624
0.597
MLP
0.198
0.299
0.524
0.563
0.188
0.293
0.135
0.252
81.408
1.403
15
0.523
0.506
0.633
0.620
0.456
0.470
0.530
0.522
76.533
1.647
30
0.680
0.600
0.653
0.638
0.633
0.571
0.785
0.654
57.319
1.522
Avg
0.467
0.603
0.607
0.426
0.444
0.483
0.476
71.753
1.524
Transformer
0.394
0.439
0.759
0.683
0.382
0.430
0.358
0.426
0.521
0.549
0.496
0.497
0.745
0.673
0.465
0.514
0.522
0.586
0.586
30
0.614
0,580
0.757
0.687
0.543
0.541
0.693
0.631
0.640
0.622
Avg
0.501
0.505
0.754
0.681
0.455
0.479
0.522
0.526
0.582
0.585
21


Reviews:
[{"review_id": "wuX6eItFHt", "reviewer": "Reviewer_JXVC", "summary": "This paper focuses on handling the sudden drift in streaming data and tries to explore when to retrain after drift. A method called CALIPER has been developed for detecting concept drift occurrence and stable retraining. And a theoretical analysis of the proposed method has been given for fundamental support. The experiment on several datasets and benchmarks has been conducted, and the experiment results show the performance of the proposed method.", "questions": "1. The author should clarify the difference between the proposed method with traditional concept drift learning methods to enhance the novelty.\n\n2. The experiment should not only focus on MLP and Transformer, but also should focus on tree-based model, which performs excellently in concept drift learning.\n\n3. More comparison on the recently proposed concept drift detection method is required.", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. The idea of \"when to retrain after drift\" is interesting, the previous works usually retrain directly when drift is detected. This work focuses on identifying the right time to retrain the model to help enhance the learning stability.\n\n2. The proposed method is well designed with a detailed theoretical analysis, and the experiment is sufficient and reflects the learning performance of the proposed method.", "weaknesses": "1. In the experiment, models like MLP and transformer have been chosen for comparison. I think tree-based models should also be chosen for comparison, since they are commonly used in traditional concept drift learning.\n\n2. The author only compares the proposed method with ADWIN, which is a traditional drift detection method, more comparisons with recently proposed drift detection methods are required.\n\n3. A parameter analysis is needed to show the robustness of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QohygaGUI4", "reviewer": "Reviewer_U28n", "summary": "The paper proposes a method for determining the right time to retrain/adapt a model after concept drift has occurred. The proposed method is computational efficient because it only uses the data from the data stream together with some hyperparameters.", "questions": "See weaknesses", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Strengths\n- Clear contribution with potentially high impact, well-grounded in the literature\n- Formal analysis of the proposed algorithm\n- Overall, the paper is well structured. Mostly easy to read, with some exceptions (see \"Weaknesses\" for suggestions on how to improve it)", "weaknesses": "The state dependence looks like a pretty strong assumption, as it assumes continuity of the state transition function. I am not sure if this is safe to assume. In particular, in high-dimensional settings. This is also reflected in Section 2.3. I miss a critical discussion on those assumptions, apart from Appendix C. Maybe one could also run experiments on datasets that differ in their dimensionality to get a better feeling for how the method performs in such cases.\n\nProblem 1: I do not understand why minimising the generalisation loss links to the data-side stopping criterion R. If I understand it correctly, they are supposed to model the same thing -- however, the generalisation error is difficult to compute in an online approach, right? If so, it is just a change of notation. I suggest clarifying this to avoid any potential confusion on the reader's side.\n\nLine 216: If I understood it correctly, the following steps have to be executed for every theta in the locality grid, right? If so, please state this explicitly!\n\nWhat is the reasoning behind line 223 \"Proceed only if ...\"? Do you want to ensure that the data set contains enough samples (see statement in line 238)? I suggest adding a short clarification. In general, it is important to ensure that all steps in the algorithm, the reader understands why certain things are done.\n\nA few comments on the GitHub:\n- Please provide more information in the README, in particular, describing the outline of the repository (i.e., where to find what).\n- Please put more comments and docstrings in the code\n\nMinor:\n- Acronym WLR in Figure 1.c not introduced -- becomes obvious later in the paper. However, I recommend introducing it when it is first used.\n- Problem 1: While I am familiar with epsilon-delta notions, it might be good to add a brief explanation to make the notation more accessible.\n- I suggest moving the proof sketches into the appendix -- they kind of interrupt the reading flow", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eNjd7SQMz6", "reviewer": "Reviewer_NZ6X", "summary": "This paper addresses the problem of determining when sufficient data is available to safely retrain a model after a sudden concept drift. The authors propose CALIPER, a model-agnostic and data-only test to estimate this required post-drift data size. The core idea is grounded in the concept of \"state dependence\" in dynamical systems. CALIPER employs a lightweight weighted local regression (WLR) to probe the local predictability of the post-drift data window. A retraining trigger is issued when the WLR's prediction error exhibits a monotonically non-increasing trend as the locality parameter increases, conditioned on a sufficient effective sample size (ESS). The authors provide theoretical analysis linking this trigger to state dependence and learnability, and empirical results across four datasets and three model families show that CALIPER outperforms fixed-window and incremental update strategies.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper formalizes the problem of \"post-drift data sufficiency,\" skillfully identifying the gap between drift detection and effective model adaptation. Focusing on when to retrain, rather than just if a drift occurred, is a profound and highly practical contribution to the streaming learning community.\n\n2. The core idea of leveraging state dependence—an intrinsic data property—to infer learnability is interesting. It reframes a complex model-dependent question into a simple data-driven test. Using a lightweight WLR as a proxy is an efficient and well-justified choice for streaming environments.\n\n3. The paper provides a theoretical foundation and well-designed experiment.", "weaknesses": "1. This method hinges on the assumption that the data stream is generated by a dynamical system of the form x(t+1) = f(x(t)) + noise, This assumption may not hold in many complex dynamical systems where the next state x(t+1) depends on an extended history of past states (x(t-k), ..., x(t)) [1, 2] or is affected by significant external latent factors. The paper does not provide an analysis of its robustness when this core assumption is violated, thus restricting the method's general applicability.\n    [1] Learning robust spectral dynamics for temporal domain generalization, arXiv preprint arXiv:2505.12585\n    [2] Continuous Temporal Domain Generalization, NeurIPS 2024.\n\n2. While the method is model-agnostic, it implicitly assumes that data sufficiency is a property independent of the downstream model. Intuitively, however, models with varying complexity and characteristics (e.g., a linear model vs. a deep Transformer) will need different amount of samples for stable converge.  The paper does not provide an analysis of how CALIPER's estimate correlates with the true convergence points of different models, nor does it discuss how it mitigates the influence of these model-dependent data requirements.\n\n3. The method relies on distance-based neighborhoods, which are notoriously unreliable in high-dimensional settings due to the curse of dimensionality. This could impair key components: the ESS check may require an impractically large window to meet the C(d+1) threshold, and the performance of WLR itself degrades. The experiments are conducted on relatively low-dimensional data, leaving its high-dimensional scalability unverified.\n\n4. The post-drift window X_t is re-normalized at every time step. This dynamic scaling could itself introduce non-stationarity or mask the underlying data dynamics, especially in the presence of outliers. A more stable streaming normalization scheme might be more appropriate.\n\n5. The ESS check is performed only at the tightest locality (θ_max), but ESS is a function of θ. This single check does not guarantee sufficient samples across the entire range of θ values tested for monotonicity, creating a potential logical inconsistency.\n\n6. The proof sketches for Propositions 1 and 2 rely on high-level intuitive arguments (e.g., \"if many pairs... a decrease could not persist\") rather than formal mathematical derivations. For instance, the link between the probability of violating state dependence and the expected change in localized error needs to be made explicit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)