Error-quantified Conformal Inference for Time Series
Authors: Junxi Wu
Dongjian Hu
Yajie Bao
Shu-Tao Xia
Changliang Zou
Venue: ICLR 2025 Poster (2025)
Keywords: Time Series; Uncertainty Quantification; Conformal Prediction; Distribution Shift
Primary area: learning on time series and dynamical systems
TL;DR: We propose a new online conformal inference method ECI by quantifying the extent of under/over coverage, which can react quickly to distribution shift in time series.

Abstract: Uncertainty quantification in time series prediction is challenging due to the temporal dependence and distribution shift on sequential data. Conformal prediction provides a pivotal and flexible instrument for assessing the uncertainty of machine learning models through prediction sets. Recently, a series of online conformal inference methods updated thresholds of prediction sets by performing online gradient descent on a sequence of quantile loss functions. A drawback of such methods is that they only use the information of revealed non-conformity scores via miscoverage indicators but ignore error quantification, namely the distance between the non-conformity score and the current threshold. To accurately leverage the dynamic of miscoverage error, we propose Error-quantified Conformal Inference (ECI) by smoothing the quantile loss function. ECI introduces a continuous and adaptive feedback scale with the miscoverage error, rather than simple binary feedback in existing methods. We establish a long-term coverage guarantee for ECI under arbitrary dependence and distribution shift. The extensive experimental results show that ECI can achieve valid miscoverage control and output tighter prediction sets than other baselines.

Published as a conference paper at ICLR 2025
ERROR-QUANTIFIED CONFORMAL INFERENCE FOR TIME SERIES
Junxi Wul Dongjian Hul; Yajie Bao't Shu-Tao Xia
Changliang Zou'
School of Statistics and Data Science, LPMC, KLMDASR and LEBPS, Nankai University 2 Tsinghua Shenzhen International Graduate School, Tsinghua University {wujunxi, hudongj jian @mmall nankai edu. [yajiebao, zoucl @nankai............... edu.cn. en, xiast@sz. beast@sz.tsinghute tsinnguu.. edu. on
ABSTRACT
Uncertainty quantification in time series prediction is challenging due to the tem- poral dependence and distribution shift on sequential data. Conformal inference provides a pivotal and flexible instrument for assessing the uncertainty of ma- chine learning models through prediction sets. Recently, a series of online confor- mal inference methods updated thresholds of prediction sets by performing online gradient descent on sequence of quantile loss functions. A drawback of such methods is that they only use the information of revealed non-conformity scores via miscoverage indicators but ignore error quantification, namely the distance be- tween the non-conformity score and the current threshold. To accurately leverage the dynamic of miscoverage error, we propose Error-quantifeed Conformal Infer- ence (ECI) by smoothing the quantile loss function. ECI introduces a continuous and adaptive feedback scale with the miscoverage error, rather than simple binary feedback in existing methods. We establish a long-term coverage guarantee for ECI under arbitrary dependence and distribution shift. The extensive experimen- tal results show that ECI can achieve valid miscoverage control and output tighter prediction sets than other baselines.
INTRODUCTION
Uncertainty quantification for time series is crucial across various domains including finance, cli- mate science, epidemiology, energy, supply chains, and macroeconomies, etc, especially in high- stakes areas. To achieve this goal, an ideal model is supposed to consistently produce prediction sets that are well-calibrated, meaning that over time, the proportion of sets containing true labels should align closely with the intended confidence level. Classic methods for uncertainty quantification often rely on strict parametric assumptions of time-series models like autoregressive and moving average (ARMA) models (Brockwell & Davis, 1991). Other methods like Bayesian recurrent neural net- works (Fortunato al. (017) and deep Gaussian processes (Li al. (020) are difficult to calibrate by themselves. And quantile regression models (Gasthaus et al. 2019) may "overfit" when esti- mating uncertainty. Additionally, complex machine learning models such as transformer (Vaswani et al. 2017] Li et al., [209 Gao et al., 2024) have been designed to output accurate predictions but cannot provide valid prediction sets. Hence, a systematic tool is required to perform uncertainty quantification for complex black-box models in time series data.
Conformal inference (Vovk et al.) (005) an increasingly popular framework for uncertainty quan- tification with arbitrary underlying point predictors (whether statistical, machine or deep learning). At its core, conformal prediction sets are guaranteed to contain the true label with a specified prob- ability, under solely the assumption that the data is exchangeable. This is achieved without making parametric assumptions about the underlying data distribution, thereby enhancing its applicability across a wide range of models and datasets. However, in time series data, exchangeability does not hold due to strong correlations and potential distribution shifts.
Equaal contribution. Corresponding authors

---

Published as a conference paper at ICLR 2025
Recently, growing body of research has focused developing online conformal methods for scenarios where data arrives sequentiall.. One of the most important branches is the Adaptive Con- formal Inference (ACI) proposed by Gibbs & Candes (202I), and its following works (Zaffran et al. 2022 Bhatnagar et al. 2023 Gibbs & Candès) 2024 At each time step, these methods generate a prediction set characterized by single threshold or confidence parameter that regulates the size of the set, for example C. {y y v) S gr), where S, is the non-conformity score function. After Y; is observed, they update the threshold gi through indicator [( & Cr}, which is identical to 1 (X,, Yi) > gt However, simple binary feedback cannot precisely capture the magnitude of error (x(X,, Y,) Qt. quantifying the extent of under/over coverage of Òs. For ex- ample, in the miscoverage case, an empty prediction set and a prediction set that almost covers the true label yield the same feedback value in ACI and its variants. Hence it will take longer time to correct past mistakes, see the blue curve in Figure
Coverage
100%
Coverage
75%
090
10
1000
500
090
2004
2010
2012
2014
2014
2014
Time
Figure 1: Comparison results between OGD (online (sub)gradide descent) and ECI on Google stock dataset with Prophet model. OGD uses the same feedback as ACI but updates the confidence level. The coverage averaged over a rolling window of 50 points.
In this paper, we propose Error-quantified Conformal Inference (ECI) based on an adaptive error quantification (EQ) term that provides additional smooth feedback. ECI not only uses the miscover- age indicator in the feedback, but also introduces a continuous EQ function to assess the magnitude of revealed error S,(X, Yt) - qt- Benefiting from the EQ term, our online conformal procedure will react quickly to distribution shifts in time series, see Figure | This leads to tighter prediction sets without sacrificing the miscoverage rate. We summarize our main contributions as follows:
We propose ECI for uncertainty quantification in time series. It is novel method based on adaptive updates with additional smooth feedback, quantifying the extent under/over coverage. We further propose two variants of ECI. The first introduces cutoff threshold for the EQ term to avoid over-compensation caused by small errors. Another variant integrates the error of previous steps make coverage more stable.
There mainly two theoretical results. Firstly, obtain coverage guarantee for ECI with a fixed learning rate and not restricted to long-term. We prove it by showing that every miscoverage step will be followed by several coverage steps given a proper learning rate. Secondly, for arbitrary learning rates, we give a finite-sample upper bound for the averaged miscoverage error. Both theoretical results do need any assumption on the data-generating distribution.
Extensive experimental results demonstrate that ECI and its variants provide superior per- formance in time series, including data finance, energy, and climate domains. We show that ECI maintains coverage at the target level and obtains tighter prediction sets than other sate-offthe methods.
TTo distinguish from the miscoverage error (S, V2) > q:) a, refer "error" to the term Se(X, Ya) de in the rest of our paper.
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND AND PROBLEM SETUP
2.1 CONFORMAL INFERENCE
Let f: : + Y be prediction model trained on an independent training set. Given labeled data {(X;, Ki))inn C X x y and test data X,+1 E X, the objective is to construct a confidence set for the unknown label Y.+1- To determine whether the candidate value y is reasonable estimate of Yn+1. we define a non-conformity score function 5(.,-) : X x y - R. For example, in the regression task, we may take absolute residual score, scaled residual score (Lei et al. 2018). and conformalized quantile regression score (Romano et al. 2019). Generally, S(X;, Y,) depends on the base model ff and measures how well the prediction value f(X,) conforms the true label Y, Given the nominal level a E (0, 1). split conformal prediction (Papadopoulos et al. 2002 Lei al., 2018) outputs the prediction set C(X (()++1) = (y E J:: 5()))19)) S 8), where the threshold is the (n + 1 smallest value among (5(X,, Y))= If (X,,, ) are exchangeable, we have the finite-sample coverage guarantee [[ E C((x++)) 2 at.
In practice, split conformal prediction divides the labeled data into training set for fitting the predictive model and a calibration set for computing non-conformity scores. There are other variants of conformal inference to better utilize the labeled data, like full conformal (Vovk al.) 2005). Jackknife+ (Barber et al. 202T). In many scenarios, data may exhibit a distribution shift and thus are no longer exchangeable, Chernozhukov et al. (2018) extend conformal inference ergodic cases with dependent data, but needs transformations of data to be strong mixing series. Oliveira et al. 2024) study split conformal prediction for non-exchangeable data relying on some distributional assumptions. We refer to Tibbshirani et al. (2019), Pookkopaev & Ramdas (202T), (Barber et al. (2023) and Yang et al. (2024a) for more development dealing with non-exchangeable data.
2.2 CONFORMAL INFERENCE FOR SEQUENTIAL DATA
Recently, significant efforts have been made extend conformal inference online schemes. Gibbs & Candès (2021) proposed ACI which models the distribution shift in time series as a learning prob- lem in a single parameter whose optimal value varies over time. Based on ACI, several works (Gibbs
& Candès 2024; Zaffran et al.) 2022] Bhatnagar et al. 2023 Podkopaev et al. [024]] Angelopou-
los et al. 2024] Yang et al., 2024b) used online learning techniques to adaptively adjust the size of the prediction set based recent observations. Gibbs & Candès (024) and Bhatnagar et al. (023) utilized meta-learning approaches to aggregate the results updated with multiple learning rates experts, where main algorithms were adapted from Gradu et al. (2023) and et al. (2017) respectively. In addition, |Podkopaev et al. (2024) extended the betting technique in Orabona & Pál (2016) to the conformal setting and proposed new parameter-free algorithm. The methods mentioned above can achieve a long- term coverage guarantee without any assumptions about the data-generating process. Xu & Xie [2021: (202)) casted the problem of constructing conformal prediction set as predicting the quantile of a future residual and propose algorithms to adaptively re- estimate (conditional) quantiles. However, these methods are constrained by model and distribution assumptions and potentially suffer from over-fitting problems. |Weinstein & Ramdas (2020) (2020) and Bao "eeittes.Weinstein
ctt al. 2024) investigated online selective conformal inference problem for i.i.d. data stream.
Closely related to our work is the Conformal PID algorithm of Angelopoulos et al. (2023b), which simplifies and strengthens existing analyses in online conformal inference with ideas from control theory. Our algorithm differs from theirs by replacing the integration and scorecasting with the EQ term, and is thus able to yield significantly tighter prediction sets with valid coverage. A more detailed deseription of the competing methods is included in the Appendix C
2.3 PROBLEM SETUP
Suppose the time series data {(X,, Y2) CX'x 3 are collected sequentially. At time I, our goal
construct a prediction set Cx,)) for the unseen label Y, based on the machine learning model trained on previously observed data {(X,, Y.) Aligned with the standard conformal inference methods, we use non- conformiot score function Sc(-,-) X x y + IR that may change over time. For example, in the regression task, S, y) = ly - f((x)) with the base model fe trained at time t. Then we construct the conformal prediction by
C,(X,) {u y (.(,,,)) a).
3

---

Gr. which means that Published as a conference paper at ICLR 2025
where qi is the threshold that estimates approximates (1 a--th quantile for the distribution of the non-conformity score Se(X,, Yi).
Note that if the data sequence {(X,, Ye) are ii.d. exchangeable, according to the split conformal prediction, taking 9t in Equation (I) to be the (1 a1+++ 1-th sample quantile of
(X, Yo) yields a valid coverage guarantee F(Y(+1 E C(+1(X(+1)) 2 1 a. However, in an adversarial setting where exchangeability does not hold, such as time series data with strong correlations or distribution shifts, it is very difficult to achieve such real-time coverage guarantee. Recent works (Bhatnagar et al. 2023] Angelopoulos et al. [203a] 2024, Podkopaev al. 2024)
began to consider the following long-term miscoverage control:
lim T-000
Σ 142, & C(X,)) = t=1
(2)
T
Hence, our primary target is to design an online algorithm to dynamically choose thresholds (#th21 that can achieve the control in without any distributional assumptions about data.
3
ERROR-QUANTIFIED CONFORMAL INFERENCE
3.1 ERROR QUANTIFICATION VIA SMOOTHING FEEDBACK
The online conformal inference framework developed by Gibbs & Candès (2021) adopts the online
learning technique to learn the thresholds {gr based on previously observed data. Let 8t S(X,, X)) and (g)) = (st {1 > + - a) denotes the (1 a a)-th quantile loss. When the
label Y, revealed, we can perform online (sub)gradient descent (OGD) as
4t+1 = - (q)) = q: + n a):
(3)
where err, - 1(s, > ] = 1 & Cx.))) is the miscoverage indicator and is the learning rate. The subgradient err, not cover Y, (i.e., err, = 1), OGD will increase the threshold to construct a more liberal prediction set in the next step; otherwise, OGD will construct more conservative prediction set. In fact, ACI uses the same idea but updates the confidence level of the prediction set instead. Notice that the enrolled average of history feedback Σi= (err, a)/T 'T' is exactly the averaged miscoverage we aim to control in Equation ( Essentially, the long-term coverage guarantee of OGD or ACI comes from the equivalence between feedback and control.
be regarded as the feedback after observing label Y; if Cx(X,) does
However, subgradients of quantile loss l, can only take two values a and -a, regarding the feedback value of coverage (s; S qi) and miscoverage (s; qu) respectively. As consequence, the feedback keeps the same value no matter how severe the miscoverage is or how conservative the coverage is. In other words, discrete feedback value does not exploit the information of the error.
To address this issue, we consider smoothing the feedback when updating the thresholds. Let f (x) e [0, be a smooth approximation to the indicator 1(x 0), such as Sigmoid function, Gaussian error function, etc. Then we can approximate the quantile loss (g)) via (,(4) (f(ss - -a) g) This smoothing technique was demonstrated in previous studies (Kaplan & Sun| 2017 Fernandes et al. 2021 Applying OGD on the smoothed quantile loss (g) we have the following fully smoothed update rule
g++1 = q: V qi -
- qi) - a qt Vf((s q:)].
Here we called (s; - qr) fss -9t) ) as the error quantification (EQ) term, which provides additional feedback based on the magnitude error $t qt-
To have preliminary view of the effect of EQ term, we take Sigmoid function f(x) (- as an example, and show the curve of EQ function x fxx in Figure 2 Firstly, the feedback from the EQ term has the same sign as subgradient err, a by letting it does not deviate from the direction of update in OGD. On the other hand, when s significantly deviates from the current threshold q (for example, EQ term tends to decrease as 8, q: grows. This is to prevent the subsequent prediction sets from running out of control due to single anomaly data point.
abrupt change point appears at time t). the
4

---

Published as a conference paper at ICLR 2025
XWM
Runction Valuee
-0.3
0.2
Figure 2: Dynamics of the EQ function across variable æ, where f(x) is Sigmoid function and c = 1.
However, Equation may introduce extra bias during the derivation process, as detailed in Ap- pendix A.I If we perform the fully smoothed update rule (4) we cannot directly control the aver- aged miscoverage gap Σt= orn//T - a due to the smoothing bias.
Hence keep the EQ term and replace f(; 9t) in Equation (4) with the miscoverage indicator eIT,, that is
G++1 = qr + [err, - (s+ Øo [s( 2.)
(5)
Formally, we propose Error-quantije Conformal Inference (ECT) based on (5). which leverages the miscoverage indicator and the EQ feedback. Compared with OGD in Equation (3), the feedback from EQ term is compensation over the original subgradient err; -a. Clearly, for the same pair (st, 2t). ECI can provide more feedback
simultaneously update thresholds with partially smoothed
compared with OGD and fully smoothed rule in Equation (4) We provide an ablation study in Appendix A.2 to illustrate the importance of the EQ term.
3.2 EXTENDED VERSIONS
ECI-cutoff. The EQ term reacts quickly to sudden distribution shifts like changepoints. How- ever, when q slightly larger/smaller than st, the additional adjustment provided by the EQ term may lead under/over coverage of prediction sets. For example, if 8, S, and 8 [gt Th + T/ (8t V(s (t), mea.. then adding the EQ term will cause miscoverage, i.. St41 q++1. Therefore, we introduce cutoff to our added term:
9++1 = qt + 1 (erry - a) + (st 4:) Vf(x - 1 gel > nx)
(6)
where h2 -h haaxx-uuijs |s; -syl and h is pre- pre-determined threshold, is window length.
ECI-integral. More information imput generally means better performance. Note that (5) only takes the information of one last timestep into consideration. A natural alternative is to integrate the error of more than single step. Therefore, we propose the extended update:
a++1 qi + Σ wi eir, + (s: 9:) Vf(s: - 91)
where (wihssss C [0. 1] is sequence of increasing weights with Σ -i=1 wi 1. Equation evaluates the recent empirical miscoverage frequency and degree of miscovery when deciding whether or not lower raise q:, making coverage more stable.
3.3 DISTRIBUTION-FRE COVERAGE GUARANTEES
In this section, we outline the theoretical coverage guarantees of ECI. The detailed proofs are re- ferred in Appendix B Assumption 1. For any t N4, there exists B such that s; [0, B). Assumption 2. |x [x)) < and V (x) < e cffor any & E R. where X, >> 0 are constants.
We first present two assumptions and briefly explain their feasibility.
Assumption [ assumes boundedness of scores, which is ubiquitous in online conformal literature
(Gibss & Candès 2021 Angelopoulos et al. 2023b 2024). For Assumption 2 note that a typical
choice of J is the Sigmoid function acrr with scale parameter 0. Then
[f - (((x))))) c
5

---

Published as a conference paper at ICLR 2025
e
2 [V | (cx( (cc))) < lex 1+ e
crre
Thus Assumption 2 is naturally satisfied. Moreover, if we consider ECI-cutoff and replace Vf(x) by (x) - V ( > h). then
[(((a)] c
> h))
1 + gch
which can be sufficiently small as long we set to be large.
The following theorem provides dynamic miscoverage bound when the learning rate is fixed. We prove by showing that every miscoverage step will followed by least N coverage step, where = LL that is if Y, & C, happens, then Y+++ E C.+i holds for = 1,2, ... ,N
min(n.N2) Theorem 1. Assume that > 2N c < 2N-(B+(1-o+11p) where N = 2 Under Assumptions
and 2 the prediction sets generated by (5) satisfies for any T,
T+N
1
11 E Ci) $ N
(8)
As standard choices for a are {0.01. 0.05, 0.1, if we further assume a = N-1 for some N e N, it follows from () that 217++ 1(Y C.}- S for any T, which immediately yields the long-term coverage guarantee in Equation (
We present the bound of averaged miscoverage error under adaptive learning rates. The result is also distribution--ree and only needs a proper choice of learning rate sequence.
Theorem 2. Let {nt her be an arbitrary positive sequence. Under Assumptions and the pre- diction sets generated by ( with adaptive learning rate or satisfies:
1 T
Σ er a))
(B + Mr-1)lAAril +c[B + (1 1)MT-1),
(9)
T
where ||A1:
bn +T Ine
n, n- 1. My maxigrs7 ?/--
For the first term. following the analysis in Angelopoulos et al. (2024), if the learning rate decreases over extended periods when the distribution appears stable, but then increases again, in a repetitive manner, | | / can be sufficiently small. To be specific, ||1i S 2N,/ (nte:/iingggi We), where Nr ΣT [ me} denotes the number of times the learning rate is increased. Hence, does not decay too quickly and the number of "resets" Ny is o(T). the first term in the upper bound in (2) will be within an acceptable range. For the second term, if we set e as sufficiently small value, then the second term is also limited to small value.
EXPERIMENTS
4.1 SETUP
Datasets. We evaluate four real-world datasets: Amazon stock, Google stock (Nguyen 2018), elec- tricity demand (Harries et al. 1999) and temperature in Delhi (Vrao. 2017). Besides, we evaluate the synthetic dataset under changepoint setting. In the subsequent sections, we will provide de- tailed introduction to each of these datasets.
Base predictors. We consider three diverse types of base predictors.
Prophet (Taylor & Letham 2018); As Bayesian additive model, Prophet predicts the value Y, function of time expressed Y, g(t) s(t) + h(t) + cs. where g(t) models the overall trend, stt) accounts for periodic seasonal effects. h(t) captures holiday effects, and e, represents the noise assumed to follow normal distribution.

---

Published as a conference paper at ICLR 2025
AR (AutoRegressive Model); As classic model, AR is defined as Y; - ... ...to, (0 + et, where Φ1, 02 .... Up are the parameters, and p is the order of AR model. Following Angelopoulos et al. (2023b),
set 3.
Theta (Assimakopoulos & Nikolopoulos, 2000); As decomposition- based forecasting approach, Theta model modity the curvature of time series by applying coefficient
to its second differences. We typically use 0 = O (the long-term trend) and 0 = 2 (the short-term dynamics) to decompose the original series into two components.
Baselines, We compare with four state-of-the-art methods: ACI (Gibbs & Candès 2021), OGD, SE- OGD (Bhatnagar et al.] 2023), decay-OGI (Angelopoulos et al. (2024), PID (Angelopoulos et al.) 2023b). A detailed description of existing methods can be found in Appendix C
General implement. We choose target coverage 1 a = 90% and construct asymmetric prediction sets using two-side quantile scores under a/2 respectively. For each baseline, we select the most appropriate range of learning rates for the respective datasets and present the best results in the tables. For sets, all baselines will output asymmetric sets [Y; - g/. Y, + qid with upper score qi" and lower score of under half of the coverage level a/2 respectively.
Sigmoid function and C 1. 5t} min((s-w4t1..... 8t)). For ECI-integral, we set weights w; = Σ 0.95* 7 for S S I. Specifically, PID, ECI, and its variants use adaptive learning rates 7/s = 1y. (max(st- ( } 100mmmineee (iin }) where is window length. Unless necessary changes are made, the settings for all basclines adhere the original papers and open-source codes.
Other implement. For EQ term, we set f(x) = 1+oxp(-ar))
1
For ECI-cutoff, we set h and he 0.95
(max
Overview of experimental results. We have conducted extensive experiments, including stock data in Section 4.2 electricity data in Section 4.3] climate data in Section 4.4 syn-
thetic data in Section 4.5 and ablation study of hyperparameters in Appendix D Experimen-
tal results with Transformer
the base model can be found in Appendix
More com-
prehensive experimental results and discussion on the scorecaster and learning rates can be
found in Appendix G Our code is available at /ttppss////sstto..oo/rrrtttttrrr comp://et///rrrro
Error quaatiffien -oonoooaal--neeeeeeeee
4.2 RESULTS IN FINANCE DOMAIN
We consider the uncertainty problem of forecasting stock prices, including Amazon (AMZN) and Google (GOOGL), collected over 9 years (from January 1, 2006 to December 31, 2014). Models will forecast the daily opening price of each of Amazon and Google stock on a log scale.
The quantitative results are shown in Table D and Table 2 For ACI, the occurrence of infinite sets is too frequent, due to updating ag and adopting the a₃ quauuiiee of past scores as q.- This implies that ACI may tend to conservatively expand the prediction sets in the face of more complex or volatile data to ensure high coverage rates. However, such a strategy is not always ideal in practical applications, as overly broad sets reduce the precision and utility of the predictions. For OGD and SF-OGD, they achieve a good balance between coverage rate and set width to some extent. However, their performance overly relies on the selection of learning rate and may fail under many learning rate settings. For PID, its scorecasting can be seen as helping compensate for the base predictors' predictive accuracy. Thus, it can lead to improvements in the worse base predictor case (such as Prophet). However, in the case of better base predictors, it will instead widen the length of the prediction set.
As for ECI, its performance is outstanding, especially in the control of set width. Compared to exist- ing methods, ECI can provide shorter and more accurate prediction sets with little loss of coverage. ECI-cutoff extends ECI by introducing truncation threshold to further reduce the redundancy of prediction sets. The experimental data also show that ECI-cutoff achieves the shortest set width of all methods in general. ECI-integral extends ECI by integrating the information from longer past time, leading to a better balance between coverage rate and set width.
Figure shows the coverage results Amazon stock dataset with Prophet as the base predictor. Throughout the entire period, the more effectively a method maintains the nominal level (here is 90%), the more valid the method is. We can ACI generally having larger oscillations, and OGD,
7

---

Published as a conference paper at ICLR 2025
Table 1: The experimental results the Amazon stock dataset with nominal level a 10%. The best (shortest) width results are indicated with bold text.
Prophet Model
AR Model
Theta Model
Coverage
Average width
Median
Coverage
Average width
Median
Coverage
Average
Median
Method
( (%)
width
(%)
width
( (6)
width
width
ACT
90.2
DC
46.97
89.8
0
13.77
89.7
00
12.31
OGD
89.6
55.15
30.00
89.9
19.10
15.00
89.8
18.07
14.50
SF-OGD
89.5
61.47
31.75
89.9
24.44
21.05
90.0
23.88
21.14
decay-OGD
89.9
97.22
36.20
89.7
20.23
14,01
89.2
17.49
13.46
PID
89.8
52.56
39.09
89.6
59.22
37.93
61.19
40.20
ECT
90.1
47.00
34.84
89.5
17.12
12.73
89.7
17.46
12,49
ECI-cutoff
89.7
43.46
29.98
89.3
16,91
12.63
89.6
17.19
12.48
BCI-integral
89.8
42.01
30.02
89.5
16.99
89.6
17.20
12.46
Table 2: The experimental results in the Google stock dataset with nominal level a 10%.
Prophet Model
Model
Theta Model
Coverage
Average width
Median
Coverage (%)
Average width
Median width
Coverage
Average width
Median
Method
(%)
width
(%)
width
ACI
90.0
00
66.83
89.8
00
18.64
90.5
32.78
OGD
89.7
57.60
46.00
90.7
33.76
23.00
89.9
31,49
29,50
SF-OGD
89.6
58.92
47.78
89.9
28.31
24.42
90.0
34.04
31.48
decay-OGD
89.9
77.23
50.18
90.2
46.53
26.77
90.2
55.32
33.71
PID
90.1
57.47
48.44
89.9
64.88
54.07
89.9
63.58
54.05
ECI
89.9
56.06
46.96
89.7
19.95
17.19
89.6
30.92
29.53
ECI-cutoff
89.8
53.12
44.36
89.7
19.84
17.63
89.6
30.71
28.11
ECI-integral
89.8
52.36
43.28
89.7
19.93
17.31
89.6
30.42
28.02
and SF-OGD become increasingly oscillatory over time. Figure 4 compares the prediction sets of the variants of ECI with those other methods. Consistent with the quantitative results, the variants of ECI have the shortest prediction sets.
Table 3: The experimental results in the electricity demand dataset with nominal level 10%.
Prophet Model
AR Model
Theta Model
Coverage
Average width
Median width
Coverage (%)
Average width
Median width
Coverage
Average
Median
Method
( (%)
(6)
width
width
ACT
90.1
DC
0.443
90.1
00
0.105
90.2
00
0.055
OGD
89.8
0.433
0.435
90.0
0.133
0.115
0.081
0.075
SF-OGD
89.9
0.419
0.426
90.0
0.129
0.116
90.3
0.106
0.095
decay-OGD
90.1
0.531
0.521
90.1
0.122
0.099
90.0
0.100
0.059
PID
90.1
0,207
0.177
90.0
0.434
0.432
89.9
0.413
0.411
ECT
90.0
0.384
0.382
90.0
0.117
0.098
89.9
0.071
0.055
ECI-cutoff
90.0
0.405
0.396
90.2
0.118
0.096
90.1
0.072
0.055
ECI-integral
90.1
0.402
0.398
90.0
0.117
0.098
89.9
0.072
0.055
4.3 RESULTS IN ENERGY DOMAIN
Then we consider the uncertainty problem of electricity demand. The dataset measures electricity demand in New South Wales, collected at half-hour increments from May 7th, 1996 to December 5th, 1998. All values are normalized [o, 1].
Table Bable shows that ECI and ECI-integral have the shortest prediction sets with AR and Theta model, even maintaining the highest coverage in the AR model. We can note that PID stands out with Prophet model due the scorecaster. In fact, This dataset has collected several other variables, such
8

---

Published as a conference paper at ICLR 2025
ACI Marry
OGD
Coverage
100%
Mr M Am
M
www www
75%
SF-OGD
Conformal PID Control
100%
My /ve
Coverage
75%-
deCay-OGD
ECI
100%
Coverage
73%
50%
ECI-cutoff
ECI-integral
100%
Coverage
75%
50%
2000
2010
2012
2014
2014
2018
2008
2010
2012
2014
2016
2015
Figure 3: Comparison results of coverage rate on Amazon stock dataset with Prophet model.
ACI
OGD
1000
Sets
100
SF-OGD
Conformal PID Control
Sets
decay-OGD
ECI
1000
Sets
500
ECI-cutoff
ECI-integral
1000
2009
2010
2016
2018
2006
2010
2017
2014
2016
2018
Figure 4: Comparison results of prediction sets on Amazon stock dataset with Prophet model.
the demand and price in Victoria, the amount of energy transfer between New South Wales and Victoria, and so on. These are given as covariates to the scorecaster and complement Prophet well. Other methods do not this information.

---

Published as a conference paper at ICLR 2025
4.4 RESULTS IN CLIMATE DOMAIN
Finally, consider the uncertainty problem of climate demand. The dataset measures the daily temperature in the city Delhi over 15 years (from January 2003 to April 24, 2017). Table shows that ECI-cutoff achieves best performance in general.
Table 4: The experimental results in the Delhi temperature dataset with nominal level a 10%.
Prophet Model
AR Model
Theta Model
Coverage
Average width
Median
Coverage (%)
Average width
Median width
Coverage
Average
Median
Method
( (%)
width
(6)
width
width
ACI
91.0
8.49
90.0
0o
6.06
00
6.48
OGD
90.4
7.54
7.60
90.1
6.82
6.10
90.0
6.36
6.30
SF-OGD
90.0
7.17
7.08
90.1
6,37
5.91
90.1
6.75
6.43
decay-OGD
90.1
8.84
8.35
90.0
6,36
5.67
89.9
6,56
6.18
PID
90.1
7.65
7.65
89.7
8.92
8.86
89.7
8.77
8.79
ECI
90.0
7.20
7.22
90.1
6.39
6.10
90.0
6.41
6.27
ECI-cutoff
90.1
7.01
6.96
90.1
6,28
5.97
90.0
6.27
6.17
BCI-integral
90.0
7.30
90.2
6.39
6.11
6.38
4.5 RESULTS IN SYNTHETIC DATA
In this experiment, compare the performance of our method with other basclines under synthetic changepoint setting in Barber et al. (2023). The data {X₁, Y; are generated ac- cording to linear model Y, = X; 3r + Et. X₁
NN E N/ 1). And we set: 3 (0) = (2, 0) for 1....,,.000 500; 3, = 3(0) = (0, -2 -1, 0) for / = 501, 1500; and B. = 3(2) (0, 0, for 1501, ... 2000. up to time 2000.
2000.. And two changes the coefficients happen
compare ECI and variants with some competing methods about coverage and prediction show set width. Table show the result of coverage and set width, while the base predictor is Prophet model. In general, ECI-cutoff and ECI-integral achieve best performance.
Table 5: The experimental results in the synthetic data dataset with nominal level 10%.
Prophet Model
AR Model
Theta Model
Coverage
Average width
Median width
Coverage (%)
Average width
Median width
Coverage
Average width
Median
Method
( (%)
(%)
width
ACI
89.9
8,20
89.9
0
8.20
89.9
00
8.43
OGD
90.0
8.49
8.50
89.9
8.39
8.40
89.9
8.73
8.70
SF-OGD
90.0
12.48
11.56
90.0
12.58
11.69
89.9
12.70
11.88
decay-OGD
90.0
8.30
8.22
90.0
8.26
8.21
90.0
8.57
8.60
PID
89.7
11.02
9.64
89.9
10.83
9.35
89.7
11.23
9.78
ECI
89.9
8.16
8.25
89.9
8.17
8.26
89.8
8.55
8.68
ECI-cutoff
89.8
8.31
8.44
89.9
8.14
8.19
89.8
8.51
8.59
ECI-integral
89.8
8.25
8.37
89.9
8.16
8.23
89.8
8.48
8.58
5 CONCLUSION
Several approaches have recently been introduced for online conformal inference. A significant limitation of these methods their neglect of quantifying extent of over/under coverage. In this work, we propose Error-quantified Conformal Inference (ECI) to construct prediction sets for time series data. Compared with ACI (Gibbs & Candès] 2021) and its variants, ECI introduces additional smooth feedback by measuring the magnitude of eπror tributional shifts in time series, and yield more tight conformal prediction sets. Theoretically, we establish finite-sample coverage guarantee for ECI with fixed learning rate in short interval and prove a miscoverage bound with arbitrary learning rate. Empirically, we verify our method's effectiveness and efficiency across extensive datasets.
4s- ECI can rapidly adapt to the dis-
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
We would like to thank the anonymous reviewers and area chair for their helpful comments. Changliang Zou was supported by the National Key R&D Program of China (Grant Nos. 2022YFA 1003703, 2022YFA 1003800), and the National Natural Science Foundation of China (Grant Nos. 11925106, 12231011, 12326325).
REFERENCES
Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I Jordan, and Tijana Zrnic Prediction-ppwered inference. Science, 1866667)766666666666677 674, 2023a.
Anastasios N. Angelopoulos, Emmanuel Candes, and Ryan J Tibshirani. Conformal pid control for time series prediction. Advances in Neural Information Processing Systems, 36, 2023b.
Anastasios N. Angelopoulos, Rina Barber, and Stephen Bates. Online conformal prediction with decaying step sizes. In Forty-first International Conference on Machine Learning, 2024.
Vassilis Assimakopoulos and Konstantinos Nikolopoulos. The theta model: decomposition ap- proach forecasting. International journal of forecasting, 16(4):521-535 2000.
Yajie Bao, Yuyang Huo, Haojie Ren, and Changliang Zou. CAP: general algorithm for online
selective conformal prediction with FCR control. arXiv preprint arXiv:2403.0777 2024.
Rina Foygel Barber, Emmanuel Candès, Aaditya Ramdas, and Ryan J Tibshirani. Predictive inference with the jackknife. The Annals of Statistics, 49(1):486-507) 2021.
Rina Foygel Barber, Emmanuel Candes, Aaditya Ramdas, and Ryan J Tibshirani. prediction beyond exchangeability. The Annals of Statistics, 51(2):8 816-855, 2023.
Conformal
Aadyot Bhatnagar, Huan Wang. Caiming Xiong, and Yu Bai. Improved online conformal prediction via strongly adaptive online learning. In International Conference on Machine Learning. pp- 2337-2363. PMLR, 2023.
Peter J Brockwell and Richard A Davis. Time series: theory and methods. Springer science & business media, 1991.
Victor Chernozhukov, Kaspar Wüthrich, and Zhu Yinchu. Exact and robust conformal inference methods for predictive machine learning with dependent data. In Conference On Learning Theory, pp. 732-749. PMLR, 2018.
Marcelo Fernandes, Emmanuel Guerre, and Eduardo Horta. Smoothing quantile regressions. Jour- nal of Business & Economic Statistics, 391)3338-337, 2021.
Meire Fortunato, Charles Blundell, and Oriol Vinyals. Bayesian recurrent neural networks, arXiv preprint arXiv:1704.000 774900779. 2017.
Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, and Wei Liu. Induc- ing high energy-latener of large vision-language models with verbose images. The Twelfth International Conference Learning Representations, 2024.
Jan Gasthaus, Konstantinos Benidis, Yuyang Wang, Syama Sundar Rangapuram, David Salinas, Valentin Flunkert, and Tim Januschowski. Probabilistic forecasting with spline quantile function rnns. In The 22nd international conference arlificial intelligence and statistics, pp. 1901-1910. PMLR. 2019.
Isaac Gibbs and Emmanuel Candès. Adaptive conformal inference under distribution shift. Ad- vances in Neural Information Processing Systems, 1111660-1672. 2021.
Isaac Gibbs and Emmanuel J Candès. Conformal inference for online prediction with arbitrary distribution shifts. Journal of Machine Learning Research, 1-36, 2024.
Paula Gradu, Elad Hazan, and Edgar Minasyan. Adaptive regret for control of time- tane-varying dynam-
ics. In Learning for Dynamics and Control Conference, pp. 560-572, PMLR, 2023.
11

---

Published as a conference paper at ICLR 2025
Michael Harries, New South Wales, et al. Splice-2 comparative evaluation: Electricity pricing. 1999.
Kwang-Sung Jun, Francesco Orabona, Stephen Wright, and Rebecca Willett. Improved strongly adaptive online learning using coin betting, In Artificial Intelligence and Statistics, pp. 943-951. PMLR, 2017.
David Kaplan and Yixiao Sun. Smoothed estimating equations for instrumental variables quantile regression. Econometric Theory, 33(1): 105-157, 2017.
Jing Lei, Max G'Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman. Distribution- free predictive inference for regression. Journal of the American Statistical Association, 113 (523):1094-1111, 2018.
Naiqi Li, Wenjic Li, Jifeng Sun, Yinghua Gao, Yong Jiang, and Shu-Tao Xia. Stochastic deep gaussian processes over graphs. Advances in Neural Information Processing Systems, 33:5875- 5886, 2020.
Shiyang Li, Xiaoyong Jin, Yao Xuan, Xiyou Zhou, Wenhu Chen, Yu-Xiang Wang, and Xifeng Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. Advances in Neural Information Processing Systems, 32, 2019.
Cam Nguyen. S&P 500 stock data. Kaggle, 2018.
Roberto Oliveira, Paulo Orenstein, Thiago Ramos, and Joao Vitor Romano. Split conformal pre- diction and non-exchangenneee -xchanncaaeeebee data. Journal of Machine Learning Research, 25(225):1-38, 2024.
Francesco Orabona and Dávid Pál. Coin betting and parameter-free online learning, Advances in Neural Information Processing Systems, 29, 2016.
Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confidence machines for regression. In European Conference Machine Learning, pp. 345-356. New York: Springer, 2002.
Aleksandr Podkopaev and Aaditya Ramdas. Distribution-fiee uncertainty quantification for classi- fication under label shift. In Uncertainty in Artificial Intelligence, pp. 844-853. PMLR, 2021.
Aleksandr Podkopaev, Dong Xu, and Kuang-Chin Lee. Adaptive conformal inference by betting. In International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 40886-40907. PMLR, 2024.
Yaniv Romano, Evan Patterson, and Emmanuel Candès. Conformalized quantile regression. Ad- vances Neural Information Processing Systems, 32:3543-3553, 2019.
Kean Ming Tan, Lan Wang, and Wen-Xin Zhou. High-dimensional quantile regression: Convolution smoothing and concave regularization. Journal of the Royal Statistical Society Series B: Statistical Methodology, 84(1):205-233 2022.
Sean J Taylor and Benjamin Letham. Forecasting at scale. The American Statistician, 72(1):37-45, 2018.
Ryan J Tibshirani, Rina Foygel Barber, Emmanuel Candès, and Aaditya Ramdas. Conformal pre- diction under covariate shift. Advances in Neural Information Processing Systems, 32, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, E ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Infor- mation Processing Systems, volume 30, 2017.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer Science & Business Media, 2005.
Sumanth Vrao. Daily climate time series data. Kaggle, 2017.
Asaf Weinstein and Aaditya Ramdas. Online control of the false coverage rate and false sign rate. In International Conference on Machine Learning, pp. 10193-10202, 2020.
12

---

Published as a conference paper at ICLR 2025
Chen Xu and Yao Xie. Conformal prediction interval for dynamic time-series. In International Conference on Machine Learning, pp. 11559-11569. PMLR, 2021.
Chen Xu and Yao Xie. Conformal prediction for time series, IEEE Transactions on Pattern Analysis and Machine Intelligence,
2023.
Yachong Yang, Arun Kumar Kuchibhotla, and Eric Tchetgen Tchetgen. Doubly robust calibration of prediction sets under covariate shift. Journal of the Royal Statistical Society Series B: Statistical Methodology, pp. qkae009, 2024a.
Zitong Yang. Emmanuel Candès, and Lihua Lei. Bellman conformal inference: Calibrating predic- tion intervals for time series. arXiv preprint arXiv:2402.05203, 2024b.
Margaux Zaffran, Olivier Féron, Yannig Goude, Julie Josse, and Aymeric Dieuleveut. Adaptive conformal predictions for time series. In International Conference on Machine Learning, pp- 25834-25866. PMLR, 2022.
13

---

Published as a conference paper at ICLR 2025
A SMOOTHED WAYS
A.1 EXPLANATIONS FOR EXTRA BIAS
Our target is the quantile loss (g(g) (err, as q). For simplicity, we denote h(g) err, ((g) 8, q. Note that h(g) is non-differentiabl, so we use smoothed function f(4) to approximate h(q). The fully smoothed method in Equation (4 considers:
Vh(g)g(g) V f(g) + v(a)
The two terms V(()) and g (g) both introduce error. Strictly we have
h(g + و)g( + 8) h(g 8)g(4) h(g 8)g(g) h(g)g(4)
Vh(g)a(o) = lim 6-0
h(g + 5)g(4 + 6) - h(g + 5)g(g)
h(g + 5)g(4) h(g)g(g)
lim 6-0
+ lim 6-0
5
h(g + 6) h(g) g(g)
h(g)Vo(9)) lim
5
using V approximate
23 -err, - a)
fs: (xt - q).
Hence, if we only approximate the second term, potential bias may be reduced. The fully smoothed method is better only when the error of the two terms are negatively correlated and cancel out.
A.2 FULLY SMOOTHED WAYS
There is another smoothing technique applied in the quantile regression (Fernandes et al. 2021 Tan al. 2022), which directly smooths the inidcator in subgradient. Based on this, we can have the following udpate rule
4++ = q: + 7) [f(s q:) aa.
(10)
To further validate ideas in Section 3.1 method and fully smoothed method, as shown in Figures5Joof All experimental setting are aligned with Section 4
test the experimental performance of smoothed
The update rule of fully smoothed method in Equation is
[(sL- - 90) - a + (st 4f(s - 41)
4+1 = 9t +n
And the update rule of ECI in Equation 5
g++1 qi + 1 err; a (s: fss 9.)
Compared with ECI in Equation (5), fully smoothed updates rules in Equation (4) and Equation (00) do not keep the actual value of indicator function er,, and brings the bias between smooth function f(x. 4:) and 1sx 4.). This leads to these method being conservative.
Experimental results also demonstrate it. Since fx) does not approach quickly in the early part of the period, and may can be seen that smoothed method and fully smoothed method tends to have overly high coverage in the early stages in the Figures 5 to Consequently, it results in wider sets.
be larger than a, the coverage rate not effectively approach
14

---

Published as a conference paper at ICLR 2025
ACI
Conformal RD
smoothed method
fully smoothed method
ECI
1.0
0.8
In:0.1. cvg-90.1%
Insl, cwg.89 9%
Insl, cvge92%
Insl. cvg
0.6
2.0
0.8
Coverage
0.6
[r-0.05, Cvg-90.1%
1r-05, C.g..ccg--B cwg-c9.999 89.9%
Ir-05. ovg -9.-%
Ir-0.5. cwg.co----33 99.88%
1r-0.5. cvg-90.15
3.0
0.8
cvg-90.2%
cvg
cug
1m cup
0.6
2.0
[n:00000 11000005, 099-90.3%
In:0.05, cv9-
Ir:0.05, cvgm889.0%
Ir:0.05, cvgm889.8%
Ir=0.05, cvg.88.25
0.8
2000
3000
1000
2000
3000
1000
2000
3000
1000
2000
3000
3000
2000
3000
Time
Figure 5: Coverage result on Amazon stock dataset with Prophet model.
ACI
Conformal MD yuanttppphhataalllllleeo
smoothed method "vily 11 Tutpoe///ttttyhhnpn...
fully smoothed method Maate
ECI
1.0
0.8
0.6
[mm0.1, 1. cvgm50.0%
Innl, co.o99..0
Inet, c2.792.7%
Inm3, c2g.92.1%
Irm cwg.eessi $0.00%
1.0
0.6
Coverage
In-0 0.000 cxg-90.0N
Inw0.5, nrg-89.77%
In ovg 2og-92.555
In ovg 92
In 0.5, ovgu
1.0
MM
"Wy Mir////ll
0.8
0.5
In-0.02, cxg-89.8%
1r0....cg--88 Cr.-88.6%
1r-0.2. cvg-52.9%
In-0.1. -2.-92.95
1r-0.1.cvg--B2. cVg-89.55
1.0
0.8
0.6
(mm0. 005, evg 89..5
Imm0.05, cxgm87.3%
Imm0.0% cvgm92.7%
In .0, cuge92.8%
Imm0.05 cvgm892%%
3000
3000
Time
Figure 6: Coverage result on Amazon stock dataset with AR model.
B PROOFS OF MAIN RESULTS
We show below that under Assumptions | and 2 there exists a bound for ge and EQ term, depending on learning rate т)t and the upper bound of 8.. This result is essential in proving our following results of coverage guarantces.
B. PROPOSITIONS
Proposition 1. Fix an initial threshold qı E [0, BJ. Then under Assumptions 7 and 2 ECI in 5 with arbitrary nonnegative leurning rate T) satisfies that
-(a + A)M,-1 < q. $ B B+ (1 a A)M-11 1
(11)
where Mo 0, and M, maxigrs: 1/r for 2
15

---

Published as a conference paper at ICLR 2025
ACI
Conformal RD
smoothed method "VL Metaa
fully smoothed method Aanneni
ECI
1.0
0.6
cvg=90.0%
Insl, cvgm89.71
Insl. cvg.92.7%
Insl. 92.
Ieel, cvg.30001
1.0
0.8
Coverage
[r-0.05, Cxg-89.9%
1r-055, www.0...gg-... cxg-89.6%
Ir-05. ovg 100-59.99 9%
Ir-0.5. cvg- 99..992.9%
Ir-0.5. cvg-09.99
1.0
"My Wwwe
0.8
0.6
cvg-29.7%
cug
ovg cvg-92.29
1m
1e cvg
2.0
Muble
0.8
[n: 00.000,,,00 8Vg=899.4%
In=0.05, cxg-887.5%
In:0.05, cvg-922.5
In: cvg-92..0%
2000
3000
1000
2000
3000
1000
2000
3000
1000
2000
3000
2000
3000
Time
Figure 7: Coverage result on Amazon stock dataset with Theta model.
Proof. We prove this by induction. First, Q1 E [0, B] by assumption. Next fix any t 2 1 and assume q: lies in the range specified (II. and consider Q-+1-
(1): 2
4++1 = gr + 1h (1 - a) + (s - 4.) Vf(s, 9c)
S qi n (1-a) +nd
S 8: + (1 - a -)m
< B+ (1 -1-a+ M)M
a
and g++1 2 q: 2 - M 2 MMa
(2): 8t S q::
9+11 9: + 17t (-a) + T/( - 4t) Vf(s: 4t)
2 +1| (-a) TTd
2 (a + X)Mr
and ge+1 S (1 a A)
S B + (1 a A)Mr.
Proposition 2. Under Assumptions and 2 [(st qe Vf(s 9:) cB + 1) for any t 2 1.
Proof. Based on st E [0, B] and Proposition
|s qu1 max (g: St, 9:3 S max 8t - qrP
S max B + (1 a A)M, B + (a + A)M
$ B + (1 a M
Hence (ss: - -
-
M)
B.2 PROOF OF THEOREM I
Theorem 1. Assume that r) 2NB, c
where N [2]- Under Assump-
2 (1-a+
tions and 2 the prediction set generated by eq. 5 satisfies:
T
lim T
1(Y & C.J < a.
(12)
T
t=1
16

---

Published as a conference paper at ICLR 2025
Proof. We first prove that for any t, q: implies $t+1 U+++; 1 1,2, ,N 1. Note that (Y, E C+) {|Y YI S qe) is meaningless when & < 0. Hence set gt to be max (qt, 0} after each update which does not affect the validity of our proof and assume 4t 2 0. For simplicity, denote g(x) x V (x),, then g++1 = qe + +1 (ett,
g(s - qr]].
We prove by induction. For k = 1,
a++ St-1
qt + 7) [1 - a + g6: 9:)l > 711 - a c [B + (1 a + A)m))
Observe that c. [B (1-1aat A|ml
B + (1 a + Aurk
N²
1 2:
hence
2N
8 711 - a 2 > nl 1
a) 2 0.
For < < N 1, by recursion:
41+k 814k + 111 ka) + Σ
g(s448 -((+)) - 81+k
i=0
> nn(1 ka) + +
g(8(+: - ((++)) 81+K
(st > qt 2 0)
k-1
2 11(1 (ny + C Σ
(8t+1 - 444) - -8++
i=1
2 1711 ka) - cl 1) [B + (1 - a + Xiri 81**
N c(N - 2) [B (1 a Aunj 8
N a N
(N - 2)n
7
>
2N2
2N
The last inequality is based on the assumption 8ttk S B < 9 < 2N-[B+(1-atAp)
In conclusion, we have proved that for every miscoverage step t, ie. Y, $ Cτ the next N steps of (5) will satisfy Y2t1 E CC+s, 2, - 1. Therefore, for any T
T+N Σ
1 12 $ Ci $ N
T+1
B.3 PROOF OF THEOREM 2
Theorem 2. Let (mp he21 be an arbitrary positive sequence. Under Assumptions 1 and tion set generated by Equation (5) with adaptive learning rate n7 satisfies:
the predic-
1 T
(B MT MI-1llAit + cc[B+ (1 + 11
eer, a) <
(13)
T
where ||1 | Σtz inc nا Mr maxigrsT Tr-
17

---

Published as a conference paper at ICLR 2025
Proof. Denote A1 1/1
and A
nT-1 for all 2 1,
err; a)
T (err, ai)
T
T
Σ
A
TIe (ette - (i)
T
Σ
A
(T+1
9r
+
(st - Vss - (1))
by Equation 5
T
1=
A (LT++ -+)
(8: 2:) V)) 4:)
T
T
r=
Σ
A (gT+1 1 4r)
+
Σ
8c qc) V Vs(s qr)
7
A 1
max
9T
- 4rl +
8t q V - gr)
1SST
B + -
Σ/ [B + (1 + 1)M
T
B My
+clB+ + (1 a 1)M--1
C MORE DETAILS ON EXISTING METHODS
C.I ACI
Adaptive Conformal Inference (ACT) in Algorithm| [ models the sequentially conformal inference with distribution shift as learning problem of single parameter whose optimal value is varying over time. Assume we have calibration set Deat {(X,, Y.)) and Qx(-) the fitted quantiles of the non-conformity scores:
@(p) inf
Σ
(18(x.)))50)
2p
(Deal
(X..Y.)EDeal
For prediction set C((a) (y (x(x,,) S Qi - a)), define:
B sup Y, C,(3)).
Consider pinball loss E(a,, 8r) = Po (3. - (x)), by gradient descent:
a++1 04 n0a,, (au, 3:) a: 7 1..>9,) ar 17(a err.).
ACI transforms unbounded score sequences into bounded ones, which then implies long-run cover- age for any score sequence. This may, however, come at cost: ACI can sometimes output infinite or null prediction sets (a:; < or at > 1).
C.2 OGD
Online Gradient Descent (OGD) in Algorithm 2 is iterative optimization algorithm that updates model parameters incrementally using each new data point, making it suitable for real-time and streaming data applications.
18

---

Published as a conference paper at ICLR 2025
Algorithm 1 Adaptive Conformal Inference (ACI)
Require: a E (0, 1). 7) > O, Deat, init. 01
1: for > 1 do
2: Observe input X, E X
3: Compute (1 a4)
4: Return prediction set C((ar) == {v Sx(X,,)) s
5:
Observe true label Y, É y' and compute true radius 3r := sup(B : V, E C.(3))
6:
Update predicted radius
ac+1 at + 11(a La.20.)
7: end for
Algorithm Online Gradient Descent (OGD)
Require: a E (0, 1), base predictor f. learning rate > 0, init. Q1 E IR
1: for > do
2: Observe input X₁ E X
3: Return prediction set Cr(X,,
[fe(X.) (t f((x)) + qt)
4: Observe true label Y; E y and compute true radius 8 = inf E R: Y, E (X).s)
5:
Compute quantile loss g(+) (g:) P1-a (st q()
6:
Update predicted radius
g++1 = 4: - V))
7: end for
C.3
SF-OGD
Scale-Free OGD (SF-OGD) that we summarize in Algorithm is a variant of OGD that decays its effective learning rate based cumulative past gradient norms. Suppose f is a base predictor and choose (X,,, q) (X,) q. f(()) + g] to be prediction set around fX))).
Algorithm 3 Scale-Free Online Gradient Descent (SF-OGD)
Require: 10, base predictor f. learning rate 7) init. Q1 RR
1: for 2 1 do
2: Observe input X X
3: Return prediction set Cr(X,, [f((X,) qe+1 f((X)) + gel
4: Observe true label Y E y
5:
Compute true radius s inf(s E R Y, (((
6:
Compute quantile loss g(0) (gt) = (1-a( - qi)
7:
Update predicted radius 4++1 St 11
(()) (qt)
1= | (3:)1
8: end for
C.4 DECAY-OGD
Online conformal prediction with decaying step sizes (decay-OGD) in Algorithm is variant of OGD that decays its effective learning rate based time steps. Suppose f is a base predictor and choose (x.: q) [f.(X.) q, f((x,) + q) be a prediction set around fx.)).
19

---

Published as a conference paper at ICLR 2025
Algorithm 4 Online conformal prediction with decaying step sizes (decay-OGD)
Require: (0, 1), base predictor f. learning rate 7) init. qi R
1: for 2 1 do
2: Observe input X X
3: Return prediction set (x,,, 9t) [f((x)) qe+1, f((,) qdl
4: Observe true label Y, E y
5:
Compute true radius 8; = (n (8 E R Y, (x),,
6:
Compute quantile loss ( (ge) = f1 ( gt)
7:
Compute a decaying step size 1/2 = t 2-
8:
Update predicted radius at+1 qt Te Vell (g:)
9: end for
C.5 CONFORMAL PID
Conformal PID in Algorithm 5 is bulit upon ideas from conformal prediction and control theory. It able to prospectively model conformal scores an online setting, and adapt to presence of systematic errors due to seasonality, trends, and general distribution shifts. Conformal PID consists of three parts: quantile tracking, error integration and scorecasting. For prediction set:
C = {リ E y Se(Fe, y) $ 9c).
consider the optimization:
minimize
P1-a(5t 9).
RER
1=1
Conformal PID solves it via online gradient method:
4++1 q: + n0p1-a(se 9e)
q: + 77( (s, > 4:)) - qt + . -x),
which is called quantile tracking. In parctice, the learning rate is not fixed. They choose it in an adaptive way: Tt = 7 (max($t-w+1) ... {t} min(st-w41: parameter. The error integration incorporates the past error to further stabilize the coverage:
ss))). where 1 is a scale
g- = Tt. Σ (err a)
The last step is to add up a scorecasting term: 9r, a model that can take advantage of any leftover signal that is not captured like seasonality, trends, and exogenous covariates. Scorecastor needs be trained and can Theta or other models.
Putting the three steps together is the conformal PID method:
a++1 gi + m a) +r Σ (ert - -a)
C.6 SPCI
The sequential predictive conformal inference (SPCI) outlined Algorithm 6 cast the conformal prediction set predicting the quantile of a future residual and adaptively re-estimate the conditional quantile of non-conformity scores. Suppose f is pre-trained model, Qx is estimator of (p),, the p-th quantile of the residual é |Y -fx.) SPCI sets is Ō (X₂) defined as:
(f(x.) + ((x)) + Q.(1 +9)]
20

---

Published as a conference paper at ICLR 2025
Algorithm 5 Conformal PID
Require: a (0, base predictor f. trained scorecastor g', 7 O, window length w, init. q1 E R
1: for 2 1 do
2: Observe input X, X
3: Return prediction set C = (y ( 1 41
4: Observe true label Y E y and compute score St (X Yr).
5: Compute learning rate = (max($t-wtt) ... 5e) max m 88-
6:
Compute the integrator
Σ 1=1
eer; a)
rh r1
7: Compute the scorecastor (X,)
8:
Update:
g++1 = ((())) + me(eer, a) +
Σ err; - -a)
9:
end for
where 3
minimizes set width:
8 arg min (Q:(1 a (3)) BE(0,a]
Algorithm Sequential Predictive Conformal Inference (SPCI)
Require: Training data {(X,, Y.) prediction algorithm A, significance level a, quantile
gression algorithm Q.
Output: Prediction sets (X,) = [f(x,) + (3), @i(1 a 111,,,
1: Obtain f and prediction residuals with A and [(X,, Y))
2: for >> '' do
3: Use quantile regression to obtain Q + 2
4: Obtain prediction set (X,)
5: Obtain new residual é;
6: Update residuals by sliding one index forward (i.e., add é, and remove the oldest one)
7:
end for
D ABLATION STUDY OF HYPERPARAMETERS
D.1 EFFECTS OF DIFFERENT SCALE PARAMETER IN SIGMOID FUNCTION
We explore the effects of different scale parameter c in Sigmoid function. We conduct the ablation study Amazon stock dataset. Figures and show the line graphs across different When base predictor is not well (such as Prophet), as c increases, the sets tighten, but the coverage decreases. For AR model and Theta model, there almost identical performance when e in a reasonable range. Note that, when is large, numeric overflow will encounter because of scalar power. As varies, the changes in coverage and width relatively small and the two metrics are actually trade-off. When the coverage is fixed, the width of our methods with {0.1, 0.5, 1, 1.5, 2} are consistently shorter than other methods (see Table IT.. In general, our methods are less sensitive to the choice of scale parameter due to the trade-off between coverage and width.
D.2 EFFECTS OF DIFFERENT WINDOW LENGTII
We also explore the effects of different window length w in adaptive learning rates n and adaptive cutoff threshold h, Tables show the experimental results of window length w in ECI-cutoff. In general, w = 100 has the shortest average width with coverage greater than 89.5%. is worth noting that in the synthetic dataset, there is clear trend of increasing width and coverage increases.
21

---

Published as a conference paper at ICLR 2025
Graph Coverage
00.2
Proshat
AR
Thela
90.0
69.8
0.25
0.50
0.75
1.50
1.25
1.50
1.75
2.50
Figure 8: Coverage result on different scale parameter e in Sigmoid function.
Line Graph and Average Width with Dual Y-Axres (40 and 16.75-18.75)
Aeneage 4iteh
17.75
6.4
17.30
17225
AA
17.00
Theta
40
16.75
Figure 9: Set width result on different scale parameter in Sigmoid function.
This is because the main influencing factor at this time the learning rate, and an increase w leads larger adaptive learning rate.
Table 6: The ablation experimental results of window length in the synthetic data dataset with nominal level = 10%.
Prophet Model
AR Model
Theta Model
Coverage
Average width
Median width
Coverage (%)
Average width
Median width
Coverage (6)
Average width
Median
(%)
width
10
89,4
8.18
8.21
89.4
8.22
8.30
89.4
8.41
8.43
50
89.8
8.42
8.45
89.6
8.32
8.37
89.6
8.61
8.57
100
89.8
8.31
8.44
89.9
8.14
8.19
8.51
8.59
150
89.9
8.47
8.46
89.9
8.43
8.47
89.9
8.83
8.82
200
89.9
8.56
8.54
89.9
8.33
8.40
89.9
8.83
8.84
E EXPERIMENTAL RESULTS IN SYNTHETIC DATA
Following Barber et al. 2023), we test the performance on synthetic data under a changepoint setting. The data (X,, 11 generated according to linear model Y, X Br ct. X,
22

---

Published as a conference paper at ICLR 2025
Table 7: The ablation experimental results of window length in the Amazon stock dataset with nominal level = 10%.
Prophet Model
AR Model
Theta Model
Coverage
Average
Median
Coverage
Average
Median
Coverage ())
Average width
Median
(%)
width
width
( (%)
width
width
width
10
90.1
40.98
29.68
89.9
16.05
11.76
89.7
16.24
11.84
50
89.3
36.14
27.34
89.0
16.29
12.15
88.8
16.28
11.91
100
89.7
43.46
29.98
89.3
16.91
12.63
17.19
12.48
150
89.0
45.97
33.60
89.2
16.20
12.24
89.2
16.43
12.15
200
89.1
43.99
31.66
89.4
16.24
12.30
89.1
16.28
12.48
Table 8: The ablation experimental results of window length w in the Google stock dataset with nominal level a
10%.
Prophet Model
AR Model
Theta Model
Coverage
Average
Median
Coverage
Average
Median
Coverage (%)
Average width
Median
( ())
width
width
( (%)
width
width
width
10
89.9
50.33
42.47
91.7
22.74
22.05
89.6
33.49
29.42
50
89.4
47.38
40.90
90.2
20.69
19.21
89.3
31.63
29.64
100
89.8
53.12
44.36
89.7
19.84
17.63
89.6
30.71
28.11
150
89.1
55.62
47.59
90.0
20.53
17.85
89.7
30.54
28.98
200
89.4
53.65
45.49
89.8
20.48
18.02
89.8
30.99
30.06
~ 1). And we set:
3e g(0) -2,,,,00) t= t= 500,
3e = ((1) = -0, -1,0), t= 501 1500,
3. .(2) = 10), t t= 1501 2000,
where two changes in the coefficients happen up to time 2000.
We compare ECI and variants with some competing methods about the coverage and prediction set width. Figures 10 and 11 show the result of coverage and set width, while the base predictor is Theta model.
OAD VAUN
OSD
Time
Figure 10: Coverage result on synthetic data under a changepoint setting.
F EXPERIMENTAL RESULTS WITH TRANSFORMER
As one of the most successful deep learning models, Transformer has had a significant impact on various application fields, including time series. Thus it is interesting to conduct additional
23

---

Published as a conference paper at ICLR 2025
Time
Figure 11: Set width result on synthetic data under a changepoint setting.
experiment with Transformer the base model. We set input length to 12, output length to 1, the number of encoder layers to 3, the number of decoder layers to 3, and the number of features in the encoder/decoder inputs 64.
The quantitative results are shown in Tables 9 and 10 Consistent with other base models, ECI variants have achieved the best performance on five benchmark datasets with Transformer.
Table 9: The experimental results with Transformer base model at nominal level a = 10%.
Amazon stock
Google stock
Electricity demand
Coverage
Average width
Median
Coverage
Average width
Median
Coverage
Average width
Median
Method
())
width
(%)
width
(%)
width
ACI
90.1
40.44
90.2
00
57.13
00
0.109
OGD
89.4
52.68
31.00
90.1
109.27
89.00
90.1
0.139
0.120
SF-OGD
89.3
56.56
31.75
90.1
88.30
70.55
90.3
0.141
0.114
decay-OGD
89.8
93.16
34.98
89.9
120.25
69.81
0.147
0.111
PID
89.8
55.36
39.04
90.1
78.69
58.65
89.9
0.428
0.435
ECI
89.9
49.07
33.79
89.9
70.93
55.00
90.2
0.135
0.111
ECI-cutoff
89.7
45.01
29.64
89.9
66.67
51.29
89.9
0.133
0.108
ECI-integral
89.7
45.02
29.46
90.0
68.64
52.45
90.2
0.135
0.111
Table 10: The experimental results with Transformer base model at nominal level = 10%.
Amazon stock
Google stock
Coverage
Average width
Median width
Coverage
Average width
Median width
Method
( (%)
(%)
ACI
90.3
11.69
89.9
DC
8.20
OGD
89.9
9.72
9.50
89,9
8.13
8.20
SF-OGD
90.0
10.93
9.97
90.0
12.55
11.65
decay-OGD
89.7
14.43
11.47
90.1
8.30
8.24
PID
89.9
10.02
9.75
89.7
10.75
9.05
ECI
89.9
9.15
9.16
89.9
8.09
8.15
ECI-cutoff
90.0
10.42
9.95
89.9
8.01
8.10
ECI-integral
89.9
8.87
8.60
89,9
8.04
8.13
G MORE DETAILS ON EXPERIMENTS
G. DISCUSSION ON THE SCORECASTER
is worth noting that the Conformal PID baseline outperforms ECI under the Prophet base model in Table 3 Actually, the scorecaster term of conformal PID in the baseline utilizes the relatively
24

---

Published as a conference paper at ICLR 2025
accurate Theta model (which can also be replaced by AR or Transformer) to take advantage of any leftover signal and residualize out systematic errors in the score distribution. can be regarded as an additional component that "sits top" of the base forecaster (base model). Therefore, across all test datasets, the performance of conformal PID in the Prophet model consistently outperforms that of the AR and Theta model. This is attributed to the fact that Prophet, being peree-perorning model, is complemented by the superior performance of PID's scorecaster Theta, which is better- performing model.
We maintained the settings of Table 3 to test the performance of ECI combined with the scorecaster (Theta model), as shown in Table [1] The results demonstrated that ECI+scorecaster is consistently superior over Conformal PID across three base models, thereby validating the effectiveness of the ECI update. Interestingly, for the worse-perforning Prophet base model, adding the scorecaster enhanced the performance of ECI, while the scorecaster tended to degrade performance when better- performing base models were used. This observation is also mentioned in Angelopoulos et al. (023b); "an aggressive scorecaster with little signal can actually hurt by adding variance to the new score sequence".
Table 11: The experimental results in the electricity demand dataset with nominal level a 10%. Both scorecasters of PID and ECI+scorecaster are Theta model.
Prophet Model
AR Model
Theta Model
Coverage
Average width
Median
Coverage
Average
Median
Coverage
Average
Median
Method
())
width
( (%)
width
width
( (%)
width
width
PID
90.1
0.207
0.177
90.0
0,434
0.432
89.9
0.413
0.411
ECI
90.0
0.384
0.382
90.0
0.117
0.098
89.9
0.071
0.055
ECI+scorccaster
90.3
0.193
0.166
90.0
0,420
0.428
89.9
0.395
0.409
G.2 MORE DETAILS ON LEARNING RATES IN THE EXPERIMENTS
In this section, we show more comprehensive experimental results. We set base predictor as Theta model and all experimental setting are aligned with Section Coverage and prediction set results can be seen as the following Figure 12 to Figure 19
Since the methods based on OGD are highly sensitive the learning rate, we initially select four appropriate learning rates for these methods across various datasets, and then choose the one that performs the best among these four. fact, aside from the OGD-based methods, the four learning rates for other methods remain unchanged. We compile a list that includes all the learning rates, which are
ACI :7) = {0.1, O, 0.01,
OGD :7) {10, 5, 1, 0.5, 0.1, 0.05, 0.01, 0.005),
SF-OGD :7 = {1000, 500, 1,,055001..,,,,,,,,,1,,000000
decay-OGD :7 {2000, 20,0,,00000000000000,0000,12111,
Conformal PID :1 = {1, 0,5,
ECI :7 [1, 0.1.0.1.
ECI-cutoff :" = {1, 0.5, 0.1.0......005)
ECI-integral :
Note that, except ACI and OGD, other methods use 7/. adaptive learning rate in practice. Specifi-
cally, for SF-OGD:
(g)
7i =
(g)
where (g.) quantile loss and ge the predicted radius time 4. For decay-OGD:
7 = .. 1
25

---

Published as a conference paper at ICLR 2025
where the hyperparameter c = 0.1 follows Angelopoulos et al. (024). For conformal PID, ECI,
ECI-cutoff and ECI-integral:
= n (maxfst-wt) 8t) min st-+1, ,t))),
where 8, is the non-conformality score at time & and the window length w = 100 follows An- gelopoulos al. (2023b).
www.
POOE DMPREIS
pel, Capall PC
6.
og-ates
7307
Figure 12: Coverage result Google stock dataset.
denpp DSt
Figure 13: Prediction set result on Google stock dataset.
(--LL Dig KEE
BEENS
P-L op
FoLOg SO
Figure 14: Coverage result on Amazon stock dataset.
26

---

Published as a conference paper at ICLR 2025
Mised
1-2
pua
bart
Figure 15: Prediction result Amazon stock dataset.
CEA
Vett Hegithing Lall
Inoous ogpest 26
HHd 15.0 ooprrttk
FinE S, Capirt
PHOS INg
1. Copprrt No
Time
Figure 16: Coverage result on electricity demand dataset.
Conmaga
MAMAAA AMAAAA
506 196
Figure 17: Prediction set result on electricity demand dataset.
27

---

Published as a conference paper at ICLR 2025
Lapply loch
with coppeta
1hh
Figure 18: Coverage result Delhi temperature dataset.
SPACE
MWN
W
M
MMM
MV
WV
M
TO MWW
M
WW
WW
WW
WW
MW
Figure 19: Prediction result on Delhi temperature dataset.
28


Reviews:
[{"review_id": "ajkFda4IsH", "reviewer": "Reviewer_P71t", "summary": "This paper focuses on uncertainty quantification for time-series prediction, where exchangeability does not hold due to distribution shift and sequential dependence of data-points. Online conformal inference methods can be utilized for this task. Recent body of work on online conformal inference methods uses online gradient descent on the quantile loss function to update the thresholds of prediction sets. However, the subgradients of quantile loss function, that update the threshold at every time step, only leverage information on whether the outcome at that time-step is contained in the prediction set. The update does not include information on the magnitude of miscoverage. This paper introduces a smoothed version of quantile loss function such that the subgradient carries this information. Thus, 'Error-quantified Conformal Inference' incorporates a smooth version of feedback to update the threshold that defines prediction sets. The paper also introduces variants of smoothed ECI feedback for uncertainty quantification in time-series. Theoretical results on coverage guarantees and averaged miscoverage error are obtained without any assumptions on the data-generating distribution. Experimental results demonstrate superior performance of the ECI method in terms of reduced width of prediction sets while maintaining good coverage in comparison with other important online conformal inference baselines.", "questions": "For Table 3: Did the authors try to use a similar scorecaster term within the ECI update and check if that could improve over conformal PID in the Prophet model? \n\nAlso, is there any intuition for why decay-OGD baseline beats ECI in Table 5 (Prophet model)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of a smoothed version of quantile loss function to generate a more informative feedback for online gradient descent is simple and intuitive. It contributes meaningfully to improve uncertainty quantification in the online setting. \n\nThe mathematical intuition for the idea is well-presented and contextualized within the literature on online conformal inference. \n\nThe proposed technique produces consistent improvement in the width of prediction sets while maintaining coverage guarantees in a number of time series prediction tasks.", "weaknesses": "The authors mention difficulties in calibrating the outcomes of complex machine learning models like transformer, along with other models like deep Gaussian processes. However, the proposed techniques were evaluated on simpler time series models. It would be helpful to have some results with more complex models or discuss how the method would impact them. \n\nIt would be helpful to discuss the tradeoffs between different ECI variants in the context of experimental results. \n\nFor the place where the conformal PID baseline beats the proposed method, it would be helpful to explain the scorecaster term in a couple of lines so that it is more obvious why this baseline could produce a superior performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PQ9bFPMjZ6", "reviewer": "Reviewer_f97D", "summary": "This paper proposes Error-quantified Conformal Inference (ECI), based on the ACI approach, for online conformal inference with time series data. The key idea is to augment the online gradient descent with a smoothing term that reflects the extent of the gap between the non-conformity score and the threshold at each time step, moving beyond simple binary feedback. The main results include demonstrating coverage guarantees under a fixed learning rate, given bounded conditions for the smoothing function and non-conformity scores, and presenting a reasonable bound on the coverage gap in the case of well-designed adaptive learning rates.", "questions": "See weaknesses.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- While the methods are relatively straightforward, they are explained clearly and precisely.\n- The use of smoothing feedback to actively incorporate the degree of miscoverage during the update is a compelling idea, allowing the prediction sets to adapt well to, e.g., potential distributional shifts in time series data.\n- Theoretical results are based on reasonable assumptions.", "weaknesses": "- The authors primarily select the Sigmoid function \bfor smoothing. However, as illustrated in Figure 2, when the extent of miscoverage exceeds a certain level, the EQ term actually decreases (which is also the case for the Gaussian error function). While the authors justify this with the goal of ensuring robustness, the rationale seems somewhat unconvincing.\n- In connection with the above, if the Sigmoid function is used, the parameter $c$ can be considered a hyperparameter. Also given that Theorem 1 imposes constraints on $c$ for coverage guarantees, a more in-depth discussion on its selection, beyond the brief mention in Appendix D, would be appreciated.\n- The interpretation of the miscoverage bound presented in Theorem 2 is rather ambiguous. The choice of the learning rate is a crucial aspect, yet theoretical discussion on this remains superficial.\n- Similarly, the experiments lack detailed discussion on how the learning rate, which is critical for the performance of each method, was chosen. The statement on Line 330, “we select the most appropriate range of learning rates $\\eta$ for the respective datasets and present the best results in the tables,” is not only vague but could potentially raise concerns about data leakage. More transparency in the tuning process is needed.\n- Additionally, the choices of $h$ and $w$ in the ECI-cutoff method are not discussed.\n\nOverall, while the proposed method is straightforward and built on a solid idea, there is room for more detailed discussion regarding the rationales, interpretations of the results, and specific hyperparameter tuning.\n\nMinor:\n- Line 119: Please use \\citep for Barber et al. (2021).\n- Line 238: incorrect interval order\n- Line 653: non-differential -> non-differentiable\n- Are \"coverage\"s in, e.g., Figure 3, rolling averages?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AYgQV0vZjx", "reviewer": "Reviewer_FWjo", "summary": "This work introduces Error-quantified Conformal Inference (ECI), a scheme for providing prediction intervals for time series data. Standard conformal inference requires exchangeability for statistical coverage guarantees to hold but this is generally violated in the context of time series data. Other methods to provide prediction intervals exist but many do not incorporate the magnitude of miscoverage when providing updates for future intervals. ECI utilizes this information for its updates. The authors derive a coverage guarantee for their methodology and  provide empirical results on synthetic and real-world datasets to support their claims. Compared to baselines that do not incorporate the magnitude of error they find that they are able to provide narrower intervals.", "questions": "- Do the proofs for the validity of EQ-integral and EQ-cutoff need to be adjusted from the proof of EQI?\n- For Assumption 1 should it also be \"$\\forall t$\"\n- Line 159: \"$(1-\\alpha) (1 + t^{-1})$\" is not necessarily in $[0, 1]$. For instance if $t=3$ and $\\alpha = 0.001$. How can this be a sample quantile?\n- The EQI intervals visually are much more jagged. Could this ever result in dangerous overfitting?\n- The significance of Figure 2 is unclear to me and the caption does not provide much detail. Why is this important and what is going on in this figure?\n- Would error magnitude information be more useful for better or worse performing base models $f$ relative to other methods (ACI, OGD, etc)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- General presentation of the problem is clear and this is an approach could be of interest to an active community\n- Mathematical derivations are included\n- Strong empirical results on synthetic and real-world datasets", "weaknesses": "- Derivation on line 266 is difficult to read because it is inline\n- Minor grammatical errors and occasional typos scattered throughout the manuscript\n- See questions", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zego3GVxg6", "reviewer": "Reviewer_Yq3J", "summary": "The author propose an extension over the Adaptive Conformal Inference method of Gibbs and Candes, then used in the time-series realm by Zaffran et al. by incorporating in the adaptation procedure information about the magnitude of the error in the miscoverage rate.\nAfter having stated the method and analysed its performance with some real-world applications.", "questions": "- The \"adaptive\" CI school is only one of the possible approache to deal with the absence of indipendence between observations in Conformal Inference, in fact, https://proceedings.mlr.press/v75/chernozhukov18a.html provide a fairly interesting approach, which would be interesting to see compared to the authors proposal.\n\n- A very recent contribution by Oliveira et al. https://jmlr.org/papers/v25/23-1553.html shows that the contribution of adaptivity procedures to be negligible. How do the authors comment about this?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposed extension is rather interesting, seems to provide reasonable results, and puts itself in a rather interesting area of conformal prediction.", "weaknesses": "- \"Conformal Inference for Time Series\" is an extremely broad title, which gives the impression that the authors are the first in doing something in this which is, in fact, not true.\n\n- I found the article written in a very rough and unrefined way: I have counted at least 10 instances of missing articles (the...). Moreover, the introduction contains claims that are either too bold, or require a deeper explanation. What does it mean, for instance, that \"Bayesian recurrent neural networks or deep gaussian processes are difficult to calibrate?\", or, why is it meaningful to say that \"quantile regression models may overfit when estimating uncertainty\"\n\n- I am seriously concerned about the \"stock market\" application. It seems to me that the authors aim at forecasting log stock prices, which are known to be nonstationary. In fact, the usual approach in the econometric literature is to forecast returns (i.e. log differences) which instead, tend to be a stationary time series.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)