Conformalized Time Series with Semantic Features
Authors: Baiting Chen
Zhimei Ren
Lu Cheng
Venue: NeurIPS 2024 poster (2024)
Keywords: conformal prediction
time series
neural network
uncertainty qualification
Primary area: safety_in_machine_learning
TL;DR: enhance prediction efficiency and coverage for time-series data by leveraging deep representations to dynamically adjust weights in the semantic feature space

Abstract: Conformal prediction is a powerful tool for uncertainty quantification, but its application to time-series data is constrained by the violation of the exchangeability assumption. Current solutions for time-series prediction typically operate in the output space and rely on manually selected weights to address distribution drift, leading to overly conservative predictions. To enable dynamic weight learning in the semantically rich latent space, we introduce a novel approach called Conformalized Time Series with Semantic Features (CT-SSF). CT-SSF utilizes the inductive bias in deep representation learning to dynamically adjust weights, prioritizing semantic features relevant to the current prediction. Theoretically, we show that CT-SSF surpasses previous methods defined in the output space. Experiments on synthetic and benchmark datasets demonstrate that CT-SSF significantly outperforms existing state-of-the-art (SOTA) conformal prediction techniques in terms of prediction efficiency while maintaining a valid coverage guarantee.

Conformalized Time Series with Semantic Features
Baiting Chen Department of Statistics and Data Science UCLA
Zhimei Ren Department of Statistics and Data Science
University of Pennsylvania zren@wharton, upenn.
brantchenng.c ucla.
Lu Cheng
Department of Computer Science
University of Illinois Chicago lucheng@uic. edu
Abstract
Conformal prediction is powerful tool for uncertainty quantification, but its ap- plication to time-series data is constrained by the violation of the exchangeability assumption. Current solutions for time-series prediction typically operate the output space and rely on manually selected weights to address distribution drift, leading to overly conservative predictions. To enable dynamic weight learning in the semantically rich latent space, we introduce novel approach called Con- formalized Time Series with Semantic Features (CT-SSF). CT-SSF utilizes the inductive bias in deep representation learning to dynamically adjust weights, pri- oritizing semantic features relevant the current prediction. Theoretically, we show that CT-SSF surpasses previous methods defined in the output space. Exper- iments on synthetic and benchmark datasets demonstrate that CT-SSF significantly outperforms existing state-of-the-art (SOTA) conformal prediction techniques in terms prediction efficiency while maintaining valid coverage guarantee.
Introduction
Uncertainty quantification is essential for reliable predictions in time series data 334 45). The emer- gence of "black-box' models has intensified interest in conformal prediction (CP), a technique val- ued for its model-agnostic and distribution-fir properties [44, 57]. Under the assumption of data exchangeability, the confidence bands provided by CP are theoretically guaranteed [43]. This relia- bility has led to CP's promising performance across various domains, e.g., image classification [1]. graph neural networks [56), anomaly detection [22], and natural language processing (37 16).
The fundamental assumption of exchangeability in CP is often compromised in time series data due inherent temporal dependencies, highlighted in studies such as [53] and [48]. This violation poses significant challenges in directly applying CP to time series forecasting. Existing adapta- tions of CP tailored for time series (e.g.. ( tend produce overly conservative prediction sets or intervals too wide to provide practical insight, thus limiting their practical utility. These adapta- tions often necessitate manually selected weights, which curtail their generalizability across various datasets. Moreover, the practice of calculating non-conformity scores based on output space does take full advantage of the rich representational data available in the latent space. This oversight is particularly critical given the prevalent use of neural networks (NN) in contemporary time series forecasting, which are capable of capturing deep, complex patterns in data as discussed in (55) and 39/.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

The latent space in NN offers enhanced model interpretability and the ability to capture complex, nonlinear relationships in the data that are often invisible the original input space (40)). By map- ping input data into latent feature space through the feature function, the resulting representation not only supports the model learning essential temporal patterns but also in generalizing better to unseen data by focusing on the most informative features. To overcome the inherent challenges in applying CP to time series data, our work presents a novel approach- Conformalized Time Series with Semantic Features (CT-SSF)-whie Central to our methodology is the use of weighted errors for constructing non-conformity scores in the latent space, drawing upon rich semantic features. The weights dynamically adjusted to prioritize semantic features more pertinent to the current prediction task. This approach ensures that the weights adapt to the changing importance of different data aspects over time. By shifting CP into the latent space of NN, our non-conformity scores achieve a deeper insight into the temporal dy- namics. This not only maintains the requisite coverage of predictive intervals but also significantly improves their prediction efficiency, offering a refined tool for time series forecasting that aligns with the complexities of modern data structures.
leverages latent semantic features of NN for time series.
Our contributions are summarized as follows:
We propose CP in the latent space of the underlying time-series prediction model, enabling the construction of confoonfry scores that encapsulate a more comprehensive under- standing of the data and deliver tight prediction sets.
We propose adaptive weight adjustment scheme to refine the weight learning process in the latent space, accounting for distribution shifts in time-series data by emphasizing data points more relevant to the current time step.
We conduct extensive experiments under both synthetic and realistic settings corroborate the effectiveness of the proposed algorithm, showing a 10%-20% improvement over the SOTA approach for real-world data.
2
Related Work
CP with exchangeable data CP, pioneered by [4]. has become cornerstone in uncertainty quan- tification due to its model-free and distribution-free property [35] The work in this domain can be
summarized of CP into two branches: improve the efficiency of CP ((33. 13. 51. 36. 6 27 ) and
generalize CP to different settings, such as quantile regression [32), decision tree (20]. random for-
est (21l), survival analysis [I] k-nearest neighbor [30], online learning [8] and auction design [1
Comprehensive insights into CP and its theoretical underpinnings are provided by 4
Beyond Exchangeability The foundational works of [44]] and the introductory reviews by 4 de- scribe the evolution of CP to accommodate contexts "beyond exchangcability", Notable extensions include [41] application to covariate shift, and innovative adaptations for dealing with label shift by [31]. The work of [41] and [ address calibration and test set shifts by reweighting data points. [25] show how reweighting can be extended to causal inference setups for predictive inference on indi- vidual treatment effects, and [11) show how to apply these ideas in the context of censored outcomes in survival analysis.
CP for time series data CP for time series can be divided into two primary trends that deviate from the traditional assumption of exchangeability. The first trend focuses adaptively adjusting the error rate a during the testing phase to enhance coverage accuracy. This approach is demonstrated
in works such (2. & 14 53 26, 46 52 15 38, 58)). The second trend (c.g., [41. 48 49. 7 23
24,447 47l) emphasizes manually assigning weights to historical non-conformity scores, giving greater importance to those that reflect the current scenario more accurately. To address the limitation of manual selection, the advanced HopCPT model, developed by [5), leverages the Modern Hopfield Network to capture temporal structure and learn the weights. The work closest to ours is the HopCPT ((S)). We advance HopCPT in developing the non-conformity score in the latent space with rich semantic features, enabling a deeper analysis and utilization of the inherent temporal patterns in the data. Alternative probabilistic frameworks like Mixture Density Networks [1C] and Gaussian Processes [0] offer robust methodologies for time series forecasting beyond CP. However, these methods frequently grapple with computational limitations and lack robust theoretical backing.

---

3 Background
Split Conformal Prediction One of the most common approaches in CP is the split conformal prediction method (29), which starts with a trained model and assesses its efficacy on a calibration set of paired examples ((,,, Y,) Central to split CP is the foundational assumption that the data are exchangeable, ensuring that the errors observed in the test set will be consistent with those from the calibration set. This allows for reliable empirical estimation of quantiles across both datasets. Define a non-conformity score function s(x, y) that quantifies the disagreement between predicted and ground-truth values. The coverage guarantee of CP is defined as follows:
P(Y++1 E C(X++1)) 2 - a, C(X,+1) = {v =(x S 9).
(1)
where (Xn+1, Xn+1) is a new data point, q is the [(1 an + 1)7-th smallest elements among {s(X,, Y=) and signifies predefined miscoverage rate.
Non-exchangeable Conformal Prediction Real-world settings encounter challenges such data drift and inter-dependencies, necessitating further adaptations of CP to these non-exchangeable set- tings. One common solution to use reweighting 57. For example, given a of pre-specified weights {wx) ue E [0, 1), the coverage guarantee for seminal work NexCP is articulated as follows:
P(Y.+1 E C(Xn+1)) ( 21 a- Σ (..rv(Z.Z'), 8
W
1
++
where Z (X,, (Xn, Yn), (Xn+1, +1) represents sequence of n calibration examples along with subsequent test example and úi; is the normalized weight. The term Z' denotes the se- quence after the i-th pair (X,, Y.) is swapped with the test example (X,+1, Yn+1). and drv Z2 Z quantifies the dissimilarity introduced by this swap.
To construct prediction sets, NexCP determines the quantile threshold @ by the following equation:
i i= inf
q
Σ 1f8 S 21
Eq. B is consistent with the standard CP when all weights {w; = 1. Intuitively, closer alignment to exchangeable data minimizes the calibration terms dtv(2; 2"). allowing for more precise calibra- tion. Strategic weight allocation- susioon--uchha as assigning larger weights to calibration points (æ;, U4) that have similar distributions in Z and Z' and smaller weights where distributions diverge- can yield tighter bounds the prediction sets. For time series data, this suggests that greater weights should be assigned to more recent observations to reflect their increased relevance.
Methodology
In this section, we first formally define CP for time-series prediction and then propose our framework that leverages rich semantic features in latent space modeling and employs dynamic weighting mechanism that adjusts to temporal dependencies based on these semantic features, simultaneously achieving valid coverage and high efficiency.
4.1 Problem Definition
Given a multivariate time series Z, = (xe, V1) for = 1...., T, where each X, E R is a feature vector with dimensions and yt is the corresponding real-valued target. The prediction model μ utilizes these feature vectors to produce point predictions ÿt. = p(x)). The goal of CP is to construct prediction interval Ct (Xx+1) 1) that contain the true value yt++ with confidence level of 1
P(Y+11 E C², (X(+1)) 2 ax
(4)
where the probability is over the randomness of (Zo host and x,+1- This interval aims to balance reliability with informativeness beyond ensuring the coverage, it ideal that the prediction interval is of short length.
A common choice of non-conformity score is the absolute errors ly, ,xxx)) between observed values and the model's predictions [43. (2)). The prediction interval is then calculated based on the empirical I - a quantile Qı-a of the non-conformity scores:
Ch (x.+1) (xxx41) + Q1-a ((x))) U (co)).
(5)
3

---

4.2
Non-conformity Score in the Latent Space
Given the rich semantic information in the latent space, we investigate the effectiveness of using semantic features for CP with time series data. Our base model architecture μ is a deep neural network (e.g.. RNN) which is structured around two key sub-neural networks comprised of linear layers = g f: The feature function I maps input data into latent feature space, while the prediction head g transforms these features into the forecasted outputs.
In typical supervised learning for time series, ground truth labels are available only in the output space. The latent space, by contrast, is designed capture abstract, non-obvious patterns in the data.
These features help the model's 's learning but do not correspond directly to observable labels ((19). To address the lack of ground-truth labels in the latent space, we apply the surrogate feature [40] to replace the ground-trum -ruth term when constructing the non non-conformity score. Concretely, we define the nonconformity score to be
inf 1lc f(x)lI: 8 (X,Y, =EV:B(D)=Y
1| 11-
where f and 9 are approximate f and g. This score measures the minimal discrepancy between any latent representation v that could correctly predict Y when processed by 9 and the actual latent representation fXX produced by the model from input X.
It usually complicated calculate the score in Equation 6 due to the infimum operator. One solution is to directly apply the gradient descent mechanism in [40]: + ()V(((u) Y) however, this approach overlooks the characteristics of time series data and the adaptability of the update process. Therefore, we propose a weighted gradient descent mechanism
u + u - nu V - Y)?:
(7)
where w represents vector of weights specifically designed enhance the adaptability of the update process. These weights adjust the influence of each component the gradient, enabling the latent vector u to more effectively minimize the squared error between the predicted g(u) and actual targets Y. With learning rate 7/ these weighted adjustments prevent inefficient updates, fostering better convergence and ensuring well-defined non-conformity scores in the semantic feature space.
4.3 Adaptive Weight Adjustment
To ensure precise calibration of our model for time series data, is imperative to meticulously de- fine the weight terms ő thereby enabling the algorithm to concentrate on minimizing errors where they are most critical. We propose an adaptive weight adjustment scheme in the latent space, funda- mentally inspired by the intuition that weights should be dynamically adjusted to prioritize semantic features more pertinent to the current prediction task. To effectively quantify this pertinence, we employ the principle of proximity in етror terms, where smaller discrepancies between predicted and ground-truth values suggest higher relevance (5). For time series data, the underlying processes often exhibit consistent behaviors trends. Similar errors reflect this consistency, making them reliable indicators for future occurrences. As a result, we expect that similar errors should work best to predict the current error. Note that these error terms measured in the semantic feature space, allowing us to leverage the rich representations of the base model. Our strategy to handle non- exchangeable conditions in time series is to replace the quantile of in standard CP with a weighted quantile:
Q Σ wj d.. + W.41 800
Here, Wi are weights assigned based on the similarity of their error terms to the current prediction er- ror, 8; are the non-confornity score calculated in the latent space, represents the size of calibration data, and & denotes a point mass at a which represents a discrete distribution:
[1 if =
= P(X= {o if a
In Eq. g we assign larger weights to more relevant data points which can significantly enhance the lower bound of empirical coverage (as shown in Equation 2 This insight guides our adapta- tion of the attention mechanism to assess and quantify the relevance of training data points to

---

given test instance. The attention mechanism is designed to calculate weights based on the sim- ilarity of error terms, which are then used to adjust the influence of each data point in the final prediction model ((42). The attention weights here are: t < Attentionneightssgee Attention AttentionWeights((0 ((()) Y) represents the attention weights calculated by the Transformer model, reflecting the similarity between the predicted output ÿ(u) and Y. Accordingly, the attention mecha- nism in our approach actively assigns higher weights to these similar errors, thereby optimizing the inference coverage. The adaptive weight adjustment bolsters method's capacity to consistently meet the significant level, providing a robust guarantee for its performance.
Y), and
4.4 Weighted Conformal Prediction with Semantic Features
Since the non-conformity score is initially constructed in the semantic space, is necessary to trans- late the prediction intervals to the output space for practical application. Our method incorporates band estimation techniques [40) to estimate confidence or prediction intervals around the forecasts. This approach allows for uniform uncertainty levels in the latent space, while effectively reflecting varied levels of uncertainty in the output space based on the non-linear transformation. In this work, we apply linear relaxation-basee perturbation analysis (LiPRA) ((50)) tackle this problem under deep neural network regimes. LiPRA transforms the certification problem as a linear programming problem, and solves it accordingly. Following [40], model the Band Estimation problem perturbation analysis one, where we regard the surrogate feature as a perturbation of the trained fea- ture and analyze the output bounds of the prediction head. Since LiPRA results relatively looser interval than the actual band, this method would give an upper bound estimation of the exact band length.
We summarize our framework in Algorithm 1. First, it calculates non-conformity scores within the latent space by splitting the base model into feature function and prediction head in step 3. By doing so, our method can utilize the latent features to generate precise and efficient prediction sets that tailored the specific dynamics of the data. During the calibration phase, the algorithm integrates the attention mechanism in steps 7-10 to refine the learning process of weights. This integration is crucial for dynamically prioritizing and adjusting the influence of specific features based on their relevance to the prediction task. Lastly, step 14 applies a structured process to achieve comprehensive coverage and maintain competitive intervals in our prediction sets. As result, our method can dynamically adjust weights in response to changes in data patterns.
Algorithm Conformalized Time Series with Semantic Features
Require: Datasct {Xe, Y2) test fcature { X, hette 7), M, W, a;
I: Randomly split the dataset into training Der = {(X,, Y.)ee, and calibration Dea = {(X,, Vilhele
2: Training with Der
3: Train a base deep NN model for time series = - f(-) using the training fold Der;
4: Adaptive weight adjustment
5: (f(X.) : m + 0: [Tcal Stel: + (1/n, 1/n)
6: while m M do
7: predict g(u) - (Vihem
update
9: u V (Ilğ(u) (X)).emm.un..11)))
10: m 1-+m m
11: end while
Ensure: s(X, l f(x)-l-
12: for E Ise do
13:
Calibration with Dea
14:
Caleulate the (1 o-)-th quantile Q1-a of the distribution [Dal++ (Eim1 Wy Esg 0a+1 60c)
15:
Prediction
16: Apply LiPRA on f(X.) with perturbation Qi-a and prediction head ğ, which returns C1 C1-a(X)) (Xi);
Den X;
18:
end for
Ensure: C1 (X-) for each test input.
5

---

4.5 Theoretical Guarantee
This section outlines the theoretical guarantees for CT-SSF with respect to coverage and prediction interval length. Based on Theorem 4 in [40]. we demonstrate that constructing non-conformity scores in the latent space allows CT-SSF to surpass CP for time series in the output space (e.g.. NexCP) in terms of average prediction interval length under the following mild assumption:
Assumption 1. In the oulput space, we define (v, X) as the length associated with sample X, derived from the length in the feature space. This is represented as H(v,
{g(u) R
||u - -[XX]l z }3. We assume a Hölder condition for H, stipulating that for any X, the inequality (u, X) - (u, X)1 < L/e 100-w0 -ul holds, where B > O and L>> > O are constants.
The Hölder condition for (u, ensures that small changes in the input lengths or o result in small and predictable changes in the length H. This is characterized by the constants a and L. which control the sensitivity of (112). In the context of CT-SSF, this condition helps maintain consistent lengths when transforming from the feature space to the output space (length preservation), ensures that differences between individual lengths and their quantiles are amplified (expansion), and guaran- tees that the prediction intervals remain stable and reliable across both spaces for given calibration set (quantile stability). Below is an informal statement of the theoretical guarantee of CT-SSF. The complete version of the theorem is in Appendix [C.1
Theorem 1. Under Assumption 7 if the following cubic conditions hold:
1. Length Preservation. CT-SSF method does not cost much loss in feature space.
2. Expansion. The Band Estimation operator expands the differences between individual length and their quantiles.
3. Quantile Stability. The band length is stable in both the feature space and the output space for given calibration set.
Let ii denote the weights learned from Algorithm 1. Suppose that w. 's are independent of the calibration and test nonconformity scores. Then CT-SSF outperforms CP approaches for time series in the output space in terms of average band length while maintaining a valid coverage guarantee.
Here, length preservation ensures that the transformation from feature space to output space main- tains consistent interval lengths, minimizing efficiency loss. Expansion plays a key role in reducing inefficiency, as the latent space exhibits smaller distances between individual non-conformity scores and their quantiles, reducing the computational cost of the quantile operation. Lastly, quantile stabil- ity ensures that the interval generalizable from the calibration set to the test samples. Since these conditions primarily emphasize the properties of quantiles and transformation steps rather than the reweighting, extending this framework to incorporate a weighted setting is logical and justified progression. Note that in our theoretical results (for both efficiency and coverage) essentially as- sumes the weights to fixed, where in implementation, the weights are learned from the data. remains interesting question to establish condescending theory for data-drive weights. The detailed proof's can be found in Appendix C
Experiments
5.1 Experimental Setup
Prediction Models for Time Series. CT-SSF is model agnostic, therefore, any NN-hased predic- tion models for time series.prediction models can be used the base models. To better show the advantage of our proposed method, we utilize a Recurrent Neural Network (RNN) model, which can be replaced with more advanced models like Transformers ((421).
Datasets.
Our experiments encompass evaluations on both synthetic and four real-world benchmark datasets, allowing for assessment under controlled and natural conditions. They are electricity, stock of Amazon, weather, and wind data. The details of these datasets can be found Appendix [B]
Calibration Details. During calibration, to get the best value for the number of steps M, we take a subset (e.g., one-fifth) of the calibration set as the additional validation set. We calculate the non- conformity score on the rest of the calibration set with various values of step M and then evaluate

---

the validation set to get the best M whose coverage is right above 1 a. The final trained surrogate feature close the true feature because ((v) sufficiently close the ground truth Y. In practice, the surrogate feature after optimization satisfies
(Ø(()) 21
1%
(10)
||
Eq. [0 indicates that the normalized error of predictions with the surrogate feature is less than 1%. Therefore, the surrogate feature approximates ground truth feature in the latent space.
Compared Approaches. Baselines include state-of-the-art (SOTA) methods for CP under distribu- tion drift and SOTA CP for time series: (1) Standard Split CP (143)). This is the standard split
CP method outlined Section 3] (2) NexCP ((Z)). A refined CP method for handling distribution drifts using manually selected weights. Details are outlined in Section 3 (3) FeatureCP (FCP) ((40)) Implementing CP within a latent feature space under the assumption of exchangeability: (4) HopCPT (5.). The current state-of-the-art CP for time series. HopCPT leverages Modern Hopfield Networks to build prediction intervals by identifying and utilizing historical events with similar error distributions.
Metrics. Our analysis employs two widely used metrics to assess the effectiveness of each CP method (4) Empirical Coverage Rate (Coverage): Measures the effectiveness of CP in achieving the theoretically guaranteed coverage. Average Prediction Set Size (Width): Evaluates the efficiency of CP, reflecting the compactness of the prediction intervals.
All the standard deviations are obtained over five repeated runs with different random seeds under the same base model. Code and data are available at
com comm/baitinn0052/T-S555
5.2 Simulations
We adapt the simulation method in [7] and generate time series datasets that incorporate manu- ally designed temporal dependency and hetcroskedasticity. We generate n = 1, 000 data points (X, Y,) R I by sampling X; from Gaussian distribution, X, NO, las). We set The coefficient matrix A, B, C are set to the identity matrix IM and & NO Σ) where Σ is a diagonal matrix and its elements are given by 0.5 + 0.1i.
Y ~ AX; + BX + CX,2+
Miscoverage error
Methods
0.05
0.1
0.15
Cov
Width
Cov
Width
Cov
Width
CP
94.80+1..88
10.39-L0.49
1..0
9.30=
84.90=2.83
8.24+:0.0
FCP
94.20+t0.68
1.77+0.27
89.70:#2.41
1.72+0.21
15.10=1.32
1.40:t0.21
NexCP
95.30
9.60
91.50
9.10+0.40
85.50
7.66:L0.39
HopCPT
95.10-3.22
1.54
91.28+1.40
1.43:10.78
85.10.1..04
1.10:00.72
CT-SSF
96.50+10.84
1.45
90.70
85.70 1.23
1.044
Table 1: Performance of the evaluated CP algorithms for the simulations dataset.
Table 1 presents the performance of the compared CP algorithms across three different levels of miscoverage error (a): 0.05, 0.1, and 0.15. We observe from the results that CT-SSF consistently outperforms other baselines across all tested scenarios. Specifically, CT-SSF shows a 4.61 reduction in interval width compared the SOTA approach, HopCPT, The smallest width indicates that our method can produce the most informative prediction intervals while maintaining valid empirical coverage. This efficiency attributed to the innovative construction of the non-conformity score within the latent space with adaptive weight adjustment, which enhances the ability to leverage the underlying information of the base model. Meanwhile, the superior per- formance of FCP compared to CP and NexCP aligns with the experimental results reported in [40]. However, since FCP operates under the assumption exchangeability. it may occasionally result in under-coverage, e.g., 0.05 and conservative results, though it maintains robust coverage. This underscores the need for our adaptive weight adjustments with semantic features optimize performance.
miscoverage level of 0.05,
0.10. On the other hand, NexCP tends to produce overly
Ablation Study. To further investigate the effectiveness using semantic features and adaptive weight adjustment in CT-SSF, we compare it with its three variants that use: (1) Manually selected
7

---

weights in the Semantic space (CT-MS), (2) Manually selected weights in the Output space (CT- MO). This variant reduces to the NexCP method and (3) Learned weights in the Output space (CT-
1.O). The weights are learned using the same learning scheme in CT-SSE.
0.03
Frror
Creerage
Weth
Grenngg
With
C1-60
C1-L0
C1-L
Rate:
Emor Rate:
Error Rate:
Cowrgge
Coverge
Wete
Mate
20
CT.LD
CT
CT.50
C71
CT-55 CTMS
CT.L
Figure 1: Comparisons of the variants of CT-SSF The blue bar chart represents Width and the gray bar represents Coverage, the blue line represents target coverage.
Figure demonstrates the effectiveness of the CT-SSF method compared to its variants across var- ied miscoverage risks from 0.05 to 0.3. CT-SSF consistently outperforms the other methods in terms of prediction interval width, indicating its superior efficiency in generating compact predic- tion sets while maintaining robust empirical coverage. Relative to CT-MO and CT-LO, our method achieves more than a 50% reduction in the width of the prediction intervals, benefit derived from exploiting the rich latent information in the feature space. Additionally, in comparison with CT-MS, our method shows an approximate 10% reduction in interval width. These results demonstrate the significant impact of adaptive weight adjustments on the efficiency of prediction intervals, further establishing CT-SSF as a highly effective CP method for time series forecasting.
CT-SSF's performance can be attributed to its specialized approach to learning weights in the la- tent space. Unlike methods such as NexCP and CT-MS, which apply predefined weight schemes, CT-SSF's dynamic adaptation to the underlying data structure allows for more precise adaptation of the model to the specific data characteristics. Unlike HopCPT, which learns the weights but con- structs non-conformity scores in the output space, CT-SSF builds these scores directly within the latent space to utilize underlying latent features effeetively and better capture the data variability and structural dependencies, resulting in more reliable and informative prediction intervals.
5.3 Real-world Data
Real-world experiments conducted using benchmark time series data including electricity (l7), stock market (114, 21,, weather ([28, 54)). and wind speed forecasting (148).
CT-SSF
HopCPT
NexCP
FCP
CP
Cov
90.59=1.74
20.88:1..39
91.67:10.79
91.26:10.99
91.26=E1.15
Elec
Width
0,21+10.03
0.23+10.55
0.74
0.28=00055
0.83+10.01
Cov
91.58=2.233
90.7913.06
91.03:122.7
91.5842.07
91.74=+1.98
Stock
Width
0.19--0.03
0.23++0.04
1.43-10.12
0.29-00110
1.51+-0.1
Cov
12.20
90.11+0..68
10.14++0.26
90.06+0022
10.13-0.26
Weather
Width
0.012-10.022
0.017=10004
0.03410.006
0.023=0.004
0.034+0.007
Cov
9..32=1.31
90,06:22.49
90.0612.08
90.19
89,54
Wind
width
0.52+10.03
0.54+0.04
2.7610.11
0.59=0.13
2.65+0.11
Table 2: Performance of the evaluated CP algorithms for the real data. The specified miscoverage
level a = 0.1 for all experiments. Results for different a can be found in Appendix B
8

---

Results in revel that CT-SSF maintains valid coverage rates while excelling in minimizing prediction interval widths. Across all datasets, CT-SSF shows a 10%-20% reduction in prediction intervals compared to the SOTA HopCPT. While NexCP is designed for non-exchangeable settings, the need for manually selected weights limits its practical application. For instance, in the wind dataset, improperly chosen weights result in larger prediction intervals compared to standard CP, despite meeting theoretical coverage guarantees. We further conduct an ablation study to compare the variants of CT-SSF, similar to that for synthetic data. Results in Appendix [E show that CT-SSF outperforms other variants. This demonstrates the effectiveness of utilizing semantic features and implementing adaptive weight adjustments. While our experiments are based on RNN, we show in Appendix that using a different base model can lead to similar findings.
2nd
3rd
4th
5th
6th
Cov
89.75=
90.94 1.02
90.59:11.74
90.22-+-.022
90.21+
Elec
Width
0.23++0.09
0.29:10.02
0.21+-0.03
0.28-t0.02
0.30=0002
Cov
93.50:11.52
88.65=2.44
91.58:122.23
88.50:+2.42
88.41+2.60
Stock
Width
0.39:10.12
0.26:#0.09
0.19::0.03
0.21=0.07
0.21=0.07
Cov
90.12:10.31
90.07=0.34
90.12::0.26
90.06:10.56
90.06::0.54
Weather
Width
0.018=0.004
0.034:000110
0.012+10.002
0.014+0.002
0.014=0.002
Cov
89.67-22.42
90.32+1.511
89.28+1.88
1..63+
88.90+
Wind
Width
1.00+.023
0.52+00.03
0.54
0.59+
0.65
Table Performance of the CT-SSF with different f and g configurations. The specified miscover- age level is a = 0.1 for all experiments.
The impact of choosing f and g. Finally, we analyze the potential impact of the selection of the semantic feature space on the performance of CT-SSF. Our base model is an 8-layer RNN, and we conduct experiments on semantic spaces ranging from the 2nd to the 6th layer, i.e., using the first 2-6 layers as f and the rest as g. Results in Tinicate that the performance of CT-SSF is influenced by the configuration of f and g. as evidenced by the variation in the length of prediction intervals. key observation is that optimal performance typically arises from the middle layers (specifically, the 3rd and 4th layers). This suggests that f with too few layers may not capture enough semantic information, and g with limited number of layers lacks adequate predictive strength. Additionally, the coverage and interval length results similar for the 5th and 6th layers across all four datasets. This similarity may be due to the comparable level of information interpreted in these feature spaces. Given all these observations, it is important to perform cross- vols-dvnli to identify the best semantic space for the highest efficiency when deployed in various applications.
6 Conclusions and Limitations
In this work, we introduce CT-SSF, a novel approach designed to overcome the challenges of apply- ing CP directly to time-series data. Our method shifts the calculation of non-conformity scores the semantic feature space, effectively capturing the complex, nonlinear relationships in time-series data that are often overlooked in the output space. By employing a reweighting scheme and dynamically adjusting weights based on the importance of various semantic features, CT-SSF achieves valid cov- erage with high efficiency. We provide theoretical analyses demonstrating that CT-SSF outperforms CP methods that operate in the output space. Experimental results show that CT-SSF significantly outperforms cxisting SOTA CP methods for time-series data.
Our approach to CP for time series forecasting presents certain limitations that warrant further explo- ration and mitigation. Firstly, its reliance on NNs excludes simpler models like ridge regression or random forests. This limitation, while justified by the prevalent use of NN in time series forecasting, narrows the scope of our methodology's 's wide applicability. Secondly, the robustness of our predic- tion set lengths is sensitive to the configuration of the latent space to construct f and g. Our current solution uses a validation set to help separate f and g, rendering less data for calibration. Future enhancements could include the development of adaptive methods that dynamically refine the latent space to enhance the stability of the predictions across diverse data scenarios, thereby addressing the current limitations and expanding the utility of our approach.
9

---

Acknowledgements
This work supported by the National Science Foundation (NSF) Grant #2312862, National Insti- tutes of Health (NIH) #RO1AG091766) and Cisco gift grant.
References
[1] Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, and Michael Jordan. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint arXiv:2009, 2020.
[2] Anastasios Angelopoulos, Emmanuel Candes, and Ryan J Tibssiiani.. Conformal pid control for time series prediction. Advances in Neural Information Processing Systems, 36, 2024.
[3] Anastasios Angelopoulos, Rina Foygel Barber, and Stephen Bates. Online conformal pre- diction with decaying step sizes. arXiv preprint arXiv:2402.01133 20244
[4] Anastasios N Angelopoulos and Stephen Bates. A gentle introduction conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.075// 2021.
[5] Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal prediction for time series with modern hopfield networks. Advances in Neural Information Processing Sys- tems, 36, 2023.
[6] Yu Bai, Song Mei, Huan Wang, Yingbo Zhou, and Caiming Xiong, Efficient and differentiable conformal prediction with general function classes arXiv preprint arXiv:2202. 1091, 2022.
[7] Rina Foygel Barber, Emmanuel J Candes, Aaditya Ramdas, and Ryan J Tibshirani. Conformal prediction beyond exchangeability. The Annals of Statistics, 51 2023.
[8] Aadyot Bhatnagar, Huan Wang, Caiming Xiong, and Yu Bai. Improved online conformal pre- diction via strongly adaptive online learning. In International Conference Machine Learn- ing, pages 2337-2331 2263. PMLR, 2023.
[9] Ilias Bilionis and Nicholas Zabaras. Multi-output local gaussian process regression: Appli- cations to uncertainty quantification. Journal of Computational Physics, 231((7):571--5746 2012.
[10] Christopher M Bishop. Mixture density networks. 1994.
[11] Emmanuel Candès, Lihua Lei, and Zhimei Ren. Conformalized survival analysis. Journal of the Royal Statistical Society Series B: Statistical Methodology, 85(1):24-45, 2023.
[12] Lawrence C Evans. Partial differential equations american mathematical society. Providence,
RI, 2, 1998.
[13] Adam Fisch, Tal Schuster, Tommi Jaakkola, and Regina Barzilay. Efficient conformal pre- diction via cascaded inference with expanded admission. arXiv preprint arXiv:2007.03114 2020.
[14] Isaac Gibbs and Emmanuel Candes. Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34:1660-1672, 2021.
[15] Isaac Gibbs and Emmanuel J Candès. Conformal inference for online prediction with arbitrary distribution shifts. Journal of Machine Learning Research, 25(162): 2024.
[16] Yu Gui, Ying Jin, and Zhimei Ren. Conformal alignment: Knowing when to trust foundation models with guarantees. arXiv preprint arXiv:2405,, 10301, 2024.
[17] Jiale Han and Xiaowu Dai. Conformal online auction design. arXiv preprint arXiv:2405.0777 070388 2024.
[18] Michael Harries, New South Wales, al. Splice-2 comparative evaluation: Electricity pricing. 1999.
10

---

[19] Bernard Hernandez, Oliver Stiff, Damien K Ming. Chanh Ho Quang. Vuong Nguyen Lam, Tuan Nguyen Minh, Chau Nguyen Van Vinh, Nguyet Nguyen Minh. Huy Nguyen Quang, Lam Phung Khanh, et al. Learning meaningful latent space representations for patient risk stratification: Model development and validation for dengue and other acute febrile illness. Frontiers in Digital Health, 5:1057467, 2023.
[20] Ulf Johansson, Henrik Boström, and Tuve Löfström. Conformal prediction using decision trees. In 2013 IEEE 13th international conference on data mining, pages 330-339. IEEE, 2013.
[21] Ulf Johansson, Henrik Boström, Tuve Löfström, and Henrik Linusson. Regression conformal prediction with random forests. Machine learning. 97:155-176, 2014.
[22] Rikard Laxhammar and Göran Falkman. Inductive conformal anomaly detection for sequential detection of anomalous sub-trajectories. Annals of Mathematics and Artificial Intelligence, 74:67-94, 2015.
[23] Jonghyeok Lee, Chen Xu, and Yao Xie. Kernel-based optimally weighted conformal prediction intervals. arXiv preprint 2405 16828, 2024.
[24] Junghwan Lee, Chen Xu, and Yao Xie. Transformer conformal prediction for time series. arXiv preprint arXiv:2406.00 05332,, 2024.
[25] Lihua Lei and Emmanuel J Candès. Conformal inference of counterfactuals and individual treatment effects. Journal of the Royal Statistical Society Series B: Statistical Methodology, (3(5):911-938) 2021.
[26] Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. Conformal prediction with temporal quantile adjustments. Advances in Neural Information Processing Systems, 35:31017-31030, 2022.
[27] Eugene Ndiaye. Stable conformal prediction sets. International Conference on Machine Learning, pages 16462-16479. PMLR, 2022.
[28) Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint 1IXv:22211.14330. 2022.
[29] Harris Papadopoulos, Kostas Proedrou, Volodya Vovk, and Alex Gammerman. Inductive confi- dence machines for regression. Machine learning: ECML 2002: 13th European conference on machine learning Helsinki, Finland, August 19-23, 2002 proceedings 13, pages 345-356. Springer, 2002.
[30] Harris Papadopoulos, Vladimir Vovk, and Alexander Gammerman. Regression conformal prediction with nearest neighbours. Journal of Artificial Intelligence Research, 15-81, 2011.
[31] Aleksandr Podkopaev and Aaditya Ramdas. Distribution-free uncertainty quantification for classification under label shift. In Uncertainty in artificial intelligence, pages 844-853, PMLR, 2021.
[32] Yaniv Romano, Evan Patterson, and Emmanuel Candes. Conformalized quantile regression. Advances in neural information processing systems, 32, 2019.
[33] Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 33:3581-3591, 2020.
[34] David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilis- tie forecasting with autoregressive recurrent networks. International journal of forecasting, 36(3):1181-1191 2020.
[35] Glenn Shafer and Vladimir Vovk. A tutorial conformal prediction. Journal of Machine Learning Research, 9(3), 2008
[36] David Stutz, Ali Taylan Cemgil, Arnaud Doucet, et Learning optimal conformal classifiers. arXiv preprint arXiv:2 10.09/92 2021.

---

[37] Jiayuan Su, Jing Luo, Hongwei Wang, and Lu Cheng. Api is enough: Conformal prediction for large language models without logit-access. arXiv preprint arXiv:2403.01216, 2024.
[38] Johan Hallberg Szabadváry. Adaptive conformal inference for multi-step ahead time-series forecasting online. arXiv preprint arXiv:2409. 14792, 2024.
[39] Ahmed Tealab. Time series forecasting using artificial neural networks methodologies: A systematic review. Future Computing and Informatics Journal, 3(2):334 340, 2018.
[40] Jiaye Teng, Chuan Wen, Dinghuai Zhang, Yoshua Bengio, Yang Gao, and Yang Yuan. Predic- tive inference with feature conformal prediction. arXiv preprint arXiv:2210.00175 2022.
[41] Ryan Tibshirani, Rina Foygel Barber, Emmanuel Candes, and Aaditya Ramdas. Conformal prediction under covariate shift. Advances in neural information processing systems, 32, 2019.
[42] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[43] Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world, volume 29, Springer, 2005.
[44] Volodya Vovk, Alexander Gammerman, and Craig Saunders. Machine-learning applications of algorithmic randomness. 1999.
[45] Bin Wang, Jie Lu, Zheng Yan, Huaishao Luo, Tianrui Li, Yu Zheng, and Guangquan Zhang. Deep uncertainty quantification: A machine learning approach for weather forecasting. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2087-2095, 2019..
[46] Xiaoqian Wang and Rob Hyndman. Online conformal inference for multi-step time series forecasting, arXiv preprint a:Xiv:2410.13115, 2024.
[47] Chen Xu, Hanyang Jiang, and Yao Xie. Conformal prediction for multi-dimensional time series by ellipsoidal sets. arXiv preprint arXiv:2403.03850, 2024.
[48] Chen Xu and Yao Xie. Conformal prediction interval for dynamic time-series. In International Conference on Machine Learning, pages 11559-11569. PMLR, 2021.
[49] Chen Xu and Yao Xie. Sequential predictive conformal inference for time series. In Interna- tional Conference on Machine Learning, pages 38707-38727. PMLR, 2023.
[50] Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33:1129-1144 2020.
[51] Yachong Yang and Arun Kumar Kuchibhotla. Finite-sample efficient conformal prediction. arXiv preprint arXiv:2 104. 13871, 2021.
[52] Zitong Yang, Emmanuel Candès, and Lihua Lei. Bellman conformal inference: Calibrating prediction intervals for time series. arXiv preprint AXXv:2402.002203 2024.
[53] Margaux Zaffran, Olivier Féron, Yannig Goude, Julie Josse, and Aymeric Dieuleveut. Adaptive conformal predictions for time series. In International Conference on Machine Learning, pages 25834-25866. PMLR, 2022.
[54] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[55] Peter Zhang. Time series forecasting using a hybrid arima and neural network model. Neu- rocomputing, 10:159-175, 2003.
12

---

[56] Tianyi Zhao, Jian Kang, and Lu Cheng. Conformalized link prediction on graph neural net- works. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4490-4499, 2024.
[57] Xiaofan Zhou, Baiting Chen, Yu Gui, and Lu Cheng. Conformal prediction: A data perspective. arXiv preprint arXi::2410.06494, 2024.
[58] Yanfei Zhou, Lars Lindemann, and Matteo Sesia. Conformalized adaptive forecasting of het- erogeneous trajectories. arXiv preprint arXiv:2402.09633 2024.
[59] Shixiang Zhu, Hanyu Zhang, Yao Xie, and Pascal Van Hentenryck. spatio-temporal prediction with application to wind power generation. Stti:2008.13288. 2021.
Multi-resolution arXiv preprint
13

---

A Comparisons to existing methods
This section contains descriptions of the relationship of our method to existing approaches of con- formal prediction for time series data. There are generally three primary approaches designed to manage these challenges and enhance the reliability and validity of the CP in time series: reweight- ing, updating non-conformity scores and updating miscoverage level. We introduce representative work for each of these categories here. For a more detailed analysis of CP methods for time series, please refer to [57]-
Reweighting assigns relevance-basee based weights to data points to align the data distribution closer to a target distribution. NexCP [7] exponentially decayed weights to emphasize recent observations, but these lack adaptability. HopCPT [S] improves on this by using Modern Hopfield Network (MHN) for Aimilarity-baser reweighting. It assigns weights to past time steps based on their relevance to the current time step. Encoded inputs are processed with learned weight matrices, and a hyperpa- rameter adjusts the focus of the probabilistic distribution. These weights create weighted conformal prediction intervals by discounting extremal quantiles of relative errors.
The second technique. updating non-conformity scores leverages the most recent T data points and continuously updates prediction intervals as new data becomes available. For example, EnbPI [48] updates the non-conformity score with sequential error patterns to adapt the intervals dynamically. And SPCI [ replaces the empirical quantile with an estimate by a conditional quantile estimator to effectively address serial dependencies among residuals in sequential analysis.
The last main direction for CP in time series focuses on adaptively adjusting the significance level during test time to account for mis-coverage. This method can dynamically adjust the size of prediction sets in an online setting where the data generating distribution is allowed to vary over time in an unknown fashion. For example, the update rule for the quantile level a in ACI [14] is:
0+++1 at + 7(a err:):
(11)
where y is step size parameter, and err, indicates if Y2 was not included č2 (at). The approach ensures that the prediction intervals adjust over time to account for shifts in the data distribution, maintaining the desired coverage probability.
Our work falls under reweighting scheme, with a focus on weighted conformal prediction in the latent feature space. This approach enables to capture the underlying data structure more pre- cisely, leading to tighter prediction intervals. We leverage the standard attention mechanism for its simplicity and computational efficiency, while also allowing for dynamic adjustment of data point significance. Furthermore. our method effectively integrates attention- based weight updates within the latent feature space, where the learned rich representations capture the intrinsic characteristics of the data with greater accuracy.
B Extended Experiments
Datasets Details
Datasets from four different domains are used in the experiments. Details are summarized below.
Electricity. The Electricity Demand Forecasting dataset, introduced by Harries in 1999 (118), tracks electricity usage and pricing in New South Wales and Victoria, Australia, with half-hourly data from May 7, 1996, to December 5, 1998. For our experiment, we focus on four key variables: nswprice and vicprice (electricity prices in New South Wales and Victoria) and nswdemand and vicdemand (electricity demand in each state). Our response variable, transfer, measures the electricity transferred between the two states. We select data from 9:00 AM 12:00 PM to reduce daily fluctuations and discard an initial period with constant transfer values, resulting in 3,444 time points for analysis.
Amazon Stock. The Amazon stock dataset provides detailed historical data on Amazon's 's stock performance. This dataset includes key financial metrics such as the opening price, highest price, lowest price, closing price, and trading volume for each trading day. The dataset captures daily stock prices at various intervals, providing a comprehensive view of Amazon's 's stock market behavior. These metrics are crucial for analyzing trends, volatility,
14

---

and the overall performance of Amazon's stock over time. The data is structured with columns representing the opening price (open). the highest price during the day (high), the lowest price during the day (low), the closing price (close), and the volume of shares traded (volume).
Weather. The weather dataset records meteorological data every 10 minutes throughout the entire year of 2020, encompassing 21 key indicators that provide comprehensive overview of weather conditions. Key indicators include air temperature, humidity, wind direction, rainfall, solar radiation. This dataset offers detailed, granular insights into weather patterns and conditions, making it valuable for meteorological analysis and forecasting.
Wind Speed. The wind speed data collected wind farms operated by the Midcon- tinent Independent System Operator (MISO) in the US (1591). The dataset records wind speed measurements updated every 15 minutes over a one-week period in September 2020. This high-frequency data provides detailed insights into wind speed variations and is crucial for analyzing and modeling wind energy production.
Additional Experiments We conduct extended experiments the real-world data for RNN and a different base model for time series prediction, Transformer. Results with different miscoverage levels can be seen in Table I and Table 5 respectively. We also perform ablation study similar to Section [52) for real data in Figure 2 CT-SSF consistently delivers the shortest prediction intervals across all datasets and varying error rates when utilizing Transformer as the base model. Although it does not perform optimally for weather and wind data at a 0.05 error rate with an RNN base, CT- SSF still achieves results comparable the top-performing method, HopCPT. These experiments support the conclusion that CT-SSF can produce competitive prediction intervals while maintaining coverage guarantee.
15

---

error rate
CT-SSF
HopCPT
NexCP
FCP
CP
Cov
95.32::0.02
95.18:+1.40
95.27:10.54
95.52=1.04
95.47=11.09
0.05
Width
0.23+0.04
0.26+0.07
0.98++0.04
0.34=0.06
0.99=00022
Cov 84,87+1181
84,90+1.775
85.57+0.70
86.67+1.48
86.27+1,477
Elec
0.15
Width
0.20=+0.03
0.23:+0.05
0.62:+0.04
0.25=0.04
0.73+:0.01
Cov
80.12:22.06
1..79:11.88
80,93:10.70
81.04:1..77
81.1611.97
0.20
Width
0.1910003
0.20
0.60
0.22=0.04
0.65110.01
Cov
95.47+11551
95.32+2.21
95.23+1.27
95.31+1.33
45.23+1.33
0.05
Width
0.30:10.04
0.31
2.13:10.28
0.45=0.17
2.27:10.22
Cov 86.75:+2.77
85.71+2..09
86.03--2.09
86.90=2.40
86.42++2..3
Stock
0.15
Width
0.18+0.00
0,20+0.0M
1,18+0.12
0,24=0.07
+-21
Cov 80.87+11.00
81.74+33.50
81.03++3.56
82.93+22.02
82.61+11338
0.20
Width 0.15:10.02
0.18:0.04
0.87
0.20=0006
1.07:10.02
Cov 95.08=+0.20
94.96:+0.66
05.14+0.20
95.01=0.117
94.97.0.17
0.05
Width 0.018+0.002
0.017+0.002
00.08
0.023-0.004
0.044+0.0009
Cov 85.24:t0.37
85.29 05.29
85.37+00.54
84.90=0.22
84.90+0.16
Weather
0.15
Width 0.012:10.002
0.014:10.002
0.032+10.005
0.015=0.003
0.029=0.006
Cov 80.31=00.26
80.38=00.78
80.20=0..50
79.89=0.21
79.89+0.28
0.20
Width
0.011+0.002
0.013+0.002
0.031+0.000
0.013-00003
0.025+00005
Cov
95.03-11.58
96.07=1.224
95,94
93.85=1.73
94.3841..6
0.05
Width 0.67+00.6
0.66#0.10
3.28:40224
0.69=0.14
3.15:10.28
Cov 84.18+2..00
85.36+22.63
85.75+3.077
84.96=1.43
13.13+2..59
Wind
0.15
Width 0.42++0.0
0.45+00.03
2.25++0.17
0.52-0012
2.28++0.07
Cov 37.3
78.56:16.16
79.60:14.50
78.82:11877
76.20+13.99
0.20
Width
0.37:10.05
0.40+0.06
1,98:10.09
0.46=0011
2.05:10.12
Table 4: Performance of the evaluated CP algorithms for the real data of RNN. The standard devia- tion is obtained over five repeated runs with different random seeds.
C Theoretical Guarantee
This section provides theoretical guarantees for CT-SSF regarding coverage and interval length (i.e., efficiency). We use the seminar work NexCP for illustration and the results below can be applied to other CP for time series in the output space as well.
C.1 Efficiency
Notations. Let P denote the population distribution. Let D ~ Pn, denote the calibration set with sample size n, where we overload the notation P., to denote the distribution of a set with samples drawn from distribution P. Given the model g f with feature extractor f and prediction head we assume g is continuous. We also overload the notation Qi (V) to denote the (1 qq--uae of the set V U {oo). Here, the set V represents the empirical distribution after reweighting, i.c., Ei Wh do; ü doc Additionally, let M[] denote the mean of set, and let set minus real number denote the broadcast operation.
NexCP. Let V (uf). denote the individual lengths in the output space, given the calibra- tion set Den- Specifically, of = 2|1 Uel. where yı denotes the true response of sample and y; denotes the corresponding prediction. Since NexCP returns band length with the a quantile of the non-conformity score, the resulting average band length is derived (V2
16

---

error rate
CT-SSF
HopCPT
NexCP
FCP
CP
Cov
95.42:::.4 1.43
15.50:11.43
95.58:10.42
95.58=1.06
95.47=11.09
0.05
Width
0.19:00.06
0.26+0.07
0.78++0.03
0.28=0.05
0.98==0022
Cov
90.59+11.74
90.88+2..99
90,22+2.02
91.23+0.86
91.26+1.00
Elec
0.10
Width
0.17:10.05
0.23:+0.05
0.62:+0.04
0.24
0.84:10.01
Cov
1..0
8..74::1.23
1..30111.56
84.9611.77
1.72
0.15
Width
0.15110.0
0.2110000
0.58+103
0.21=0.02
0.73
Cov 95.16+1.61
95.24+1.22
95.31+1.21
95.07+1.43
95.23+1.33
0.05
Width
0.21+10.4
0.30+10.05
1.83:10005
0.35=0.11
2.27
Cov 91.67+22.07
90.79+22.09
91.27=+1.11
91.42+2214
91.74++.98
Stock
0.10
Width
0.16+0.04
0,23++.0M
1,23++0116
0,24=0.08
1.51+0.17
Cov 85.71+2.114
86.74+22.78
86.03++2.09
96.50+22377
86.43+22.53
0.15
Width 0.14+0.003
0.20+t0.04
0.89
0.19
1.21::0.06
Cov 94.97400.17
95.04+00.24
95.08+0.24
94.96=0.16
94.97-0.17
0.05
width 0.019+0.004
0.020+0.006
00.084
0.023-0.003
0.044+000009
Cov 90.111+0.19
90.20:+0.50
90.23++0.37
90.19+3.72
90.13+0.22
Weather
0.10
Width 0,015:10033
0.017:10.005
0.032+10.006
0.021=0.003
0.035=10.007
Cov 84.89+00.11
85.29=00.48
85.37=00.55
85.24=0.37
84.90+0.116
0.15
Width 0.012+0.003
0.014+00000
0.027+0.004
0.015-00003
0.029+0.006
Cov 95.81=11.78
1.503
95.81
94.25=2.00
94.3811..6
0.05
Width 0.47:10.04
0.67+00.55
3.76:+0.15
0.55=0.12
3.15:10.28
Cov 89.80+22.50
90.06+1.722
90.12+1.23
90.33+1.51
1..54+
Wind
0.10
Width 0.37++0.04
0.52+0.04
2.91+0.111
0.46-0.10
2.65++0.14
Cov
84.83:12.78
84.18:22.90
85.62-2..89
83.14=2.59
84.83++1.20
0.15
Width 0.39+0.03
0.42:10.04
2.68:10.37
0.41
2.28:10.07
Table 5:
Performance of the evaluated CP algorithms for the real data with Transformer as the base model. The standard deviation obtained over five repeated runs with different random seeds.
CT-SSE, Let v {uf Fieteo be the individual lengths in the semantic feature space given the calibration Dea. The resulting band length in CT-SSF is denoted by
Ex Exiast
H (Qi-ooVV Xtest)
We propose formal description of the cubic conditions and provide the proof following the structure of 44].
Theorem 1 Assume Assumption (honpptiooooos....... holds. Additionally, assume that there exist constants
and c > 0, such that the feature space satisfies the following cubic conditions:
Length Preservation. Semantic-Feature CP does not incur a significant loss the feature space in a quantile manner:
ED-P, Q1-a (H D)) < Ep-P, Qi-a +
2. Expansion. The operator H(v, X) expands the differences between individual lengths and their quantiles, namely,
LE M (I-a -V3)
< En pa M (H(V)))) H[(3)]
min
-c 2mx2max(11)
17

---

0.6
1.2
60
Coverage
1.0
60
Coverage
Widt
0.4
0.6
40
0.2
0.4
0.0
CT-S3F
CTMM
CT4O
CT-MD
0.0
CT-S5F
CTMS
CT4,0
CT-RG
Error Rate: 0.1
Error Rate: 0.1
0.035
2.5
0.025
2.0
With
0.020
Coverage
1.5
60
Coverage
0.015
40
40
1.0
0.010
0.005
20
0.5
20
CT-SSF
CT:MS
CTLO
CTMM
CT-SSF
CTMMS
CTLO
CTMOO
Figure 2: Comparisons of the variants of CT-SSF using the real datasets. The specified miscoverage level is a = 0.1. The blue bar chart represents Width and the gray bar represents Coverage, the blue line represents the target coverage.
3. Quantile Stability. Given calibration set Dea. the quantile of the band length is stable in both the feature space and output space, namely,
ED-Po (V Q2-a(V s nn
and
En-P, (V Qi-o VT.
Let Wi denote the weights learned from Algorithm Suppose that thys are independent of the calibration and test nonconformity scores. CT-SSF provably outperforms NexCP in terms of average band length, namely,
E (V
<
Q1-a(VBB_.)
where the expectation taken over the calibration fold and the testing point (X Ytest).
Before proceeding with the proof, we note that Theorem extends Theorem 4 from [40] to accom- modate exhhanchaeee settings. This adaptation feasible because the foundational assumptions and methodologies hinge primarily quantile calculations rather than the specific characteristics of the underlying distributions. Thus, we can leverage the framework their proof to construct our argument effectively.
Proof. By the expansion assumption,
min(a.1)
LEpM (V)) vo]
EDM (ALa((((HVV H(V) D)] -e- mmx((L,1
vn
We rewrite it as
min(a.1)
LD (HO)
LEDM (1)a v2]= -V3]
Vn
Due to the Hölder condition, we have that
(H()) M(HVV, D)) + LM Q1-a (V)) V/8",
18

---

therefore
EDM (H(Q D)] < EDDIIa (H(V): Di) m min
Therefore, due to Assumption 1, we have that
min(1,a)
EpMH(QI-o(V)))..) EnQi-a - (1,
Additionally, according to the quantile stability assumption, we have that
EDM ((Q1-a (
vn
and
ED (Q1-a(VV)) Qi-a vn
Therefore,
EH(Q1-a (V X') = (EMMH(QI-aVV D)
min
max (1,
c
+ D
+ vn
v
n
< Q1 Q1
C.2 Coverage Guarantee
Since the coverage for CP depends on the ranking of the nonconformity measures rather than their actual values (35. 37 we can extend the work of [ to the semantic feature space, where we construct the nonconformity score and compute the corresponding quantile value.
Theorem (Coverage Gap; Adaptation from 71 Theorem 2). Let ACov denote the coverage gap between the empirical coverage and the desired coverage level. Let w, denote the weights learned from Algorithm I. Suppose that w, 's are independent of the calibration and test nonconformity scores. Then,
ACov 2 - E w. dr (R(())
i=1
where drv represents the total variation distance, and R(Z) denotes the sequence of absolute residuals on Z and 7: denotes a sequence where the test point Z44 swapped with the i-th calibration point. Note that these sequences represented by the semantic features here.
This coverage gap suggests that assigning lower weights to time steps with high total variation distance help maintain small coverage gaps. reinforeing the theoretical foundation of our method. CT-SSF employs an attention mechanism to allocate higher weights to data points from the same error distribution as the current and lower weights otherwise. According Theorem 2 our dynamic weighting strategy can effectively minimize the coverage gap.
D Potentional Social Impact
Our method significantly can enhance decision-making making in finance, healtheare, and environmental monitoring by improving the accuracy and reliability of time series predictions. The success of our method, however, depends on the quality of training data used. This highlights the critical need for robust data management to maintain data integrity in dynamic environments. By upholding high data quality standards, our method ensures reliable support across these vital sectors, fostering trust in automated decision-make systems.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction introduce our contributions enhance pre- diction efficiency and coverage for time-series data by leveraging deep representations to dynamically adjust weights in the semantic feature space.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our work in section 6.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The au- thors should reflect on how these assumptions might be violated practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-tex system might not be used reliably provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach ad- dress problems of privacy and faimeess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
20

---

Answer: [Yes]
Justification:
We provide the assumptions and proof of the theory section 5 and Appendix
C.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We discuss these information in section 5 and the appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way accomplish this, but reproducibility detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of model checkpoint, other means that are appropriate to the research performed.
also be provided via
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
21

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We provide open access to our code section 5.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurIPS code and
data submission guidelines for more de-
tails.
While we encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips
for more details. The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
reproducible, they
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify these conditions in section 5 and appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question:
Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We provide standard errors for the experimental results with different random
seed.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should prefer- ably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
We discuss the information for computer resources to reproduce the experi- ments in appendix B.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental rums well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: :tttt::://eerips... http://wurrrp..ccpuuiieee cc/public/ htcc:puubic/Bcccccuuuudldlllll Ethicsuuidelinees
Answer: [Yes]
Justification: The research upholds standards of research integrity, ensuring ethical con- siderations in data handling and experimentation, promoting inclusivity, maintaining trans- parency for reproducibility,
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential social impact of our research in appendix D.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g,, deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitiga- tion strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (c.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
The paper appropriately credits the creators and original owners of all assets used, such code, data, and models, and clearly mentions the license and terms of use for each.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

---

If assets are released, the license, copyright information, and terms of use in the pack- age should be provided. For popular datasets, wwtpsssappppsososoeooocoo cow.co/////c//o com/datasets has curated licenses for some datasets. Their licensing guide can help determine the li- cense of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper deseribe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
25

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
26


Reviews:
[{"review_id": "61nsDAft1j", "reviewer": "Reviewer_iFYG", "summary": "The paper presents a conformal prediction approach for time series data in the latent space of the prediction model – Conformalized Time Series with Semantic Features (CT-SSF). Non-conformity scores constructed in the latent space are expected to capture deeper insights of the data and temporal dynamics and improve prediction efficiency. The paper also proposes an adaptive weight adjustment scheme in the latent space to dynamically adjust weights such that larger weights are assigned to more relevant data points. Authors perform experiments on synthetic and real-world datasets to demonstrate the effectiveness of their proposed method compared to existing conformal prediction methods for distribution drift and time series.", "questions": "1. Can the authors elaborate on how $\\tilde{w}$ is updated in Algorithm 1?\n\n2. Minor typo: p6 l243 -> calibration", "limitations": "The authors discuss limitations in the Limitations section. Authors briefly discuss the potential social impact in A.3.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Originality**: The paper extends HopCPT [1], a conformal prediction method for time series data, by constructing non-conformity scores in the latent space using ideas and methods from [2] (e.g., band estimation). The main original contribution in my opinion is the weighted gradient mechanism. While CT-SSF combines methods from [1] and [2] for the most part, this novel combination is effective and demonstrates performance improvement over existing methods. \n\n\n**Quality**: The paper is technically sound. The claims are well supported by theory and empirical evaluation over multiple datasets and comparison with baselines. The paper also includes ablation studies to demonstrate the effectiveness of the method.\n\n\n**Clarity**: The paper is well-written, organized, and easy to follow for the most part.\n\n**Significance**: The proposed method constructing non-conformity scores in the latent space provably outperforms conformal prediction methods in the output space (e.g., NexCP) in terms of average prediction interval length under the stated assumptions. The empirical analysis also demonstrates consistent results. I believe these results will be of interest to the uncertainty quantification as well as broader community.\n\n\n**References**\n\n[1] Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal prediction for time series with modern hopfield networks. Advances in Neural Information Processing Systems, 36, 2023.\n\n[2] Jiaye Teng, Chuan Wen, Dinghuai Zhang, Yoshua Bengio, Yang Gao, and Yang Yuan. Predictive inference with feature conformal prediction. arXiv preprint arXiv:2210.00173, 2022.", "weaknesses": "1. Limited discussion of past work: While the paper largely borrows ideas from [1, 2], I feel the text lacks appropriate context for complete understanding. There is limited explanation of the band estimation technique as well as HopCPT. I understand the main paper might not have space but this can be added to the appendix.\n\n2. There is less clarity on what the assumptions for Theorem 1 rely on and how they translate in practice. It would be helpful to discuss the robustness of the assumptions and implications if they are violated (in theory or practice) since this is the main result of the paper.\n\n\n**References**\n\n[1] Andreas Auer, Martin Gauch, Daniel Klotz, and Sepp Hochreiter. Conformal prediction for time series with modern hopfield networks. Advances in Neural Information Processing Systems, 36, 2023.\n\n[2] Jiaye Teng, Chuan Wen, Dinghuai Zhang, Yoshua Bengio, Yang Gao, and Yang Yuan. Predictive inference with feature conformal prediction. arXiv preprint arXiv:2210.00173, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We extend our sincere appreciation for your valuable feedback and suggestions. Regarding your concerns, we would like to offer further clarification.\n\nQ1: Can the authors elaborate on how $\\tilde{w}$ is updated in Algorithm 1?\n\nA1: The weights $\\tilde{w}$ are updated based on the attention mechanism of the Transformer model. The weights are adjusted dynamically to reflect the significance of each data point in the prediction task.\nThe attention weights from the Transformer model are used to assign higher importance to data points that have more influence on the prediction:\n$\\tilde{w} \\leftarrow \\text{AttentionWeights}(\\hat{g}(u), Y).$\nHere, $\\text{AttentionWeights}(\\hat{g}(u), Y)$ represents the attention weights calculated by the Transformer model, reflecting the similarity between the predicted output $\\hat{g}(u)$ and $Y$. Finally, we adjust the latent vector $u$ using the weighted gradient descent mechanism. This step ensures that the updates to $u$ are influenced by the dynamically adjusted weights $\\tilde{w}$, focusing the learning process on the most critical data points. We will add these explanations in the future version of this work.\n\nQ2: Limited discussion of past work.\n\nA2: \nThanks for the suggestion. We acknowledge that our initial submission did not provide a comprehensive discussion of the previous works due to the page limit. \n\nTo remedy this, we have added more detailed descriptions of current CP approaches for time series data in the appendix, which is also attached below.  \n\nThere are generally three primary approaches designed to manage these challenges and enhance the reliability and validity of the CP in time series: reweighting, updating non-conformity scores and updating significance level. \n\nReweighting assigns relevance-based weights to data points to align the data distribution closer to a target distribution. NexCP uses exponentially decayed weights to emphasize recent observations, but these lack adaptability. HopCPT improves on this by using a Modern Hopfield Network (MHN) for similarity-based reweighting. It assigns weights to past time steps based on their relevance to the current time step. Encoded inputs are processed with learned weight matrices, and a hyperparameter adjusts the focus of the probabilistic distribution. These weights create weighted conformal prediction intervals by discounting extremal quantiles of relative errors.\n\nThe second technique, updating non-conformity scores leverages the most recent $T$ data points and continuously updates prediction intervals as new data becomes available. For example, EnbPI uniquely updates the non-conformity score with sequential error patterns to adapt the intervals dynamically. And SPCI replaces the empirical quantile with an estimate by a conditional quantile estimator to effectively address serial dependencies among residuals in sequential analysis. \n\nThe last main direction for CP in time series focuses on adaptively adjusting the significance level $\\alpha$ during test time to account for mis-coverage. This method can dynamically adjust the size of prediction sets in an online setting where the data generating distribution is allowed to vary over time in an unknown fashion.\nFor example, the update rule for the quantile level $ \\alpha $ in ACI is: $\\alpha_{t+1} = \\alpha_t + \\gamma(\\alpha - \\text{err}_t)$, \nwhere $ \\gamma $ is a step size parameter, and $ \\text{err}_t $ indicates if $ Y_t $ was not included in $ \\hat{C}_t(\\alpha_t) $. The approach ensures that the prediction intervals adjust over time to account for shifts in the data distribution, maintaining the desired coverage probability. \n\nQ3: There is less clarity on what the assumptions for Theorem 1 rely on and how they translate in practice. It would be helpful to discuss the robustness of the assumptions and implications if they are violated (in theory or practice) since this is the main result of the paper.\n\nA3: We understand the importance of clearly discussing the assumptions underlying Theorem 1 and their practical implications. Below, we have provided an enhanced explanation to address the robustness of the assumptions and the potential consequences if they are violated.\n\nAssumption 1, Length Preservation in the Feature Space, ensures that the transformation from the feature space to the output space maintains consistent interval lengths. This assumption is generally robust with smooth and continuous feature extraction methods, but abrupt changes can lead to inconsistent intervals, affecting prediction reliability. Assumption 2, Expansion by Band Estimation Operator, captures differences in interval lengths and quantiles. It is robust if the operator is well-calibrated, though sensitive to calibration data quality and quantity. Violations may result in inaccurate prediction intervals, compromising coverage guarantees. This assumption directly influences the performance of our method, so we conduct new experiments to validate this assumption in Table 2 in the attached PDF. Results show that the latent space has a smaller distance between individual non-conformity scores and their quantiles, which reduces the cost of the quantile operation. Finally, Assumption 3, Quantile Stability, ensures stable band lengths across feature and output spaces, crucial for consistent coverage. This assumption is typically robust with well-behaved data distributions but can be challenged by extreme values, leading to unstable intervals and undermining coverage validity. To mitigate these risks, we recommend using continuity-preserving feature extraction methods, regularly recalibrating the band estimation operator with diverse data, and employing robust statistical methods for outlier detection. We will add these explanations in the future version of this work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Yzd4GRgukJ", "reviewer": "Reviewer_phaJ", "summary": "This paper proposes Conformalized Time Series with Semantic Features (CT-SSF), a conformal prediction method that constructs prediction intervals for time series data using nonconformity scores computed in the latent feature space of neural networks. CT-SSF further assigns time-dependent weights to the scores based on their similarity to the current prediction errors.", "questions": "I have several clarifying questions regarding the methodology and experiments outlined in the paper:\n\nMethodology: \nIf my understanding is correct, CTSSF is an incremental methodology based on Nex-CP (using weighted conformal prediction), FeatureCP (implementing CP within a latent feature space instead of in the output space), and HopCPT (using the same intuition of designing weights according to how similar previous errors are to the current error). I’m confused about the last part—how exactly do you update the weight? Did you rely on the MHN attention mechanism as in the HopCPT paper? Please correct me if I misunderstood or overlooked anything - I would appreciate it if the authors could highlight the unique challenges faced in this project. If the authors could clarify these challenges and provide experimental results with a larger number of independent repetitions to enhance the significance of their results (please see 'Experimental Significance' under weaknesses), I would be happy to increase my score.\n\nExperiment:\nDid you compare your results with other benchmarks used for conformal time series forecasting, such as the EnbPI [1] method and the SPCI [2] method?\n\nHow many seeds did you use for real data experiments?\n\nReference: \n[1]: “Conformal Prediction Interval for Dynamic Time-Series.” Chen Xu and Yao Xie, PMLR, 2021\n[2]: “Sequential Predictive Conformal Inference for Time Series.” Chen Xu and Yao Xie, PLMR, 2023", "limitations": "The author addressed the limitations of their work.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The idea of adapting time-dependent weights for the nonconformity score computed in semantic feature space is novel. \n\nThe authors conducted experiments on both synthetic and real datasets to demonstrate that, on average, the width of the prediction interval is shorter than those obtained from existing approaches. Additionally, the authors conducted an ablation study to investigate the performance of each core component in the CT-SSF method.", "weaknesses": "Clarity: The clarity of the writing can be improved. For example: In the experimental setup (Section 5.1), it is mentioned that CT-SSF is model-agnostic, allowing any prediction models for time series to be used as the base models. In contrast, the conclusion states that one of the limitations of CT-SSF is its reliance on NNs, excluding simpler models like ridge regression or random forest. The clarity of the paper would benefit from more rigorous writing, such as restricting the first statement to 'any NN-based prediction models for time series.'\n\nExperimental Significance: The standard errors in the synthetic data experiments are based on only five repetitive experiments. As a result, the confidence intervals on widths are too wide to demonstrate significant advantages of the CT-SSF method. The experimental results would be more convincing if more repetitive experiments were conducted with additional seeds.\n\nTheoretical contribution and unique challenges: As the authors have acknowledged, the theorem and its proof in this paper are identical to Theorem 4 in Teng et al. (2022). Therefore, the unique challenges faced in this project are not clearly articulated. Please refer to question number 1 for further clarifications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback on our manuscript. We appreciate your comments and have taken them into consideration to improve our paper. Below, we do our best to address the reviewer's questions adequately such that we could receive a better score.\n\nQ1: Comparisons with prior works and unique challenges. \n\nA1: Our work is indeed greatly inspired by Nex-CP, FeatureCP, and HopCPT. However, CT-SSF is not incrementally a combination of the three. A simple solution would be to combine Nex-CP with FeatureCP. However, Nex-CP employs pre-defined weights instead of data-adaptive weights, which prevents it from fully capturing the intrinsic characteristics of the original time series. While Nex-CP uses weighted conformal prediction and FeatureCP operates within a latent feature space, CT-SSF uniquely integrates these concepts by applying adaptive reweighting directly within the latent feature space. This integration allows for more precise capturing of the underlying data structure, leading to tighter prediction intervals and improved prediction accuracy.\n\nCompared to HopCPT, our weight update mechanism uses a standard simple attention weights mechanism rather than the MHN attention mechanism from the HopCPT paper. We chose the standard attention mechanism for its simplicity and computational efficiency, which still allows for dynamic adjustment of data point significance. Despite using a typical attention mechanism, our results demonstrate that CT-SSF outperforms HopCPT in both performance and computational efficiency because the attetion-based mechanism effectively prioritizes relevant data points and a simple standard attention mechanism can reduce computational overhead. The simplicity and efficiency of the standard attention mechanism enable faster computations and more scalable implementations without sacrificing accuracy.\n\nThe primary unique challenge we faced in this project was ensuring the effective integration of attention-based weight updates within the latent feature space. Achieving this required innovative solutions to ensure that the dynamic adjustments made by the attention mechanism did not introduce significant distortions or inconsistencies in the prediction intervals. To address this, we employed continuity-preserving feature extraction methods, such as RNNs, to maintain the intrinsic structure of the data. Additionally, we utilized Transformer models for dynamically adjusting weights, which allowed us to effectively prioritize data points based on their relevance to the current prediction task. \n\nQ2: Experimental Significance.\n\nA2: Thanks for the suggestion.  \nWe did not include comparisons with the EnbPI and SPCI benchmarks in our current study because HopCPT was shown to significantly outperform these methods in the original paper. However, we understand the importance of comprehensive benchmarking. Therefore, we have conducted experiments on EnbPI and SPCI with 10 repetitive trials, and the results are presented in Table 1 in the attached PDF. Our findings indicate that CT-SSF demonstrates a 10\\%-20\\% reduction in prediction intervals compared to the SOTA HopCPT across all datasets, and over a 20\\% reduction for SPCI and EnbPI.\n\nFor the real data experiments, we used five different seeds to ensure the robustness of our results. We agree that using more seeds could further strengthen our findings. Therefore, we increased the number of repetitions to 10 for the new results in Table 1 in the attached PDF. We see that variance is reduced as expected and CT-SSF still outperforms all other baselines. \n\nQ3: Clarity.\n\nA3: Thank you for pointing this out. We acknowledge the inconsistency in our statements regarding the model-agnostic nature of CT-SSF. We have revised the statement in Section 5.1 to specify that CT-SSF is compatible with any NN-based prediction models for time series. The revision is below:\n\"CT-SSF is model agnostic, therefore, any NN-based prediction models for time series prediction models can be used as the base models. To better show the advantage of our proposed method, we utilize a Recurrent Neural Network (RNN) model, which can be replaced with more advanced models like Transformers.\"\n\nOnce again, we appreciate the reviewer's precious time. We are eager to engage in further discussions to clear out any confusion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TqtKIQGsrn", "reviewer": "Reviewer_aroC", "summary": "The paper presents a novel approach called Conformalized Time Series with Semantic Features (CT-SSF) for improving uncertainty quantification in time-series forecasting. \nThe authors propose leveraging latent semantic features through deep representation learning to dynamically adjust weights for conformal prediction. \nThis approach aims to address the limitations of existing methods that rely on manually selected weights in the output space. \nThe paper demonstrates that CT-SSF achieves tighter prediction intervals with valid coverage guarantees, outperforming state-of-the-art methods in both synthetic and real-world datasets.", "questions": "Q1 (cr. W1). Please compare and discuss the impact of switching from output space (single dimension) to the latent space (multiple dimension) on the conformal prediction, e.g., whether and why this may or may not incur an increase in the calibration examples.\n\nQ2 (cr. W2). Please add experimental results for CNN and transformers.\n\nQ3 (cr. W3). Please discuss, report and compare the efficiency.\n\nQ4. In Figure 1, there is no order among the methods, hence a line chart is improper.", "limitations": "The author(s) have discussed in the paper.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1. The paper introduces an interesting combination of conformal prediction and deep representation learning, focusing on the latent space rather than the output space.\n\nS2. This paper provides theoretical analysis on the validity of the proposed method.\n\nS3. The experimental results demonstrate improvement on the conformal width.", "weaknesses": "W1. The latent space is high-dimensional compared to the output space.\nConsidering high-dimensional statistics is usually much more complicated compared to single-dimensional space, it would be interesting to discuss theoretically or empirically the tightness of the conformal scores in the latent space.\n\nW2. The latent space structure can be very different for different types of architectures.\nHence, the empirical results on different types of deep architectures, e.g., rnn, cnn, transformers, are necessary but missing.\n\nW3. The efficiency of the proposed conformalization method is missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable and informative suggestions and would clarify the questions and concerns in the following.\n\nQ1: Please compare and discuss the impact of switching from output space (single dimension) to the latent space (multiple dimension) on the conformal prediction, e.g., whether and why this may or may not incur an increase in the calibration examples.\n\nA1: The main impact of switching CP to the feature space is that in the feature space, it leads to a smaller distance between individual non-conformity scores and their quantiles (i.e., the second argument of the cubic conditions), which is key to improving prediction efficiency in this context. Intuitively, feature space is more semantically rich, providing more fine-grained information to capture the underlying data structure and relationships more accurately, leading to better prediction efficiency. We also provide new experimental results in Table 2 in the attached PDF to compare the average distance between each sample and their quantile in the latent space. The results validate that the distance in feature space is significantly smaller than that in output space, which aligns well with our hypotheses.\n\nAdditionally, to further analyze the relationship between the distance in the feature space and the prediction efficiency, we plot the relationship between efficiency (band length) and the cubic metric in Figure 1 in the attached PDF. Specifically, the cubic metric here represents the core statement in the cubic condition (statement 2), which implies a metric form like $\\\\mathbb{M}[Q_{1-\\\\alpha}V^f_{D_{ca}} - V^f_{D_{ca}}]$. The results demonstrate that the cubic condition is closely related to the performance of CT-SSF. For each given dataset, the cubic metric  negatively related to the band length and thus positively related to efficiency.\n\nIn addition, we conducted additional experiments to analyze the impact of the number of calibration examples on performance, as shown in the following table. Our findings indicate that our method is quite robust to the calibration set size, producing consistently better prediction efficiency. We believe this is because the latent space representations are highly efficient and can capture the essential features of the data well. Consequently, our method does not incur an increase in the calibration examples.\n\n**Table:** The impact of calibration set size on the performance of the evaluated CP algorithms for the real data with CNN as the base model. The specified miscoverage level is $\\alpha = 0.1$ for all experiments. The standard deviation is obtained over 10 repeated runs with different random seeds.\n\n|       |         | CT-SSF            | HopCPT            | 100 Calibration  | 200 Calibration  | 300 Calibration  |\n|-------|---------|-------------------|-------------------|------------------|------------------|------------------|\n| **Elec**  | **Cov**   | 90.4±2.20        | 90.2±1.63        | 89.7±3.50        | 89.6±2.11        | 89.6±2.02        |\n|       | **Width** | 0.23±0.04        | 0.32±0.06        | 0.23±0.03        | 0.23±0.03        | 0.23±0.03        |\n| **Stock** | **Cov**   | 91.8±1.82        | 90.6±2.09        | 89.7±2.28        | 90.3±1.70        | 91.1±1.70        |\n|       | **Width** | 0.33±0.08        | 0.53±0.15        | 0.31±0.09        | 0.31±0.08        | 0.32±0.08        |\n| **Weather** | **Cov**   | 89.8±0.27        | 89.9±0.25        | 88.0±0.47        | 89.3±0.32        | 88.8±0.34        |\n|       | **Width** | 0.02±0.005       | 0.04±0.007       | 0.02±0.005       | 0.02±0.005       | 0.02±0.005       |\n| **Wind** | **Cov**   | 88.5±1.83        | 89.2±3.41        | 88.0±2.20        | 88.5±2.50        | 89.0±3.40        |\n|       | **Width** | 0.36±0.05        | 0.60±0.10        | 0.40±0.09        | 0.35±0.10        | 0.40±0.11        |\n\nQ2: Please add experimental results for CNN and transformers.\n\nA2: Thanks for the suggestion. We agree that empirical results on different types of deep architectures are necessary. Experimental results for Transformers have been provided in Table 5, Appendix A.1, which suggest that CT-SSF demonstrates approximately a 10\\% reduction in prediction intervals compared to HopCPT across all datasets. Additionally, we conducted new experiments on CNN, and the results can be seen in Table 1 in the attached PDF. We observed that CT-SSF still generates 10\\% to 20\\% shorter prediction intervals than HopCPT across all datasets.\n\nQ3: Please discuss, report and compare the efficiency.\n\nA3: We compared the prediction efficiency of CT-SSF with other baselines, such as HopCPT and NexCP. Our results, as shown in Tables 1 and 2 of the original paper, indicate that CT-SSF achieves approximately a 5-10\\% reduction in average prediction interval length across both synthetic and real-world datasets. In terms of computational efficiency, the runtime varies significantly depending on the size of the datasets. For example, with electricity data, our method takes approximately 40 seconds, while HopCPT takes around 3 minutes, and all other baselines take less than 5 seconds for a single experiment (i.e., one seed, not including training the prediction model). The longer runtime for our method and HopCPT is due to the complexity of the adaptive reweighting and attention mechanisms used, which involve more intensive computations compared to simpler baseline methods. \n\nQ4: In Figure 1, there is no order among the methods, hence a line chart is improper.\n\nA4: We agree with your observation regarding Figure 1. We will remove the line chart and replace it with a scatter plot, to accurately represent the data.\n\nOnce again, we appreciate the reviewer's precious time. We are eager to engage in further discussions to clear out any confusion.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nHtARDggCH", "reviewer": "Reviewer_8Q7C", "summary": "The paper proposes producing conformal intervals using non-conformity scores defined in latent space, demonstrating improved performance against existing CP methods that work in output space.", "questions": "No more questions", "limitations": "See weaknesses above.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Adopting CP in feature space rather than latent space and demonstrating performance gains is novel.", "weaknesses": "1. Model-agnostic: the authors claim CT-SSF is also model agnostic, yet the experimental results in Table 3 clearly show variation in terms of different f and g configurations. There arises the question on the wide applicability and computational efficiency of the method, especially if searching for the \"optimal\" (f,g) is costly.\n\n2. Experimental comparison: how are the baselines implemented? Are the same RNN model used as the base model for the baselines? It is important to highlight what are kept the same across baselines in order to explain the performance gain of CT-SSF. If multiple components are varying, it is hard to attribute the performance gain to the use of latent features in CT-SSF.\n\n3. Theory:\n\n    - Section 4.5 tries to explain the theoretical guarantee of the proposed method, yet it is too hasty to be clear. The key question of why interval widths can be reduced lacks an intuitive explanation, and given the experimental results, it also seems such reduction depends on (f,g) combination, which is not clearly explained in the theory section.\n\n   - What are assumptions on the original series? Does the guarantees always hold for any time series?\n\nInstead of presenting Theorem 1 in this casual form, it is better to give an informal theoretical statements. The exact bound can be described in big-O notation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your valuable and informative suggestions and would clarify the questions and concerns in the following.\n\nQ1: Model-agnostic and f,g splitting criterion.\n\nA1: CT-SSF is model agnostic because CP is naturally model agnostic and any NN-based time series prediction models can be used as the base models.\n\nOne splitting criterion of $f$ and $g$ to significantly reduce computational efficiency is using cubic metrics of different splitting points for selection. As shown in Figure 1 in the attached PDF, we plot the relationship between prediction efficiency (band length) and the cubic metric. The cubic metric here represents the core statement in the cubic condition (statement 2), which implies a metric form like $\\mathbb{M}[Q_{1-\\alpha}V^f_{D_{ca}}- V^f_{D_{ca}}]$. We find that the cubic condition is closely related to the performance of CT-SSF. For each given dataset, the cubic metric is negatively related to the band length and thus positively related to prediction efficiency. Given this positive relationship, it is possible to use the cubic metric as the criterion for cross-validation instead of directly optimizing for prediction efficiency. By focusing on the cubic metric during cross-validation, we can reduce the number of necessary computations and improve the overall efficiency of the model selection process.\n\nAdditionally, the splitting criterion of $f$ and $g$ can follow standard cross-validation to find the best splitting point, like standard hyperparameter search. The process can be conducted on a smaller subset of the data to quickly estimate the best splitting point, thereby reducing the overall computational burden. Despite the variation, the experimental results on different splitting points consistently outperform other CP for time series benchmarks.\n\nQ2: Experimental Comparison.\n\nA2: A consistent RNN model is used as the base model across all baselines. After training the RNN model, it is fixed to ensure a fair comparison between different CP methods. This consistency allows us to attribute the performance gains specifically to the use of latent features in CT-SSF. The results demonstrate the excellence of CT-SSF in achieving superior prediction interval performance. To see if CT-SSF's improvement is consistent across different base models, we also conducted experiments on Transformers (see Table 5, Appendix A.1) and new experiments on CNN (see Table 1 in the attached PDF). Results for both architectures suggest that CT-SSF can outperform HopCPT with 10\\% to 20\\% shorter prediction intervals across all datasets.\n\nQ3: Theory.\n\nA3: A formal and detailed discussion regarding the theorem can be found in Appendix A.2. Here, we briefly summarize it.\n\nTheorem 1. Assume Assumption 1 holds. Additionally, we assume that there exist constants $ \\epsilon > 0 $ and $ c > 0 $, such that the feature space satisfies the following cubic conditions:\n1. Length Preservation. Semantic-Feature CP does not incur a significant loss in the feature space in a quantile manner, namely,\n$   \\\\mathbb{E} Q_{1-\\alpha}(H(V^f_D, D)) \\geq \\mathbb{E} Q_{1-\\alpha}(V^o_D) + \\epsilon.$\n2. Expansion. The operator $ H(v, X) $ expands the differences between individual lengths and their quantiles, namely,\n$L\\\\mathbb{E} [ \\\\mathbb{M} [ Q_{1-\\alpha}(V^f_D)- (V^f_D)]^\\alpha]<\\mathbb{E}[ \\mathbb{M} [ Q_{1-\\alpha}(H(V^f_D)) - H(V^f_D)]- \\epsilon - 2 \\max \\\\{L, 1\\\\}( \\frac{c}{\\sqrt{n}})^{\\min\\\\{\\alpha,1\\\\}}.$\n3. Quantile Stability. Given a calibration set $D^{'}$, the quantile of the band length is stable in both the feature space and output space, namely,$\\mathbb{E} [ Q_{1-\\alpha}(V^f_D) - Q_{1-\\alpha}(V^f_{D^{'}} )]\\leq \\frac{c}{\\sqrt{n}} $ and $\\mathbb{E} [ Q_{1-\\alpha}(V^o_D) - Q_{1-\\alpha}(V^o_{D^{'}} )]\\leq \\frac{c}{\\sqrt{n}} $.\n\nCT-SSF provably outperforms NexCP in terms of average band length, namely, $\\mathbb{E} [ H(Q_{1-\\alpha}(V^f_{D_{ca}}), X_{test}) ] < Q_{1-\\alpha}(V^o_{D_{ca}})$, where the expectation is taken over the calibration fold and the testing point $ (X_{test}, Y_{test}) $.\n\nThe reduction in interval widths is primarily attributed to the use of latent features, which enable a more precise capture of the underlying data structure. Different from most existing conformal prediction algorithms, which regard the base model as a black-box model, feature-level operations allow seeing the training process via the trained feature. For a well-trained base model, feature-level techniques improve efficiency by utilizing the powerful feature embedding abilities of well-trained neural networks.\n\nThe cubic conditions assume the latent space has a smaller distance between individual non-conformity scores and their quantiles, which reduces the cost of the quantile operation. We provide experimental results in Table 2 in the attached PDF comparing the average distance between each sample and their quantile in the latent space. The results show that the feature space has a smaller distance between individual non-conformity scores and their quantiles, which is key to improving prediction efficiency. Additionally, Figure 1 in the attached PDF demonstrates that the cubic metric is negatively related to the band length and thus positively related to efficiency. Different configurations of $f$ and $g$ will generate different cubic metrics, leading to variations in the reduction.\n\nThanks for the great idea of providing an exact bound described in big-O notation for the prediction interval! While deriving the exact bound is challenging due to the inherent variability and complexity of the method (e.g., the structure of the base model, the latent space, the specific implementation of the attention mechanism, as well as the transformation from the latent space to the output space), we will surely consider adding the exact bounds in our future version of this work.\n\nOur method doesn't rely on specific assumptions about the underlying time series, the same as the majority of prior works in CP for time series.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We would like to thank all reviewers for their thorough reviews, helpful suggestions, and constructive comments. To summarize, we made the following main changes to the manuscript to follow the suggestions and comments of the reviewers:
1. To illustrate the generalizability and performance of CT-SSF, we have added additional experimental results using CNN as the base model with more repetitions. Results in Table 1 in the attached PDF suggest a 10\% to 20\% reduction in prediction intervals compared to CP for time series benchmarks and standard CP.
2. We conducted additional experiments to bridge the gap between theory and empirical results. Table 2 in the attached PDF compares the quantile and non-conformity scores in the latent space and output space, suggesting that the latent space has a smaller distance. Experimental results in Figure 1 in the attached PDF indicate that prediction efficiency is positively related to the cubic metric. These results offer enhanced insights into the theoretical framework.
3. Recognizing the importance of discussing past work, we have added detailed descriptions of CP for time series in the appendix.
4. We revised typo errors and inconsistencies, and removed the line chart in the ablation study.

Meta-review: The paper studies conformal prediction in the context of time series prediction problems. After the rebuttal and discussion phase there is a clear convergence of (the majority) of reviewers (who engaged in discussion) towards a positive impression of the paper (in terms of soundness, experimental validation and clarity). Considering the reviewer comments and the rebuttal, and including the score increases, I am recommending acceptance at this point, but I am also encouraging the authors to take all comments seriously and adjust the manuscript accordingly.
Decision: Accept (poster)