Online Time Series Forecasting with Theoretical Guarantees
Authors: Zijian Li
Changze Zhou
Minghao Fu
Sanjay Manjunath
Fan Feng
Guangyi Chen
Yingyao Hu
Ruichu Cai
Kun Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Online Time Series Forecasting
Primary area: general_machine_learning

Abstract: This paper is concerned with online time series forecasting, where unknown distribution shifts occur over time, i.e., latent variables influence the mapping from historical to future observations. To develop an automated way of online time series forecasting, we propose a Theoretical framework for Online Time-series forecasting (TOT in short) with theoretical guarantees. Specifically, we prove that supplying a forecaster with latent variables tightens the Bayes risk—the benefit endures under estimation uncertainty of latent variables and grows as the latent variables achieve a more precise identifiability. To better introduce latent variables into online forecasting algorithms, we further propose to identify latent variables with minimal adjacent observations. Based on these results, we devise a model-agnostic blueprint by employing a temporal decoder to match the distribution of observed variables and two independent noise estimators to model the causal inference of latent variables and mixing procedures of observed variables, respectively. Experiment results on synthetic data support our theoretical claims. Moreover, plug-in implementations built on several baselines yield general improvement across multiple benchmarks, highlighting the effectiveness in real-world applications.

Online Time Series Forecasting with Theoretical Guarantees
Zijian Li!:2
Changze Zhou
Minghao Fu
Sanjay Manjunath²
Fan Feng
Guangyi Chen 1.2
Yingyao Hu Ruichu Cai Kun Zhang! 1.21
CCrneeie Mellon University
2 Mohamed bin Zayed University of Artificial Intelligence
Guangdong University of Technology
Johns Hopkins University
University of California San Diego
Abstract
This paper is concerned with online time series forecasting, where unknown dis- tribution shifts occur over time, i.e., latent variables influence the mapping from historical future observations. To develop an automated way of online time series forecasting, we propose Theoretical framework for Online Time-series forecasting (TOT in short) with theoretical guarantees. Specifically, we prove that supplying forecaster with latent variables tightens the Bayes risk-the benefit endures under estimation uncertainty of latent variables and grows as the latent variables achieve more precise identifiability. To better introduce latent variables into online forecasting algorithms, we further propose to identify latent variables with minimal adjacent observations. Based these results, we devise model- agnostic blueprint by employing a temporal decoder to match the distribution observed variables and two independent noise estimators to model the causal infer- ence of latent variables and mixing procedures of observed variables, respectively. Experiment results synthetic data support theoretical claims. Morcover, plug- in implementations built on several baselines yield general improvement across multiple benchmarks, highlighting the effectiveness in real-world applications.
Introduction
Time series data stream in sequentially like endless tide. Online time-series forecasting [Anava et al.. 2013, Pham et al., 2023, Lau et al.] aims to leverage recent observed variables Xt-r:t to predict the future segment X++1:T. In real-world scenarios, each observation x; may be partially observed, while a set of unobserved latent variables z; governs the evolving relationship between past and future. These latent variables introduce unknown distribution shifts [Zhang et al., 2024b, Zhao and Shen, 2024], i.e., the conditional distribution p(x++1:7 xt-r:)) changes over time, resulting in suboptimal performance of time series forecasting algorithms. Therefore, how adapt these distribution shifts automatically is key challenge for online time series forecasting.
To overcome this challenge, different methods proposed to handle the time- time-varying distributions. Specifically, some methods devise different model architectures to adapt to nonstationarity. For example, Pham et al. [2022] considers that the fast adaptation capability of neural networks can handle the distribution changes, so they propose the fast and slow learning networks to balance fast adaptation recent changes while preserving the old knowledge. Another similar idea is yee Ava Lau et al. [2025], which harnesses slow and fast streams for coarse predictions and near-future forecasting, respectively. Another solution focuses on concept drift estimation. For example, Zhang al. [2024b] detects the temporal distribution shift and updates the model in an aggressive way. Considering that
'Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

the ground-trute future values are delayed until after the forecast horizon, Zhao and Shen [2024] first estimates the concept drift and then incorporates it into forecasters. And Cai et [2025] estimates the nonstationarity with sparsity constraint by assuming the temporal distribution shifts are led by the unknown interventions. Please refer to Appendix for more discussion of the related works.
Despite demonstrating empirical gains in mitigating temporal shifts, these methods leave some theoretical questions unanswered. Specifically, existing approaches incorporating distribution shifts into model architectures [Pham et al., 2023, yee Ava Lau et al., 2025, Zhang et al., 2024b, Zhao and Shen, 2024] offer few theoretical guarantees explaining why conditioning on such estimated shifts yields a systematic reduction in forecasting error. Although recent work Cai et al. [2025] provides theoretical results, it often imposes strict conditions on the data generation process, i.e. nonstationarity is led by interventions on latent variables. Moreover, the uncertainty inherent in distribution shift estimation is rarely considered in the generalization bounds of error risk, leaving the discrepancy between empirical and ground-truth generalization risks uncharacterized. Furthermore, how to shrink the aforementioned gap theoretically and empirically remains underexplored. Therefore, general theoretical framework for online time-series forecasting is urgently needed.
In this paper, we propose Thretical framework for Online Time-series forecasting (TOT) with theoretical guarantees. Specifically, we first consider a general time series generation process, where the temporal distribution shifts are led by latent variables z, Sequentially, we show that conditioning the forecaster on Z, tightens the Bayes risk; the reduction persists under estimation noise and improves as identifiability sharpens. We further show that both the latent variables and their causal dynamics can be identified using only four adjacent observations, yielding a concrete, minimal-dan criterion. Building on these theoretical results, we devise model-agnostic blueprint with temporal decoder to match the marginal distributions of observations and two independent noise estimators to model the temporal dynamics of observed and latent variables. Experiment results on synthetic data verify our theoretical results, and plug-in versions atop different baselines achieve general improvement across several benchmarks. The key contributions of our work are summarized as follows:
We establish formal risk- bound guarantees for online forecasting under latent-driven distribution shift, furmishing explicit theoretical guidance to enhance forecasting performance.
We prove that both latent states and their temporal causal dynamics can be uniquely identified from only four consecutive observations.
We propose a plug-and-pll model architecture and achieve general improvement several benchmark datasets of online time series forecasting.
2 Problem Setup
We first introduce a data generation process as shown in Figure 1. Specifically, we let x; {.r,1 .r } be n o-dimension random vector that represents the observations. We further assume that they are generated by the historical observations x,- hidden variables z, = {z,,1 }, and independent noise è via a nonlinear mixing function g- Moreover, the latent variables 2,, is generated by the time-delayed parents Paa(z,,), instantancous parents Pa. (2,,)), and independent noise <ř,, via latent causal in- fluence f. Putting them together, the data generation process can be formulated Equation (I).
2=-2
Zz
Zee1
X--1
X
Xe+1
Figure I: Illustration of the gen- eration process for time series data, where the mapping of ob- served variables X,-1 -> x, is influenced by latent variables zt.
X1 = g(z X1-1, C.A))
21,5 f, (Pag(2).)) Pa (31,)) G..))
G. ~ Pet. ~ P (1)
Nonlinear Mixing Procedure
Latent Cousal Influence
To better understand our theoretical results, we provide the definitions of subspace-wise [Von Küigel- gen et al., 2021] and component wise identifiability [Kong et al., 2022]. Please refer to Appendix B for the description of the notations.
Definition 1 (Block-witg Identifiability of Latent Variables z.). The biock-wise identifiability of 2: E Rn means that for ground-truth Zt, there exists 2n and an invertible function H. : R. + IR."
such that 2 H1 (2)).

---

Definition 2 (Component-wise Identifiability of Latent Variables 21,). The component-vir wise identifiabilityy of 2, E RR" is that for each Ze,s, E {1.... .n.. there exists corresponding estimated component 3 E (1,"", , and invertible function h; R - R, such that Zt.; = hn(().)))
3 Theoretical Framework for Online Time Series Forecasting
Based on the aforementioned data generation process, we present theoretical results for online time series forecasting. Specifically, we first demonstrate that incorporating latent variables z, reduces forecasting risk, and that improved identifiability of these latent variables narrows the gap between estimated and ground-truth risk (Theorem 1). To ensure identifiability, we first identify the joint distribution of X, and 2, by matching the marginal distribution of four consecutive observations (Theorem 2). Subsequently, by imposing a sparse constraint on the estimated mixing procedure, i.c. &, we further establish component identifiability of the latent variables z (Theorem 3).
3.1 Predictive-Risk Analysis
We begin with the predictive risk analysis regarding different types of imputs of time series forecaster. Theorem 1. (Predictive-Ree Risk Reduction via Temporal Latent Variables) Let Xx, Zy and 2, be the ob- served variables, ground-truth latent variables, and the estimated latent variables, respectively. We let Xt-rit = {Xt-T,-.-- Xx} be the historical (T -ttep observed variables. Moreover, we let Ro, Rx and R2 be the expected mean squared error for the models that consider {x,-ra) {xt-r:t: ) and (xI- Zr respectively. Then, in general, we have Ro 2 RR 2 Rzz and if Ze is identifiable we have R. > *2² Rz.
Discussion:
The proof can be found in Appendix C.I. This risk bound can be derived by leveraging the law of total expectation and the law of total variance to decompose the expected mean squared error. Intuitively, Theorem highlights the critical role latent variables play in reducing predictive risk for online time series forecasting. Specifically, it reveals three distinct scenarios:
First, if the ground-truth latent variables z, have no influence on the observed variables xt, the predictive risk remains unchanged irrespective of whether latent variables are considered, i.c. R. = R2 = R2 However, this scenario rarely occurs in practice because z, having no influence on the mapping X + Xt+1 implies that the observed time series data is stationary, and it is typically challenging collect all relevant observations without any influence from exogenous variables.
Second, when latent variables do influence observed variables, incorporating the ground-truth latent variables reduces the predictive risk compared to using observations alone, leading R. Rz. Practically, however, ground-truth latent variables 2, are unknown, and we can only access the estimated latent variables 2,- If these latent variables are fully identifiable, the estimated latent variables can achieve the same risk reduction as the true latent variables, resulting in Ro > R₂ = Rz
Third, if the estimated latent variables 2, partially identify the ground-truth latent variables, meaning there exists at least one dimension j E 1, n for which, for all E 1,....,,, n, no function h; satisfies 2 h2(2).). Consequently, the estimated latent variables 2, capture only certain aspects of the temporal dynamics while omitting others due to incomplete identifiability, resulting in R. > R₂ R₂ In the worst case, when the estimated latent variables completely non- identifiable (e.g., 2, are purely random noise), we have R. R₂ >> Rz.
Based on the aforementioned discussion, we have the following two takeaway conclusions.
Incorporating latent variables to characterize distribution shifts helps reduce forecasting error. ii The more accurately the latent variables are identified, the lower the predictive risk.
3.2 Identify Joint Distribution of Latent and Observed Variables
Based on the aforementioned results, we further propose to identify the latent variables from observa- tions. By leveraging four consecutive observed variables, i.c., 24 X2, and X,+1, can find that the block (X,, 2;) is block-wise identifiable. For a better explanation these results, we first introduce the definition of the linear operator follows:
Definition (Linear Operator Hu and Schennach [2008], Dunford and Schwartz [1971]. Consider two random variables a and b with support A and B, the linear operator Lbja is defined as
3

---

mapping from a probability function Pa in some function space F(A) onto the probability function Pb Lbja Pa some funetion space F(B),
F(A) - F(B) pb = Lbja o Pn - Pbla(lla))aaaaa
Theorem (Block-wise Identification under Adjaceett Observed Variables.) Suppose that the observed and latent variables follow the data generation process. By matching the true joint distribution of 4 adjacent observed variables, i.e., (xt-2,X1-1,X, X(+1), we further consider the following assumptions:
Al (Bound and Continuous Density): The joint distribution of x, a and their marginal and condi- tional densities are bounded and continuous.
A2 (Injectivity): There exists observed variables X₁ such that for any E Xx there exist Xt-1 X and neighborhood 2 N around (X,, x1-1) such that, for any (11) N Ls. 1 is injective; Lx 2.. Lixe 22--2 is injective for any E Xe and x E X1-1, respectively.
A3 (Uniqueness of Spectral Decomposition) For any Xx E X2 and any 2, + Ee E Zr, there exists XI-1 E E1-1 and corresponding neighborhood N" satisfying Assumption A2 such that, for some (Xe, xx-1) E with x; + Xs, X F Xt-1:
k(X,, Xts X X t C 0o for any Ze E Zq and some constant C.
ii k(x,,, N,, Xx x,- Ze) * ks (x,, X,, X, X,- 2e), where
1x 21 (x: Xx- 1, Zz Px |x
(2c
11-1, 2t)
1xx1, X,, x X1
(3)
Pxe|xe-1118
(x X1-1, Zi } Px
(x1
X1-1, Zt)
Suppose that we have learned (g. f. pε) to achieve Equations (I), then the combination of Markov state 2,, X2 is identifiable, i.e., [2,, &c} = H (Z,, where H is invertible and differentiable.
Implication and Proof Sketch. This theory based on Hu and Shum [2012] and it shows that the block (Xe, Ze) can be identified via consecutive observations. Please find the proof in Appendix C.2. Although both [Fu et al., 2025] and our theory employ the technique of eeeenaalueeeigenunntiooon decomposition, our result is general case of Fu et al. [2025]. which demonstrates the identifiability results under the assumption that there is no causal relationship between the observed variables. Moreover, we further relax the monotonicity and normalization assumption (Please find the details of this assumption in Appendix C.4) in Hu and Shum [2012), which requires that the function form of p(x++1 [2, x,) 2, is known. This adjustment allows our conclusion to better align with real- world time series data.
The proof can be summarized into the following three steps. First, under the data generation process in Figure I, we establish the relationship between the linear operators acting on the observed and latent variables. Sequentially, by introducing the neighborhood of (x,-1, Xt), we derive an eigenvalue- cigenfunction decomposition for the observations, accounting for different transition probabilities of observed variables. Finally, by leveraging the uniqueness property of spectral decomposition (Theorem XV.4.3.5 Dunford and Schwartz [1971]). we demonstrate that the latent variables are bisck-wisc identifiable when the marginal distribution of the observed variables is matched.
Connection with Previous Results. To better understand our results and highlight the contributions of our work in comparison to previous studies, we further discuss the different intuitions between our work and prior works. In Fu et al. [2025], due to the absence of causal edges between the observed variables x,, conditioning are conditionally independent, which allows us the use different observations to describe the variance of latent variable. Mcanwhile, because there are causal edges between the observed variables, and the latent variable z, influences the mapping X + Xt, we use different transitions of observations, i.c., p(xx [x, 2,) to describe the variance of 2,. making the identifiability of latent variables possible.
2r, the three consecutive observations (i.e., Xt-1. 1, Xx, Xx+
Discussion on Assumptions. We provide a detailed explanation of the assumptions to clarify their real-world implications and enhance the understanding of our results.
2 Pleasse refer to Appendix C.5 for the definition of neighborhood.

---

First, the assumption of the bound and continuous conditional densities is a standard assumption in the works of identifiability of latent variables [Hu and Schennach, 2008, Hu, 2008, Fu et al.. 2025]. It implies that the transition probability densities of observed and latent variables are bounded and continuous. This assumption easy to meet practice, for instance, the temperature records in the climate data are changing continuously within reasonable range.
Lba denotes the mapping from Pa to Pb. and the injectivity of the linear operator means that there is enough variation in the density of b for different distributions of For example, /x-xxxx-- have sufficient influence on the future observations, i.e., the historical values of temperature should have a strong influence on the future values. Please refer to Appendix C.3 for more examples of the injectivity of linear operators.
Second, according Definition 3, the linear operator L
means that the historical observations
Third, the third assumption implies that the changing of latent variables 2, produces visibly different
Paylan 1 (3ea xx- 1.,).
influence on the mapping from X;-1 to X2- Concretely, we let Pz,.1 Px (Xa (x.)1.
and
18-- (X, 1.2)) Pa,2 = PR, 11) be the probability of transitioning rate from two different historical state (i.e., X and x) to the same current states (x, or xt). Assumption A3 says that for any two different values of latent values, there exist different historical states x,-1: X so that at least one of these ratios changes. Intuitively, this means the latent variable must have a sufficiently large influence on X - X₁. For example, if z, encodes the information of seasons and x, denotes the temperature, then the typical day-to-day temperature jump in winter versus summer will differ markedly. Since A3 asks for the existence of any two such states, it is easy to be satisfied in several real-world scenarios with continuous x,. More discussion can be found Appendix C.6.
3.3 Component-wise Identification of Latent Variables
Based on the block-wise identifiability of (z,, from Theorem 2, we further show that is component-win identifiable. For a better understanding of our results, we first provide the definition of the Intimate Neighbor Set [Li et al., 2024b, Zhang et al., 2024a] as follows.
Definition 4 (Intimate Neighbor Set Li et al. [2024b], Zhang et al. [2024a]). Consider Markov network My over variables set U, and the intimate neighbor set of variable ut, is
V Man(uee A {ut. Wt. adjacent to ut
and is also adjacent to all other neighbors of Ut,5, Ut. Eu, (uex))
Based on this definition, we consider u, = {,--1, X,-1, Ze, Zt,x;, xt}, then the identifiability can be ensured with the help of historical information (x:-1, 7c-1) and the sparse mixing procedure.
Theorem 3. (Component-wisee Identification of 2, under sparse mixing procedure.) For a series of observations x, E TR" and estimated latent variables 2, E IR" with the corresponding process f, (.))),) ÿ. suppose the marginal distribution of observed variables is matched. Let Mu, be the Markov network over u, & (zt-1,Xt-11 21, Besides the common assumptions like smooth, positive density, and sufficient variability assumptions in [Li et al., 2025], we further assume:
A4 (Sparse Mixing Procedure); For any Zt,t E Zz, the intimate neighbor set of Zi, is an empty set.
When the observational equivalence is achieved with the minimal number of edges of the estimated mixing procedure, there exists a permutation π. of the estimated latent variables. such that Zt,s and (1))()) is one-to-one corresponding, i.e., Z₁,5 is component-wit identifiable.
Proof Sketch and Connection with Existing Results. Proof can be found in the Appendix C.7. For simplicity, we omit the subscript in conditional distributions. We exploit the sparsity of the mixing procedure from 2, Xx. Concretely, a latent component Zt, does not contribute an observed variable Xt,j. where i E {1, ... n.. there is no causal edge between them. In the Markov network over ut implying the conditional independence of 24,5 11. æt. ju, (Z1,,, J. Vi, j E (1...... ...... n.. This conditional independence implies that the corresponding mixed second-order order partial derivative of p(z;, X₂ 11-1, x;- is zero, i.c. 8 p/ v, 1.x, 1) = O, which can further yields a linear system via sufficient changes assumptions in Li et al. [2025] and further results in identifiability of 2, More discussion can be found in Appendix C.8.
dz,,
5

---

While our argument leverages the same technique used in Li al.
O/O
Observed/Latent Variables
Sparse Latent Process
[2025], deriving zero-derivative con- ditions from sparsity in the Markov network, our contribution is orthogo- nal three main respects: 1) Noisy Mixing Procedure. Li et al. [2025] assume an invertible, noise-free mix- ing from latent to observed vari- ables. In contrast, we allow addi- tional noise in the mixing procedure 24 + Xr, thereby accommodating measurement error in x,- Since the real-world datasets always contain measurement uncertainty, explicitly accounts for observation noise, making our theoretical results suitable for real-world scenarios. 2) Temporal relations among Observations. Previous methods on temporal causal representation learning Li et al. [2025], Yao et al. [2021. 2022] usually assume that there are no causal links between X,-1 and x, due the difficulties of reconstructing Z, directly. However, our method allows the temporal relationships among observed variables, which is more suitable for real-world scenarios.
Deterministic/Cane Process
Sparse Mixing Process
RE
(b)
model
Process of IDOL.
Process of Our Method.
Figure 2:
Illustration of the difference between sparse causal influence and sparse mixing procedure.
3) Different Sparsity Assumptions. Although IDOL Li et al. [2024b] and LSTD Cai et al. [2025] also leverage the sparsity assumption, we would like to highlight that these sparsity assumptions are different. Specifically, IDOL imposes sparsity constraints on the latent process as shown in Figure 2 (a). We instead assume sparsity in the mixing procedure from Z, to as shown in Figure (b). According to the data generation process shown in Equation (I), there is instantaneous relations within Xt, the intimate set of any 24,4 are more easy to be empty, implying that the sparsity condition our work is simpler to met in practice.
4 Model-Agnostic Blueprint
The aforementioned results tell us the importance of in- troducing the latent variables into online time series forecasting and how to identify them theoretically. To show how to identify the latent variables empirically, we further devise general neural architecture as shown in Figure 3, where the gray blocks can be replaced with different backbone networks by decomposing the forecasting models into encoders and forecasters. We let X1:: and X(+1:7 be historical and future observa- tion, respectively. Since the proposed architecture is built upon the variational autoencoder [Kingma, 2013, Zhang al., 2018, Blei al., 2017], we first derive the evidence lower bound (ELBO) in Equation (4).
$ L
Ly
by
& .. R
8
Rr
Latent Transition Estimator
Decoder
Forecaster
L EC1
2 ... 2 2
Observed Transition Estimator
Encoder
ELBO = Egtzer ( In p( (
X1
I. and T.y 3Dx (alzaal X (()((()))
Figure 3: Illustration of the proposed plug-and-play architecture, where the gray blocks (encoder and forecaster) can be replaced with different backbones. The dashed lines denote the backpropagation.
Lie
where 3 denotes the hyper-paramette and DEL L the Kulbback-Leibler divergence. ((21:r((x1:) ) is used to approximate the prior distribution and implemented by encoders. p(x (1:r) is used to reconstruct the historical observation x1 and predict the future observation X++1-T, which are implemented by decoders and forecasters. The encoder s(.) and decoder ())-) can be formalized as:.
ži. 6(x1:)), Eud (2(1))
(5)

---

And the details of the forecaster can be found in Subsection 4.2.
4.1 Latent and Observed Transition Estimators
To estimate the prior distribution of latent variables 2t, further model the transitions 2 - Zz and 2t, X - X by estimating the independent noise ? and G. respectively. For the latent transition, first let r be set of trainable inverse transition function that take žt, and 2,-1 input 22 and estimate the noise term Et,5 i.e., E7, (Z,,i, Zz-1). And each r; is implemented by Multi-layer
Perceptron networks (MLPs). As result, we can develop a transformation Fiz (24-1, 24)
(2 ff,, whose Jacobian is Jx,- J J where -
0
Or
and J - diag 83
dr
Sequentially, we have Equation (6) via the change of variables formula.
dr
pog,(2 2 log 1(2--11 { log 02g,
(6)
According to the generation process, the noise e., should be independent of 2,- so we can enforce the independence of the estimated noise term é And Equation (6) can be further rewritten as
Or
logp(21::) log
Σ
Σ log
log
Ō21,1
where p(ér,i) is assumed a Gaussian distribution. Details of prior derivation are in Appendix D.1.3.
According the ELBO in Equation (4), we model the mapping from 21.7 to X1:T via any neural architecture directly. But the temporal relations might be omitted since they are not explicitly modeled, resulting suboptimal performance. Therefore, we employ similar solution to model the transition of observed variables. Specifically, we use another MLP to implement rf to estimate noise term & i.e., E, r (2,, 1, xt.)). Therefore, we can devise another transformation
0
No X(1,xx)), + (2,,x,-1, 2P) and the corresponding Jacobian J, = 0
0 where
Jo
J = ddagg
0r (z.,))
And we further have Equation (8):
Or dzi,;
Or
1og,(x,,X,-1 2,) = log.lo/pp---1. 27) +og
logp(x,/x.-11 p()) +og
(8)
As results, can model the relationship among 2,, Xt- and x, by enforcing the independence of the estimated noise term E,,, In practice, we assume p( also follows a Gaussian distribution, so we employ a prior Gaussian distribution to constrain p(E).) which is denoted by L
4.2 Observation Residual Forecaster and Sparsity Constraint on Mixing Procedure
According to Theorem 1, we should leverage the historical observations and the estimated latent variables for forecasting. Hence, different from previous methods [Cai et al., 2023, Fu et al., 2025] that only use latent variables, we devise a "residual" forecaster network as shown in Equation (9).
*1+1-T <p(2++1:7 m7(x1=)),
where 77 is an MLP for dimensionality reduction. Guided by the results of Theorem 3, the mixing procedure is assumed to be sparse. However, without further constraints, the estimated mixing struc- ture induced by the MLP-based architecture r? may dense since we only restrict the independence of the estimated noise. Moreover, the spurious causal edges may lead to the incorrect estimation of p(x):)) which further results in the suboptimal forecasting performance. To solve this problem, we propose to enforce the sparsity of the estimated mixing procedure by applying LI penalty on the partial derivative of & with respect to 2,, which are shown as follows:
Σ
0i,,
(10)
1m

---

Table Experiment results on simulation data for identifiability and forecasting evaluation.
Mathod
TOT
IDOL.
TDRI.
Inputs
Ours (xc, and Rt)
Upper Round (xy and xx)
Roseline (Xy)
09935)000034) 0.9524(000088)
0,375)(00245)
0.3572
0,5027(0.0045)
0.877%0.0662)
0,000500
0.85990000022
0.80730007736
0,43950.0097)
0.4195000017)
D. 0.653200 (062)
(a) MCC results for identifiability evaluation.
(b) MSE results for forecasting evaluation.
Table 2: Mean Square Error (MSE) results on the different datasets and different backbones.
Models
Len
LSTD
LSTD+ TOT
proceed-
procsed-T+ TOT
OneNet
OneNet+ TOT
OneNet-T
OneNet-T+ TOT
Online-T
Online-T-
TOT
1
0.377
0.374
1.537
1,001
0.380
0.361
0.411
0.364
0.502
0.385
ETTh2
24
0,541
0,532
2.908
2.360
0.532
0.525
0.772
0.691
0,830
0.733
48
0.616
0.564
4.056
3,956
0.609
0.562
0,806
0.773
1.183
0.874
1
0.081
0,081
0.106
0.102
0.082
0.082
0.082
0.077
0,085
0.077
ETTml
24
0.102
0.531
0.530
0.096
0.154
0.258
0.221
48
0.115
0.117
0.704
0.697
0.108
0.098
0.223
0.177
0,283
0.246
0.153
0.313
0.171
0.150
0,206
WTH
24
0.136
0.116
0.707
0.703
0.175
0.149
0.293
0.263
0,308
0.265
0.157
0.959
0.956
0.158
0.263
0,302
1
2.112
2.038
3,270
3,131
2.351
2.301
2.470
2.211
3,309
2.172
ECL.
24
5.907
5,852
2,000
4.713
4.671
11,339
48
1.411
1.413
7.192
7.403
2.201
2.143
4.567
4.445
11,534
4,574
0.231
0.333
0.312
0.241
0.222
0.236
0.211
0.334
0.212
Traffic
24
0,398
0,397
0.413
0,410
0.438
0.354
0.425
0,401
0.481
0.386
0.426
0.421
0.454
0.454
0.473
0.377
0.451
0.407
0,503
0,413
1
0.013
0.013
0.012
0.009
0.017
0.015
0,031
0.017
0.113
0,010
Exchange
24
0.039
0.037
0.129
0.102
0.047
0.030
0.060
0.042
0.1
0,035
48
0,043
0,042
0.267
0,195
0.062
0.039
0.065
0.051
0.168
0,040
Finally, the total loss of the proposed method can be summarized as follows:
Lotal Ly + aLr B(L -L
) + 7b
(1I)
where 0, and y are hyper-parameters.
Experiment
5.1 Synthetic Experiment
Data Generation We generate the simulated time series data with the fixed latent and observed causal process. We provide different synthetic datasets with different time lags, different dimensions of latent variables, latent structure, and mixing structure, respectively. Due to the page limitation, we put other synthetic experiment results in Appendix D.1.5 and report the results of Dataset A and B in the main text. Dataset A strictly follows the data generation process in Figure For dataset B, we do not consider the causal influence of x;-1 when generating x,, i.c., x; g (Z,, <f). Please refer to Appendix 1.I.1 1 for the details of the synthetic data and implementation details of our method.
Baselines. To evaluate our theoretical claims, we consider two different tasks. Specifically, to evaluate Theorem 2 and 3, we consider the identifiability performance of latent variables with Mean Correlation Coefficient (MCC) metric and choose IDOL Li et al. [2025] and TDRL Yao et al. [2022] as baselines. To evaluate Theorem we consider the performance of forecasting with Mean Square Error (MSE) metric. We choose MLPs as a forecasting model. Moreover, we let the models with only Xt, with x,, 2, and with xt, 2, baselines, our method and upper bound, respectively. We repeat each method over three different random seeds and report the mean and standard deviation.
Results and Discussion. Experiment results of the simulation datasets are shown Table 1. The identification results in Table (a) show that 1) the proposed TOT can well identify the latent variables, while the other methods can hardly achieve since they do not model the temporal causal inference of observations. 2) Even when the temporal causal inference of observations is omitted, TOT still achieves a better performance, since IDOL and TDRL are not suitable to the noisy mixing procedure. The forecasting results in Table (b) also align with the results from Theorem 1. Specifically, the MSE results of our method and upper bound method outperform than that of baseline, i.e., only considering the historical observations, demonstrating the importance of latent variables. We also find that our forecasting results are close to but weaker than that of the upper bound method, indirectly reflecting that the proposed method can well identify the latent variables.
8

---

Table Mean Absolute Error (MAE) results on the different datasets and different backbones.
Models
Len
LSTD
LSTD+
proceedT
proced-T+ TOT
OneNet
OneNet+ TOT
OneNet-T
OneNet-T+ TOT
Online-T
Online-T-
TOT
TOT
0.347
0.346
0.447
0.401
0.348
0.346
0.374
0.350
0.436
0.348
ETTh2
24
0.411
0.390
0.659
0.615
0.407
0.403
0.511
0.508
0,547
0.490
48
0.423
0.420
0.767
0.728
0.435
0.543
0.521
0,589
0.530
1
0.187
0.187
0.190
0.185
0.187
0.184
0.191
0.183
0,214
0.180
ETTml
24
0.217
0.447
0,442
0.224
0.288
0.346
48
0.249
0.249
0.521
0.505
0.238
0.229
0.371
0,312
0.403
0.369
0.200
0.200
0.143
0.192
0.221
0.198
0.276
WTH
24
0.223
0.207
0.382
0.376
0.225
0.229
0.345
0.333
0.367
0.326
0.239
0.340
0.362
1
0.226
0.221
0.286
0.282
0.254
0.253
0,411
0.302
0.635
0.214
ECL
0.380
1.196
48
0.294
0.289
0.431
0.422
0.348
0.357
0.534
0.402
1.235
0.343
1
0.225
0.224
0.268
0.256
0.240
0.221
0.236
0.203
0,284
0.200
Traffic
24
0,316
0.313
0.291
0.289
0.346
0.300
0.346
0.313
0.385
0.298
0.332
0.308
0.318
0.355
0.322
0,380
0,070
0.069
0.063
0.051
0.085
0.078
0.117
0,085
0.169
0,055
Exchange
24
0.132
0.129
0.211
0.189
0.148
0.118
0.166
0.137
0,213
0.124
48
0,142
0,142
0.300
0.260
0.170
0.137
0.173
0.152
0.258
0,132
0.045
ExchangelMSE)
0.145
ExchangelMAA)
0.90
(MS)
0.48
ETTh2(MMA)
0.461
0.042
0.041
0.140
0.137
0.80
0.46
0.039
0.039
0.135
0.70
0.44
0.130
0.033
0.034
0.125
0.130
0.122
0,124
0.50
0.523
0.40
0.42
0.399
0.402
0.030
0.120
0.40
0.38
0.027
10T
TOT:Sp
Tor
ToT-de
0.115
TOT
Tor-Sp
ToT
ToT-de
0.30
107
5p
ToTde
0.36
10T
TOT:5p
TO
do
Figure 4: Ablation study on the Exchange and ETTh2 datasets.
5.2
Real-world Experiment
5.3 Experiment Setup
Datasets We follow the setting of Wen et al. [2023] and consider the following datasets. ETT is an electricity transformer temperature dataset, which contains two separate datasets (ETTh2, ETTmI]. Exchange is the daily exchange rate dataset from eight foreign countries. Weather is recorded the Weather Station the Max Planek Institute for Biogeochemistry in Germany. ECL is an eectricity-consumine load dataset with the electricity consumption. Traffic is a dataset of traffic speeds collected from the California Transportation Agencies Performance Measurement System.
Baseline: We consider the following methods as our backbone networks: OneNet Wen et al. [2024] (including two model variants), Procced-T (TCN is abbreviated as T) Zhao and Shen [2024], Online- Zinkevich [2003], and LSTD Cai et al. [2025]. For each method, we only modify the model architecture and keep the training strategy fixed. We repeat each method over three random seeds. Since some methods report the best results on the original paper, we show the best results on the main text. We also provide the experiment results with
and variance over three random seeds in Appendix D.2.4. Please refer to Appendix D.2.2 and D.2.3 for implementation details and more experiment results based other backbone networks, respectively.
5.4 Results and Discussions
Experiment results of different benchmarks under different backbone networks are shown in Table 2 and 3. According to the experiment results, we can draw the following conclusions: (1) Our framework consistently outperforms all backbones on most datasets, demonstrating in real-world scenarios. (2) The gains with the OneNet backbone are larger than with LSTD or Proceed, since OneNet models only dependencies of observed variables, whereas LSTD and Proceed already leverage some information of distribution shifts led by latent variables. (3) Compared to L.SD, our improvements are smaller on a few tasks like ECL., this is because L.S employs similar neural architecture to identify latent variables. However, still match or exceed its performance
effectiveness

---

most benchmarks since our method is suitable for more flexible and realistic real-world scenarios. We further devise three model variants named TOT-Sp, TOT-kI, and TOT-de, which remove the sparsity constraint I.g, KI. divergence L.L, and the reconstruction loss I,, respectively. Experiment results in Figure reflect the necessity of each loss term and model architecture.
6 Conclusion
This paper presents a general theoretical and practical framework for online time-series forecasting under distribution shifts caused by latent variables. We first show that introducing these latent variables yields provable reduction in Bayes risk, and that this benefit scales with the precision of latent-state identifiability. We then show that both the latent states and their underlying causal transitions can be uniquely identified from just four consecutive observations, under mild injectivity and variability conditions. Building on these results, we design a plug-and-plas architecture with an additional temporal decoder and two independent noise estimators. Extensive experiments synthetic and real-world benchmarks confirm our theoretical guarantees and demonstrate consistent improvements several baselines. Future work aims to extend these results to related tasks, such as causal discovery on time series data and video understanding. Limitation: Our theoretical results assume fully observed, discretely sampled data. Extending this framework to continuous, irregular, or multi-rate sampling time series data remains an important and meaningful direction.
7 Acknowledgment
The authors would like to thank the anonymous reviewers for helpful comments and suggestions during the reviewing process. The authors would like to acknowledge the support from NSF Award No. 2229881, AI Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract ROIHL1
and grants from Quris AI. Florin Court Capital, and MBZUAI-WIS Joint Program, and the Al Deira Causal Education project. Moreover, this research was supported in part by National Science and Technology Major Project (2021ZD01 TEIZD0111500 1501),, National Science Fund for Excellent Young Scholars (62122022), Natural Science Foundation of China (U24A20233, 62206064, 62206061, 62476163, 62406078), Guangdong Basic and Applied Basic Research Foundation (2024A 1901, 2023 2023B1515120022
References
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. Conference and Workshop Neural Information Processing Systems, 32, 2019a. last.
Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In IEEE Conference on Computer Vision and Pattern Recognition, pages 11254-11263 2019b. last.
Oren Anava, Elad Hazan, Shie Mannor, and Ohad Shamir. Online learning for time series prediction. In Conference on learning theory, pages 172-184. Proceedings Machine Learning Research, 2013.
Sergul Aydore, Tianhao Zhu, and Dean P Foster. Dynamic local regret for non-convex online forecasting. Conference and Workshop on Neural Information Processing Systems, 32, 2019.
Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint 100.0180 2018.
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859 877, 2017.
Ruichu Cai, Zhiyi Huang, Wei Chen, Zhifeng Hao, and Kun Zhang Causal discovery with latent confounders based on higher-order cumulants. In International conference on machine learning, pages 3380-3407, PMLR, 2023.
10

---

Ruichu Cai, Haiqin Huang, Zhifan Jiang, Zijian Li, Changze Zhou, Yuequn Liu, Yuming Liu, and Zhifeng Hao. Disentangling long-show short term state under unknown interventions for online time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 15641-15644 2025.
Raymond Carroll, Xiaohong Chen, and Yingyao Hu. Identification and estimation of nonlinear models using two samples with nonclassical measurement errors. Journal of nonparametric statistics, 22(4):379-399, 2010.
Pierre Comon. Independent component analysis, concept? Signal processing. 36(3):287-314, 1994.
Imant Daunhawer, Alice Bizeul, Emanuele Palumbo, Alexander Marx, and Julia E Vogt. Identifiability results for multimodal contrastive learning. International Conference on Learning Representations 2023, 2023.
Nelson Dunford and Jacob T. Schwartz. Linear Operators. John Wiley Sons, New York, 1971.
Minghao Fu, Biwei Huang, Zijian Li, Yujia Zheng, Ignavier Ng, Guangyi Chen, Yingyao Hu, and Kun Zhang. Learning general causal structures with hidden dynamic process for climate analysis. arXiv preprint arXiv:2501.12500 2025.
Alex Graves and Alex Graves. Long short-term memory. Supervised sequence labelling with recurrent neural networks, pages 37-45, 2012.
Luigi Gresele, Paul K Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf. The incomplete rosetta stone problem: Identifiability results for multi-viee nonlinear ica. In UAL pages 217-227. Proceedings of Machine Learning Research, 2020.
Stephen Grossberg. Adaptive resonance theory: How a brain learns consciously attend, learn, and recognize a changing world. Neural networks, 37:1-47, 2013.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv:2111 0tt:221110009900 100396, 2021a.
Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré. Combining recurrent, convolutional, and continuous-time models with linear state space layers. Conference and Workshop Neural Information Processing Systems, 34:572-585, 2021b.
Albert Gu, Karan Goel, Ankit Gupta, and Christopher Ré. On the parameterization and initialization of diagonal state space models. Conference and Workshop on Neural Information Processing 2022.
Systems,
San Gultekin and John Paisley. Online forecasting matrix factorization. IEEE Transactions on Signal Processing, 67:5):1223-111 2018
Hermanni Hälvä and Aapo Hyvarinen. Hidden markov nonlinear ica: Unsupervised lcarning from nonstationary time series. In Conference UAI, pages 939-948. Proceedings of Machine Learning Research, 2020.
Yingyao Hu. Identification and estimation of nonlinear models with misclassification error using instrumental variables: A general solution. Journal of Econometrics. 144(1):27-61, 2008
Yingyao Hu and Susanne M. Schennach. Instrumental variable treatment of nonclassical mea- surement eиror models. Econometrica, 76(1):195-216, 2008. doi: https://do..or//0..111 0012-9682.2008 0000000..08....... URL https://om/imellbrary wiley www.y..co/doo//eeeeeeeeo/a wwtm///oo/ab//..... 1111/j. 2012-96833 2008 00823.
Yingyao Hu and Matthew Shum. Nonparametric identification of dynamic models with unobserved
state variables. Journal of Econometrics, 171(1):32 44, 2012.
Zenan Huang, Haobo Wang, Junbo Zhao, and Nenggan Zheng. Latent processes identification from multi-view time series. International Joint Conference on Artificial Intelligence-23, 2023.

---

Aapo Hyvärinen. Independent component analysis: recent advances. Philos. Trans. Royal Soc. A. 37101444)201104444 2013.
Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.
Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources, In Artificial Intelligence and Statistics, pages 460-469 469. PMLR, 2017.
Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks. (2(3):4994339 1999.
Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. The 22nd International Conference on Artificial Intelligence and Statistics, pages 859-868 PMLR, 2019.
Aapo Hyvärinen, Ilyes Khemakhem, and Ricardo Monti. Identifiability of latent-variable and cuuucurralequatioon models: from linear nonlinear. Ann Inst Stat Math, 76(1):1-33, 2024.
Ilyes Khemakhem. Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pages 2207-2217. PMLR, 2020a.
Ilyes Khemakhem, Ricardo Monti, Diederik Kingma, and Aapo Hyvarinen. Ice-beem: Identifiable conditional energy- baaeed deep models based on nonlinear ica. Conference and Workshop on Neural Information Processing Systems, 33:12768- 2020b.
Diederik Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. Reformer: The efficient transformer. arXiv
preprint arXiv:2001 2,44510001.04451,, 2020.
Lingjing Kong. Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akin- wande, and Kun Zhang. Partial disentanglement for domain adaptation. International Conference on Machine Learning, pages 1455-11 1442. Proceedings of Machine I Learning Research, 2022.
Lingjing Kong, Biwei Huang, Feng Xie, Eric Xing, Yuejie Chi, and Kun Zhang. Identification of nonlinear latent hierarchical models. Advances in Neural Information Processing Systems, 36: 2010-2032, 2023.
Sébastien Lachapelle and Simon Lacoste-Julien. Partial disentanglement via mechanism sparsity. CRL 2022, 2022.
Sébastien Lachapelle, Tristan Deleu, Divyat Mahajan, Ioannis Mitliagkas, Yoshua Bengio, Simon Lacoste-Julien and Quentin Bertrand. Synergies between disentanglement and sparsity: General- ization and identifiability in multi-task learning. In International Conference on Machine Learning. pages
Proceedings of Machine Learning Research, 2023.
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. The 41st international ACM SIGIR conference on research & development in information retrieval, pages 95-104, 2018
Ying-yee Ava Lau, Zhiwen Shao, and Yan Yeung. Fast and slow streams for online time series forecasting without information leakage. The Thirteenth International Conference on Learning Representations.
Te-Won Lee and Te-Won Lee. Independent component analysis. Springer, 1998.
Zijian Li, Zunhong Xu, Ruichu Cai, Zhenhui Yang. Yuguang Yan, Zhifeng Hao, Guangyi Chen, and Kun Zhang. Identifying semantic component for robust molecular property prediction. arXiv preprint arXiv:231 11 1,.04377 2023.
Zijian Li, Ruichu Cai, Zhenhui Yang, Haiqin Huang, Guangyi Chen, Yifan Shen, Zhengming Chen, Xiangehen Song, Zhifeng Hao, and Kun Zhang. When and how: Learning identifiable latent states for nonstationary time series forecasting. arXiv preprint 12767, 2024a.
12

---

Zijian Li, Yifan Shen, Kaitao Zheng, Ruichu Cai, Xiangchen Song, Mingming Gong, Zhifeng Hao, Zhengmao Zhu, Guangyi Chen, and Kun Zhang. On the identification of temporally causal representation with instantaneous dependence. arXiv preprint arXiv:2405.1532, 2024b.
Zijian Li, Yifan Shen, Kaitao Zheng, Ruichu Cai, Xiangchen Song, Mingming Gong, Guangyi Chen, and Kun Zhang. On the identification of temporal causal representation with instantaneous dependence. In The Thirteenth International Conference on Learning Representations, 2025. URL attps://openrerve/
Juan Lin. Factorizing multivariate function classes. Advances in neural information processing systems, 10, 1997.
Long-Ji Lin. Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning. 8:293-321, 1992.
Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Stratis Gavves. Citris: Causal identifiability from temporal intervened sequences. In International Conference on Machine Learning, pages 13557-13603. Proceedings of Machine Learning Research, 2022.
Chenghao Liu, Steven CH Hoi, Peilin Zhao, and Jianling Sun. Online arima algorithms for time series prediction. In Association for the Advancement of Artificial Intelligence, volume 30, 2016.
Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex X Liu, and Schahram Dust- dar. Pyraformer: Low-complexity pyramidal attention for long-rang range time series modeling and forecasting. In International conference on learning representations, 2021.
Yuejiang Liu, Alexandre Alahi, Chris Russell, Max Horn, Dominik Zietlow, Bernhard Schölkopf, and Francesco Locatello. Causal triplet: An open challenge for intervention-cente causal representation learning. In Conference on CLeaR, pages 553-573. Proceedings of Machine Learning Research, 2023.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Conference and Workshop on Neural Information Processing Systems, 30, 2017.
Zhenting Luan, Haoning Wang, Liping Zhang, Shansuo Liang, and Wei Han. Online prediction for streaming tensor time series. arXiv preprint arXiv:2403. 10.3.88322 18320, 2024.
Lutz Mattner. Some incomplete but boundedly complete location families. The Annals of Statistics, pages 2158-2162, 1993.
Mohamed Mejri, Chandramouli Amarnath, and Abhijit Chatterjee. novel hyperdimensional com- puting framework for online time series forecasting on the edge. arXiv preprint arXiv:2402.01999, 2024.
Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio. N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint 1xiv:1905.11437 2019.
Zijie Pan, Yushan Jiang, Dongjin Song, Sahil Garg, Kashif Rasul, Anderson Schneider, and Yuriy Nevmyvaka. Structural knowledge informed continual multivariate time series forecasting. arXiv preprint arXiv:2402. 122222 2024.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven CH Hoi. Learning fast and slow for online time series forecasting. arXiv preprint 2rXiv:2202.1167. 2022.
Quang Pham, Chenghao Liu, Doyen Sahoo, and Steven Hoi. Learning fast and slow for online time series forecasting. In The Eleventh International Conference Learning Representations, 2023. URL httpp:://ppereeteeee
David Salinas, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Deepar: Probabilistic forecasting with autoregressive recurrent networks. Int. J. Forecast, 36(3):1181-1199) 2020.
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning. Proceedings of the IEEE, 109(5):612-6)) 2021.
13

---

Xiangchen Song, Weiran Yao, Yewen Fan, Xinshuai Dong, Guangyi Chen, Juan Carlos Niebles, Eric Xing, and Kun Zhang. Temporally disentangled representation learning under unknown nonstationarity. Conference and Workshop on Neural Information Processing Systems, 36, 2024.
Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances neural information processing systems, 34:16451-16461 2021.
Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. Mien: Multi-scale local and global context modeling for long-term series forecasting. In The eleventh international conference on learning representations.
Qingsong Wen, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan, et al. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. Advances in Neural Information Processing Systems, 36:69949 2023.
Qingsong Wen, Weiqi Chen, Liang Sun, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan, al. Onenet: Enhancing time series forecasting models under concept drift by online ensembling. Conference and Workshop on Neural Information Processing Systems, 36, 2024.
Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with correlattnn for long-term series forecasting. Conference and Workshop on Neural Information Processing Systems. 34:22419-22430 2021.
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Timesnet: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:22 10.022 2022.
Shaoan Xie, Lingjig Kong, Mingming Gong, and Kun Zhang. Multi-domain image generation and translation with identifiability guarantees. The Eleventh International Conference on Learning Representations, 2023.
Hanqi Yan, Lingjing Kong, Lin Gui, Yuejie Chi, Eric Xing, Yulan He, and Kun Zhang. Counterfactual generation with identifiability guarantees. Advances in Neural Information Processing Systems, 36, 2024.
Dingling Yao, Danru Xu, Sébastien Lachapelle, Sara Magliacane, Perouz Taslakian, Georg Martius, Julius von Kügelgen,, and Francesco Locatello. Multi-view causal representation learning with partial observability, arXiv preprint arXiv:231 1.04056, 2023.
Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. arXiv preprint #rXiv:2110.05422 2021.
Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. Advances in Neural Information Processing Systems, 35:26492-26503, 2022.
Ying yee Ava Lau, Zhiwen Shao, and Dit- Yan Yeung. Fast and slow streams for online time series forecasting without information leakage. In The Thirteenth International Conference on Learning Representations, 2025. URL https::/openrevieee wttpre/rrriieeeeeeeooouuuiiiieie www.:oruummoeeee/////o Id=InnEEygM.....
Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Association for the Advancement of Artificial Intelligence, volume 37, pages 11121-11128, 2023.
Cheng Zhang, Judith Bütepage, Hedvig Kjellström, and Stephan Mandt. Advances in variational inference. IEEE transactions pattern analysis and machine intelligence, 41 (188:200882002 2018.
Kun Zhang and Laiwan Chan. Kernel-based nonlinear independent component analysis. In International Conference on Independent Component Analysis and Signal Separation, pages 301- 308. Springer, 2007.
14

---

Kun Zhang, Shaoan Xie, Ignavier Ng. and Yujia Zheng. Causal representation learning from multiple distributions: general setting. arXiv preprint arXiv:2402.00052, 2024a.
Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less is more: Fast multivariate time series forecasting with light sampling-oriented mlp structures, arXiv preprint arXiv:2207.0111 2022.
YiFan Zhang, Weiqi Chen, Zhaoyang Zhu, Dalin Qin, Liang Sun, Xue Wang, Qingsong Wen, Zhang Zhang, Liang Wang, and Rong Jin. Addressing concept shift in online time series forecasting: Detect-then-adapt, arXiv preprint arXiv:2403, 14949, 2024b.
Lifan Zhao and Yanyan Shen. Proactive model adaptation against concept drift for online time series forecasting. arXiv preprint arXiv:2412.08435, 2024.
Yujia Zheng and Kun Zhang. Generalizing nonlinear ica beyond structural sparsity. Advances in Neural Information Processing Systems, 36:13366-13355. 2023.
Yujia Zheng, Ignavier Ng, and Kun Zhang. On the identifiability of nonlinear ica: Sparsity and beyond. Advances neural information processing systems, 35:16411-16422, 2022.
Haoyi Zhou, Shanghang Zhang. Jieqi Peng. Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Association for the Advancement of Artificial Intelligence, volume 35, pages 11106-11115, 2021.
Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th international conference on machine learning (ieml-03), pages 928-939- 9366,, 2003.
15

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificcton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our works address the identification problem of hierarchical temporal causal representation.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the viewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
16

---

Justification:
We have provided the limitation in the conclusion explicitly,
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
We have provided the full set of assumptions and complete proof.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We have provided implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We have provided the code in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/ gguides/CodeSuumiiee
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: SoodeeSbmissione
/ //nips. /twpss.ccppbb.... www.c//wwwccggudeeeeeeeooeeepeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
18

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer:
[Yes]
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We repeat all the experiments over three random seeds and report the mean and standard deviation.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-simma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided in the implementation details.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
19

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: //tppr////eurpss.... cc wpppplic//thisssiiieeeeoooppppee
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have provided it in the appendíx.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
20

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
21

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https : //teerrps..... cttpiipps...//ooooooee cc/Conffernces/2200 /LLM) for what should or should be described.
22

---

Supplement to
"Online Time Series Forecasting with Theoretical Guarantees"
Appendix organization:
Related Works
24
A.1 Time Series Forecasting...
24
A.2 Online Time Series Forecasting
24
A.3
Continual Learning
24
A.4
Causal Representation Learning
24
B Notations
25
Proof
26
C.1 Proof of Theorem
26
C.2 Proof of Theorem 2.
................................
27
C33 More Discussion of injective linear operators
C,4 Monotonicity and Normalization Assumption
31
C.5 Definition of Neighborhood.
31
C.6 More Discussion of Uniqueness of Spectral Decomposition
................................
C.7 Proof of Theorem 3.
32
C.8 More Discussion on the Sparse Mixing Procedure
36
Experiment Details
36
D.1
Synthetic Experiment
36
D.1.1
Data Generation Process
36
................................
D.1.2
Evaluation Metric
................................
36
D.1.3
Prior Likelihood Derivation
................................
37
D.1.4
Evident Lower Bound
................................
38
D.1.5
More Synthetic Experiment Results
................................
38
D.2
Real-world Experiment
................................
38
D.2.1
Dataset Description
38
D.2.2
Implementation Details
38
D.2.3 More Experiment Results
43
D.2.4
Experiment Results Mean and Standard Deviation
44
Broader Impacts
45
23

---

A Related Works
A.1 Time Series Forecasting
Recent advancements in time series forecasting have been driven by the application of deep learning techniques, which have proven to be highly effective in this area. These methods can be broadly categorized into several groups. First, Recurrent Neural Networks (RNNs) are commonly used for capturing temporal dependencies by leveraging their recursive structure and memory to model hidden state transitions over time [Graves and Graves, 2012, Lai et al., 2018, Salinas et al., 2020]. Another popular approach is based on Temporal Convolutional Networks (TCNs), which employ a shared convolutional kernel to model hierarchical temporal patterns and extract relevant features [Bai et al., 2018, Wang et al., Wu et al., 2022]. Additionally, simpler yet highly effective methods, such as Multi-Layer Perceptrons (MLP) [Oreshkin et al., 2019, Zeng al., 2023, Zhang et al., 2022, Li et al., 2024a] and state-space models [Gu et al., 2022, 2021b,a], have also been utilized in forecasting tasks. Among these, Transformer-based models have emerged as particularly noteworthy, demonstrating significant progress in the time series forecasting domain [Kitaev et al., 2020, Liu et al., 2021, Wu al., 2021, Zhou et al., 2021]. Despite the success of these methods, they are generally designed for offline data processing, which limits their applicability to real-time- online training scenarios.
A.2 Online Time Series Forecasting
The rapid growth of training data and the need for real-time updates have made online time series forecasting more popular than offline methods [Anava et al., 2013, Liu et al., 2016, Gultekin and Paisley, 2018, Aydore et al., 2019]. Recent approaches include Pan et al. [2024], which uses structural consistency regularization and memory replay to retain temporal dependencies, and Luan et al. [2024], which applies tensor factorization for low- complexxy online updates. Additionally, Mejri et al. [2024] addresses nonlinear forecasting by mapping -wwdimensional series to high-dimensiooal spaces for better adaptation. Online forecasting is widely used in practice due to continuous data and frequent concept drift. Models are trained over multiple rounds, where they predict and incorporate new observations to refine performance. Recent work, such as [Pham et al., 2022, Cai et al., 2025, yee Ava Lau al., 2025] and Wen et al. [2024], focuses optimizing fast adaptation and information retention. However, simultaneously adapting to new data while retaining past knowledge can lead to suboptimal results, highlighting the need decouple long- and short-term dependencies for improved predictions. However, most of these methods rarely explore the theoretical guarantees for online time series forecasting.
A.3 Continual Learning
Our work is also related to continual learning. Continual learning is an emerging field focused on developing intelligent systems that can sequentially learn tasks with limited access to prior experience [Lopez-Paz and Ranzato, 2017]. A key challenge in continual learning is balancing the retention of knowledge from current tasks with the ficxibility to learn future tasks, known as the stability- plasticity dilemma [Lin, 1992, Grossberg, 2013]. Inspired by neuroscience, various continual learning algorithms have been developed. This approach aligns with the needs of online time series forecasting, where continuous learning allows models to update in real time as new data arrives, improving their ability to adapt to changing data dynamics and enhancing forecasting accuracy.
A.4 Causal Representation Learning
To ensure the identifiability of latent variables, Independent Component Analysis (ICA) has been widely used for causal representation learning [Yao et al., 2023, Schölkopf et al., 2021, Liu et al., 2023. Gresele et al., 2020]. Traditional ICA methods assume a linear mixing function between latent and observed variables [Comon, 1994, Hyvärinen, 2013, Lee and Lee, 1998, Zhang and Chan, 2007], but this is often impractical. To address this, studies have proposed assumptions for nonlinear ICA, such as sparse generation processes and auxiliary variables [Zheng et al., 2022, Hyvärinen and Pajunen, 1999, Hyvärinen et al., 2024, Khemakhem et al., 2020b, Li et al., 2023]. For example, Aapo et al. confirmed identifiability by assuming latent sources belong to the exponential family, with auxiliary variables like domain and time indices [Khemakhem et al., 2020a, Hyvarinen and Morioka, 2016, 2017, Havarinen al., 2019]. In contrast, Zhang al. showed that nonlinear ICA can achieve
24

---

component-wise identifiability without the exponential family assumption [Kong et al., 2022, Xie al., 2023, Kong al., 2023, Yan al., 2024].
Other studies also use sparsity assumptions to achieve identifiability without supervised signals. For instance, Lachapelle al. applied sparsity regularization discover latent components [Lachapelle et al., 2023, Lachapelle and Lacoste-Julien, 2022], while Zhang et al. used sparse structures to maintain identifiability under distribution shifts [Zhang al., 2024a]. Nonlinear ICA has been used for time series identifiability [Hyvarinen and Morioka, 2016, Yan et al., 2024, Huang et al., 2023, Hälvä and Hyvarimen, 2020, Lippe et al., 2022]. Aapo et al. used variance changes detect contrastive learning was applied
nonstationary time series data identifiability, while permutation-bast for stationary time series [Hyvarinen and Morioka. 2016]. More recently, techniques like TDRL [Yao et al., 2022], LEAP [Yao et al., 2021] and IDOL [Li et al., 2025] incorporated independent noise and variability features. Additionally, Song et identified latent variables without domain- specific observations [Song et al., 2024), and Imant et al. used multimodal comparative learning for modality identifiability [Daunhawer et al., 2023]. Yao al. showed that multi- paalii-prrppectvve causal representations remain identifiable despite incomplete observations [Yao al., 2023]. However, these methods typically assume invertibility in the mixing process. This paper relaxes that assumption and provides identifiability guarantees for online time series forecasting.
B
Notations
This section collects the notations used in the theorem proofs for clarity and consistency.
Table A4: List of notations, explanations, and corresponding values.
Index
Explanation
Support
n
Number of variables
n E N+ i, k, b, E {1,-..,0, E N+
i,J, k
Index of latent variables
Time index
Variable
X2
Support of observed variables in time-index t Support of latent variables Observed variables in time-index t
X, C R
Z,
2, C g"
X
X E R"
Z;
Latent variables in time-index t
2, E R
Ur
Xt Zr, Xt)
ut E p
E
Independent noise of mixing procedure
Et ~ Pep
Independent noise of the latent transition of Z₁,
tti Pet.:
2
Function
P-
b)
Density function of given b Nonlinear mixing function
g())
R 122x+1 - IR"
f((-)
Transition function of Zι,i
R nn+1 IR
h(-)
Invertible transformation from to 2,
R" - R
(.))
Permutation function
R" + IR.
F
Function space
Mu
Markov network over u, Encoder for 2!
th
Decoder
r
Noise estimator of e Noise estimator of ET,
ri
Symbol
Bayes Risk
J. k
Jacobian matrix of r
25

---

C Proof
C.1 Proof of Theorem
Theorem Al. (Predictive-Risk Reduction via Temporal Latent Variables) Let X1, Zt. and 2s be the observed variables, ground-truth latent variables, and the estimated latent variables, respec- tively. We let Xt-rit =
{X ,X+) {x be the historical (T + 1)-step observed variables. More- over, let Ro; Ry: and R₂ be the expected mean squared enror for the models that consider (x--ra): 1x, 2 and (x 2 respectively. Then, in general, we have Ro 2 RR 2 Ra, and 2; identifiable we have Ro Rr Rz.
Proof. Suppose that the observed variables x,, ground-truth latent variables 24, and estimated latent variables 2, follow the data generation process shown in Figure 1. We let F= (x == a (xt- (xt-z:t: 2t), and Fz == ox1,-1:t 2) be the information o-algebras generated by the variables available the forecaster in the three settings (only observed variables, observed and ground truth latent variables, observed and the estimated latent variables). And the corresponding optimal
Bayes forecaster be formalized x |[xx+1/x(-ra]] x F[xx+1|xxx-00 ze]. and
(2) E[xx+1 [x-rat, Re]. respectively. Then we let Ro, Rz, and R₂ be the corresponding Bayes risk. By using the law of total expectation, we have:
Ro Ex,,Ixx-.xXX++ x(42)) Ex
[Bx (x- [[x++
x 11
(A1)
Ex
Var(xx++1 X (x)]):)]
R₂ = Ex Xe [xx+1 - -x(41) Ex de Ex 12/111100 - [xx+1
xx+11 )3]
(x))
(A2)
Ex Var -1 2r)l,
R₂ [(x+1 xx+1) ) Ex &
(2)
Ex 1xt- [[x++
(2)
(A3)
Ex rer [Var x:: 2,)
By using the law of total variance, Equation (A1) can be written
Ex ric
Var(xx= rx)]
E Ex (ar (xt Zt]
Ro
(A4)
F Fx Var,
(E X++1 X1 Xt- zc)l
and
Exi vat
[Var(x++1 (x- = Ex, [Va (x-- 2:)]
Ro
R4
(AS)
E Ex.-r. Vara,
(E(x,++ Xy_ r.ts 2,1)
Suppose in Equation (A4), O. That implies Varz,/x (E[x141 Xt- 2e)) = O, meaning that
2. does not have any influence the mapping X:-rit -> X which false because z, Ml X Xt-rat- This leads to a contradiction, which implies e > 0 and hence R. > Ru.
Consider Equation (A5). Here, in general, for any 2, we have 2 0 and hence Ro > R2.
Then we leverage the law of total variance again and have:
Varz, Ix- (B(x,+1 X 2z))
-x mid [Vrrz, Xx -... [E] X1 Xt r:t: 2x]) + Varz, x (E |xi B[xe+ Xx- 2el)
-Eg,|xe [Var,,
.. [E[x;-1XX 1 z, + Var x1 EE [[ X (122 201)
(A6)
Then we take the expectation on both sides of Equation (A6) and have
Exi aa [Vaz,, Xe- (E (E(x Xt- 2x]) Ex, Varz, xx- (El [ (e 2n])]
(A7)
E2, [Vazz, ru Eix X1- r:t: 2:])]
26

---

Similarly,
E2, xt-rit (arz |x, v. (B) [B[x++1 ( X zel)]
means that
(x a. (E(x++1 Xe-rit zi]) D, implying that B[x++1 Xt-rit; Zel is constant, i.c.. Z; and 2, have a one-to-one correspondence. Therefore, in general, 2 e, and iff z, is identifiable.
By combining Equation (A4) and (A5), in general, have R. 2 R₂ Rz, and if 2, is identifiable we have Ro > R2= = Rz.
C.2 Proof of Theorem 2.
For better understanding of our proof, begin by introducing an additional operator represent the point-wise distributional transformation. For generality, we denote two variables as a and b, with corresponding support A and B.
Definition (Diagonal Operator) Consider two random variable a and b, density functions pe and are defined on some support A and B, respectively. The diagonal operator Dbja maps the density function Pa to another density function Da Pa defined by the pointwise multiplication of the function pbja at fixed point b:
Pbia p) Dija o Pa; where Doja (bja (bl I).
(A8)
Theorem A2. (Block-wise Identification under Adjacent Observed Variables.) Suppose that the observed and latent variables follow the data generation process. By matching the true joint distribution of 4 adjacent observed variables, i.e., {x1-2, X1,1, X(+1). we further consider the following assumptions:
Al (Bound and Continuous Density): The joint distribution of x, z and their marginal and condi- tional densities are bounded and continuous.
A2 (Injectivity): There exists observed variables X₁ such that for any x, E Xr there exist X;-1 E 2 and a neighborhood N2 around (x1,X1-1) such that, for any (4,))111 E N Lx.xcnlx 2.8, injective; Lxq11xxie Lx/xx 2.x, 12.1xx injective for any X, E X, and x,-1 X-1, respectively.
A3 (Uniqueness of Spectral Decomposition) For any E X₁ and any # 2, E Zú, there exists xi-1 X1-1 and corresponding neighborhood N" satisfying Assumption A2 such that, for some
(x,, x(-1) Nr with X₁ # Xt, X;-1 Xt-1i
kxxi, Nty Xt X 1 k1)
C do for any 2h Z, and constant C.
ii k(x,, X Xt 2,) * (x N2, X2 81- 21), where
Pxi 1 (x
Xt-1, Zz Pxe, ber-
(Xe Xx- Zi)
k(x,, x,,
(A9)
Pxi 1:24 Xx
X,-1 Ze Pxx (xc,
1
X Zz)
Suppose that we have learned (g, f, pε) to achieve Equations (1), then the combination of Markov state Zt, Xt is identifiable, i.e., [Ze, &c] = (2,,, xx), where H is invertible and differentiable.
'Please refer to Appendix C.5 for the definition of neighborhood.
27

---

Proof. By the definition of data generation process, the observed density Px++1 Ne, Nt- 1.-22 equals
Pre+i X1-1
Pxc+i Xe de- 1 Xx
dz dZ
If
Px 1 PR. Xe- Xe-2 dzz
/f Presi Ne 2e Pre - Be Pai Xe- X d2
If Preci 2i Pxe 2, Pae (x, Px d2,d2;-1-
[/ Pxi+i Ne te Per te Pee dz
According to the property of Markov process,
1x De Dec- ( SPanies iske 2,x1 d dac
Px 1,X:Xi 1.31
f Pxet Ire Pxee Ie- 1. PR: Prcxxi 18k1 dze.
(A10)
In operator notation, given values of (Xt, Xe-1) E Ne x 14-1, this
Lx. Xt- 1 Lixes xx zz Dxx/xi-1 Lz. X1
(A11)
After obtaining the representation of observed density function, furthermore, the structure of Markov process implies the following equalities:
P211186011 1:2:
Px++ 2x 1xx, dZz,,
=f 2x1,2.2x 1 Pr 1886 18t4 dz--1-
PR 1.31
(A12)
In operator notation, for fixed Xe, Xt-1. the above equations are expressed:
Xx T.x.- 1xx
Tx 1-8-22
Txx,, (xi- 1.2--1 112
(A13)
Substituting the second line into the first, we get
1x 188-2
Tixet1lxx 2 Txx X 18
+> Lix. X 1 Lz, 1.8 L
X (xet/xe.2
-1
Lxx Nt. 1:8
(A14)
The second line uses Assumption A2. Next, eliminate Lz--1 Ne- Ne-2 from the above. Again, using the conditional independence of Markov process, we have:
11-11 Px. Px-xxx-1,81-22 = Px.|xx-1 2i 1
dz
(A15)
which can be represented in terms operator (for fixed xx- 1) as:
Lx..x.- X Lxx X1 1836 Li 1 11
Loz, 1:X: 1
L 10--1 Lxxxx 18Nt
(A16)
The R.H.S. applies Assumption A2. Hence, substituting the above into Eq. A14, we obtain the desired representation:
1-1 Lxx 2c-1 xe 1. Lxx, 11xt Xens (e. L 1
1x. x141 bei Lxx 1. N. Ne-2 L x1 1:20-2 Lixe Ne- Re-1"
(A17)
28

---

The second line applies Assumption A2 to post-multiply by Lx, 1x,xx-1 -18-2"1----2" while in the third line, we postmultiply both sides by Lx, 1
For each Xt, choose X and a neighborhood N around (Xt, xt-1) to satisty Assumption A2 and A2, and pick a (x+) within the neighborhood N to satisfy Assumption A2. Because (&e; x(-1) E N also Xt-1)) (Xt, xt-1) E N By the representation of observations in Eq. A11, we have
1X Lx x Dx,1xxxxx 1e. X1-
The first term the R.H.S., Ixxe1|xx, R. does not depend X₃-1, 11- and the last term Lz,, 11-,; does not depend on x,. This feature suggests that, by evaluating Eq. A1 at the four pairs of points (xx,X,-1) x. (x,, xt-1), (x1)),-1), (xe, 8t-1), each pair of equations will share one operator in common. Specifically;
Lx 1 Lx X1 2e Dx X2- 1 Liz, 11-211
(A18)
Txx, 81:00 11 118--2
[x R Dx,(xe- I2, 18-221
(A19)
Lx,- 1 Lx 81. Dx,181-11 Li 2. Re- 12-22
(A20)
12 Lx 8.. Dx.181-1118 L 2e N2-22
(A21)
Assumption A2 implies that Lixas R. is invertible. Moreover, Assumption A2 implies
Pxe 1x--1 (X: x 2t) for all Zr so that D2.1x--1.2, invertible. We can then solve
for I x2 1xx from Eq. A19 as
D2 8 1 Zt xe-i & Lx 1 1.81
1-1
Lz 1.21-2"
(A22)
Plugging this expression into Eq. A18 leads to
Lxx++1 Nr. Xt- 1x 1 D
R 1x Lx
L
Lxxe 11,44188xx111 21-11-1
(A23)
Lemma 1 of Hu and Schennach [2008] shows that, given the injectivity of Lxs 2.81 1,81, in
Assumption A2, we can postmultiply by L-1 x1,11 1.2--2 to obtain:
Xe-2 xx+ 1 Lx Dx,1x,,1 1x,1x,110 x1
D
L 18 (A24)
Similarly, manipulations of Eq. A20 and Eq. A21 lead to
= Ix 1 XXt1 &
Tix.4. (8, Dx, 1. x//8, 1 "xcilxeem (A25)
D-1
I=
Assumption A2 guarantees that, for any X4, (X,, X, xc-1) exist so that Eq. A24 and Eq. A25 are valid operations. Finally, we postmultiply Eq. A24 by Eq. A25 to obtain:
AB Lx Ix.. Dxi 12.10-11.0 D (Ixe+ 18.. Lxx (2))))
DR.181-1.24 D=1 Dx.Re-n.e
Xeas 18..2.
=
Lx 1x11118011
D-1 (xxixx am x Dx.sx,, 1:21 D De(8811115
X X2,
1-1
Lx Dxy, 1.
(A26)
x111 2/
where
(Dx.R.x (zi) (Dxilo 1.0 D 1 Dx, 1.a, D-1
12 (zc)
Pxe x (Xt Xt-1, Ze Pxs Pxxebe Ixc (Xc,
X, 1, 21)
h((z))
Px, xi- m. / The X;-1, Ze Poe, 1x (X::
R4- Zt)
= k(x,, X4, X4- X Zz (z(e).
(A27)
By matching the marginal distribution of observed variables, we can define the operator ÁB as the estimated counterpart of the operator AB, constructed using the estimated densities of x,-2. 8,-1, x,, x and 2, Since both the marginal and conditional distributions of the observed variables
29

---

are matched, the true model and the estimated model yield the same distribution over the observed variables. Therefore, we also have:
ÁB = AB.
(A28)
Eq. A26 implies that the observed operator AB has an inherent cigenvalue-cigeefunttioo de- composition, with the cigenvalues corresponding to the function ,(,,,, X2, x; x; Z,) and the cigenfunctions corresponding to the density Px,-1|x,,2, (- X,, Zz). Furthermore, Eq. A28 implies
that AB and AB admit the same cigendecomposttions, which are similar to the decomposition in Hu and Schennach [2008] or Carroll et al. [2010]. Assumption A2 ensures that this decomposition is unique. Specifically, the operator AB the I.H.SS. has the same spectrum as the diagonal operator xx,,,..., xx,,xx1 111 Assumption A2 guarantees that the spectrum of the diagonal operator is bounded. Since operator is bounded by the largest element of its spectrum, Assumption A2 also implies that the operator AB is bounded, whence we can apply Theorem XV.4.3.5 from Dunford and Schwartz [1971] to show the uniqueness of the spectral decomposition of bounded linear operators:
Lxp41 I..2, = CL 22, P-1 Dx 1.K, 1.a, = -PDx, fe. &
(A29)
where C is a scalar accounting for scaling indeterminacy and P is a permutation on the order of discussed in [Dunford and Schwartz, 1971]. These forms of indeterminacy are analogous to those in eigendecomposition, which can be viewed as a finite- dinte-dimensooall special case. We will show how to resolve the indeterminacies in eigen(spectral) decomposition.
elements in xx &
First, Eq. A29 itself does not imply that the eigenvalues k(x,, Xx, X,-1, x,- Zr) are distinct for different values Z₁. When the eigenvalues are the same for multiple values of 2,, the corresponding eigenfunctions are only determined up to an arbitrary linear combination, implying that they are not identified. Assumption A2 rules out this possibility, and implies that for each x can find values &1, X, 1,, such that the eigenvalues are distinct across all Z!.
Second, since the normalizing condition
San. 1 ddc,+1 1 1
(A30)
must hold for every 2,, one only solution set C' = 1, that is, the scaling indeterminacy is resolved.
Ultimately, the unorder of eigenvalue/eeigentunctiosss distributions Xt, X,-1 X hence, the operator, /xx,, corresponding to the set {Px..1
left. We have match the observational
Xt. } for all xt. 2t, admits a unique solution (orderibng ambiguity of eigendecompositionn only changes the entry position):
(Px |x, 2. (: Xx, 2+) = (Px ( (x++1
Xx, 2,))
for all x,, Zt. Xx, 2,
(A31)
Due to the set is unorder, the only way to match the R.H.S. with the L.H.S. in consistent order is to exchange the conditioning variables, that is,
(px (: x, (1)
Zi (()), Px xizi ( (preiile (x))) 2(11),, Pxeti &
(2) (2)
2z .... xt
(2) (2)
xx Zz
...}
Peeet (x. (. x 2. (x())),, Px
(1))
(m(2)) (n())) ...
2,
|Pac |8. 2, (- & (TT(1))
2 (=(1))), P x1 (2)) 2 (
where superscript (.) denotes the index of the conditioning variables [xt, 2r). and π is reindexing the conditioning variables. We use relabeling map H to represent its corresponding value mapping:
H(xx, Px
= 2x. Xt, 2t), for all X1, Ze, X1, 2n
(A32)
By Assumption A2, different x* corresponds to different P (. H (xx, 2z)), there is no repeated element (Px.t1xx. (. Hx,,, 2 (and (Px41180.2 & 2.))). Hence, the relabelling map H is one-to-one.
Furthermore, Assumption A2 implies that Px.. (- (xe, determines unique H (X1, ze).
The same holds for the Px..1/s,, 1 Xe, 2,), implying that
1x z. (. H Hxx, Z:)) 12.41180,2 Xxc, 21) = xc, 2, H H 21),
(A33)
30

---

implying that X,, Z, is block-wist -iice identifiable.
Next, suppose the implemented MIP used in the transition module is differentiable, then assert that there exists functional such that M [Px.-al ( X2, ze)] H(x,, 2,) for all 2, E 2, and X₁ Xr, where H is differentiable, that is, we can learn differentiable function H that
M 1& ( X z+)] M ax ( (X,, 2,)) H (x,, zt),
(A34)
which is equal to X 2n only if H is differentiable.
L
C.3 More Discussion of injective linear operators
linear operator can be intuitively understood as function that maps one distribution of random variables to another. Specifically, when assume the injectivity of linear operator in the context of nonparametric identification, we are asserting that distinct input distributions of the operator correspond to distinct oulput distributions. This injectivity ensures that there is no ambiguity in the transformation from the input space to the output space, making the operator's behavior predictable and identifiable. An example from real-world scenario can be seen in weather forecasting. The temperature on given day can be influenced by several previous days' temperatures. If we view the relationship between past and future temperatures each unique pattern of past temperatures leads to a distinct forecast for the future temperature. The injectivity of this operator ensures that the mapping from past weather data future forecasts does not result in ambiguity, allowing for more accurate and reliable predictions.
linear operator, injectivity would mean that
For better understanding of this assumption, we provide several examples that describe the mapping from Pa + Pb. where a and are random variables.
g(a). where g is an invertible function. Example 2 (Additive Transformation). b = a where p(c) must not vanish everywhere after the Fourier transform (Theorem 2.1 Mattnr [1993]).
Example 1 (Inverse Transformation).
Example 3. b ({) + e, where the conditions from Examples I and 2 are required.
Example (Post-linear Transformation). b = 91 (g())())+) +e), a post-nonlimear model with invertible nonlinear functions 91, 92. combining the assumptions in Examples 1-3.
Example (Nonlinear Transformation with Exponential Family). g(a, e). where the joint distribution p(a.J b) follows exponential family.
Example (General Nonlinear Transformation). b= g(a e), general nonlinear formulation.
Certain deviations from the nonlinear additive model (Example 3), e.g., polynomial perturbations, can still be tractable.
C.4 Monotonicity and Normalization Assumption
Assumption 1 (Monotonicity and Normalization Assumption [Hu and Shum, 2012)). For any
X X₁, there exists a known functional G such that [Px.n Xx, (-1x,, 2:)] is monotonic in z, normalize 2, = [Px.miXx, (. (:1x,: ):
C.5 Definition of Neighborhood
Definition 6 (Neighborhood), Givien point in metric space, and a positive number r, the neighborhood No of is defined as:
(²)x) = (y d(zz y) < <
(A35)
C.6 More Discussion Uniqueness Spectral Decomposition
This assumption essentially states that, in order to identify the latent variables of the system, it is necessary observe four different transitions the observed variables that are governed by the same latent variables. For a better understanding of this assumption, we provide an economic model. Consider an economic model where x, represents economic regime (such as recession or period of growth). To accurately identify the economic regime, we would need to observe inflation under four distinct scenarios: transitions from high- inflation state to a fow-inflation state, and from low-inflation state to high-inflation state, under
inflation rate at time I, and z; represents
31

---

different historical conditions. These four observed inflation transitions allow us to identify whether the economy is in a recession or growth phase, based on the changes inflation behavior.
This assumption is straightforward to satisfy in real-world economic modeling, especially when there is access to sufficient historical inflation data. In practice, there are often multiple transitions between inflation states over time, corresponding to shifts in the economic regime (e.g., moving from high inflation during an economic boom to low inflation during a recession). By collecting enough observations across different periods of economic change, this assumption can be easily fulfilled, ensuring that we can identify the underlying economic regime with confidence.
C.7 Proof of Theorem 3.
Lemma A1. (Component-wise Identification of 2: with instantaneous dependencies under sparse causal influence on latent dynamics.)Li et al. [2025] For series of observed variables
Xt E IR and estimated latent variables à, E R" with the corresponding process f p(E), g. where g is invertible, suppose the process subject to observational equivalence x, = =(z)). Let C & {z ] E R and Me. be the variable set of two consecutive timestamps and the corre- sponding Markav network, respectively. Suppose the following assumptions hold:
A4 (Smooth and Positive Density): The conditional probability function of the latent variables e, is smooth and positive, i... p(ce 22)) is third-order differentiable and ((c) > over R²",
A5 (Sufficient Variability): Denote
|Mai as the number of edges in Markov network Me. Let
mr(m) = 8 logp(ca/z(-2)
8 logp(ce 71-2)
d0t -2m 2,m
(A36)
8 log p(c)/2222) 8 log 21-2) 1
8 (ogglci((zz22)
Ocr 221
Ocr, (L.)EE(MM.)'
where @ denotes concatenation operation and (i,5) E E(Mc,) denotes all pairwise indice such that Ct,j, are adjacent in Me..- For m [1,...,. ,nl, there exist 4n + [Me, different values of 2,m such that the 4n + MMe,l values of vector functions w(m) are linearly independent.
A6 (Latent Process Sparsity): For any Zzt E Zt. the intimate neighbor set of za is an empty set.
When the observational equivalence is achieved with the minimal number of edges of the estimated Markov network of Me, there exists a permutation πt of the estimated latent variables, such that z and (x)e is one-to-one corresponding, i.e., za is component
identifiable.
Proof. The proof can be summarized into three steps. First, we leverage the sparsity among latent variables to show the relationships between ground-truth and estimated latent variables. Sequentially, we show that the estimated Markov network Me. is isomorphic to the ground woound-truth truth Markov networks MM,- Finally, show that the latent variables are component-win identifiable under the sparse mixture procedure condition.
Stepl: Relationships between Ground-truth and Estimated Latent Variables. We start from the matched marginal distribution to develop the relationship between Z, and 2, as follows:
p(x.) = p(xx) - p()(2.) p(g(2.)) - p( (2.)) = p(z.) - p(1, (2.)) = p(a.), (A37)
where 9 Z + X' denotes the estimated mixing function, and h := g- o 9 is the transformation between the ground-truth latent variables and the estimated ones. Since ğ and y are invertible, is invertible as well. Since Equation (A37) holds true for all time steps, there must exist an invertible function be such that (hhh(ee)) p(c:), whose Jacobian matrix at time step / is
J Jh
J Jhat
(A38)
Jh
Then for each value of x,-2, the Jacobian matrix of the mapping from (x--2, ét) to (Xx-2, C:) can be written as follows:
0
J hort
32

---

where * denotes any matrix. Since x can be fully characterized by itself, the left top and right top block are and respectively, and the determinant of this Jacobian matrix is the same as Jhatl Therefore, we have:
p(c,, x1-2) = p(c,, X x
(A39)
Dividing both sides of Equation (A39) by p(x--2), we further have: x( = p(c Since p(c, 2x p(celg(Ze - pc (z-2), and similarly p(ex|x;:2)) - (ppe)))pppllllee (r)))) we have: (gg((c(( log (c + log [Jn..l.
(A40)
(A41)
Let Ci E2 be two different variables that are not adjacent in the estimated Markov network Me. over éN (2,-1, 21). We conduct the first-order derivative w.r.. čτ, and have
i log p(ce)((12)
2n log ((g(((c)))2)) dea + Olog JJn.t] 0āg,
(A42)
dc,,
deg
dē,
1=1
We further conduct the second-order derivative w.r.t. GG and êr, then we have: 8 log Sogp(Crllee p(c(((()))22)
8 logg(cal2e-2) der, der,
de, 0dg, 8ūs,
(A43)
2n Dlogp(cs de.. + 82 log [Jn..
da, 06,, d
Since G,A, G..1 are adjacent in Me. G and âτ, are conditionally independent given è K G.1.) Utilizing the fact that conditional independence can lead to zero cross derivative [Lin, 1997], for each value of 2.-2, we have
8 Poggpcc/2)22)
8 log logp(Cu,xl: (e Cr.1: 2(-2) + d
8 loogp(r,,l Mar, Ce.i. 2(-2) 00,,
2 log p(e,) Cr 2(-2) = 0.
(A44)
Bring in Equation (A44), Equation (A43) can be further derived as
8 log p(c, 22)
dey da,,
82 Plggpcc./zz
der. Ser.j
dct
06g,
de,
D6. 06.,1
(..))))MM..)
ii=j
(ii)aa, and cr are udjacent in Mei
2n +
82 log pepp,c//122)
de,, dei.j
Oer,
Di,,
Tic.1
(i ande,, o4.5 not adjacent Moc
a log plogp(c.lzx-2)
Pars Olog JJnne
dee,
1=1
(A45)
where (j.t) E E(Mce) denotes that o, and Crg are adjacent in Me. Similar to Equation (A44), we have pp(c:(er-2) when Ct,i, Gtj adjacent in MMe. Thus, Equation (A45) be rewritten
dca.doe..
as
2n 82 log p(c)((112) der dars
2n
8a Plogp(colzt- der dor.j
Σ
act
0d,, De,,
dc,
0d, de,,
==1 j.(.)))E(MM))
2n Olg
Pag dee,
alog [Jnnel
+
Ocrs
06,,
1=1
(A46)
33

---

Then for each m = 1, ......... of Equation (A46) and have:
1........ n and each value of 22-2,m we conduct partial derivative on both sides
0 pogp(ci/zee22
dess der,
2n
0 log p(gg/c//z)))) dc,, -2m
dea dc, 00, dee,
E
den,
(.()))
0² log p(cc Z(-2)
Oct
Dc,, 0 2.m
001,100111
(A47)
Finally we have
2n 8 log pog (2-2)
dcs, der
2n 82 PPloplccelzz (c)
02 021-2.m 2,m
dds d6,,
dc, 06s,
(A48)
83 log logp(ccllue2) (21-2) der Sc,, O2 0 -.m 2.m
der. deg
+ de,, det.
E
0ōe
dū,, 06,,1
According to Assumption A2, construct 4n [Mel different equations with different values of 2,22m. and the coefficients the equation system they form are linearly independent. To ensure that the right-hand side of the equations always O, the only solution is
der, dcr. =0,
(A49)
dor, 06r1
de,
Ocrj
der.
Oc,, =0,
(A50)
0i,,
06,,1 Ddi,
ae, =0.
(A51)
Bringing Eq A49 into Eq A50, at least one product must be zero, thus the other must be zero as well. That is,
da,, der. =0. 0e, dēr,1
(A52)
According the aforementioned results, for any two different entries čg &, é, that are not adjacent in the Markov network Me, over estimated ét we draw the following conclusions.
(i) Equation (A49) implies that, each ground-truth latent variable Ct C₁ is function of at most one of čt,k and C.J-
(ii) Equation (A52) implies that, for each pair of ground-truth latent variables Gr2, and Crjj that adjacent in Me, over C₁ they can not be function of è and टτ,1 respectively.
Step2: Isomorphism of Markov Networks First, we demonstrate that there always exists a row permutation for each invertible matrix such that the permuted diagonal entries are non-zero [Zhang al., 2024a]. By contradiction, if the product of the diagonal entry of invertible matrix A is zero for every row permutation, then we have Equation
det A)
Σ (gn(() II (oon)a)
(A53)
oES..
by the Leibniz formula, where Sn is the of n-permutations. Thus, we have
II 1=1
Co = 0, Yo L Sn,
(A54)
which indicates that del(A) (A) and is non-invertible. contradicts the assumption that A is invertible, and a row permutation where the permuted diagonal entries are non-zero must exist. Since
34

---

hg is invertible, for Z, at time step i, there exists permuted version of the estimated latent variables, such that
dzg,
i= ... ,,,
(A55)
(e
where is the corresponding permutation at time step t. Since C₁ = (2 2z,, Ze+1), by applying Te-1, Tyy Tt+1, we have n' such that
dc,, (0))
* #0, i= 1= ,3,. .n..
(A56)
Second, we demonstrate that Me, is identical Map' where May' denotes the Markov network of the permuted version of N (ê,).
Step3: Component-wise Identification of Latent Variables Finally, we prove that the latent variables are component-wiss identifiable. On the one hand, for any pair of (i. such that Cr Ct.j adjacent in Mc. while (()) (i) are not adjacent in Mag' according to Equation (A52), we have der,i Bet,
(') ol) O, which is a contradiction with how '' constructed. Thus, any edge presents in Mc, must exist in Mag' " On the other hand, since observational equivalence can be achieved by the true latent process (g: J P.Pe),, the true latent process is clearly the solution with minimal edges.
Under the sparsity constraint on the edges of Mag', " the permuted estimated Markov network Me;' must be identical the true Markov network Me. Thus, we claim that
(i) the estimated Markov network Me. is isomorphic to the ground- truth Markov network Me.
Sequentially, under the same permutation Te, further give the proof that 2τ, only the function of (,)), Since the permutation happens on each time step respectively, the cross-time disentanglement prevented clearly.
Now let us focus on instantaneous disentanglement. Suppose there exists pair of indices i,
(1,... m). According to Equation (A55), we have 8ag, = and 8=L.5 0. Let us discuss case by case.
(2,
If Re,i is not adjacent to St.j. we have (() is not adjacent to m)) well according to the conclusion of identical Markov network. Using Equation (A49), we have Szi, (h.
Sec.s S 0, which leads (8e, (a,T+())
0.
If 24, is adjacent to 2tj. have 2,,,,() (i) is adjacent to (), When the Assumption A3 (Sparse Latent Process) is assured, i.e. the intimate neighbor set of 24,; is empty, there exists least one pair of (V', k) such that z is adjacent žτ, but not adjacent zr.j- Similarly, we have the same structure on the estimated Markov network, which means that Ee, (k)) is adjacent to (t)) but not adjacent $t, Using Equation (A52) have Dzc.k
(2, (np O, which leads to Th,
dzl.i
DzL = 0.
In conclusion, we always have )5,
Thus, we have reached the conclusion that
(ii) there exists permutation π of the estimated latent variables, such that 21,5 and &1,7(() is one-to-one corresponding, i.c., 2,, is component- ammennett-wise identifiable.
Theorem A3. (Component-wise Identification of 2, under sparse mixing procedure.) For a series of observations x, E R. and estimated latent variables 2, E R2 with the corresponding process ferp(c) 9. suppose the marginal distribution of observed variables matched. Let Mu, be the Markov network over u, & (24-1, Xt-1; 2t, Xt} and Mu,. Besides the similar assumptions like smooth, positive density, and sufficient variability assumptions, we further assume:
35

---

A7 (Sparse Mixing Procedure); For any Zit Z,, the intimate neighbor set of Zu is an empty set.
When the observational equivalence is achieved with the minimal number of edges of the estimated mixing procedure, there exists permutation x of the estimated latent variables, such that zu and (iye is one-to-one corresponding, i.e., 7it component-wer identifuble.
Proof. By reusing Theorem 2 with more observations, 1, - Ze, Xx) is also block-wise identi- fiable. So we have:
p L1, X1, 2 1, x = p(z:: X1, 2 84- J
p 24 &e 2
11-1, 21-1, = p(ze, X & J..
(A57)
mp & 2, 1, = In p(z x |, 11) + In Jxl
where h 2 X, 2 denotes the invertible transformation. J stands for the absolute value of the Jacobian matrix determinant of h. For any Zr.j. suppose that there exist &e,k that žr,j does not contribute to the mixture &t,
By using the sparse mixing procedure assumption (A7), we can constrain the sparsity of the estimated mixing function, such that there exist two different estimated latent variables ú, & and úz that are not adjacent in the estimated Markov networks Mu, and 8 log p(ū, & 0. Sequentially, U0,
can replace p(c|2(-1) in Lemma with p(ù, 2, 11) and then by reusing the proof process of Lemma can prove that is component-wise identifiable.
C.8 More Discussion on the Sparse Mixing Procedure
Although recent works like Zheng et al. [2022], Zheng and Zhang [2023] also utilize the sparse mixing process from z, to x; to achieve identifiability, our assumption is easier satisfy compared to these methods. The primary reason for this is that our generative process allows for noise in the mixing process from z, to X,. thereby accounting for measurement errors in the observed data. In contrast, methods like Zheng et al. [2022], Zheng and Zhang [2023] require the additional assumption that the mixing process is invertible and free from noise.
D Experiment Details
D.1 Synthetic Experiment
D.1.1 Data Generation Process
We follow Equation (1) to generate the synthetic data, As for the temporally latent processes, we use MLPs with the activation function of LeakyReLU to model the sparse time-delayed. That is:
2,,5 (Leaky Z,-1, 0.2) + VeL.s z1,4s) E2,6 ex
(A58)
where Wi. is the i-th row of W* and Vei, is the first i columns in the i-th row of V. Moreover, each independent noise et,; is sampled from the distribution of normal distribution. We further let data generation process from latent variables to observed variables be MLPs with LeakyReLU units. And the generation procedure can be formulated as follows:
Leaky Re LU LU (Leaky Re LU (0.2 x
-.oee---W... 0.2) +2, 0.2) (A59)
where Wx and Wm, denote the weights of mixing function. We provide 4 datasets from A to D, whose settings are shown in Table A5.
The total size of the dataset is 100,000, with 1,024 samples designated as the validation set. The remaining samples are the training set.
D.1.2 Evaluation Metric
To evaluate the identifiability performance of our method under instantaneous dependencies, we employ the Mean Correlation Coefficient (MCC) between the ground-truth and the estimated 2,
36

---

Table A5: Details of different synthetic datasets.
Dimension of
Causal Edge
Time Lag
Latent Variables
among Observations
A
5
yes
5
1
C
5
2
yes
10
1
yes
higher MCC denotes better identification performance the model can achieve. In addition, also draw the estimated latent causal process to validate our method. Since the estimated transition function will be transformation of the ground truth, we do compare their exact values, but only the activated entries.
D.1.3 Prior Likelihood Derivation
We first consider the prior of In p(Z1.). We start with an illustrative example of stationary latent causal processes with two time-delay latent variables, i.c. z, [2+,1 2t,2] with maximum time lag 1, = 1, i.e., žt,; = (,,,11,6,,) with mutually independent noises. Then we write this latent process as transformation map (note that we overload the notation f for transition functions and for the transformation map):
21-1,1
2t-1,1
Zt- 1,2
Rt- 1,2
1t,1
f1,1
2t,2
6r,2
By applying the change of variables formula to the map f. we can evaluate the joint distribution of the latent variables p(2t-1,1: 2t-1,2; 2r,1; 21,2) as
p 1.1, 2t 1.2, s1.1, 21.2) p(2 21 1,2, 66,1,
(A60)
Jrl
where Jy is the Jacobian matrix of the map where the instantaneous dependencies assumed be a low-triangular matrix:
0
0
Jr
0.1-1.3 02r.2
Øcc.1
0a1.2
8ar,2
1.1
Tae 1.2
Given that this Jacobian is triangular, we can efficiently compute its determinant as II. Hard Fur- thermore, because the noise terms are mutually independent, and hence c,, 1 Ct.j for j + and 1 Zt-1, with the RHS of Equation (A60) follows
ex.
p(21-111 1,2 21,1, (1,2) = p(z p(21-111111 1,1, 21 x ((.2 11, x p(er))
(A61)
Jrl
JJr]
Finally, generalize this example and derive the prior likelihood below. Let (1,,,,1,,,,,... be set of learned inverse transition functions that take the estimated latent causal variables, and output the noise terms, i.e., é., rz(ā,, )2. Then design transformation A B with low-triangular Jacobian as follows:
Tn x
22 L. ... 2c 2c]
mapped to 22 L.-..
11-1, Ec,s]
with JA-BB
diag
Brasi 7750.3
B
(A62)
Similar to Equation (A61), we can obtain the joint distribution of the estimated dynamics subspace as:
log log,((A) = logp(žz-L. 2 + Σ log p(é,,) + log([de((JAA-B)))
(A63)
Because of murually independent noise assumption
37

---

Finally, we have:
3r.
log (2)-( = Σ p(e,,) +
Σ log
(A64)
Since the prior of 1 litt11 (2 (R--1) with the assumption of first-order Markov assumption, estimate p(2++1:T (1:+)) similar way.
D.1.4 Evident Lower Bound
In this subsection, we show the evident lower bound. We first factorize the conditional distribution according to the Bayes theorem.
(np(()))) = in P(X1.T Zi:T)
E (i In p(xI.T. Zi.7 )g Z1.T X1.))
p(z XI.T)
=
p(zI.T X1.T ) Bir x1:4)
(A65)
2 Belar iiiirrlxu In ((XITIZZ)))
DKz(a(ZI.T X14) [p(z.)) ELBO.
L, and Ly
1KL
D.1.5 More Synthetic Experiment Results
We repeat each experiment with different random seeds. We further consider CariNG as basclines, experiment results are shown in Table A6.
Table A6: MCC results of synthetic datasets.
TOT
IDOL
CariNG
TDRL
A
0.9258(0.0034)
0.3788(0.0245)
0.7354(0.0345)
0.3572(0.0523)
B
0.9324(0.0078)
0.8593(0.0092)
0.0823(0.009))
(.8073)0.0786)
C
0.9322(0.0052)
0.6073(0.0952))
0.7084(0.0361)
0.7134(0.0346)
D
0.8433(0.0140)
0.7800(0.0387)
0.7371(0.0804)
0.7747(0.0690)
D.2
Real-world Experiment
D.2.1 Dataset Description
ETT Zhou et al. [2021] is an electricity transformer temperature dataset collected from two separated counties in China, which contains two separate datasets (ETTh2, ETTm1] for one hour level.
Exchange
Lai et [2018] the daily exchange rate dataset from of eight foreign coun- tries including Australia, British, Canada, Switzerland, China, Japan, New Zealand, and Singapore ranging from 1990 to.
ECL 4 is an electricity consuming load dataset with the electricity consumption (kWh) collected from 321 clients.
Traffic is dataset of traffic speeds collected from the California Transportation Agencies (CallTrans) Performance Measurement System (PeMS), which contains data collected from 325 sensors located throughout the Bay Area.
Weather provides 10-minute summaries from an automated rooftop station the Max Planck Institute for Biogeochemistry in Jena, Germany.
D.2.2 Implementation Details
The implementations of our method based on different backbones are shown in Table A7 to All.
"hups://pems.don dottpeemmmm.dscco.aooom wot.ca.goo...coo/.oo.oo
https://wwwwbbcceeeo www.:eeeeeeeaaa...eeeeeeeeeeee
38

---

Table A7: LoTD+ToT Architecture details. T, length of time series. x input dimension. n: latent dimension. LeakyReLU; Leaky Rectified Linear Unit. ReLU: Rectified Linear Unit. Tanh: Hyperbolic tangent function.
Configuration
Description
Output
Latent Variable Encoder
Input:X::t
Observed time series Convld Incurons,LcakyRe... T neurons,_LcakRRLLa
Batch Sizextx x dimension
Dense
Batch Sizex640x xx]
Dense
Batch Sizextx [x|]
Dense
Batch Size T X
Dense
512 necrons,LeaakRRee
Batch Sizex 12
Dense
Leuuons,,LeakyReluu
Batch Size xtx X
Dense
T neurons,LeakkReL.u
Batch Size T |
Latent Variable Decoder
Input:2::r
Latent Variable X, neurons,,cakyReel..
Batch Sizextx
Dense
Batch Size xtx [X]
n
Dimensionality reduction
In:ut:xi:
Observed time series
Batch Sizextx Xx] Batch Sizexts Xx
Linear
neurons, ReLu
Regressor
Input: [z::: (x(1))))
Latent Variable
Batch Size T Tx Batch SizexTx511 Batch Size T Xxe
Dense
512 Leuroon,LLakkRRll
Dense
x, LekkyRl
Latent Transition Estimator
Input: 21:T
Latent Variable
Batch Sizex (n+1)x 128
Dense
128 neurons, Lea
Dense
128 neurons,Leaky ReLU 128 neurons,,eakyyReLU
128 128
Dense
128 x 128
Dense
neuron
Batch Size
Jacobian Compute
Compute log(det(J))
Batch Size
r
Observed Transition Estimator
Input: Z1:T
Latent Variable
Batch (n+z) (n+1)x 128
Dense
128 Leurons,LLakyyEEi 128 Leakons, 128 neurons,
Dense
128 128
Dense
128 128
Dense
neuron
Batch Size
Jacobian Compute
Compute log(det(J))
Batch Size
39

---

Table A8: OneNet+ToT Architecture details. T, length of time series. xx]: input dimension. n. latent dimension. LeakyReLU; Leaky Rectified Linear Unit. ReLU: Rectified Linear Unit. Tanh: Hyperbolic tangent function.
Configuration
Description
Output
Latent Variable Encoder
Input:X1:: 1st
Observed time series
Batch Sizextx dimension Batch Sizexnx Xg
Linear
neurons
Convolution neural networks
320 neurons
Batch Size xizex320xx x Xx)
Latent Variable Decoder
In:ut:21:
Latent Variable neurons,Relaa ReLu
Batch Sizex320x [x|] Batch Sizextx X
Linear
Dimensionality reduction
1n:
Observed time series 320 neurons,ReLuu
Batch Sizextx X Batch S x20 X1]
Linear
Regressor
Input: ( (x))))))
Latent Variable
Batch Sizex640x x Xx]
Linear
Xxy neurons, ReLU
Batch SizexTx [x]
Linear
neurons,ReLu
Batch Sizexnx X
Convolution neural networks
320 neurons
Batch Size x20xx20xx [x]]
Linear
T neurons,ReLu
Batch Size Tx [
r?
Latent Transition Estimator
Input: Z1:T
Latent Variable
Batch (n+ex (n+1)x 128
Dense
128 Leakons 128 Leukyons.LeakReLLUU 128 neurons,,LeakyReLU
Dense
128x 128
Dense
128x 128
Dense
neuron
Batch Size 1
Jacobian Compute
Compute log(det(J))
Batch Size
Observed Transition Estimator
Input: Z1:7
Latent Variable
Batch (izex(n+1)
Dense
128 neurons,
(n+1) 128
Dense
128 Leakknn,
128 128
Dense
128 neurons,LeakkRReLU
128 128
Dense
neuron
Batch Sizex
Jacobian Compute
Compute log(det(J))
Batch Size
40

---

Table A9: OneNet-T+TOT Architecture details. T, length of time series. xx|| input dimension. n. latent dimension. LeakyReLU; Leaky Rectified Linear Unit. ReLU: Rectified Linear Unit. Tanh: Hyperbolic tangent function.
Configuration
Description
Output
Latent Variable Encoder
In:ut:X1:t
Observed time series
Batch Sizextx dimension Batch Sizexnx Xg
Linear
neurons, neurons,Rel.u ReLu T neurons,10 layers
Dilation convolution
Batch Sizex Tx X
Latent Variable Decoder
Input:21:
Latent Variable neurons, ReLu
Batch Sizex320x (x|] Batch Sizextx X
Linear
1
Dimensionality reduction
1n
Observed time series neurons,Re ReLu neurons, 10 layers
Batch Sizextx X
Linear
Batch S X
Dilation convolution
Batch Size X
Regressor
Input: (2:25 (x1:))
Latent Variable
Batch SizexTx2|l
Linear
X1] neurons.ReLU
Batch Size X
Linear
neurons.ReLu
Batch Sizexnx X
Convolution neural networks
320 neurons
Batch Size x 320 XX]
Linear
neulons,Rel.u
Batch Size SizexTxx X
r
Latent Transition Estimator
Input: Z1:T
Latent Variable
Batch Sizex(n+1) (n+1)x 128
Dense
128 neurons.Leaky
Dense
128 neurons..LeakkReLL 128 LeuKo
128x 128
Dense
128 128
Dense
neuron
Batch Size 1
Jacobian Compute
Compute log(det(J))
Batch Size
Observed Transition Estimator
Input: Z1:T
Latent Variable
Batch (n+1)
Dense
128 Leurkons,LeakyReLL 128 Leukkns.LeakkReLL 128 neurons,_eallyReL.
(n+1) 128
Dense
128 128
Dense
128x 128
Dense
neuron
Batch Size
Jacobian Compute
Compute log(det(J))
Batch Size
41

---

Table A10: online-T+TOT Architecture details. T, length of time series. |x|: input dimension. n: latent dimension. LeakyReLU; Leaky Rectified Linear Unit. ReLU: Rectified Linear Unit. Tanh: Hyperbolic tangent function.
Configuration
Deseription
Output
Latent Variable Encoder
In
Observed time series neurons, Relu
Batch Sizextx x dimension
Linear
Batch Sizexnx Xx]
Convolution neural networks
320 newrons
Batch Sizex320x X
Linear
neurons,
Batch Sizextx X;
Latent Variable Decoder
In:ut:21:t
Latent Variable neurons,ReLu
Batch Sizex320x [x]
Linear
Batch Sizextx Xx]
Dimensionality reduction
Input:x1:4
Observed time series neurons,
Batch Size X
Linear
Batch Sizextx X
Regressor
Input: b1.7 (Xx:e)
Latent Variable
Batch Sizex2tX Xg
Moving average
kernel size,stride=1
Batch Size X
Dilation convolution
320 neurons,5 layers,ReLu patch length=6,ReLu neurons
Batch Size x20x [x|]
Padding
Batch Size x X2 x patch length xpatch num
Transformer
Batch Size X2 xnxppatc num
Linear
T Rellu
Batch Size T X
Latent Transition Estimator
Input: Z1:T
Latent Variable
Batch Sizex(n+1) (n+1) 128
Dense
128 Lea
Dense
128 neurons,LeakyRLLU
128 x 128
Dense
128 neurons,LeakyReL.U
128x 128
Dense
neuron
Batch Size
Jacobian Compute
Compute log(det(J))
Batch Size
Observed Transition Estimator
Input: Z1:T
Latent Variable
Batch Sizex (n+1)x 128
Dense
128 Leurons.LeakyReL1. 128 Rea 128 LeukoR,
Dense
128x 128
Dense
128x 128
Dense
neuron
Batch Sizex
Jacobian Compute
Compute log(det(J))
Batch Size
42

---

Table A11: TToT Architecture details. T, length of time series. |x,|: imput dimension. nt latent dimension. LeakyReLU; Leaky Rectified Linear Unit. ReLU: Rectified Linear Unit. Tanh: Hyperbolic tangent function.
Configuration
Description
Output
Latent Variable Encoder
Input:Xi:r
Observed time series n neurons,Rella T neurons, 10 layers
Batch Sizextx x dimension
Linear
Batch Sizexn [Xp]
Dilation convolution
Batch Size Tx X
Latent Variable Decoder
Input:Z1:r
Latent Variable neurons,Rel.u
Batch Sizex320> |xel Batch Size [x,]
Linear
Dimensionality reduction
Input:Xi:t
Observed time series n neurons,Rela T neurons, 10 layers
Batch Size [x|] Batch Sizexnx X
Linear
Dilation convolution
Batch Size XTx X
Regressor
Input: (2::1 (X):)]
Latent Variable
Batch xizexTx2 X
Linear
[ neurons, ReLu
Batch Size Tx [
Latent Transition Estimator
Input: 21:T
Latent Variable
Batch Sizex(n+1) (n+1)x122 128
Dense
128 Leurons,LeakyRRL.UU
Dense
128 neurons.
128 x 128
Dense
128 LeaK
128 128
Dense
1 neuron
Batch Size 1
Jacobian Compute
Compute log(det(J))
Batch Size
Observed Transition Estimator
Input: Z1.T
Latent Variable
Batch Sizex (n+1)x 128
Dense
128 neurons, Leakkons,LakyReL.U 128 neurons,,eeakRRLLU
Dense
128x 128
Dense
128 neurons.
128x x 128
Dense
1 neuron
Batch Size 1
Jacobian Compute
Compute log(det(J))
Batch Size
D.2.3
More Experiment Results
We further consider MIR Aljundi et al. [2019a] and TFCL Aljundi et al. [2019b] as the backbone networks, experimental results shown in Table A12.
Table A12: MSE and MAE results of different datasets on TFCL and MIR backbone.
Midel
Len
TPCL-TOT
MIR
MIR-TOT
Modals
Les
TECL.
TECL~TOT
MIR-TOT
MSB
MAF
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAB
MSE
MAB
MS5
MAB
MSE
MAB
0.557
0.472
8.463
0.362
0.486
0.41
0.447
0,37%
2.732
0.524
3.815
01.44
2.575
0.504
3.395
0.589
ETT12
24
0.546
0.548
0.325
0.554
0.812
0.541
0.652
0.465
ECL
24
12494
1,256
18,083
1.165
9.265
1.056
6.142
1,841
45
1,208
0.592
0.57
0.555
1.100
0.555
0.842
0.536
48
12.13
1,303
10.685
1,075
9.411
1,079
6.479
1.190
0:087
0.198
0,081
0.187
0.085
0.197
0J0K3
6.198
0.305
0.297
6.364
0.263
0.298
0.254
0.291
0.267
ETTml
24
0.211
0.341
9.186
0.32
0.192
0.325
0.132
0.267
Traffic
24
0.44]
0.493
0.389
0.314
0.651
0.643
0.39
0.330
46
0.236
0.963
0.196
0.331
0.210
0.542
0.129
0.265
0.438
0.531
0.383
0.316
0.500
0.907
0.419
0.345
0.177
0.24
0.154
0.179
0.244
0.105
0.153
0.142
0.015
0.118
0
0.162
0.363
0.291
0.355
Exchange
0.088
0.227
0.104
0.304
0.067
0.387
0.961
0.101
0.047
To demonstrate that the improvements of our approach are not due to an increase in parameters, we increase the number of parameters of the baseline methods by adding additional layers to the neural
43

---

Table A13: Evaluation on Same Size of Model
Model
LSTD(Original)
LSTD(Same size)
LSTD+TOT
Matric
MSE
MAE
MSE
MAE
MSE
MAE
0.377
0.347
0.378
0.349
0.374
0.346
ETTh2
24
0.543
0.411
0.778
0.465
0.532
0.390
48
0.616
0.423
0.620
0.443
0.564
0.420
0.153
0.200
0.155
0.202
0.153
0.200
WTH
24
0.136
0.223
0.126
0.215
0.116
0.207
48
0.157
0.242
0.168
0.251
0.152
0.239
2.112
0.226
2.109
0.234
2.038
0.221
ECL
1.422
0.292
1.420
0.285
1.390
0.278
48
1,411
0.294
1,442
0.303
1.413
0.289
0.231
0.225
0.231
0.225
0.229
0.224
Traffic
24
0.398
0.316
0,402
0.319
0.397
0.313
48
0.426
0.332
0.427
0.333
0.421
0.328
0.013
0.070
0,013
0.071
0.013
0.069
Exchange
24
0.039
0.132
0.040
0.135
0.037
0.129
0.043
0.142
0.043
0.142
0.042
0.142
network, making the number of parameters of our method and the baseline methods comparable. Experiment results are shown in Table A13. According to the experiment results, we can find that our method still achieves the general improvement.
D.2.4 Experiment Results of Mean and Standard Deviation
The mean and standard deviation of MAE and MSE are shown in Table A14, A15, A16, and A17, respectively.
Table A14 14: Mean values of MSE on different datasets.
Makb
Las
LATD
LSTD-IST
Precced-T
Praseot-T-Is
Dus/itt
OsuNei-TOr
Oache-T
OwwNe-TT-01
MBR
MIR-TOT
Oalte-T
Oulls-T-TVR
TECL
CLCL-
0.774
1.577
1.186
0.117
11.65
3:51
0,391
8.051
0.01
0.511
E.366
ETTB3
24
0.343
0.54)
2,935
2.444
0.345
0.513
0.543
6,077
0
8.507
0.822
6.57
0.211
6,00
0.801
11.000
11802
00779
0.82
0.383
0.16
0.155
0.115
18:02
0.503
0.100
0.80
0215
01085
0.225
8.155
0.210
0.55
0.340
0.00
01.55
0,853
11.30
0.357
01551
0151
0.23
2.755
WTH
01.99
0.136
11207
0.61
0.13
11.00
0.36
0.385
$.165
6372
0311
0.26
1.577
5.807
3740
2835
4500
11:00
HAN12
1.700
1.658
7.092
3.500
3.15
3.98
4.72
3.284
11987
110
0.254
0,331
0.335
0.33%
0.341
0.299
0.35
0225
0.255
8.25
0.354
0.355
6,39
8:311
0.422
0.012
0151
8.05
0.50
0.16
Exdnang:
0339
0,86
0129
0.115
0.40
034
0180
6041
0.164
81713
0.115
6.06
0.58
6/811
0399
00307
0338
0856
0165
0/556
010
8165
0.168
0100
Table A15: Mean values of MAE on different datasetss
Lus LATD
LSTI=TAT
Proeed-T [| Pranett---Is
OasNet
DuuNe--TO Oadhas1
Unchet-T-077
MBB-TO7
Oulla--T
Ostne-1-007
TFCCL
TFCL-TOT
0.307
9.347
0.147
8.401
3.541
0.3.7
B.361
1.:1.
0.333
0.553
0.352
0/066
8.301
0.423
8.437
0.367
8.733
0.445
0.435
0.355
8.330
0.372
0.485
01535
0.549
0.20]
0
KTTbl
8217
0.340
0.447
8.443
0.34
0.236
0.347
8.36
0.32
0.352
01336
0.345
0356
02299
0.321
1.507
0342
6357
0.30
0.388
0.443
0.140
3330
01383
0.386
01212
0.522
0.300
01210
0.381
6100
0.34
8250
9.347
0.833
0.000
0337
0.355
0.354
0.65
0.38
0.35
0.358
0.39
6335
0.66
BCL
01200
0.33
0.357
8.384
3340
14380
0.425
0300
1.85
1.305
1.311
0.330
1114
1
01227
8.237
0.255
0.203
3340
01226
0.255
6215
0.254
0.589
01250
0.382
0200
6.244
01341
8.338
0.388
0.111
3111
11108
0.555
8.30
0.30
0.68
01180
0.335
6451
6313
0.83
3165
0.55
Esdnang:
01122
0.138
0.21
0.51
3.148
0.355
0.155
6139
0.254
0.15
0215
0.395
0227
6191
44

---

Table A16: Standard deviation of MSE on different datasets.
LATD
LSTI+RVT
Preced-
Oscho-1
1NIK-10T
Oston-T
[OO |IPCL
TPCL-TOT
0.311
01003
6.1751
0.0014
2.044
3,00%
08331
61123
0.168
22342
0,053
6.3527
22048
0.994
01000
09614
0.835
0.0455
0.863
03137
06627
0.561
3,032
0.333
0.133
D2005
08001
KTTml
DAK0O
0.001
00191
00028
8.915
38084
01025
08311
01100
00142
31881
0.051
8.143
B1674
050054
0.200
02040
0.300
01144
WTH
0.6030
3.611
0.307
0.90
51604
D0005
0.991
0186
01000
0.911
0.141
3.890
08117
01
00905
37084
0.905
D1605
60091
0331.0
01060
036/13
0,0433
0.344
05331
00019
01003
23332
0,300
530
D1464
0.8275
02262
03857
0.301
0.9935
03032
0.349
00025
22092
0,599
02081
D2382
Tratk:
0.394
00115
02004
0.906
0.654
3.0%
00018
0.391
32134
0.271
0.055
32015
0.900
02650
0.809
33863
0824
03015
01000
0.990
10000
0.0001
01011
0.0055
0100
08397
00030
00000
31085
0,900
00018
D1081
DURC4
0.903
DUX07
54005
0.570
00137
30002
08362
00003
00130
32067
0.001
69036
30067
Table A17: Standard deviation of MAE on different datasets.
[Tee] LATD
1STI=ToT | Preeesd/7
PrawedTvlloF
OOwlee-TOT
|MRR-TT
Ouline-F
Oslee-T-FOr T-TOT
TPCS
-TRCE-TOT
EVTR2
D1001
EAKE33
03004
0JK03
0.851
(URCS
011174
000234
0
0.0470
2.003
D/K01
01013
DU0E6
00123
00075
3.04)
D0018
DAKC
03002
0.6
00860
00980
00907
00006
0.275
00025
00015
0.955
31087
DJ003
01603
01623
08825
010222
0.335
0.003
33045
D3000
E3607
03004
21603
8827
01022
01000
17877
0.203
00155
3.0045
BCL
D3044
03004
03804
0 0.625
00001
00030
01201
0.2085
00122
C/663
0.66%
2,755
D/665
01403
8644
D/808
01013
00116
0.005
3.000
D/803
01913
01091
01053
0.036
D001
0001
0030
D100
0.200
0000
D1011
0.075
310M
D/012
03644
08544
(/24)
01013
0.002
0UKC3
00011
01015
0.885
28062
Baslunge
D/004
C1917
01013
S1001
0.035
0A054
0001
09001
0.000
00023
0.051
30041
0/210
E Broader Impacts
The proposed method for online time series forecasting presents novel approach to address the challenges posed by distribution shifts in temporal data, By leveraging the identification of latent variables and their causal transitions, our framework demonstrates a provable reduction in Bayes risk, with significant improvements in forecasting accuracy, making the method highly applicable to real-time forecasting tasks in fields such as finance, healthcare, and energy management.
Our method not only outperforms existing models like IDOL and TDRL in both synthetic and real-world experiments, but it also enhances the scalability and adaptability of forecasting systems in dynamic environments. The theoretical advancements, coupled with the plug-and-play architecture, facilitate seamless integration into existing forecasting pipelines, further promoting the use of causal modeling in practical scenarios.
Furthermore, the broader implications of this work extend beyond time series forecasting. The ability to identify and utilize latent variables in real-time systems opens the door to new applications in domains such as anomaly detection, predictive maintenance, and environmental monitoring. With its potential for improving decision-male making processes in critical industries, foundation for future advancements in online forecasting and causal representation learning, thus contributing to the evolution of machine learning models that can effectively handle complex, dynamic data in real-world environments.
method sets strong
45


Reviews:
[{"review_id": "D4DxKW5j5y", "reviewer": "Reviewer_uFSc", "summary": "The focus of this paper is forecasting time series online, in the presence of distribution shifts that are governed by some kind of unobserved latent dynamics. Here, by \"distribution shifts\", the authors refer to both changes in the latent state as well as changes in how the observed variables depend on the latent state. In contrast to previous works, the authors consider the possibility of there being a sparse dependence structure of the observed variables on the latent variables at a given point in time. This, in turn, leads to identifiability of the latent variables and is shown to have beneficial downstream effects on forecasting performance. Put differently, it is more beneficial to have a sparse dependence structure that is identifiable than not using a dependence structure at all, in the presence of distribution shift. When there is no distribution shift, it is shown that the use of such a model does not result in performance decay. The paper also looks at some theoretical results having to do with identifiability and concludes with empirical examples that illustrate the performance of the proposed methodology.", "questions": "It would be nice to have a cleaner description of the \"online\" component: how are updates done once a single new data point comes in?\n\nFollowing up on the above. Can we try online learning in non-stationary environments? This methodology seems well-suitable for this kind of approach. And, do multi-step-ahead forecasting in such environments would be the ultimate robustness test of this methodology.\n\nDo we explicitly care about the dynamics of the z_t's at any point? Or only about which z_t's are relevant at t?", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: the model's structural assumptions seem well-founded and the framework appears to be able to handle scenarios with fairly complex distribution shifts. Careful consideration is given to the identifiability problem of latent variables. The authors also consider the problem of predictive risk and its relationship with the latent variable structure. The use of both observations and latent variables in constructing a prediction leads to more accurate forecasting possibilities.\n\nWeaknesses: the authors should spend a bit more discussing a simple experiment within the confines of the main text. No details are given on the data generation process, e.g., time series length, the type of sparsity, and so on. This makes it hard to develop an intuition for how the method works. The \"sparsity of the mixing procedure\" should also be made more clear. The way I understood it, the sparsity is in having each z_i only influence a few of the x_i's. Is this indeed the case? This would mean that each z_i must be \"active\" but at the same time there is a limitation on how many x_i's it can be connected to? So even if the distribution changes in a way that some x_i becomes highly correlated with a block of x_i's driven by a latent variable, the sparsity constraint can prevent this x_i from being associated with that z_i?", "was_revised": true, "final_justification": "The authors' comprehensive response and additional experiments have helped me understand aspects of the paper better, and as a result, I have adjusted my score.", "rebuttal": "Dear Reviewer uFSc, we sincerely appreciate your informative feedback and helpful suggestions that make our paper more complete. We have added the new experiments following your suggestions, provided more discussions about the sparse mixing procedure and online forecasting, as well as modified the paper and appendix accordingly. Please see our point-to-point response below.\n\n>Q1. The authors should spend a bit more discussing a simple experiment within the confines of the main text. No details are given on the data generation process, e.g., time series length, the type of sparsity, and so on. This makes it hard to develop an intuition for how the method works. \n\nA1: Thanks for your insightful suggestions, which make our experiment more solid. In light of your suggestion, we have added more simulation experiments. Specifically, we have considered four datasets with different types of sparsity: (1). the sparse latent process but the dense mixing procedure; (2). the dense latent process but the sparse mixing procedure; (3). the sparse latent process and the sparse mixing procedure; and (4). The dense latent process and the dense mixing procedure. All of these datasets contain 10000 5-dimensional time series samples with a length of 7. Experiment results are shown as follows: \n\n|dataset|MCC|\n|-|-|\n|(1)|0.9521(0.032)|\n|(2)|0.9432(0.019)|\n|(3)|0.9542(0.085)|\n|(4)|0.8143(0.173)|\n\nWe can find that the latent variables are hard to identify when both the mixing and latent dynamics are sparse.\n\n>Q2: The \"sparsity of the mixing procedure\" should also be made more clear. The way I understood it, the sparsity is in having each z_i only influence a few of the x_i's. Is this indeed the case? This would mean that each z_i must be \"active\" but at the same time there is a limitation on how many x_i's it can be connected to? \n\nA2: Thank you for your question, which indeed helps us improve the readability of our paper. You are correct that the sparse mixing procedure means each $z_{t,i}$ only influences a limited number of $x_{t,j}$. However, we would like to clarify that our sparsity requirement is mild. Specifically, we only assume that the Markov network over $z_t, x_t$ is sparse, rather than imposing strong structural sparsity across the entire latent process. Given a toy example as follows\n$$ \n\\\\begin{array}{ccc} \nz_{t-1,1} & \\\\leftarrow & z_{t-1,2} \\\\\\\\ \n\\\\downarrow & \\\\searrow & \\\\downarrow \\\\\\\\ \nz_{t,1} & \\\\leftarrow & z_{t,2} \\\\\\\\ \n\\\\downarrow & \\\\searrow & \\\\downarrow \\\\\\\\ \nx_{t,1} & \\\\leftarrow & x_{t,2} \n\\\\end{array} \n$$\n,the sparsity assumption is already satisfied although $z_{t,1}$ is fully connected to $x_{t,1}$ and $x_{t,2}$. This is because the corresponding Markov network is shown as follows\n$$ \n\\\\begin{array}{ccc} \nz_{t-1,1} &—& z_{t-1,2} \\\\\\\\ \n| & \\\\diagdown & | \\\\\\\\ \nz_{t,1} &—& z_{t,2} \\\\\\\\ \n| & \\\\diagdown & | \\\\\\\\ \nx_{t,1} &—& x_{t,2} \n\\\\end{array} \n$$\nis also sparse. And the intimate neighbor set (defined in Line 202-204) of latent variables $z_{t,1}$ and $z_{t,2}$  are empty sets, which implies identifiability by using sufficient changes from historical information.\n\nThis mild assumption is sufficient to guarantee the component-wise identifiability of latent variables, as shown in Theorem 3. In light of your insightful question, we have provided a detailed explanation with the aforementioned graphical example in the updated version.\n\n>Q3: So even if the distribution changes in a way that some x_i becomes highly correlated with a block of x_i's driven by a latent variable, the sparsity constraint can prevent this x_i from being associated with that z_i?\n\nA3: Thanks for your question! We would like to highlight that the sparsity constraint does not mean to prevent some observations from being associated with others. \n\nLet us using the same example in A2, $x_{t,1}$ is indirectly influenced by $z_{t,2}$ through $x_{t,2}$. In this case, $x_{t,1}$ is associated with $z_{t,2}$ even if the data generation process is sparse.\n\n>Q4: It would be nice to have a cleaner description of the \"online\" component: how are updates done once a single new data point comes in?\n\nA4: Thank you for this helpful suggestion. We follow the definition of online time series forecasting from [1,2], where the forecasting model generates future predictions using a moving window and updates the model immediately after the corresponding ground truth time series data arrive.\n\nFor a better understanding, we assume a first-order Markov process for both observations and latent variables as follows:\n$$\n\\\\begin{aligned}\n& z_1 \\\\rightarrow z_2 \\\\rightarrow z_3 \\\\rightarrow z_4 \\\\rightarrow z_5 \\\\rightarrow z_6 \\\\\\\\\n& \\\\downarrow\\\\quad\\\\;\\\\;\\\\;\\\\downarrow\\\\\\;\\\\;\\\\;\\\\quad\\downarrow\\\\quad\\\\\\;\\\\;\\\\;\\\\downarrow\\\\quad\\\\;\\\\;\\\\;\\\\downarrow\\\\quad\\\\;\\\\;\\\\;\\\\;\\\\downarrow \\\\\\\\\n& x_1 \\\\rightarrow x_2 \\\\rightarrow x_3 \\\\rightarrow x_4 \\\\rightarrow x_5 \\\\rightarrow x_6\n\\\\end{aligned}\n$$\nwhere we use the new arrived $(x_5)$ and the historical observations in moving window, e.g. $(x_1,x_2,x_3,x_4)$ to predict the values of $x_6$.\nBased on this example, the update procedure proceeds as follows:\n1. **Latent Variables Identification**: When new data arrive, we use only the most recent adjacent observations to identify the latent variables. For example, given $(x_1, x_2, x_3, x_4, x_5)$, we can estimate $\\hat{z}_3$ and $\\hat{z}_4$ from $(x_1, x_2, x_3, x_4)$ and $(x_2, x_3, x_4, x_5)$, respectively (leverage our consecutive four observed variables).\n2. **Latent Variables Estimation**: Online transition update: With the newly identified latent states, we learn the latent transition $p(z_t|z_{t-1})$ incrementally (e.g., updating from $(z_3,z_4)$ to infer $z_5, z_6$) as the transition of latent variables is stationary.\n3. **Prediction with Latent Variables**: Finally, we combine the estimated latent variables $z_6$ with the latest available observations, i.e., $(x_1, x_2, x_3, x_4, x_5)$ to forecast the future observation $x_6$. \n\nThanks for your insightful question. We have provided a detailed example and an algorithm flow chart in the updated version.\n\n[1] Lau, Ying-yee Ava, Zhiwen Shao, and Dit-Yan Yeung. \"Fast and Slow Streams for Online Time Series Forecasting Without Information Leakage.\" ICLR 2025.  \n[2] Zhao, Lifan, and Yanyan Shen. \"Proactive model adaptation against concept drift for online time series forecasting.\" KDD 2025.\n\n>Q5: Following up on the above. Can we try online learning in non-stationary environments? This methodology seems well-suitable for this kind of approach. And, do multi-step-ahead forecasting in such environments would be the ultimate robustness test of this methodology.\n\nA5: Thank you for this insightful suggestion. Indeed, our methodology is general and naturally applicable to non-stationary environments. However, we would like to clarify the difference between our online forecasting setting and conventional non-stationary forecasting. In the nonstationary time series forecasting, we can leverage more historical time series data. In online forecasting, we cannot leverage the entire historical data; instead, we only have access to sequentially arriving data streams and must rely on the most recent observations for updating. Our theoretical contribution lies in showing how to identify the latent variables that drive non-stationarity with minimal adjacent observations, which is critical for real-time online updates.\n\nWe appreciate your suggestion to further evaluate our method on non-stationary time series forecasting. Specifically, we follow the experimental setting of [1] and use Koopa[1], FITS[2], and SOILD[3] as baselines. We evaluate our method on the ECL, ETTh2, Exchange, ILI, and Traffic datasets. The results are presented below.\n\n|||TOT||Koopa||FITS||SOILD||\n|-|-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|Metric||MSE|MAE|MSE|MAE|MSE|MAE|MSE|MAE|\n|ECL|48|0.129|0.228|0.13|0.234|0.147|0.25|0.183|0.262|\n||96|0.131|0.223|0.136|0.236|0.134|0.256|0.188|0.254|\n||144|0.147|0.239|0.149|0.247|0.227|0.341|0.185|0.297|\n||192|0.157|0.25|0.156|0.254|0.198|0.265|0.217|0.284|\n|ETTh2|48|0.225|0.298|0.226|0.3|0.256|0.329|0.268|0.328|\n||96|0.284|0.34|0.297|0.349|0.315|0.365|0.34|0.361|\n||144|0.312|0.365|0.333|0.381|0.351|0.391|0.355|0.429|\n||192|0.336|0.379|0.356|0.393|0.358|0.398|0.387|0.467|\n|Exchange|48|0.042|0.141|0.042|0.143|0.054|0.165|0.047|0.179|\n||96|0.086|0.205|0.083|0.207|0.096|0.146|0.103|0.25|\n||144|0.125|0.254|0.13|0.261|0.145|0.304|0.148|0.311|\n||192|0.164|0.296|0.184|0.309|0.181|0.265|0.2|0.356|\n|ILI|24|1.456|0.778|1.621|0.8|1.633|0.851|1.577|0.845|\n||36|1.79|0.839|1.803|0.855|1.952|0.969|2.048|0.952|\n||48|1.746|0.885|1.768|0.903|2.256|1.069|2.029|1.117|\n||60|1.831|0.89|1.743|0.891|2.084|1.352|1.994|1.003|\n|Traffic|48|0.36|0.231|0.415|0.274|0.482|0.251|0.481|0.374|\n||96|0.323|0.218|0.401|0.275|0.361|0.234|0.401|0.33|\n||144|0.315|0.217|0.397|0.276|0.358|0.259|0.428|0.299|\n||192|0.315|0.22|0.403|0.284|0.359|0.317|0.42|0.297|\n\nWe can find that our method can still achieve ideal performance in non-stationary environments. \n\n[1] Liu. et al. \"Koopa: Learning non-stationary time series dynamics with koopman predictors.\" NeurIPS 2023  \n[2] Xu. et al. \"FITS: Modeling time series with $10 k $ parameters.\"ICLR2024   \n[3] Chen et al. \"Calibration of time-series forecasting: Detecting and adapting context-driven distribution shift.\"KDD2024. \n\n>Q6: Do we explicitly care about the dynamics of the $z_t$'s at any point? Or only about which $z_t$'s are relevant at t?\n\nA6: Thank you for this thoughtful question. We do care about the dynamics of $z_t$ and we explicitly model the dynamics of $z_t$ in **Section 4.1**. Specifically, when identifying the latent variables, we explicitly estimate the independent noise terms across different time steps, i.e., $\\epsilon_{t,i} = r(z_{t,i}, z_{t-1})$. This procedure essentially models the temporal dynamics $z_{t-1}\\rightarrow z_t$. In light of your questions, we have highlighted it in the final version."}, {"review_id": "Jn2JtYUuhq", "reviewer": "Reviewer_hiQT", "summary": "This paper proposes a theoretical framework for online time series forecasting (TOT) to address unknown distribution shifts caused by latent variables. The authors theoretically prove that incorporating latent variables tightens the Bayes risk, with the benefit increasing as latent variable identifiability improves. They show that latent variables and their causal dynamics can be uniquely identified using only four consecutive observations. Based on these results, a model-agnostic architecture is designed, featuring a temporal decoder and two noise estimators. Experiments on synthetic and real-world datasets validate the theoretical claims, demonstrating improved forecasting performance across multiple benchmarks.", "questions": "1. Will the sampling frequency affect this module, thus causing the assumption conditions to be violated and making four observation points insufficient for identifying the latent variables?\n2. This framework seems very universal. Can it be used for offline model training?", "limitations": "The authors wrote about the limitation in the conclusion.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strengths  \n1. **Rigorous Theoretical Foundation**: The paper provides a solid theoretical framework, including risk-bound guarantees and identifiability proofs for latent variables. The derivation of using four adjacent observations to identify latent variables is innovative and addresses a key challenge in nonstationary time series modeling. The theoretical results are well-supported by mathematical proofs (Appendix C), enhancing the credibility of the framework.  \n\n2. **Data Efficiency and Generalizability**: The method’s ability to identify latent variables with minimal observations (four consecutive points) makes it highly data-efficient, which is valuable for real-world scenarios with limited data. The model-agnostic blueprint allows integration with various backbones (e.g., OneNet, LSTD), demonstrating broad applicability and practical utility across different forecasting tasks.  \n\n3. **Empirical Validation**: The experiments are comprehensive, covering both synthetic and real-world datasets (e.g., ETT, Exchange, Traffic). The results show consistent improvements over baselines, with clear performance metrics (MSE, MAE) and statistical significance (repeated runs with error bars). The ablation study (Figure 4) further validates the necessity of each component, providing insights into the model’s design.  \n\n### Weaknesses  \n1. **Assumption Constraints**: The theoretical results rely on assumptions such as bounded continuous densities (A1) and injective linear operators (A2), which may not hold in highly nonstationary or noisy real-world scenarios. For example, abrupt distribution shifts (e.g., sudden policy changes) could violate the continuity assumption, potentially limiting the framework’s robustness in extreme cases.", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer hiQT, we sincerely appreciate your encouraging and insightful feedback, which has significantly helped us clarify the boundary of our method and make our experiment solid. Please find our point-by-point responses below.\n\n\n>Q1: The theoretical results rely on assumptions such as bounded continuous densities (A1) and injective linear operators (A2), which may not hold in highly nonstationary or noisy real-world scenarios. For example, abrupt distribution shifts (e.g., sudden policy changes) could violate the continuity assumption, potentially limiting the framework’s robustness in extreme cases.\n\nA1: Thank you for raising this important concern. We would like to clarify that the continuity assumption in (A1) refers specifically to the probability density functions of the latent and observed variables, not to the continuity of the data generation process. Therefore, even in the presence of abrupt distribution shifts, such as sudden policy changes, our theoretical results remain valid as long as the latent and observed variables admit bounded continuous densities. Moreover, such conditions still hold in many realistic nonstationary or noisy settings, where the the distributions do not become degenerate.\n\n\n>Q2: Will the sampling frequency affect this module, thus causing the assumption conditions to be violated and making four observation points insufficient for identifying the latent variables?\n\nA2: Thank you for the thoughtful question. The sampling frequency can indeed affect the identifiability of latent variables, particularly when it induces a form of subsampling in which some observation variables become unobservable. In such cases, the corresponding latent variables may no longer be fully identifiable unless additional observations are incorporated.\n\nFor example, consider the following generative structure:\n$$\n\\\\begin{array}{ccccccccccc}\nz_1 & \\\\rightarrow & z_2 & \\\\rightarrow & z_3 & \\\\rightarrow & z_4 \\\\\\\\\n\\\\downarrow & & \\\\downarrow & & \\\\downarrow & & \\\\downarrow \\\\\\\\\nx_1 & \\\\rightarrow & x_2 & \\\\rightarrow & x_3 & \\\\rightarrow & x_4\n\\\\end{array}\n$$\nIf $x_2$ becomes unobservable due to subsampling or sensor failure, then the identifiability of $z_1$ and $z_3$ may require additional observable variables beyond the original four observed variables. However, for online forecasting tasks, even if some latent variables are only partially identified, our method can still leverage the identifiable components to make effective predictions of future observations.\n\n\n>Q3: This framework seems very universal. Can it be used for offline model training?\n\n\nA3: Thank you for the great suggestion, which extends the boundary of our contributions. Indeed, our methodology is universal and can naturally be applied to offline model training, where the distributions of historical time series change over time. In light of your questions, we conducted additional experiments on the offline training scenarios with non-stationary time series. Specifically, we follow the setting of [1], and consider Koopa [1], FITS [2], SOILD [3] as baselines. Moreover, we evaluate our method on ECL, ETTh2, Exchange, ILI, and Traffic datasets. Experiment results are shown as follows:\n\n|  |  | TOT |  | Koopa |  | FITS |  | DLinear+SOLID |  |\n|---|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| Metric |  | MSE | MAE | MSE | MAE | MSE | MAE | MSE | MAE |\n| ECL | 48 | 0.130 | 0.228 | 0.13 | 0.234 | 0.147 | 0.25 | 0.183 | 0.262 |\n|  | 96 | 0.131 | 0.223 | 0.136 | 0.236 | 0.134 | 0.256 | 0.188 | 0.254 |\n|  | 144 | 0.147 | 0.239 | 0.149 | 0.247 | 0.227 | 0.341 | 0.185 | 0.297 |\n|  | 192 | 0.157 | 0.25 | 0.156 | 0.254 | 0.198 | 0.265 | 0.217 | 0.284 |\n| ETTh2 | 48 | 0.225 | 0.298 | 0.226 | 0.3 | 0.256 | 0.329 | 0.268 | 0.328 |\n|  | 96 | 0.284 | 0.34 | 0.297 | 0.349 | 0.315 | 0.365 | 0.34 | 0.361 |\n|  | 144 | 0.312 | 0.365 | 0.333 | 0.381 | 0.351 | 0.391 | 0.355 | 0.429 |\n|  | 192 | 0.336 | 0.379 | 0.356 | 0.393 | 0.358 | 0.398 | 0.387 | 0.467 |\n| Exchange | 48 | 0.042 | 0.141 | 0.042 | 0.143 | 0.054 | 0.165 | 0.047 | 0.179 |\n|  | 96 | 0.086 | 0.205 | 0.083 | 0.207 | 0.096 | 0.146 | 0.103 | 0.25 |\n|  | 144 | 0.125 | 0.254 | 0.13 | 0.261 | 0.145 | 0.304 | 0.148 | 0.311 |\n|  | 192 | 0.164 | 0.296 | 0.184 | 0.309 | 0.181 | 0.265 | 0.2 | 0.356 |\n| ILI | 24 | 1.456 | 0.778 | 1.621 | 0.8 | 1.633 | 0.851 | 1.577 | 0.845 |\n|  | 36 | 1.79 | 0.839 | 1.803 | 0.855 | 1.952 | 0.969 | 2.048 | 0.952 |\n|  | 48 | 1.746 | 0.885 | 1.768 | 0.903 | 2.256 | 1.069 | 2.029 | 1.117 |\n|  | 60 | 1.831 | 0.89 | 1.743 | 0.891 | 2.084 | 1.352 | 1.994 | 1.003 |\n| Traffic | 48 | 0.36 | 0.231 | 0.415 | 0.274 | 0.482 | 0.251 | 0.481 | 0.374 |\n|  | 96 | 0.323 | 0.218 | 0.401 | 0.275 | 0.361 | 0.234 | 0.401 | 0.33 |\n|  | 144 | 0.315 | 0.217 | 0.397 | 0.276 | 0.358 | 0.259 | 0.428 | 0.299 |\n|  | 192 | 0.315 | 0.22 | 0.403 | 0.284 | 0.359 | 0.317 | 0.42 | 0.297 |\n\nWe can find that our method can still achieve the ideal performance in offline scenarios.\n\n[1] Liu, Yong, et al. \"Koopa: Learning non-stationary time series dynamics with Koopman predictors.\" NeurIPS 2023    \n[2] Xu, Zhijian, Ailing Zeng, and Qiang Xu. \"FITS: Modeling time series with $10 k $ parameters.\"ICLR2024     \n[3] Chen, Mouxiang, et al. \"Calibration of time-series forecasting: Detecting and adapting context-driven distribution shift.\" KDD2024."}, {"review_id": "4N26Rg3ote", "reviewer": "Reviewer_2i9r", "summary": "This paper proposes TOT (Theoretical framework for Online Time-series forecasting), addressing distribution shifts in online time series forecasting caused by latent variables. The work aims to provide theoretical guarantees showing that incorporating latent variables reduces Bayes risk, prove identifiability of latent variables from four consecutive observations, and develop a model-agnostic architecture combining temporal decoders with noise estimators.", "questions": "- The paper mentions that \"latent variables introduce unknown distribution shifts\", can you elaborate? Is the paper also interested in shifts in the data space?\n\n- Can you elaborate more on time-delayed parents and instantaneus parents from Section 2>\n\n- How can practitioners verify assumptions A2-A4 in real datasets? Can you provide diagnostic tools or relaxed conditions? What happens to identifiability guarantees when assumptions are violated, and how robust is the method to such violations?\n\n- What are the computational requirements compared to baseline methods? The architecture involves multiple components (encoder, decoder, two noise estimators, forecaster) - how does this scale with sequence length and dimensionality? Can you provide runtime comparisons?\n\n-  Why are exactly 4 consecutive observations needed for identifiability? Is this the theoretical minimum, or could the framework work with 3 or 5 observations? How does performance degrade with fewer observations?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "This paper presents a well-executed theoretical framework for online time series forecasting that successfully bridges formal guarantees with practical implementation. The work demonstrates strong technical quality through its coherent three-theorem structure, where Theorem 1 establishes risk reduction via latent variables, Theorem 2 proves block-wise identifiability from just 4 observations, and Theorem 3 shows component-wise identifiability under sparsity constraints. The theoretical development is mathematically fine with clear proofs, and the paper offers an experimental validation through comprehensive synthetic and real-world experiments that consistently support the theoretical claims. \n\nI believe the paper suffers from several significant limitations that constrain its practical impact and theoretical novelty. The identifiability results rely on restrictive assumptions (A1-A4) including injectivity conditions, spectral decomposition uniqueness, and sparse mixing procedures that may not hold in real applications, yet the paper provides no guidance on how practitioners can verify these assumptions or assess their validity. The theoretical contributions are quite incremental, adapting existing eigenvalue-eigenfunction decomposition techniques from causal representation learning rather than developing fundamentally new methods. The work's scope is also limited to discrete, regularly sampled time series, excluding continuous or irregular sampling scenarios.", "was_revised": true, "final_justification": "After the rebuttal, \n\n- (+) I am convinced of the assumptions and their practicality and limitations in real-world settings.\n- (+) The contributions of the paper are explained clearly.\n- (+) Runtime analysis seems to show the efficiency and low overhead that was added.\n- (-) I am skeptical about using the term \"shift\" in $p(x_t)$.\n\nHence, I am bumping my score from 3 to 4.", "rebuttal": "Dear Reviewer 2i9r, we highly appreciate the valuable comments and helpful suggestions on our paper and the time dedicated to reviewing it. Below, please see our point-to-point responses.\n\n>Q1&Q7: As for the explanation of assumptions. \n\nA1&Q7: Thank you for your question. We would like to respectfully clarify that these conditions are standards in the field of latent variables identifiability [1,2,3]. Moreover, we have already provided detailed discussions of the practical implications and real-world validity of these assumptions in the original paper **(Lines 175–197) and Appendix C**. Specifically:\n\n- Bound and Continuous Density: This assumption only requires that the transition probability densities of are bounded and continuous. It is easy to satisfy in practice because, with sufficient data, the estimated probability density functions are naturally bounded. For instance, temperature records in climate data change continuously within a reasonable range. This assumption is naturally satisfied since we use VAE-based model.\n\n- Injectivity: The injectivity condition only requires that there is sufficient variation in the density of one variable for different distributions of its parent variables. For example, $L_{x_{t},x_{t+1}\\mid x_{t-1},x_{t-2}}$ implies that historical observations have enough influence on future observations—e.g., historical temperature values strongly influence future temperature values. We further provide multiple examples of the injectivity of linear operators in Appendix C.3. However, this assumption is not testable. In a word, when the mapping from latent variables to observations is not fully injective, our model can only recover the injective components of the latent variables.\n\n- Uniqueness of Spectral Decomposition: This assumption asks that changes in the latent variables $z_t$ produce a sufficiently large and distinguishable effect on the mapping $x_{t-1} \\to x_t$. For example, if $z_t$ encodes seasonal information and $x_t$ denotes temperature, the day-to-day temperature jump in winter versus summer differs markedly. Since A3 only requires the existence of any two such distinguishable states, it is satisfied in many real-world scenarios with continuous $x_t$. More details are provided in Appendix C.6. Importantly, this decomposition uniqueness condition is empirically testable. After identifying the latent variables, we can use the observed variables to directly test whether the decomposition is unique by using tensor unique decomposition tools.\n\n- Sparse Mixing Procedure: The Sparse mixing procedure assumption is also common in real-world scenarios. Specifically, it denotes that the Markov network over $z_t$ and $x_t$ is sparse. For example, in the field of financial data, consider $z_{t,i}$ and $x_t$ as one policy and stocks, and this policy may not have an influence on all the stocks. In principle, this assumption is not testable. When it does not hold, we may no longer achieve component-wise identifiability. However, our method can still ensure block-wise identifiability of $z_t$, which is sufficient to model how $z_t$ influences the mapping from $x_{t-1}$ to $x_t$. This allows the framework to remain effective for online forecasting, even when full sparsity is violated.\n\nIn light of your comment, we have provided a further discussion in the updated version.\n\n[1] Fu, et al. \"Identification of Nonparametric Dynamic Causal Structure and Latent Process in Climate System.     \n[2] Lachapelle et al. \"Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA.\"  \n[3] Hu et al. \"Instrumental variable treatment of nonclassical measurement error models.\" \n\n>Q2: As for the contribution.\n\nA2: Thank you for your comment, which helps us highlight our contributions. We would like to clarify that our contribution lies in addressing a fundamentally different and practically motivated **real-world problem**. Specifically, we aim to address the online time series forecasting, which has been rarely handled in the field of time series forecasting. To address this problem, we have to formulate the specific model and then tailor the techniques, like eigenvalue–eigenfunction decomposition, and incorporate further constraints to achieve our identifiability results. \n\nMoreover, we would like to highlight the contributions of Theorems 1 and 3, which are, to the best of our knowledge, entirely new:  \n  1. **Theorem 1** formally establishes how latent variables affect predictive performance under nonstationarity.  \n  2. **Theorem 3** introduces the sparse mixing procedure into temporal causal representation learning and proves that latent variables are identifiable under this mild and practically meaningful assumption. \n\n>Q4: As for the work's scope.\n\nA4: Thank you. We would like to clarify that we have already explicitly acknowledged this limitation in the manuscript (see **Lines 370-372**). Our current theoretical analysis and experiments focus on regularly sampled time series, as it allows for a clear formulation of the identifiability conditions.  \n\nImportantly, within this scope, our framework is general; it accommodates a wide range of time series data with a discrete time step. And we believe it is unrealistic to expect a single theoretical framework to cover all possible data types, especially in the early stages of developing identifiability theory for latent-variable models. Extending our framework to continuous or irregularly sampled time series is indeed an interesting direction, and we have emphasized this as future work in the revised manuscript.\n\n>Q5: The paper mentions that \"latent variables introduce unknown distribution shifts\", can you elaborate? Is the paper also interested in shifts in the data space?\n\nA5: Thanks a lot. By \"latent variables introduce unknown distribution shifts\" we refer to the fact that the evolution of latent variables influences the causal process of observed variables, which in turn changes their marginal distribution over time. Specifically, in the data generation process  \n$$\n\\\\begin{array}{ccc}  \nz_{t} & \\\\rightarrow & z_{t+1} \\\\\\\\  \n\\\\downarrow & & \\\\downarrow \\\\\\\\  \nx_{t} & \\\\rightarrow & x_{t+1}  \n\\\\end{array}  \n$$\nthe transition $x_t \\rightarrow x_{t+1}$ is governed by $z_t$. As $z_t$ evolves, the marginal distribution \\(p(x_t)\\) also changes over time, leading to distribution shifts. Because \\(z_t\\) is unobserved, these distribution shifts are \"unknown\".\n\nRegarding the second part of your question, our work focuses on **distribution shifts in the data space (observed variables)**. We have explicitly highlighted  how latent variables have influence on the evolution of the observed distribution in the updated version.\n\n>Q6: As for time-delayed and instantaneus parents \n\nA6: Thank you for your question. Let us consider the following causal graph:  \n$$\n\\\\begin{array}{ccc}\nz_{t,1} & \\\\rightarrow & z_{t+1,1} \\\\\\\\\n\\\\downarrow &  & \\\\downarrow \\\\\\\\\nz_{t,2} & \\\\rightarrow & z_{t+1,2}\n\\\\end{array}\n$$\nIn this example:  \n- Time-delayed parents:  \nThe historical latent variables, such as $z_{t,1}$ and $z_{t,2}$, are time-delayed parents of $z_{t+1,1}$ and $z_{t+1,2}$, respectively, because they influence these variables **through temporal transitions** $z_{t,1} \\to z_{t+1,1}$\n\n- Instantaneous parents:  \nAt the current time step $t$, $z_{t,1}$ is an instantaneous parent of $z_{t,2}$, as it directly influences $z_{t,2}$ within the same timestep. Similarly, $z_{t+1,1}$ is an instantaneous parent of $z_{t+1,2}$.\n\n>Q8: As for computational requirements compared to baseline methods.\n\nA8: Thanks! We have provided a detailed clarification regarding computational requirements, scalability with sequence length and dimensionality, and runtime comparisons.\n\n1. **Experimental environment:**  \nAll experiments were conducted on a single NVIDIA GTX 3090 (24GB) GPU and an Intel i7-7700k CPU with 64GB memory.\n2. **Scalability of the architecture:**  \nAs discussed in Line 252, our framework is **model-agnostic** and can be combined with different backbone networks. The computational scaling thus mainly depends on the chosen backbone:  \n   - **Encoder and forecaster:** Their scalability with sequence length and input dimensionality follows the standard complexity of the selected backbone (e.g., LSTD or OneNet).\n   - **Decoder:** The decoder is implemented as a simple MLP and is independent of sequence length; it only scales with the output dimensionality.\n   - **Noise estimators:** Each latent variable dimension corresponds to two noise estimators. Thus, the complexity of these components grows **linearly** with the latent dimensionality.\n3. **Runtime comparisons:**  \nWe have conducted runtime evaluations across different datasets and backbones to provide a fair comparison with baseline methods. The table below shows the average runtime (seconds) per training epoch for LSTD and Proceed backbones.\n\n|Model|LSTD|LSTD+TOT|Proceed-T|Proceed-T+TOT|\n|-|-|-|-|-|\n|ETTh2|12.969|6.122|28.513|55.02|\n|Exchange|7.701|3.351|12.12|13.352|\n\n>Q9: 4 consecutive observations for identifiability.\n\nA9: Thank you for this insightful question. As discussed in **Lines 152–159**, Theorem 2 implies that at least 4 consecutive observations are theoretically required to describe the variation of the latent variables $z_t$. This is the theoretical minimum for identifiability in our framework. Specifically, with only 3 consecutive observations, the distributional changes are insufficient to uniquely recover the latent transitions, making $z_t$ unidentifiable under the assumptions in Theorem 2. And using more than 4 observations (e.g., 5) is allowed and does not violate the theoretical framework. Following your suggestion, we conducted additional experiments by varying the number of observations used for identification. Experiment results are shown as follows:\n \n|len(x)| MCC|\n|-|-|\n|1|0.7232(0.1784)|\n|2|0.8435(0.1326)|\n|3|0.9224(0.0841)|\n|4|0.9543(0.0541)|\n|5|0.9578(0.0439)|"}, {"review_id": "Z3dTzigtMa", "reviewer": "Reviewer_CFrf", "summary": "This paper proposes a theoretical and practical framework called TOT for online time series forecasting, addressing the challenge of temporal distribution shifts caused by latent variables. It establishes rigorous theoretical guarantees, showing that incorporating latent variables into forecasting significantly reduces predictive risk. The authors further prove identifiability conditions, demonstrating that both latent states and their causal dynamics can be inferred using minimal consecutive observations. Additionally, the paper presents a plug-and-play model architecture involving a temporal decoder and noise estimators. Extensive experiments on synthetic and real-world datasets validate the theoretical insights and show consistent performance improvements over existing methods.", "questions": "See summary or Weaknesses.", "limitations": "See summary or Weaknesses.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "1.\tThe authors are recommended to verify their citation formatting carefully, as it appears inconsistent with standard practice. Additionally, citations in the main text do not link correctly to the corresponding references listed.\n\n2.\tThe authors need to clarify the rationale for emphasizing \"online time-series forecasting\" explicitly. If there is no substantial distinction from general time-series forecasting, it would be beneficial to unify the terminology, enhancing clarity and facilitating progress within the field.\n\n3.\tPlease explicitly state the theoretical framework’s practical implications, detailing how it theoretically informs real-world applications and contributes to advancing the research area.\n\n4.\tThe experimental validation presented in the manuscript is insufficient, as it employs only two metrics (MSE and MAE). It is advised to repeat experiments multiple times, reporting both mean and standard deviation of the results to ensure robustness.\n\n5.\tFrom the current numerical results, the proposed method does not exhibit significant improvements in forecasting performance. Additional experimental evidence should be provided to clearly demonstrate the advantages of the proposed approach.\n\n6.\tVisualization experiments are crucial for time-series forecasting tasks; hence, the authors should consider including them to enhance interpretability and clarity.", "was_revised": true, "final_justification": "I have carefully read the author's reply and I have decided to keep my score.", "rebuttal": "Dear Review CFrf, we sincerely appreciate your informative feedback and helpful suggestions that helped clarify our contributions, improve the readability of our theories, and the completeness of our experiments. Please see our point-to-point response below.\n\n>Q1: The authors are recommended to verify their citation formatting carefully, as it appears inconsistent with standard practice. Additionally, citations in the main text do not link correctly to the corresponding references listed.\n\nA1: Thank you for pointing this out. We appreciate your careful reading. We have thoroughly checked all citations and corrected inconsistencies. Additionally, we have fixed the incorrect linking between in-text citations and the reference list in the revised manuscript. \n\n>Q2: The authors need to clarify the rationale for emphasizing \"online time-series forecasting\" explicitly. If there is no substantial distinction from general time-series forecasting, it would be beneficial to unify the terminology, enhancing clarity and facilitating progress within the field.\n\nA2: Thank you for this constructive comment, which helps us better clarify the scope of our work.  \n\nThe explicit emphasis on **“online time-series forecasting”** is important because the setting differs substantially from general time-series forecasting. In online forecasting:\n- We **cannot access the entire historical data**; instead, we can only use sequentially arriving data streams and rely on the most recent observations within a moving window for forecasting.\n- Handling **nonstationarity** in such a constrained setting is crucial. One of our key theoretical contributions (Theorem 2) proves that **at least 4 consecutive observations** are sufficient to identify the latent variables driving nonstationarity, which aligns directly with the requirements of online forecasting.\n\nWe agree that our methodology is general and also applicable to nonstationary time-series forecasting in offline settings. Following your suggestion, we have applied our method to standard nonstationary forecasting tasks, and the experiment results are shown as follows:\n\n|||TOT||koopa||FITS||DLinear+SOILD||\n|---|---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n|Metric||MSE|MAE|MSE|MAE|MSE|MAE|MSE|MAE|\n|ECL|48|0.129|0.228|0.13|0.234|0.147|0.25|0.183|0.262|\n||96|0.131|0.223|0.136|0.236|0.134|0.256|0.188|0.254|\n||144|0.147|0.239|0.149|0.247|0.227|0.341|0.185|0.297|\n||192|0.157|0.25|0.156|0.254|0.198|0.265|0.217|0.284|\n|ETTh2|48|0.225|0.298|0.226|0.3|0.256|0.329|0.268|0.328|\n||96|0.284|0.34|0.297|0.349|0.315|0.365|0.34|0.361|\n||144|0.312|0.365|0.333|0.381|0.351|0.391|0.355|0.429|\n||192|0.336|0.379|0.356|0.393|0.358|0.398|0.387|0.467|\n|Exchange|48|0.042|0.141|0.042|0.143|0.054|0.165|0.047|0.179|\n||96|0.086|0.205|0.083|0.207|0.096|0.146|0.103|0.25|\n||144|0.125|0.254|0.13|0.261|0.145|0.304|0.148|0.311|\n||192|0.164|0.296|0.184|0.309|0.181|0.265|0.2|0.356|\n|ILI|24|1.456|0.778|1.621|0.8|1.633|0.851|1.577|0.845|\n||36|1.79|0.839|1.803|0.855|1.952|0.969|2.048|0.952|\n||48|1.746|0.885|1.768|0.903|2.256|1.069|2.029|1.117|\n||60|1.831|0.89|1.743|0.891|2.084|1.352|1.994|1.003|\n|Traffic|48|0.36|0.231|0.415|0.274|0.482|0.251|0.481|0.374|\n||96|0.323|0.218|0.401|0.275|0.361|0.234|0.401|0.33|\n||144|0.315|0.217|0.397|0.276|0.358|0.259|0.428|0.299|\n||192|0.315|0.22|0.403|0.284|0.359|0.317|0.42|0.297|\n\nIn light of your suggestions, we have revised the manuscript to make this distinction clearer while also emphasizing the broader applicability of our framework.  \n\n>Q3: Please explicitly state the theoretical framework’s practical implications, detailing how it theoretically informs real-world applications and contributes to advancing the research area.\n\nA3: Thank you for this question. We would like to respectfully clarify that we have already provided detailed discussions of the theoretical framework’s **practical implications** and real-world validity **in the original paper (Lines 175–197) and Appendix C**. Please kindly let us know if you cannot find it. Specifically:  \n\n- **Assumption 1 (Bounded and continuous densities):**  \nThis assumption only requires that the transition probability densities of observed and latent variables are bounded and continuous, which is easy to satisfy in practice. With sufficient data, the estimated probability density functions are naturally bounded. For instance, temperature records in climate data change continuously within a reasonable range.\n- **Assumption 2 (Injectivity of linear operators):**  \nThis condition only requires sufficient variation in the density of one variable for different distributions of its parent variables. For example:  \n$$\nL_{x_{t}, x_{t+1} \\mid x_{t-1}, x_{t-2}}\n$$  \nimplies that historical observations have a strong influence on future observations—e.g., historical temperature values strongly influence future temperature values. Multiple practical examples of injectivity verification are provided in Appendix C.3.  \n- **Assumption 3 (Spectral decomposition uniqueness):**  \nThis assumption requires that changes in the latent variables \\(z_t\\) produce a sufficiently large and distinguishable effect on the mapping \\(x_{t-1} \\to x_t\\). For example, if \\(z_t\\) encodes seasonal information and \\(x_t\\) denotes temperature, the day-to-day temperature change in winter versus summer differs markedly. Since this assumption only requires the existence of any two such distinguishable states, it is satisfied in many real-world scenarios with continuous \\(x_t\\) (details in Appendix C.6).\n- **Assumption 4 (Sparse mixing procedure):**  \nThis assumption is mild and common in real-world scenarios. For instance, in financial data, if \\(z_{t,i}\\) denotes a policy and \\(x_t\\) denotes stock prices, a single policy typically does not influence all stocks.  \nThis assumption can also be empirically checked by evaluating the Jacobian matrix of \\(x_t\\) with respect to \\(z_t\\) during reconstruction; if the estimated Jacobian is sparse, it validates the sparse mixing procedure.  \n\nIn light of your comment, we have further emphasized these points in the revised version to make the practical implications of the theoretical framework clearer.  \n\n>Q4: The experimental validation presented in the manuscript is insufficient, as it employs only two metrics (MSE and MAE). It is advised to repeat experiments multiple times, reporting both mean and standard deviation of the results to ensure robustness.\n\nA4: Thank you for your suggestion. We would like to clarify that we have already addressed this concern in the current manuscript. Specifically, as stated in **Lines 343–344**, since some methods report the best results in their original papers, we show the best results in the main text. We also provide the experimental results with the mean and variance over three random seeds in Appendix D.2.4. Please kindly let us know if you cannot find it.  \n\nThus, the robustness of our method has been verified, and the corresponding results are reported in the appendix for completeness.\n\n>Q5: From the current numerical results, the proposed method does not exhibit significant improvements in forecasting performance. Additional experimental evidence should be provided to clearly demonstrate the advantages of the proposed approach.\n\nA5: Thank you for your comment. We respectfully disagree with the statement that our method does not exhibit significant improvements in forecasting performance.  \n\nAs summarized below, we have computed the **average relative improvements** of our method over different backbones and datasets:  \n\n|Backbone| MSE Improvement|MAE Improvement|\n|-|-|-|\n|LSTD|1.79%|1.91%|\n|Proceed-T|8.54%|4.85%|\n|OneNet|11.40%|5.88%|\n|OneNet-T|14.14%|12.32%|\n|Online-T|34.31%|31.19%|\n\nThese results clearly show that our method consistently improves forecasting performance across **all tested backbones**. Notably, the improvement is larger for relatively older or simpler backbones, demonstrating that our framework is **general and complementary** to different model architectures.\n\nFollowing your suggestion, we further evaluated additional backbone networks, including ER[1] and DERpp[2]. Experiment results are shown as follows:\n\n|DERpp|DERpp++|ER|ER++|\n|:-:|:-:|:-:|:-:|\n|0.508|0.733|0.508|0.456|\n|0.828|0.741|0.808|0.762|\n|1.157|1.006|1.136|0.926|\n|0.083|0.078|0.086|0.079|\n|0.196|0.191|0.202|0.197|\n|0.208|0.212|0.22|0.183|\n|0.174|0.161|0.18|0.153|\n|0.287|0.281|0.293|0.279|\n|0.294|0.289|0.297|0.284|\n\n[1] Chaudhry,et al. On tiny episodic memories in continual learning.  \n[2] Buzzega et al. Dark experience for general continual\nlearning: a strong, simple baseline. \n\n>Q6: Visualization Results.\n\nA6: Thank you for this helpful suggestion. We fully agree that visualization is important for improving interpretability in time-series forecasting. \n\nDue to policy restrictions, we are unable to include the figures in the manuscript. However, to address this concern, we have presented the forecasting trends in a tabular format, which conveys the same information as line plots. Additionally, we have included corresponding analyses to highlight the interpretability of the results. \n\n|Timestep|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|\n|-|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|---:|---:|---:|---:|---:|-:|-:|-:|-:|-:|\n|GroundTrue|0.078|-0.139|-0.440|-0.693|-0.681|-0.488|0.053|0.174|-0.500|0.029|-0.247|-0.151|0.330|0.427|0.306|0.198|0.679|1.125|0.788|0.571|-0.982|0.090|0.667|0.451|\n|Baseline|-0.917|-0.598|-0.501|-0.970|-1.050|-1.350|-1.257|-1.242|-1.432|-1.292|-1.213|-1.244|-0.894|-1.069|-1.120|-1.377|-1.138|-0.926|-1.011|-0.528|-0.689|-0.768|-0.424|-0.878|\n|Ours|-0.172|-1.015|-0.710|-1.076|-1.273|-1.137|-0.928|-0.857|-0.928|-0.581|-0.401|-0.174|-0.396|-0.244|-0.145|0.128|-0.086|-0.100|0.325|0.209|0.221|-0.137|-0.136|-0.661|\n\nThe visualization results from the above table show that our method achieves a better performance."}]

Author rebuttal:

Meta-review: The paper introduces a framework for online time series forecasting under latent-driven distribution shifts, with theoretical guarantees on reduced risk and identifiability, supported by empirical validation.  Its main strengths are strong theoretical grounding, novel identifiability results, model-agnostic design, and consistent empirical improvements.  Weaknesses include restrictive assumptions, clarity issues on the online component and sparsity, and initially limited experimental reporting.  The decision to accept is based on the combination of rigorous theory and practical relevance, which outweigh incremental aspects and assumptions.  During rebuttal, the authors provided clarifications, additional experiments, and stronger justifications, satisfying most reviewers; only one (less-engaged) reviewer remained skeptical, but consensus leaned toward recognizing the work’s solid and impactful contributions.
Decision: Accept (poster)