Title: Towards Identifiability of Hierarchical Temporal Causal Representation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: causal representation learning
Abstract: Modeling hierarchical latent dynamics behind time series data is critical for capturing temporal dependencies across multiple levels of abstraction in real-world tasks. However, existing temporal causal representation learning methods fail to capture such dynamics, as they fail to recover the joint distribution of hierarchical latent variables from \textit{single-timestep observed variables}. 
Interestingly, we find that the joint distribution of hierarchical latent variables can be uniquely determined using three conditionally independent observations. Building on this insight, we propose a Causally Hierarchical Latent Dynamic (CHiLD) identification framework. Our approach first employs temporal contextual observed variables to identify the joint distribution of multi-layer latent variables. Sequentially, we exploit the natural sparsity of the hierarchical structure among latent variables to identify latent variables within each layer. Guided by the theoretical results, we develop a time series generative model grounded in variational inference. This model incorporates a contextual encoder to reconstruct multi-layer latent variables and normalize flow-based hierarchical prior networks to impose the independent noise condition of hierarchical latent dynamics. Empirical evaluations on both synthetic and real-world datasets validate our theoretical claims and demonstrate the effectiveness of CHiLD in modeling hierarchical latent dynamics.

Towards Identifiability of Hierarchical Temporal Causal Representation Learning

Abstract

Modeling hierarchical latent dynamics behind time series data is critical for capturing temporal dependencies across multiple levels of abstraction in real-world tasks. However, existing temporal causal representation learning methods fail to capture such dynamics, as they fail to recover the joint distribution of hierarchical latent variables from single-timestep observed variables . Interestingly, we find that the joint distribution of hierarchical latent variables can be uniquely determined using three conditionally independent observations. Building on this insight, we propose a C ausally Hi erarchical L atent D ynamic ( CHiLD ) identification framework. Our approach first employs temporal contextual observed variables to identify the joint distribution of multi-layer latent variables. Sequentially, we exploit the natural sparsity of the hierarchical structure among latent variables to identify latent variables within each layer. Guided by the theoretical results, we develop a time series generative model grounded in variational inference. This model incorporates a contextual encoder to reconstruct multi-layer latent variables and normalize flowbased hierarchical prior networks to impose the independent noise condition of hierarchical latent dynamics. Empirical evaluations on both synthetic and realworld datasets validate our theoretical claims and demonstrate the effectiveness of CHiLD in modeling hierarchical latent dynamics. 3

1 Introduction

Hierarchical temporal structures are pervasive in time series data such as weather records and stock prices. These structures arise from latent processes operating at multi-level abstraction-for example, seasonal, monthly, and daily variations in climate data. Understanding the hierarchical latent dynamics underlying time series remains a fundamental challenge [68, 4, 70] and has received growing attention. The core of this problem is the identification of latent processes evolving across different levels of abstraction from observed data.

To identify the temporal latent process, several methods have considered Independent Component Analysis (ICA) [30, 8, 27] to identify latent variables. To extend to nonlinear scenarios, researchers use various assumptions, such as sufficient changes [37, 72, 47, 77], to ensure the independent variation of latent variables. Specifically, some approaches leverage auxiliary variables [18, 19, 28, 29, 37, 31] to achieve strong identifiability of latent variables. In the context of time-series data, others utilize historical observations as surrogates for historical latent variables to induce sufficient changes [73, 29].

Recent advances have further tackled the challenge of identifying latent dynamics with instantaneous dependencies under assumptions like interventions [54], grouping observations [57], and the sparse causal influence [48]. Please refer to more discussion of the related works and real-world implications of hierarchical latent dynamics in Appendix E and C, respectively.

Despite recent advances, these methods predominantly assume a single layer of latent variables and will fail to identify hierarchical latent dynamics. As a result, under a hierarchical structure, using historical observation as a surrogate to leverage the sufficient changes condition is not applicable anymore since the high-level variables will introduce noise. Consequently, methods under the single-layer assumption [28, 74] face challenges in recovering the joint distribution of hierarchical latent variables from single-timestep observations, particularly when the observations are derived from lower-level variables. This limitation prevents these methods from using historical observations as surrogates to meet the sufficient changes condition, resulting in suboptimal identifiability.

The point above highlights the urgent need for temporal causal representation learning with hierarchical latent dynamics is to recover the joint distribution of hierarchical latent variables. Interestingly, we find that the distribution of latent variables can be uniquely determined with the help of three temporally adjacent

𝑧

!"#

#

𝑧

!

#

𝑧

!

$

x

!"#

x

!

𝑧

!"#

$

𝑧

!%#

#

𝑧

!%#

$

x

!%#

…

…

…

observations [23, 25]. Furthermore, we demonstrate that even with hierarchical structures in the latent processes, the joint distribution of multi-layer latent variables can be identified by incorporating additional temporal contextual observations. Therefore, we can derive a hierarchical dynamic process underlying the observed data via the natural sparsity of the hierarchical latent structure, facilitating time series generation grounded in a causal understanding of the process.

Based on this insight, we therefore establish a C ausally Hi erarchical L atent D ynamic ( CHiLD ) identification framework by harnessing the temporal contextual observations and natural sparsity of the hierarchical structure among latent variables. Additionally, to connect the theoretical results with a practical algorithm, we develop a time series generative model based on a variational autoencoder with a contextual encoder and normalizing flow-based hierarchical prior networks. Specifically, to identify the joint distribution of multi-layer latent variables, the contextual encoder transforms the historical, current, and future observations into the hierarchical latent variables at the current timestamp. Moreover, to enforce the independent noise condition of hierarchical latent dynamics, the normalizing flow-based hierarchical prior networks determine the prior distribution of hierarchical latent variables. These two specialized modules together boost the identification of hierarchical temporal causal representation learning. We validate our approach through simulation experiments to demonstrate identifiability and evaluate its performance on eleven time-series generation benchmarks for both generation quality and controllability. The impressive quantitative and qualitative results underscore the effectiveness of our method.

2 Preliminaries

2.1 Hierarchical Data Generation Process

We begin with the time series generation process with a hierarchical latent causal process as shown in Figure 1 4 . To facilitate clarity, we adopt the terminology widely used in ICA literature like observed variables x t , latent variables z t , and the mixing function g . In the context of hierarchical latent dynamics, we let z t = { z 1 t , · · · , z l t , · · · , z L t } , where z l t ∈ R n denotes the l -th layer latent variables. Suppose that we have observed variables with discrete timestamps X = { x 1 , · · · , x t , · · · , x T } , where x t ∈ R n is only generated from first layer latent variables z 1 t ∈ Z ⊆ R n (low-level latent

variables) via a nonlinear mixing function g and independent noise ϵ 0 t . Mathematically, we have:

For the l -th layer, i -th dimension latent variable at t timestamp z l t,i , l ∈ { 1 , · · · , L -1 } is generated via the latent causal procedure, which is assumed to be related to time-delayed parent Pa d ( z l t,i ) and the hierarchical parents Pa h ( z l t,i ) , respectively. Formally, it can be written via a structural equation model as follows

where ϵ l t,i is the temporally and spatially independent noise sampled from p ϵ l t,i . And the L -th layer latent variables z L t is only related to the time-delayed parent Pa d ( z L t,i ) , which is formalized as follows:

Note that the hierarchical structure among latent variables of data generation process in Figure 1 and Equation (1)-(3) is naturally sparse, since there are no direct causal relations between z l t and z l +1 t -1 , meaning that z l t is independent of z l +1 t -1 conditional on other latent variables.

Based on the generation process above, we further define the block-wise identifiability and componentwise identifiability of latent variables in Definition 1 and 2, respectively.

Definition 1 ( Block-wise Identifiability of Latent Variables z l t [69]) . The block-wise identifiability of z l t ∈ R n means that for ground-truth z l t , there exists ˆ z l t ∈ R n and an invertible function h : R n → R n , such that z l t = h (ˆ z l t ) .

Definition 2 ( Component-wise Identifiability of Latent Variables z l t,i [42]) . The component-wise identifiability of z l t ∈ R n is that for each z l t,i , i ∈ { 1 , · · · , n } , there exists a corresponding estimated component ˆ z l t,j , j ∈ { 1 , · · · , n } and an invertible function h l i : R → R , such that z l t,i = h l i (ˆ z l t,j ) .

2.2 Why previous theoretical results can hardly identify the hierarchical latent variables?

Based on the data generation process described in Equations (1)-(3), we provide a detailed explanation of why previous methods fail to identify hierarchical latent variables. First, the previous methods for temporal causal representation learning [28, 29, 73, 74] usually assume that the observed variables x t are generated from a single-layer latent variables z t via an invertible and deterministic mixing function g , e.g., x t = g ( z t ) , implying that we can recover the distribution of latent variables from observed variables by matching the marginal distribution of observed variables. Sequentially, these methods can leverage the historical observation x t -1 as a surrogate for z t -1 to meet the sufficient change condition and further achieve identifiability.

However, when the data generation process follows a hierarchical latent dynamics (we suppose there are two latent layers for convenience), since the low-level latent variables, e.g., z 1 t are generated from the high-level latent variables e.g., z 2 t , and noise ϵ 1 t , we cannot recover the joint distribution of z t = { z 1 t , z 2 t } due to the additional noise ϵ 1 t . Therefore, previous methods cannot consider x t -1 as the surrogate of z t -1 , and hence cannot achieve identification under the hierarchical latent dynamics.

3 Identifiability of Hierarchical Latent Variables

In this section, we show how to identify the hierarchical latent dynamics under temporal contextual observations. Specifically, we first establish the block-wise identifiability results of multi-layer latent variables z t = { z 1 t , · · · , z L t } with 2 L +1 adjacent observed variables (Theorem 1). Sequentially, we leverage the estimated multi-layer latent variables ˆ z t as a surrogate of the true variables z t to meet the sufficient change condition. Then we leverage the connection between natural sparsity of the hierarchical structure among latent variables and cross derivative [52] to achieve block-wise identifiability for latent variables within each layer (Theorem 2). Finally, we further exploit the previous results [74] to achieve component-wise identifiability of latent variables z l t,i (Lemma 1).

3.1 Identifiability of Multi-layer latent variables z t

In this subsection, we show the identifiability results of multi-layer latent variables z t . For a better explanation, we first provide the definition of the linear operator [23, 25] as follows.

Definition 3 ( Linear Operator [23, 11]) . Consider two random variables a and b with support A and B , the linear operator L b | a is defined as a mapping from a probability function p a in some function space F ( A ) onto the probability function p b = L b | a ◦ p a in some function space F ( B ) ,

Based on the definition of linear operator, we show that the multi-layer latent variables z t = { z 1 t , · · · , z L t } can be block-wise identifiability as follows.

Theorem 1. ( Block-wise Identifiability of Latent Variables z t in Hierarchical Latent Process. ) Suppose the observed and L -layer latent variables follow the data generation process in Figure 1. By matching the true joint distribution of 2 L + 1 number of adjacent observed variables, i.e., { x t -L , · · · , x t , · · · , x t + L } , we further make the following assumptions:

(i) The joint distribution of x , z , and their marginal and conditional densities are bounded and continuous.

(ii) The linear operators L x t +1 , ··· , x t + L | z t and L x t -L , ··· , x t -1 | x t +1 , ··· , x t + L are injective for bounded function space.

̸

(iii) For all z t , z ′ t ∈ Z t with z t = z ′ t , the set { x t : p ( x t | z t ) = p ( x t | z ′ t ) } has positive probability.

̸

Suppose that we have learned (ˆ g, ˆ f, p ˆ ϵ ) to achieve Equations (1) and (2), then the latent variables z t = { z 1 t , · · · , z L t } are block-wise identifiable.

Intuition and Proof Sketch. Compared with previous works that leverage spectral decomposition to achieve identifiability [23, 14], we do not require the monotonicity and normalization assumption [23] (see Appendix B.4) and quantitatively formulate the relations between required observations and the latent layers. Intuitively, Theorem 1 shows that at least 2 L +1 adjacent observations are the minimal information for the identification of z t under the data generation process described in Equations (1)-(3). This is because of the injectivity of the two operators, where an operator that maps some functional space onto lower dimensions cannot be injective by Definition 3. The proof can be summarized in the two steps. First, by using the fact that { x t -L , · · · , x t -1 } , { x t } , and { x t +1 , · · · , x t + L } are conditionally independent given z t , we can construct an eigenvalueeigenfunction decomposition regarding the integral operator. Next, by leveraging the relationship between the uniqueness of spectral decomposition (Theorem XV.4.3.5 [11]), latent variables are block-wise identifiable when the marginal distribution of observed variables is matched.

Implication. Theorem 1 demonstrates that temporal contextual observations can be used to identify the joint distribution of multi-layer latent variables, even when additional noises are introduced into the generation process from high-level latent variables to observations. As an additional benefit, the same identification result can be achieved even when the mixing process from latent to observed variables is non-deterministic and involves independent noise.

Discussion on Assumptions. To clarify our theoretical results, we provide an explanation of these assumptions and their relevance to real-world scenarios. 1) The assumption of the bounded and continuous conditional densities is standard in the literature on the identification of latent variables under measurement error [23, 25], meaning that the observed and latent variables are bounded and change continuously. For example, in the financial markets example, the stock prices (observed variables) and the interest rates (latent variables) usually change continuously. And their values are usually within a certain reasonable range. 2) The injective linear operator assumption is also used in the literature on the identification of latent variables under measurement error [14]. (More examples of this condition can be found in Appendix B.3). Intuitively, an injective operator L b | a implies that there is enough variation in the density of b for different distributions of a . Using the same financial markets example, the injectivity of L x t +1 , ··· , x t + L | z t means that the profitability (i.e., z t ) of a company has sufficient influence on the stock prices of that company (i.e., x t +1 , · · · , x t + L ). And L x t -L , ··· , x t -1 | x t +1 , ··· , x t + L means that the stock price of a company has a significant impact on its future stock price. Therefore, this assumption is easy to meet in practice. 3) The third assumption means that the density of x t can be different when the values of z t are different, which is also easy to meet. For example, under different interest rates, the variance of the stock price is usually different.

3.2 Identifiabiliy of Single-layer Latent Variables z l t

Theorem 2. ( Block-wise Identifiability of Latent Variables z l t in any l -th Layer. ) For a series of observed variables x t ∈ R n and estimated latent variables ˆ z l t ∈ R n with the corresponding process ˆ f i , ˆ P (ˆ ϵ ) , ˆ g , suppose that the process subject to observational equivalence x t = ˆ g (ˆ z 1 t , ˆ ϵ 0 t ) . We let c t ≜ { z t -1 , z t } ∈ R 2 × L × n and that M c t be the variable set of two consecutive timestamps and the corresponding Markov network respectively, and further employ following assumptions:

(i) (Smooth and Positive Density [74, 42]): The conditional probability function of the latent variables c t is smooth and positive.

(ii) (Sufficient Variability) [48]: Denote |M c t | as the number of edges in Markov network M c t . Let

where ⊕ denotes concatenation operation and ( i, j ) ∈ E ( M c t ) denotes all pairwise indice such that c t,i , c t,j are adjacent in M c t . For m ∈ [1 , · · · , n ] , there exist 4 n + |M c t | different values of z t -2 ,m , such that the 4 n + |M c t | values of vector functions w ( m ) are linearly independent.

Then for latent variables z l t at the l -th layer, z l t is block-wise identifiable without permutation, i.e., there exists ˆ z l t and an invertible function h l : R n → R n , such that z l t = h l (ˆ z l t ) .

Intuition and Proof Sketch. Proof can be found in Appendix A3. This proof is built upon the results of Theorem 1, since we cannot achieve the joint distribution of latent variables in the hierarchical structure. By considering latent variables c t = { z t -1 , z t } from two adjacent timestamps, the cross-layer latent variables are conditionally independent, e.g, z l t,i ⊥ ⊥ z l +1 t -1 ,j | c t \{ z l t,i , z l +1 t -1 ,j } , ∀ i, j ∈ { 1 , · · · , n } .

2

And such conditional independence implies ∂ log p ( c t ) ∂ z l t,i ∂ z l +1 t -1 ,j = 0 by leveraging the connection between conditional independence and cross derivatives [52]. Moreover, since we have achieved the blockwise identifiability of z t , we can leverage the contextual observation as the surrogates of historical latent variables and further introduce the sufficient variability assumption [48]. As a result, we can construct a full-rank linear system with the unique solution. By further exploiting the hierarchical temporal structure of latent variables, we can show that the estimated ˆ z l t can be block-wise identifiable without permutation, i.e, there exists an invertible function h l : R n → R n , such that z l t = h l (ˆ z l t ) .

Compared with Previous Theoretical Results. Although both [48] and Theorem 2 use the sparsity influence of latent dynamics to achieve identifiability, yet our contribution goes substantially further. First, IDOL [48] assumes an invertible mixing procedure, which begins from matching the marginal distribution of x t . While our method allows that partial latent variables contribute to the observations with a noise-contaminated mixing procedure, so it is built on the block-wise identification from Theorem 1. Second, IDOL shows the component-wise identifiability with permutation. However, our CHiLD shows the block-wise identifiable without permutation (please find it at Lemma A2 of Appendix B.5). This result is stronger than that of IDOL. We further show that each component within each layer z l t,i is component-wise identifiable.

Discussion on Assumptions. We further explain the assumptions and their real-world implications. First, the smooth and positive density assumption is a commonly adopted assumption in nonlinear ICA studies [37, 74], implying that the latent variables can change continuously based on historical information. This assumption can be easily met in real-world scenarios like financial markets. For example, the exchange rates usually change smoothly over time.

Second, the sufficient variability assumption is widely employed in existing results on the identifiability of temporally causal representation learning [73, 74, 5, 48], reflecting the changeability of latent variables. Take stock price forecasting as an example, the latent variables may represent quantitative factors such as the price-to-earnings ratio, trading volume spread at different time steps. The linear independence of the latent variables implies that changes in one factor, such as the price-to-earnings ratio, cannot be linearly represented by trading volume. Moreover, while the sufficiency assumption forms the foundation of identifiability theory, it is not overly restrictive in practical applications.

Lemma 1. ( Component-wise Identifiability of Latent Variables z l t,i in any l -th Layer [74, 42]) For a series of observed variables x t ∈ R n and estimated latent variables ˆ z t ∈ R n × L with the corresponding process ˆ f i , ˆ P (ˆ ϵ ) , ˆ g , suppose that the process subject to observational equivalence x t = ˆ g (ˆ z t , ˆ ϵ 0 t ) . Besides the conditional independence assumption, i.e., log p ( z l t | z l t -1 , z l +1 t ) = ∑ n i =1 log p ( z l t,i | z l t -1 , z l +1 t ) , we further assume the sufficient variability for each latent component. For any z l t ∈ Z l t ⊆ R n and ˆ u = { ˆ z l t -1 , ˆ z l +1 t } there exist 2 n +1 different values of ˆ u , m = 0 , · · · , 2 n , such that these 2 n vectors v l,m -v l, 0 are linearly independent, where v l,m is defined as:

Then for i -th latent variable z l t,i at the l -th layer, z l t,i is component-wise identifiability, i.e., there exists ˆ z l t,j and an invertible function h l t : R → R , such that ˆ z l t,j = h l t ( z l t,i ) .

Discussion. Please refer to the proof in Appendix B.7. Compared with existing works [28, 29, 74, 42] that also leverage similar assumptions (i.e., conditional independence and sufficient variability assumptions) to achieve component-wise identification, the main difference is that our method simultaneously leverage the time-delayed and hierarchical parents to meet the sufficient variability condition instead of domain or historical information.

3.3 Comparison with Existing Results of Causal Representation Learning

Compared to recent methods for causal representation learning, the proposed CHiLD model better aligns with real-world scenarios from multiple perspectives, as summarized in Table 1. Specifically, TDRL [74] and PCL [29] struggle to handle time-series data with hierarchical latent dynamics and noisy mixing processes. Although CaRiNG [5] also leverages temporal contextual information for identification, it remains limited as it fails to account for noisy mixing processes and cannot identify latent dynamics with instantaneous dependencies. More recently, Zhang et al. [77] and Li et al. [48] have employed sparse causal influences to identify latent variables with instantaneous dependencies. However, these methods neither address the challenges posed by noisy mixing processes nor identify latent variables within hierarchical structures. Another closely related work, Fu et al. [14], also utilizes unique eigenvalue-eigenfunction decomposition to achieve identifiability. However, this method is specifically designed for causal discovery in climate data and does not account for hierarchical latent dynamics. In contrast, our model extends previous results by establishing the relationship between the minimal required temporal observations and the size of hierarchical latent variables.

4 Approach

In this section, we propose the VAE-based model [39, 22] for time series generation as shown in Figure 2. Specifically, we begin by estimating the hierarchical latent variables using a contextual encoder which processes a sequence of observations, x t -L : t + L 5 , and outputs the estimated latent variables ˆ z t . Sequentially, the reconstructed observations are generated from the latent space via a step-wise decoder, which uses ˆ z t to produce ˆ x t .

To capture the hierarchical latent dynamics, we employ the Kullback-Leibler divergence between the posterior distribution of the estimated latent variables and a prior distribution, which coincides with the natural sparsity of hierarchical structure in Theorem 2 and the conditional independence condition in Lemma 1. Moreover, we devise a hierarchical prior network to estimate the prior distribution of latent

! x

!

! x

"

Dec

Enc

𝑧

!

!

𝑧

!

"

𝑧

!

#

x

!

x

"

Dec

Enc

𝑧

"

!

𝑧

"

"

𝑧

"

#

𝑓

!

𝑓

#

Hierarchical

Latent

Variables

𝑓

"

Contextual

Hierarchical

Encoder

Step-wise

Decoder

…

…

…

! x

$

x

#

Dec

Enc

𝑧

$

!

𝑧

$

"

𝑧

$

#

…

𝑓

!

𝑓

#

𝑓

"

…

…

…

𝑧

%&!

#

𝑧

%&!

!

𝑧

%&!

#&!

…

Hierarchical Prior

Networks

𝑧

%

!

𝑧

%

#

𝑧

%

#&!

𝑓

#&!

…

𝜖

%

#&!

variables, which is based on the normalizing flow [61, 63] and converts the prior distribution into a Gaussian noise like [74, 5]. Please refer to Appendix F.3 for implementation details.

4.1 Contextual Hierarchical Encoder and Step-wise Decoder

Guided by Theorem 1, we use the temporal context observations to identify multi-layer latent variables. Therefore, we devise a contextual hierarchical encoder for the inference process in Equation (7):

which ϕ 1 encodes the observations to the first latent layer, and adopts a similar form of 1D convolutional neural networks. Subsequently, each ϕ L encode the preceding latent layer, ˆ z L -1 t , to extract the deeper or higher-level components, yielding ˆ z L t . L can be considered as the size of the receptive field. Accordingly, we employ a neural architecture based on temporal convolutional neural networks (TCN) [44, 45] as the contextual encoder. Given the estimated latent variables ˆ z t , we further develop a step-wise decoder that only takes the bottom-layer ˆ z t as input to generate ˆ x t for each time-step within the receptive field, as shown in Equation (8).

4.2 Hierarchical Prior Networks

To make the estimated latent variables capture the hierarchical property and the conditional independent property , one straightforward solution is to minimize the Kullback-Leibler divergence between the posterior distribution and the prior distribution. However, the prior distribution is unknown and if we estimate the prior density directly, arbitrary density functions may be generated, leading to an incorrect posterior distribution. To address this challenge, motivated by [74, 5], who use normalizing flow [61] to model the causal procedure of latent variables by restricting the independence of estimated noise, we propose the hierarchical prior networks to estimate the prior distribution of latent variables. Specifically, given time delay τ , we let r l t = { r l t, 1 , · · · , r l t,i , · · · , r l t,n } be a set of learned inverse transition functions that take ˆ z l t,i , ˆ z l t -τ , and ˆ z l +1 t as input and generates the noise term, i.e., ˆ ϵ l t,i = r l t,i (ˆ z l t,i , ˆ z l t -τ , ˆ z l +1 t ) . And r l t is implemented by MLPs. Sequentially, we can devise a transformation κ : { ˆ z l t -τ , ˆ z l +1 t , ˆ z l t } → { ˆ z l t -τ , ˆ z l +1 t , ˆ ϵ l t } with its corresponding Jacobian

formula, we have:

By dividing p (ˆ z l t -τ , ˆ z l +1 t ) on both sides, the prior distribution can be derived as follows:

where p (ˆ ϵ l t,i ) follow Gaussian distributions. For the reconstruction likelihood L R , we use the meansquared error (MSE) to measure the discrepancy between the generated and original observations.

Further Discussion: Please note that although CaRiNG [5] also leverages the contextual observation in a V AE-based framework, the main difference between CaRiNG and CHiLD can be summarized into the following two folds. 1) The model design comes from different theoretical guidance. Motivated by the nonlinear ICA-based theoretical results, CaRiNG leverages the contextual observations to recover the lost latent information and address the challenge of a non-invertible mixing function. In contrast, our method is guided by the uniqueness of spectral decomposition and overcomes the difficulties of hierarchical latent dynamics. 2) The prior distribution estimations of these two methods are also different. CaRiNG assumes single-layer latent variables and uses the time-delayed and current latent variables to estimate noise. Meanwhile, the CHiLD allows hierarchical latent dynamics and hence harnesses the current, time-delayed, as well as hierarchical latent variables for noise estimation.

4.3 Optimization

Finally, we train the proposed model by optimizing the Evidence Lower Bound (ELBO) as follows

where D KL denotes the Kullback-Leibler divergence and the detailed derivation can be found in Appendix F.2. For the reconstruction loss L R , we use the mean-squared error (MSE) to measure the discrepancy between the generated and original observations. And we leverage the Equation (10) to approximate the KL divergence.

5 Experiment

5.1 Experiments on Synthetic Data

Data Generation. We generate synthetic time series data based on a fixed latent causal process, as defined in Equations (1)-(3). To evaluate the proposed theoretical results, we provide seven different datasets from A to G, with different numbers of latent variables, different complexities of generation processes, and different latent layers. Note that Dataset B consists of single-layer latent variables, while other datasets adhere to the assumptions underlying the proposed theoretical framework. Details of the data generation process and evaluation metrics can be found in Appendix G.1.

Baselines. To evaluate the effectiveness of our method, we consider the following baselines. First, we consider the standard β -VAE [20] and FactorVAE [38], which do not use historical information. Then we consider TDRL [74], iVAE [37], TCL [28], PCL [29], and SlowVAE [40], which use temporal information but do not contain instantaneous dependency. We also consider CaRiNG [5] and IDOL [48], which leverage contextual observations and consider instantaneous dependency, respectively. We use mean correlation coefficient (MCC) as the evaluation metric.

Results and Discussion. Experiment results are shown in Table 2. According to the experiment results, we can find that: 1) the proposed CHiLD achieves the highest MCC performance in Datasets A, C, D, and E; 2) the CHiLD and the recently proposed methods like IDOL and CaRiNG achieve good performance in Dataset B since it does not contain hierarchical latent dynamics. 3) In the more challenging Dataset F and G (datasets with large dimensions and latent layers), although the

Context-FID

Weather

WeatherBench

CESM2

Box

Gestures

Throwcatch

Discussion

Purchases

WalkDog

CHiLD

0.507(0.042)

0.078(0.017)

0.018(0.003)

0.092(0.011)

0.032(0.004)

0.022(0.003)

0.029(0.005)

0.021(0.009)

0.016(0.005)

KoVAE

2.131(0.637)

0.171(0.032)

0.198(0.030)

0.056(0.009)

0.079(0.023)

0.183(0.068)

0.235(0.036)

0.143(0.052)

0.137(0.034)

Diffusion-TS

3.826(0.915)

1.712(0.163)

0.407(0.037)

0.258(0.017)

0.088(0.006)

0.151(0.011)

0.487(0.116)

0.300(0.048)

0.334(0.037)

TimeGAN

6.434(1.398)

1.564(0.269)

3.255(0.232)

0.959(0.087)

0.830(0.081)

3.810(0.345)

3.761(0.393)

1.829(0.225)

3.838(0.440)

IDOL

1.694(0.485)

0.142(0.017)

0.125(0.026)

0.018(0.004)

0.025(0.002)

0.082(0.033)

0.029(0.014)

0.051(0.022)

cwVAE

5.061(1.359)

0.409(0.056)

0.238(0.044)

0.215(0.026)

0.329(0.051)

0.284(0.025)

0.919(0.132)

1.147(0.146)

0.565(0.126)

TimeVAE

3.910(0.858)

0.221(0.029)

0.108(0.029)

0.203(0.032)

0.438(0.163)

0.414(0.134)

0.067(0.020)

0.074(0.008)

0.082(0.012)

Correlational Score

Weather

WeatherBench

CESM2

Box

Gestures

Throwcatch

Discussion

Purchases

WalkDog

CHiLD

0.165(0.004)

5.190(0.016)

1.250(0.009)

1.599(0.021)

0.089(0.003)

0.065(0.001)

0.014(0.000)

0.100(0.003)

0.126(0.001)

KoVAE

1.620(0.012)

20.384(0.082)

10.087(0.017)

1.758(0.038)

1.015(0.030)

1.268(0.051)

4.060(0.023)

2.689(0.054)

3.015(0.021)

Diffusion-TS

1.932(0.021)

63.291(0.630)

16.659(0.231)

3.407(0.125)

1.724(0.220)

2.101(0.243)

5.851(0.109)

3.513(0.157)

3.701(0.037)

TimeGAN

3.438(0.020)

57.475(1.213)

20.641(0.082)

7.373(0.937)

10.917(1.620)

16.419(1.334)

27.876(0.485)

25.362(1.868)

21.848(0.230)

IDOL

1.195(0.014)

34.773(0.059)

9.125(0.016)

1.849(0.035)

0.634(0.008)

0.559(0.007)

1.295(0.010)

1.148(0.008)

2.067(0.016)

cwVAE

0.584(0.013)

34.690(0.131)

0.812(0.026)

0.926(0.064)

0.755(0.025)

8.624(0.032)

7.741(0.055)

TimeVAE

0.707(0.005)

24.873(0.091)

6.363(0.022)

0.891(0.018)

1.046(0.047)

1.063(0.046)

0.810(0.002)

0.568(0.002)

0.664(0.003)

(a) CHiLD

(c) DiffusionTS

(d)TimeGAN

(b) TimeVAE

Figure 3: Interpolation visualization of different models. For each method, after training the model, only one latent variable is gradually changed while keeping the other variables fixed. The images of each method from left to right represent the gradual increase of the latent variable.

MCC performance of the proposed CHiLD is lower, it can still outperform the existing methods for temporal causal representation learning, like IDOL, which supports our theoretical claims.

5.2 Experiments on Real-world Data

5.2.1 Experiment Setup

Datasets. We consider the task of unconditional time series generation and use the following datasets: Stock, ETTh1, fMRI, MuJoCo, and two human motion datasets, Human3.6M and HumanEva-I. For Human3.6M [32], we choose 3 motions: Discussion, Purchases, and WalkingDog. For HumanEva-I [65], we choose Box, Gesture, and Throwcatch. We further consider three climate datasets: Weather, WeatherBench, and CESM2. Please refer to Appendix G.2.1 and D for the dataset description and the connection between time series generation and modeling hierarchical temporal latent dynamics. Baselines. First, we consider the V AE-based methods like cwVAE [64], KoVAE [58] and TimeVAE [9]. We also consider other types of generative models like Diffusion-TS [76] and TimeGAN [75]. Moreover, we consider the causal representation-based method like IDOL [48]. We choose ContextFréchet Inception Distance (Context-FID) and the Correlational Score as evaluation metrics. We repeat each experiment over 3 random seeds and publish the mean and standard deviation. Please refer to Appendix G.2.2 for the introduction to the evaluation metric.

5.2.2 Quantitative Results

The experimental results in Table 3 show that the proposed CHiLD method significantly outperforms all baselines across most datasets, such as Box and Discussion, highlighting its potential for time series generation. While TimeVAE and KoVAE also employ VAE-based architectures, they assume independent latent variables. In contrast, CHiLD enforces a hierarchical latent structure, enabling it to model high-level dependence and hence generate more realistic time series data. Other time series generation baselines, such as Diffusion-TS and TimeGAN, fail to deliver optimal performance, particularly on human motion datasets. This can be attributed to the higher dimensionality of human motion data compared to datasets like ETTh1, which implies more complex latent dynamics. Please refer to Appendix G.3.2 and G.3.1 for more experiment results and the ablation studies.

5.2.3 Qualitative Results

We also provide the interpolation visualization results. Specifically, after training each model, we change one latent variable gradually and keep the other fixed.

Controllable Generation. To evaluate controllable generation, we compare TimeVAE, TimeGAN, and DiffusionTS in the Discussion motion generation dataset. As shown in Figure 3. TimeGAN performs the worst, as the generated motion patterns are difficult to recognize as human shapes. TimeVAE produces more structured movements but lacks coordination. While DiffusionTS generates recognizable human

(a) Interpolation visualizations

of High-Level Latent Variables

(b) Interpolation Visualizations

of Low-Level Latent Variables

Changing Joints

Stable Joints

shapes, the variations in motion with respect to the latent variable are limited, indicating weak controllability. In contrast, CHiLD method generates high-quality human motions and enables coordinated changes in limb movements through latent variable control. Please find more interpolation results in the form of GIF visualization from the supplementary material.

Hierarchical Generation. We interpolate both high-level and low-level latent variables in CHiLD and generate corresponding motions. As shown in Figure 4, green represents stable joints, while red indicates joints that change with the latent variable. The results show that high-level latent variables influence a broader range of joints compared to low-level ones, demonstrating that our method effectively captures hierarchical abstractions in time-series data.

6 Conclusion

This paper introduces an identification framework for temporal causal representation learning under hierarchical latent dynamics. The proposed method leverages temporal contextual observations to estimate the joint distributions of multi-layer latent variables and exploits the sparsity of hierarchical structures to achieve precise latent variable identification within each layer. A V AE-based generative model, incorporating a contextual encoder and normalizing flow-based hierarchical priors, enforces independent noise conditions while enhancing the capacity to generate realistic time series data. Empirical evaluations on both synthetic and real-world datasets validate the theoretical guarantees and demonstrate the effectiveness of our approach. Future works will extend this framework to more complex domains, including controllable video generation, as well as applications in neuroscience, finance, and climate modeling. Limitation: However, the component-wise identifiability result of our work also relies on other assumptions like layer-wise conditional independence. How to further relax these assumptions would be an interesting future direction.

7 Acknowledgment

The authors would like to thank the anonymous reviewers for helpful comments and suggestions during the reviewing process. The authors would like to acknowledge the support from NSF Award No. 2229881, AI Institute for Societal Decision Making (AI-SDM), the National Institutes of Health (NIH) under Contract R01HL159805, and grants from Quris AI, Florin Court Capital, and MBZUAI-WISJoint Program, and the Al Deira Causal Education project. Moreover, this research was supported in part by National Science and Technology Major Project (2021ZD0111501), National Science Fund for Excellent Young Scholars (62122022), Natural Science Foundation of China (U24A20233, 62206064, 62206061, 62476163, 62406078), Guangdong Basic and Applied Basic Research Foundation (2024A1515011901, 2023A04J1700, 2023B1515120020).

References

Ahmed Alaa, Alex James Chan, and Mihaela van der Schaar. Generative time-series modeling with fourier flows. In International Conference on Learning Representations, 2021.

Johannes Bill, Hrag Pailian, Samuel J Gershman, and Jan Drugowitsch. Hierarchical structure is employed by humans during visual motion perception. Proceedings of the National Academy of Sciences, 117(39):24581-24589, 2020.

Raymond J Carroll, Xiaohong Chen, and Yingyao Hu. Identification and estimation of nonlinear models using two samples with nonclassical measurement errors. Journal of nonparametric statistics, 22(4):379-399, 2010.

Cristian I. Challu, Peihong Jiang, Ying Nian Wu, and Laurent Callot. Deep generative model with hierarchical latent factors for time series anomaly detection. In Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera, editors, Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 1643-1654. PMLR, 28-30 Mar 2022. URL https://proceedings.mlr.press/ v151/challu22a.html .

Guangyi Chen, Yifan Shen, Zhenhao Chen, Xiangchen Song, Yuewen Sun, Weiran Yao, Xiao Liu, and Kun Zhang. Caring: Learning temporal causal representation under non-invertible generation process. arXiv preprint arXiv:2401.14535, 2024.

Victor Chernozhukov, Guido W Imbens, and Whitney K Newey. Instrumental variable estimation of nonseparable models. Journal of Econometrics, 139(1):4-14, 2007.

Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio. A recurrent latent variable model for sequential data. Advances in neural information processing systems, 28, 2015.

Pierre Comon. Independent component analysis, a new concept? Signal processing, 36(3): 287-314, 1994.

Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. Timevae: a variational auto-encoder for multivariate time series generation (2021). arXiv preprint arXiv:2111.08095.

Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. Timevae: A variational auto-encoder for multivariate time series generation. arXiv preprint arXiv:2111.08095, 2021.

Nelson Dunford and Jacob T Schwartz. Linear operators, part 1: general theory, volume 10. John Wiley & Sons, 1988.

Cristóbal Esteban, Stephanie L Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional gans. arXiv preprint arXiv:1706.02633, 2017.

Elizabeth Fons, Alejandro Sztrajman, Yousef El-Laham, Alexandros Iosifidis, and Svitlana Vyetrenko. Hypertime: Implicit neural representation for time series. arXiv preprint arXiv:2208.05836, 2022.

Minghao Fu, Biwei Huang, Zijian Li, Yujia Zheng, Ignavier Ng, Guangyi Chen, Yingyao Hu, and Kun Zhang. Learning general causal structures with hidden dynamic process for climate analysis. arXiv preprint arXiv:2501.12500, 2025.

Asadullah Hill Galib, Pang-Ning Tan, and Lifeng Luo. Fide: Frequency-inflated conditional diffusion model for extreme-aware time series generation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.

Samuel J Gershman, Joshua B Tenenbaum, and Frank Jäkel. Discovering hierarchical motion structure. Vision research, 126:232-241, 2016.

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-144, 2020.

Hermanni Hälvä and Aapo Hyvarinen. Hidden markov nonlinear ica: Unsupervised learning from nonstationary time series. In Conference on Uncertainty in Artificial Intelligence, pages 939-948. PMLR, 2020.

Hermanni Hälvä, Sylvain Le Corff, Luc Lehéricy, Jonathan So, Yongjie Zhu, Elisabeth Gassiat, and Aapo Hyvarinen. Disentangling identifiable features from noisy data with structured nonlinear ica. Advances in Neural Information Processing Systems, 34:1624-1633, 2021.

Irina Higgins, Loic Matthey, Arka Pal, Christopher P Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. ICLR (Poster), 3, 2017.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.

Matthew D Hoffman and David M Blei. Structured stochastic variational inference. In Artificial Intelligence and Statistics, pages 361-369, 2015.

Yingyao Hu and Susanne M. Schennach. Instrumental variable treatment of nonclassical measurement error models. Econometrica, 76(1):195-216, 2008. doi: https://doi.org/10. 1111/j.0012-9682.2008.00823.x. URL https://onlinelibrary.wiley.com/doi/abs/10. 1111/j.0012-9682.2008.00823.x .

Yingyao Hu and Susanne M Schennach. Instrumental variable treatment of nonclassical measurement error models. Econometrica, 76(1):195-216, 2008.

Yingyao Hu and Matthew Shum. Nonparametric identification of dynamic models with unobserved state variables. Journal of Econometrics, 171(1):32-44, 2012. ISSN 0304-4076. doi: https://doi.org/10.1016/j.jeconom.2012.05.023. URL https://www.sciencedirect.com/ science/article/pii/S0304407612001479 .

Yingyao Hu and Matthew Shum. Nonparametric identification of dynamic models with unobserved state variables. Journal of Econometrics, 171(1):32-44, 2012.

Aapo Hyvärinen. Independent component analysis: recent advances. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1984): 20110534, 2013.

Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in neural information processing systems, 29, 2016.

Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics, pages 460-469. PMLR, 2017.

Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411-430, 2000.

Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859-868. PMLR, 2019.

Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(7):1325-1339, 2014. doi: 10. 1109/TPAMI.2013.248.

Daniel Jarrett, Ioana Bica, and Mihaela van der Schaar. Time-series generation by contrastive imitation. Advances in neural information processing systems, 34:28968-28982, 2021.

Nadir Jeevanjee, Pedram Hassanzadeh, Spencer Hill, and Aditi Sheshadri. A perspective on climate model hierarchies. Journal of Advances in Modeling Earth Systems, 9(4):1760-1771, 2017.

Paul Jeha, Michael Bohlke-Schneider, Pedro Mercado, Shubham Kapoor, Rajbir Singh Nirwan, Valentin Flunkert, Jan Gasthaus, and Tim Januschowski. Psa-gan: Progressive self attention gans for synthetic time series. In The Tenth International Conference on Learning Representations, 2022.

Baoyu Jing, Shuqi Gu, Tianyu Chen, Zhiyu Yang, Dongsheng Li, Jingrui He, and Kan Ren. Towards editing time series. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.

Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics, pages 2207-2217. PMLR, 2020.

Hyunjik Kim and Andriy Mnih. Disentangling by factorising. In International conference on machine learning, pages 2649-2658. PMLR, 2018.

Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.

David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding. arXiv preprint arXiv:2007.10930, 2020.

Marcel Kollovieh, Abdul Fatir Ansari, Michael Bohlke-Schneider, Jasper Zschiegner, Hao Wang, and Yuyang Bernie Wang. Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting. Advances in Neural Information Processing Systems, 36, 2024.

Lingjing Kong, Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akinwande, and Kun Zhang. Partial disentanglement for domain adaptation. In International conference on machine learning, pages 11455-11472. PMLR, 2022.

Rainer Kress, Vladimir Maz'ya, and Vladimir Kozlov. Linear integral equations, volume 82. Springer, 1989.

Colin Lea, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks: A unified approach to action segmentation. In Computer Vision-ECCV 2016 Workshops: Amsterdam, The Netherlands, October 8-10 and 15-16, 2016, Proceedings, Part III 14, pages 47-54. Springer, 2016.

Colin Lea, Michael D Flynn, Rene Vidal, Austin Reiter, and Gregory D Hager. Temporal convolutional networks for action segmentation and detection. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 156-165, 2017.

Hongming Li, Shujian Yu, and Jose Principe. Causal recurrent variational autoencoder for medical time series generation. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 8562-8570, 2023.

Zijian Li, Ruichu Cai, Guangyi Chen, Boyang Sun, Zhifeng Hao, and Kun Zhang. Subspace identification for multi-source domain adaptation. Advances in Neural Information Processing Systems, 36, 2024.

Zijian Li, Yifan Shen, Kaitao Zheng, Ruichu Cai, Xiangchen Song, Mingming Gong, Zhengmao Zhu, Guangyi Chen, and Kun Zhang. On the identification of temporally causal representation with instantaneous dependence. arXiv preprint arXiv:2405.15325, 2024.

Zijian Li, Shunxing Fan, Yujia Zheng, Ignavier Ng, Shaoan Xie, Guangyi Chen, Xinshuai Dong, Ruichu Cai, and Kun Zhang. Synergy between sufficient changes and sparse mixing procedure for disentangled representation learning. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=G1r2rBkUdu .

Zijian Li, Zunhong Xu, Ruichu Cai, Zhenhui Yang, Yuguang Yan, Zhifeng Hao, Guangyi Chen, and Kun Zhang. Identifying semantic component for robust molecular property prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.

Shujian Liao, Hao Ni, Lukasz Szpruch, Magnus Wiese, Marc Sabate-Vidales, and Baoren Xiao. Conditional sig-wasserstein gans for time series generation. arXiv preprint arXiv:2006.05421, 2020.

Juan Lin. Factorizing multivariate function classes. In M. Jordan, M. Kearns, and S. Solla, editors, Advances in Neural Information Processing Systems, volume 10. MIT Press, 1997. URL https://proceedings.neurips.cc/paper_files/paper/1997/ file/8fb21ee7a2207526da55a679f0332de2-Paper.pdf .

Juan Lin. Factorizing multivariate function classes. Advances in neural information processing systems, 10, 1997.

Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Efstratios Gavves. Causal representation learning for instantaneous and temporal effects in interactive systems. In The Eleventh International Conference on Learning Representations, 2023.

Lutz Mattner. Some incomplete but boundedly complete location families. The Annals of Statistics, pages 2158-2162, 1993.

Olof Mogren. C-rnn-gan: Continuous recurrent neural networks with adversarial training. arxiv 2016. arXiv preprint arXiv:1611.09904.

Hiroshi Morioka and Aapo Hyvärinen. Causal representation learning made identifiable by grouping of observational variables. arXiv preprint arXiv:2310.15709, 2023.

Ilan Naiman, N Benjamin Erichson, Pu Ren, Michael W Mahoney, and Omri Azencot. Generative modeling of regular and irregular time series data via koopman vaes. arXiv preprint arXiv:2310.02619, 2023.

Whitney K Newey and James L Powell. Instrumental variable estimation of nonparametric models. Econometrica, 71(5):1565-1578, 2003.

Juan Carlos Niebles and Li Fei-Fei. A hierarchical model of shape and appearance for human action classification. In 2007 IEEE Conference on computer vision and pattern recognition, pages 1-8. IEEE, 2007.

George Papamakarios, Eric Nalisnick, Danilo Jimenez Rezende, Shakir Mohamed, and Balaji Lakshminarayanan. Normalizing flows for probabilistic modeling and inference. Journal of Machine Learning Research, 22(57):1-64, 2021.

Hengzhi Pei, Kan Ren, Yuqing Yang, Chang Liu, Tao Qin, and Dongsheng Li. Towards generating real-world time series data. In 2021 IEEE International Conference on Data Mining (ICDM), pages 469-478. IEEE, 2021.

Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In International conference on machine learning, pages 1530-1538. PMLR, 2015.

Vaibhav Saxena, Jimmy Ba, and Danijar Hafner. Clockwork variational autoencoders. Advances in Neural Information Processing Systems, 34:29246-29257, 2021.

Leonid Sigal, Alexandru O. Balan, and Michael J. Black. Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. Int. J. Comput. Vis., 87(1-2):4-27, 2010. doi: 10.1007/S11263-009-0273-6.

Kaleb E Smith and Anthony O Smith. Conditional gan for timeseries generation. arXiv preprint arXiv:2006.16477, 2020.

Yuewen Sun, Lingjing Kong, Guangyi Chen, Loka Li, Gongxu Luo, Zijian Li, Yixuan Zhang, Yujia Zheng, Mengyue Yang, Petar Stojanov, et al. Causal representation learning from multimodal biological observations. arXiv preprint arXiv:2411.06518, 2024.

Ruey S Tsay. Analysis of financial time series. John wiley & sons, 2005.

Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances in neural information processing systems, 34: 16451-16467, 2021.

Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. Micn: Multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations, 2022.

Christopher K Wikle. Hierarchical bayesian models for predicting the spread of ecological processes. Ecology, 84(6):1382-1394, 2003.

Shaoan Xie, Lingjing Kong, Mingming Gong, and Kun Zhang. Multi-domain image generation and translation with identifiability guarantees. In The Eleventh International Conference on Learning Representations, 2022.

Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. arXiv preprint arXiv:2110.05428, 2021.

Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. Advances in Neural Information Processing Systems, 35:26492-26503, 2022.

Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. Time-series generative adversarial networks. Advances in neural information processing systems, 32, 2019.

Xinyu Yuan and Yan Qiao. Diffusion-ts: Interpretable diffusion for general time series generation. arXiv preprint arXiv:2403.01742, 2024.

Kun Zhang, Shaoan Xie, Ignavier Ng, and Yujia Zheng. Causal representation learning from multiple distributions: A general setting. arXiv preprint arXiv:2402.05052, 2024.

Yujia Zheng and Kun Zhang. Generalizing nonlinear ica beyond structural sparsity. Advances in Neural Information Processing Systems, 36:13326-13355, 2023.

Yujia Zheng, Ignavier Ng, and Kun Zhang. On the identifiability of nonlinear ica: Sparsity and beyond. Advances in Neural Information Processing Systems, 35:16411-16422, 2022.

Chen Zhicheng, FENG SHIBO, Zhong Zhang, Xi Xiao, Xingyu Gao, and Peilin Zhao. Sdformer: Similarity-driven discrete transformer for time series generation. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.

NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.

Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:

You should answer [Yes] , [No] , or [NA] .

[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.

Please provide a short (1-2 sentence) justification right after your answer (even for NA).

The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation. While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.

IMPORTANT, please:

Delete this instruction block, but keep the section heading 'NeurIPS Paper Checklist" ,

Keep the checklist subsection headings, questions/answers and guidelines below.

Do not modify the questions and only use the provided macros for your answers .

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our works address the identification problem of hierarchical temporal causal representation.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the viewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have provided the limitation in the conclusion explicitly.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have provided the full set of assumptions and a complete proof.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have provided implementation details.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided the code in the supplementary material.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We repeat all the experiments over three random seeds and report the mean and standard deviation.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We have provided in the implementation details.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: We have provided it in the appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification:

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Supplement to ' Towards Identifiability of Hierarchical Temporal Causal Representation Learning '

A Useful Definitions and Lemmas

Definition 4. (Diagonal Operator) Consider two random variable a and b , density functions p a and p b are defined on some support A and B , respectively. The diagonal operator D b | a maps the density function p a to another density function D b | a ◦ p a defined by the pointwise multiplication of the function p b | a at a fixed point b :

Definition 5. (Completeness) A family of distribution p ( a | b ) is complete if the only solution p ( a ) to

is p ( a ) = 0 . In other words, no matter the range of an operator is on finite or infinite, it is complete if its null space 6 or kernel is a zero set. Completeness is always used to phrase the sufficient and necessary condition for injective linear operator [59, 6].

Theorem A1. (Theorem XV 4.5 in [11] Part III ) A bounded operator T is a spectral operator if and only if it is the sum T = S + N of a bounded scalar type operator S and a quasi-nilpotent operator N commuting with S . Furthermore, this decomposition is unique and T and S have the same spectrum and the same resolution of the identity.

Lemma A1. (Lemma 1 in [24]) Under Assumption 1, if L z | x is injective, then L -1 x | z exists and is densely defined over G ( X ) (for G = L 1 , L 1 bnd ).

Properties of linear operator. We outline useful properties of the linear operator to facilitate understanding of our proof:

(Inverse) If linear operator: L b | a exists a left-inverse L -1 b | a , such L -1 b | a ◦ L b | a ◦ p a = p a for all a ∈ A . Analogously, if L b | a exists a right-inverse L -1 b | a , such L b | a ◦ L -1 b | a ◦ p a = p a for all a ∈ A . If L b | a is bijective, there exists left-inverse and right-inverse which are the same.

(Injective) L b | a is said to be an injective linear operator if its L -1 b | a is defined over the range of the operator L b | a [43]. If so, under assumption 1 (ii ), L -1 a | b exists and is densely defined over F ( A ) . [24].

̸

(Composition) Given two linear operators L c | b : F ( B ) →F ( C ) and L c | a : F ( A ) →F ( C ) , with the function space supports defined uniformly on the range of supports for the domain spaces as characterized by L b | a , it follows that L c | a = L c | b ◦ L b | a . Furthermore, the properties of linearity and associativity are preserved in the operation of linear operators. However, it is crucial to note the non-commutativity of these operators, i.e., L c | b L b | a = L b | a L c | b , indicating the significance of the order of application.

Definition 6 (Markov Network) . Markov network is an undirected graph G = ( V, E ) with a set of random variables x v ∈ V , where any two non-adjacent variables like x a and x b are conditionally independent given all other variables. That is,

Markov Networks and Directed Acyclic Graphs (DAGs) are both graphical models employed to represent joint distributions and to illustrate conditional independence properties. Based on the definition of Markov Network, we further define Vector-Node Markov Network as follows:

Definition 7 (Vector-Node Markov Network) . A Markov network where each node represents a vector, rather than a scalar variable. In this structure, the joint distribution is factorized over a graph G = ( V, E ) , where V is the set of nodes, and each v ∈ V corresponds to a multidimensional vector x v . Given any two non-adjacent vectors x a and x b , any x a,i ∈ x a and x b,j ∈ x b are conditionally independent given all other variables. That is,

Definition 8 (Isomorphism of Markov networks) . We let the V ( · ) be the vertical set of any graphs, an isomorphism of Markov networks M and ˆ M is a bijection between the vertex sets of M and ˆ M

such that any two vertices u and v of M are adjacent in G if and only if f ( u ) and f ( v ) are adjacent in ˆ M .

Definition 9 ( Definition of left and right inverse. ) . Let f : x → y and g : y → x be functions. We say that g is a left inverse to f , and that f is a right inverse of g , if g ◦ f = id X , where id X is the identity function.

B Proof of Theoretical Results

B.1 Notations

This section collects the notations used in the theorem proofs for clarity and consistency. Table A4: List of notations, explanations, and corresponding values.

B.2 Proof of Block-wise Identifiability of Latent Variables z t in Hierarchical Latent Process

Theorem A2. ( Block-wise Identifiability of Latent Variables z t in Hierarchical Latent Process. ) Suppose the observed and L -layer latent variables follow the data generation process in Figure

1. By matching the true joint distribution of 2 L + 1 number of adjacent observed variables, i.e., { x t -L , · · · , x t , · · · , x t + L } , we further make the following assumptions:

(i) The joint distribution of X,Z , and their marginal and conditional densities are bounded and continuous.

(ii) The linear operators L x t + L , ··· , x t +1 | z t and L x t -1 , ··· , x t -L | x t +1 , ··· , x t + L are injective for bounded function space.

̸

(iii) For all z t , z ′ t ∈ Z t ( z t = z ′ t ) , the set { x t : P ( x t | z t ) = P ( x t | z ′ t ) } has positive probability.

̸

Suppose that we have learned (ˆ g, ˆ f, P (ˆ z t )) to achieve Equation (1) and (2), then the latent variables z t = { z 1 t , · · · , z L t } are block-wise identifiable.

Proof. We first follow Hu et al. [24] framework to prove that z t is block-wise identifiable given sufficient observation. Sequentially, we prove that we require at least 2 L + 1 adjacent observed variables to achieve block-wise identifiability.

Given time series data with T timesteps X = { x 1 , · · · , x t , · · · , x T } and L -layer latent variables, we let x <t and x >t be { x 1 , · · · , x t -1 } and { x t +1 , · · · , x T } , respectively. Note that the length of x <t and x >t are larger than L , i.e., | x <t | > L and | x >t | > L . Sequentially, according to the data generation process in Figure 1, we have:

Sequentially, the observed P ( x t -1 ) and joint distribution P ( x >t , x t , x <t ) directly indicates P ( x >t , x t | x <t ) , and we have:

We further incorporate the integration over X <t as follows:

According to the definition of linear operator, we have:

By combining Equation (A7) and (A8), we have:

which implies the operator equivalence:

Sequentially, we further integrate out x t and have:

and it results in:

According to assumption (ii), the linear operator L x >t | z t is injective, Equation (A12) can be rewritten as:

By combining Equation (A10) and (A13), we have

By leveraging Lemma A1, if L x <t | x >t is injective, then L -1 x >t | x <t exists. Therefore, we have:

Then we can leverage assumption (iii) and the linear operator is bounded. Consequently, L x >t , x t | x <t L -1 x >t | x <t is also bounded, which satisfies the condition of Theorem A1, and hence the the operator L x >t | z t D x t | z t L -1 x >t | z t have a unique spectral decomposition, where L x >t | z t and D x t | z t correspond to eigenfunctions and eigenvalues, respectively.

Since both the marginal and conditional distributions of the observed variables are matched, the true model and the estimated model yield the same distribution over the observed variables. Therefore, we also have:

where the L.H.S corresponds to the true model and the R.H.S corresponds to the estimated model. Moreover, L ˆ x >t , ˆ x t | ˆ x <t L -1 ˆ x >t | ˆ x <t also have the unique decomposition, so the L.H.S of the Equation (A16) can be written as:

Integrating Equation A15 and Equation A17, and noting that their L.H.S. are identical, it follows that they share the same spectral decomposition. This yields

where C is a scalar accounting for scaling indeterminacy and P is a permutation on the order of elements in D ˆ x t | ˆ z t , as discussed in [11]. These forms of indeterminacy are analogous to those in eigendecomposition, which can be viewed as a finite-dimensional special case. P is a mapping from distribution to distribution

Since the normalizing condition

must hold for every ˆ z t , one only solution is to set C = 1 .

Hence, D ˆ x t | ˆ z t and D x t | z t are identical up to a permutation on their repsective elements. We use unordered sets to express this equivalence:

Due to the set being unordered, the only way to match the R.H.S. with the L.H.S. in a consistent order is to exchange the conditioning variables, that is,

where superscript ( · ) denotes the index of a conditioning variable, and π is reindexing the conditioning variables. We use a relabeling map h to represent its corresponding value mapping:

̸

Since K ˆ z t , z t , L -1 x >t | ˆ z t , and L z t | x >t are continuous, h is continuous and differentiable. Moreover, by leveraging Assumption (iii), different values of z , i.e., z (1) t , z (2 t imply p ( x t | z (1) t ) = p ( x t | z (2) t ) . So we can construct a function F : Z → p ( x t | z t ) , and we have:

̸

̸

implying that F is injective. Moreover, by using Equation (A22), we have F ( z t ) = F ( h (ˆ z t )) , which implies z t = h (ˆ z t ) .

The aforementioned result leverage x <t , x t , and x >t as three different measurement of z t , where | x <t | ≫| z t | , | x >t | ≫| z t | and | x t | < | z t | . It may imply that when the x t cannot provide enough information to recover z t , we can seek more information from x <t and x >t .

Sequentially, we further prove that when the observed and L -layer latent variables follow the data generation process in Equation (1) and (2), we require at least 2 L +1 adjacent observed variables, i.e., { x t -L , · · · , x t , · · · , x t + L } to make z t block-wise identifiable. We prove it by contradiction as follows.

Suppose we have 2 L adjacent observations, which can be divided into two cases: 1) { x t -L +1 , · · · , x t , · · · , x t + L } and { x t -L , · · · , x t , · · · , x t + L -1 } . In the first case, suppose the dimension of x t and that of any layer of latent variables z l t are n , the dimension of x t -L +1 , · · · , x t -1 is ( L -1) × n and the dimension of z t is L × n , conflicting with the assumption that L x t +1 , ··· , x t + L ′ | z t is injective. In the second case, the dimensions of x t -L , · · · , x t -1 and x t +1 , · · · , x t + L -1 are L × n and ( L -1) × n , respectively, conflicting with the assumption that L x t -L ′ , ··· , x t -1 | x t +1 , ··· , x t + L is injective. As a result, we require at least 2 L +1 adjacent observations, i.e., { x t -L , · · · , x t , · · · , x t + L } to make z t block-wise identifiable.

B.3 Examples of injective linear operators

The assumption of the injectivity of a linear operator is commonly employed in the nonparametric identification [24, 3, 26]. Intuitively, it means that different input distributions of a linear operator correspond to different output distributions of that operator. For a better understanding of this assumption, we provide several examples that describe the mapping from p a → p b , where a and b are random variables.

Example 1 ( Inverse Transformation ) . b = g ( a ) , where g is an invertible function.

Example 2 ( Additive Transformation ) . b = a + ϵ , where p ( ϵ ) must not vanish everywhere after the Fourier transform (Theorem 2.1 in [55]).

Example 3. b = g ( a ) + ϵ , where the same conditions from Examples 1 and 2 are required.

Example 4 ( Post-linear Transformation ) . b = g 1 ( g 2 ( a )+ ϵ ) , a post-nonlinear model with invertible nonlinear functions g 1 , g 2 , combining the assumptions in Examples 1-3 .

Example 5 ( Nonlinear Transformation with Exponential Family ) . b = g ( a, ϵ ) , where the joint distribution p ( a, b ) follows an exponential family.

Example 6 ( General Nonlinear Transformation ) . b = g ( a, ϵ ) , a general nonlinear formulation. Certain deviations from the nonlinear additive model ( Example 3), e.g., polynomial perturbations, can still be tractable.

B.4 Monotonicity and Normalization Assumption

Assumption 1 (Monotonicity and Normalization Assumption [26]) . For any x t ∈ X t , there exists a known functional G such that G [ p x t +1 | x t , z t ( ·| x t , z t ) ] is monotonic in z t . We normalize z t = G [ p x t +1 | x t , z t ( ·| x t , z t ) ] .

B.5 Block-wise Identifiability of Latent Variables z l t in any l -th Layer

Theorem A3. ( Block-wise Identifiability of Latent Variables z l t in any l -th Layer. ) For a series of observed variables x t ∈ R n and estimated latent variables ˆ z t ∈ R n with the corresponding process ˆ f i , ˆ P (ˆ ϵ ) , ˆ P (ˆ η ) , ˆ g , suppose that the process subject to observational equivalence x t = ˆ g (ˆ z t , ˆ η t ) . We

let c t ≜ { z t -1 , z t } ∈ R 2 × L × n and that M c t be the variable set of two consecutive timestamps and the corresponding node-vector Markov network respectively, and further employ following assumptions:

(i) (Smooth and Positive Density): The conditional probability function of the latent variables c t is smooth and positive, i.e., p ( c t | z t -2 ) is third-order differentiable and p ( c t | z t -2 ) > 0 over R 2 × L × n .

(ii) (Sufficient Variability): Denote |M c t | as the number of edges in Markov network M c t . Let

where ⊕ denotes concatenation operation and ( i, j ) ∈ E ( M c t ) denotes all pairwise indice such that c t,i , c t,j are adjacent in M c t . For m ∈ [1 , · · · , n ] , there exist 4 n + |M c t | different values of z t -2 ,m , such that the 4 n + |M c t | values of vector functions w ( m ) are linearly independent.

Then for latent variables z l t at the l -th layer, z l t is block-wise identifiable, i.e., there exists ˆ z l t and an invertible function h l t : R n → R n , such that z l t = h l t (ˆ z l t ) .

Proof. By reusing Theorem A2 with more observed variables, { z t -2 , z t -1 , z t } and { z t -1 , z t } are also block-wise identifiable, implying that there exists invertible functions h 3 and h 2 , such that ˆ z t -2 , ˆ z t -1 , ˆ z t = h 3 ( z t -2 , z t -1 , z t ) and ˆ z t -1 , ˆ z t = h 2 ( z t -1 , z t ) . So we have:

And then we partition the latent variables z t into two parts z t

Let ˆ z l 1 t -1 ,k and ˆ z l 2 t,o be two variables that denote the k -th variable of z l 1 t -1 and o -th variable of z l 2 t , respectively, where l 1 > l 2 . According to the data generation process, it is not hard to find that ˆ z l 1 t -1 ,k and ˆ z l 2 t,o are not adjacent in the estimated Markov network M ˆ c t over ˆ c t = { ˆ z t -1 , ˆ z t } . We conduct the first-order derivative w.r.t. ˆ z l 1 t -1 ,k and have

We further conduct the second-order derivative w.r.t. ˆ z l 1 t -1 ,k and ˆ z l 2 t,o , then we have:

Since ˆ z l 1 t -1 ,k and ˆ z l 2 t,o are not adjacent in M ˆ c t , ˆ z l 1 t -1 ,k and ˆ z l 2 t,o are conditionally independent given ˆ c t \{ ˆ c t,k , ˆ c t,l } . Utilizing the fact that conditional independence can lead to zero cross derivative [53], for each value of ˆ z t -2 , we have:

Bring in Equation (A28), Equation (A27) can be further derived as:

where ( j, i ) ∈ E ( M c t ) denotes that c t,i and c t,j are adjacent in M c t . Similar to Equation (A28), we have ∂ 2 p ( c t | z t -2 ) ∂c t,i ∂c t,j = 0 when c t,i , c t,j are not adjacent in M c t . Thus, Equation (A29) can be rewritten as:

Then for each m = 1 , 2 , · · · , nL and each value of ˆ z t -2 ,m , we conduct partial derivative on both sides of Equation (A30) and have:

Finally, we have

According to Assumption A2, we can construct 4 nL + |M c | × n 2 different equations with different values of ˆ z t -2 ,m , and the coefficients of the equation system they form are linearly independent. To ensure that the right-hand side of the equations is always 0, the only solution is

̸

Bringing Eq A33 into Eq A34, at least one product must be zero, and the other must be zero as well. That is,

According to the aforementioned results, ˆ z l 1 t -1 ,k and ˆ z l 2 t,o be two variables that denote the k -th variable of z l 1 t -1 and o -th variable of z l 2 t , respectively, where l 1 > l 2 , we draw the following conclusions.

(i) Equation (A33) implies that, each ground-truth latent variable c l t,i in l -th block is a function of at most one of ˆ c l 1 t,k and ˆ c l 2 t,l , which are in l 1 -th and l 2 -th layer, respectively

(ii) Equation (A36) implies that, for each pair of ground-truth latent variables c l t,i and c s t,j in l -th and l -th blocks, respectively, that are adjacent in M c t over c t , they can not be a function of ˆ c l 1 t,k and ˆ c l 2 t,l respectively.

According to the data generation process, we can restrict the independent noises among blocks, and hence the estimated node-vector Markov network is isomorphic to the ground-truth node-vector Markov network.

Sequentially, we further give the proof that under the same permutation z π t , block z i t is only a function of z π ( i ) t . Since the permutation happens on each timestamp respectively, the cross-timestamp block-wise identifiability is presented clearly.

Suppose there exists a pair of indices l, s ∈ { 1 , · · · , L } . Since h 2 is invertible, there exists a permuted version of the estimated blocks, denoted as c π t , such that:

̸

̸

we have ∂ z l t,i ∂ ˆ z π ( l ) t,j = 0 and ∂ z s t,i ∂ ˆ z π ( s ) t,j = 0 . Let us discuss it case by case. We can discuss it in the following case.

If z l t is not adjacent to z s t , we have ˆ z π ( l ) t is not adjacent to ˆ z π ( s ) t . Using Equation (A33), we have ∂ c l t,i ∂ ˆ z l 1 t -1 ,k · ∂ c l t,i ∂ ˆ z l 2 t,o = 0 , which leads to ∂ z l t ∂ ˆ z π ( j ) t = 0 .

If block z l t is adjacent to block z s t , we have ˆ z π ( i ) t is adjacent to ˆ z π ( j ) t . The intimate neighbor set of z i t is empty, there exists at least one index k such that z k t is adjacent to z i t but not adjacent to z j t . Similarly, we have the same structure on the estimated Markov network, which means that ˆ z π ( k ) t is adjacent to ˆ z π ( i ) t but not adjacent to ˆ z π ( j ) t . Using Equation (A36) we have ∂ c l t,i ∂ ˆ z l 1 t -1 ,k · ∂ c s t,j ∂ ˆ z l 2 t,o = 0 , which leads to ∂ z l t,i ∂ ˆ z π ( s ) t,j = 0 .

In conclusion, we always have ∂ z l t,i ∂ ˆ z π ( s ) t,j = 0 , meaning that there exists a permutation π of the estimated blocks, such that z l and z π ( l ) is one-to-one corresponding.

t t

Sequentially, we further leverage Lemma A2 to show that there is exist an invertible function h l , such that ˆ z l t = h l ( z l t ) .

Lemma A2. (Hierarchical Structure Resolves Layer-Wise Permutation Indeterminacy) If the estimated latent causal graph represents a hierarchical structure, the layer-wise components z 1 t , z 2 t , · · · , z L t are block-wise identifiable without permutation.

Proof. Let A t represent the true causal adjacency matrix of the latent causal graph at time t in layer-wise level, and let ˆ A t represent its estimation. By definition, we have

Consider the nonzero elements of the adjacency matrices, which occur at positions where k = l +1 . If a layer-wise permutation is applied, the rows and columns of the A t are both permuted according to the same permutation. Specifically, we have

where D l +1 , 1 is a diagonal matrix representing the scaling indeterminacy, and π : { 1 , . . . , L } → { 1 , . . . , L } is a permutation function. The subscripts of ˆ A and A above indicate the following equation:

The recursive formula in Equation (A40) implies:

For π ( l ) to be a valid permutation covering all values in { 1 , . . . , L } , it is necessary that π (1) = 1 . If π (1) = 1 , the sequence π ( l ) = π (1) + ( l -1) would either exceed L or fail to include 1, violating the bijective property of π . Hence, we conclude that

indicating that the layer-wise components remain unpermuted.

B.6 Extension to Multiple Lags

For the sake of simplicity, we consider only one special case with τ = 1 in Theorem 2. Our theoretical results can actually be extended to arbitrary lags and subsequences easily. For any given time lag τ , and future horizons which is centered at z t with historical τ h and future τ f steps, i.e., c t = { z t -τ h , · · · , z t , · · · , z t + τ f } ∈ R ( τ h + τ f + n ) × n . In this case, the vector function w ( m ) in the Sufficient Variability assumption should be modified as

Besides, 2 × n × ( τ h + τ f +1) + |M c t | values of linearly independent vector functions in z l t ′ ,m for t ′ ∈ { t -τ h -1 , · · · , t -τ h -τ } and m ∈ { 1 , · · · , n } are required as well. The rest of the theorem remains the same, and the proof can be easily extended in such a setting.

B.7 Component-wise Identifiability of Latent Variables z l t,i in any l -th Layer

Lemma A3. ( Component-wise Identifiability of Latent Variables z l t,i in any l -th Layer ) For a series of observed variables x t ∈ R n and estimated latent variables ˆ z t ∈ R n with the corresponding process ˆ f i , ˆ P (ˆ ϵ ) , ˆ P (ˆ η ) , ˆ g , suppose that the process subject to observational equivalence x t = ˆ g (ˆ z t , ˆ η t ) . We employ sufficient variability assumptions as follows:

(i) (sufficient variability) For any z l t ∈ Z l t ⊆ R n and ˆ u = { ˆ z l t -1 , ˆ z l +1 t } there exist 2 n +1 different values of ˆ u , m = 0 , · · · , 2 n , such that these 2 n vectors v l,m -v l, 0 are linearly independent, where v l,m is defined as:

̸

Then for i -th latent variable z l t,i at the l -th layer, z l t,i is component-wise identifiability, i.e., there exists ˆ z l t,j and an invertible function h l t : R → R , such that ˆ z l t,j = h l t ( z l t,i ) .

Proof. According to Theorem A3, we have achieved the block-identifiability of z l t , by letting z l t -1 and z l -1 t be the temporal and hierarchical parents, we further have :

̸

where J h l t is the Jacobian matrix of the transformation associated with h l t . Sequentially, we differentiate both sides of the Equation (A45) w.r.t ˆ z l t,j and ˆ z l t,k , where j, k ∈ { 1 , · · · , n } and j = k yields

Therefore, for u = u 0 , · · · , u 2 n , we have 2 n + 1 such equations. Subtracting each equation corresponding to u 1 , · · · , u 2 n with the equation corresponding to u 0 results in 2 n equations:

̸

h t t,i where m = 1 , · · · , 2 n . As a result, under the sufficient variability assumption, the linear system is a 2 n × 2 n full-rank system, and the only solution is ∂ z l t,i ∂ ˆ z l t,j · ∂ z l t,i ∂ ˆ z l t,k = 0 and ∂ 2 z l t,i ∂ ˆ z l t,j ∂ ˆ z l t,k . And ∂ z l t,i ∂ ˆ z l t,j · ∂ z l t,i ∂ ˆ z l t,k = 0 implies that for each i = 1 , · · · , n , ∂ z l t,i ∂ ˆ z l t,j = 0 for at most one element j ∈ { 1 , · · · , n } . Therefore, there is only at most one non-zero entry in each row indexed by i in the Jacobian J h l t . Moreover, the invertibility of h l t necessitates J h l t to be full-rank which implies that there is exactly one non-zero component in each row of J l , implying that the z l is component-wise identifiable.

C Real-world Implications of Modeling Time Series Data in a Hierarchical Manner

Real-world scenarios are usually governed by hierarchical layers of temporal latent dynamics instead of a single one. Capturing the hierarchical temporal latent dynamics plays a critical role in modeling time series data. Here is two real-world scenarios.

Human Motion Modeling. Human locomotion unfolds over multiple temporal layers [16, 60, 2]:

Higher-level latent variables: Capture coarse movement categories-e.g., walking versus running or resting-capturing the switch-like changes in overall gait dynamics.

Lower-level latent variables: Resolve fine-grained kinematics such as joint angles, stride frequency, and instantaneous speed.

This decomposition isolates semantically meaningful factors: the high-level code tells what activity is occurring, while the low-level code details how it is executed. Such a structure supports downstream tasks, including activity recognition, early anomaly detection (e.g., pathological gaits), and personalized coaching systems that adapt exercises to an individual's movement profile.

Climate Modeling. Atmospheric processes also exhibit a natural hierarchy [34, 71], which is shown as follows:

Top-level latent variables capture large-scale regimes like monsoon cycles or transitions between seasons, governing the dominant energy balance of a region.

Middle-level latent variables reflect regional or intra-seasonal patterns-e.g., the twelve traditional solar terms in East Asia or monthly precipitation phases that modulate local ecosystems.

Bottom-level latent variables track short-term fluctuations in temperature, humidity, and wind that drive daily weather variability.

Modeling these tiers jointly enables scientists to disentangle long-term trends from transient disturbances, improving extended-range forecasts, pinpointing abnormal events (heat-wave onsets, unseasonal cold snaps), and facilitating attribution studies of climate change.

D Relationship between Time Series Generation and Hierarchical Dynamics

The goal of time series generation is to identify the joint distribution of observed variables of different time steps, i.e., p ( x 1 , · · · , x T ) . To achieve this goal, we should simultaneously capture long-range structure spanning hundreds of steps and fine-grained, moment-to-moment variability. Collapsing every timescale into a single latent layer forces one hidden state to shoulder both roles. This entangles long-term context with short-term detail, squanders model capacity on resolving incompatible dependencies, and obscures the semantics of the latent dimensions-making directed manipulation or counterfactual sampling nearly impossible.

By contrast, an explicit hierarchical latent structure allocates a dedicated timescale to each layer: a slow-clock top tier encodes global intent or seasonality, intermediate tiers refine meso-level patterns, and a fast-clock bottom tier captures instantaneous noise. This separation prevents semantic mixing, lets the model reuse parameters efficiently, and provides clearly interpretable handles for controllable generation (e.g., editing only high-level intent while preserving fine detail).

E Related Works

Time Series Generation Generating realistic time series data [66, 33, 1, 13, 80] is important in numerous fields, including finance, healthcare, and engineering, where access to sufficient data can be challenging. Traditional generative methods, such as Generative Adversarial Networks (GANs) [17, 56, 12, 62, 35] and Variational Autoencoders (V AEs) [39, 46, 7], have been widely explored for time series generation. TimeGAN [75], a GAN-based framework, integrates adversarial and supervised losses to effectively capture temporal dynamics, demonstrating superior performance over previous methods in terms of similarity and predictive performance. However, such GAN-based models often face challenges such as training instability and mode collapse. To overcome these limitations, V AEs have been explored as a more robust alternative. TimeVAE [10] incorporates domain knowledge to model temporal patterns like trends and seasonalities, improving both interpretability and training efficiency. Recent advances have also introduced diffusion-based models [41, 15, 21], such as Diffusion-TS [76], which leverage disentangled temporal representations and Fourier-based training objectives to generate high-quality and interpretable time series while mitigating common limitations of autoregressive approaches. Furthermore, Koopman VAEs (KoVAE) [58] leverage linear latent dynamics inspired by Koopman theory, enabling the integration of domain knowledge and stability analysis. These methods collectively illustrate the evolving landscape of time series generation, with a focus on balancing realism, interpretability, and computational efficiency. Other relevant works include [36], which synthesize time series by manipulating existing time series.

Identifiability of Causal Representation Learning Independent Component Analysis (ICA) has been widely used to identify latent variables with identifiability guarantees, traditionally assuming a linear mixing process [8, 30, 27]. However, this linearity constraint restricts its applicability in more complex scenarios. To handle nonlinear scenarios, researchers have proposed nonlinear ICA by introducing additional assumptions, such as auxiliary variables [37, 72, 47] or structural sparsity [79, 78, 49]. Specifically, methods based on auxiliary variables [28, 29, 50] typically assume

that latent sources are conditionally independent given auxiliary information, such as domain indices or temporal segments, thereby enabling identifiability. For unsupervised approaches, structural sparsity in the generative process has been exploited to recover causal latent factors without relying on auxiliary variables [79, 78, 67]. This approach ensures identifiability by enforcing sparsity constraints on the Jacobian of the mixing function, allowing for the identification up to componentwise transformations and permutations.

Temporal observations introduce unique challenges to identifiability, especially in the presence of instantaneous dependencies and dynamic latent structures. [28] achieves identifiability for stationary data using permutation-based contrastive learning, while [29] extends this approach to nonstationary time series by leveraging variability in data segments. Recent methodss [73, 74] assume conditional independence of latent variables given their time-delayed parent to identify latent causal relations. However, these assumptions often fail in real-world scenarios where instantaneous dependencies are prevalent, such as in human motion data, making them of limited applicability in such contexts. Recently, [48] proposed an identification framework for instantaneous latent dynamics, introducing a sparse influence constraint to enforce sparsity in both time-delayed and instantaneous causal relationships among latent processes. While these advances enhance identifiability, addressing hierarchical latent dependencies remains a key challenge for further improving the identifiability of causal structures in temporal data.

F Implementation Details

F.1 Prior Likelihood Derivation

We first consider the prior of ln p ( z 1: t ) . We start with an illustrative example of stationary latent causal processes with two time-delay latent variables, i.e. z t = [ z t, 1 , z t, 2 ] with maximum time lag L = 1 , i.e., z t,i = f i ( z t -1 , ϵ t,i ) with mutually independent noises. Then we write this latent process as a transformation map f (note that we overload the notation f for transition functions and for the transformation map):

By applying the change of variables formula to the map f , we can evaluate the joint distribution of the latent variables p ( z t -1 , 1 , z t -1 , 2 , z t, 1 , z t, 2 ) as

where J f is the Jacobian matrix of the map f , where the instantaneous dependencies are assumed to be a low-triangular matrix:

Given that this Jacobian is triangular, we can efficiently compute its determinant as ∏ i ∂z t,i ϵ t,i . Furthermore, because the noise terms are mutually independent, and hence ϵ t,i ⊥ ϵ t,j for j = i and ϵ t ⊥ z t -1 , so we can with the RHS of Equation (A48) as follows

̸

Finally, we generalize this example and derive the prior likelihood below. Let { r i } i =1 , 2 , 3 , ··· be a set of learned inverse transition functions that take the estimated latent causal variables, and output the noise terms, i.e., ˆ ϵ t,i = r i (ˆ z t,i , { ˆ z t -τ } ) . Then we design a transformation A → B with low-triangular Jacobian as follows:

Similar to Equation (A49), we can obtain the joint distribution of the estimated dynamics subspace as:

Because of mutually independent noise assumption

Since the prior of p (ˆ z t +1: T | ˆ z 1: t ) = ∏ T i = t +1 p (ˆ z i | ˆ z i -1 ) with the assumption of first-order Markov assumption, we can estimate p (ˆ z t +1: T | ˆ z 1: t ) in a similar way.

F.2 Evident Lower Bound

In this subsection, we show the evident lower bound. We first factorize the conditional distribution according to the Bayes theorem.

F.3 Model Details

F.3.1 Reproducibility of Simulation Experiment

For the implementation of baseline models, we utilized publicly released code for TDRL, CaRiNG. Since the source code of IDOL is not available, we implemented it based on TDRL with the sparsity constraint. And the proposed CHiLD is also modified based on the code of TDRL, and shared hyperparameters remain the same.

F.3.2 Reproducibility of Real-world Experiment

We follow the setting of [76]. The model details of the proposed method are shown in Table A5. As for other baselines like KoVAE, Diffusion-TS, TimeGAN, and TimeVAE, we employ the official implementations and use the default hyperparameters for a fair comparison. To achieve the results from the well-fit baselines, we employed the default hyperparameters and tried different values of learning rate for the best models. And the number of latent variables on each real-world dataset is shown in Table A6.

G Experiment Details

G.1 Simulation Experiment

G.1.1 Data Generation Process

As for the temporally latent processes, we use MLPs with the activation function of LeakyReLU to model the sparse time-delayed. That is:

where W ∗ i, : is the i -th row of W ∗ and V <i,i is the first i -1 columns in the i -th row of V . Moreover, each independent noise ϵ t,i is sampled from the distribution of normal distribution. We further let the data generation process from latent variables to observed variables be MLPs with the LeakyReLU units. And the generation procedure can be formulated as follows:

Finally, we have:

where the dimension of ϵ 0 t is 1.

We generate six different synthetic datasets. For dataset A, the first layer contains 4 latent variables and the second layer contains 1 latent variable. For dataset B, all the latent variables are in the first layer, since there is only one latent layer. For dataset C, the first layer contains 8 latent variables and the second layer contains 2 latent variables. For dataset D, we consider the second-order latent markov process, and the first and second layers contain 4 and 1 latent variables, respectively. For dataset E, we consider a more complex generation procedure. Specifically, each causal relation and generation using a 2-layer MLP with LeakyReLU activation, which are shown as follows:

where M ∗ is a linear layer followed by a LeakyReLU activation. For dataset F, we consider three latent layers with 1, 2, and 4 latent variables. For dataset G, we consider the case with large number of latent layers and dimensions, which contains three latent layers with 8, 8, and 8 latent variables, respectively. The total size of the dataset is 100,000, with 1,024 samples designated as the validation set. The remaining samples are the training set.

G.1.2 Evaluation Metrics

To evaluate the identifiability performance of our method under instantaneous dependencies, we employ the Mean Correlation Coefficient (MCC) between the ground-truth z t and the estimated ˆ z t . A higher MCC denotes a better identification performance the model can achieve. In addition, we also draw the estimated latent causal process to validate our method. Since the estimated transition function will be a transformation of the ground truth, we do not compare their exact values, but only the activated entries.

G.1.3 More Simulation Experiments of Different Length of Observations

To evaluate our theoretical results that L -layer latent variables require 2 L +1 consecutive observations, we designed an experiment on a synthetic two-layer model with 8 latent variables in each layer. In our experimental setting, we vary the number of consecutive observations, i.e., 1,3,5, denoted by settings J1, J3, and J5, respectively. The results are shown in Table A7:

According to the experiment results, we can find that we observe that as the length of the consecutive observation window increases, the recovery performance of the latent variables improves. In particular, when the length is 2 L +1 , i.e., 5, the identifiability result is optimal.

G.2 Real-world Experiment

G.2.1 Dataset Description

The detailed descriptions of the datasets are shown as follows:

Stock is the Google stock price data from 2004 to 2019. Each observation represents one day and has 6 features.

ETTh1 dataset contains the data collected from electricity transformers, including load and oil temperature that is recorded every 15 minutes between July 2016 and July 2018.

fMRI is a benchmark for causal discovery, which consists of realistic simulations of bloodoxygen-level-dependent (BOLD) time series.

MuJoCo (multi-joint dynamics with contact) is a time series dataset generated by the physics engine.

Huaman 3.6 is collected over 3.6 million different human poses, viewed from 4 different angles, using an accurate human motion capture system. The motions were executed by 11 professional actors, and covered a diverse set of everyday scenarios including conversations, eating, greeting, talking on the phone, posing, sitting, smoking, taking photos, waiting, and walking in various non-typical scenarios. We randomly use four motions, i.e., Gestures (Ge), Jog (J), CatchThrow (CT), and Walking (W).

HuamnEVA-I comprises 3 subjects, each performing several action categories. Each pose has 15 joints with three axis. We choose 6 motions, i.e., Discussion (D), Greeting (Gr), Purchases (P), SittingDown (SD), Walking (W), and WalkTogether (WT) for the task of human motion forecasting. Specifically, the ground truth motion of the body was captured using a commercial motion capture (MoCap) system from ViconPeak 5 The system uses reflective markers and six 1M-pixel cameras to recover the 3D position of the markers and thereby estimate the 3D articulated pose of the body. We consider the joints as latent variables and the signals recorded from the system as observations.

Weather 7 dataset offers 10-minute summaries from an automated rooftop station at the Max Planck Institute for Biogeochemistry in Jena, Germany.

WeatherBench 8 is a benchmark dataset for data-driven medium-range weather forecasting. It repackages forty years (1979-2018) of ERA5 global reanalysis into machine-learning-ready NetCDF tensors sampled every six hours. Fourteen core surface and pressure-level variables-geopotential height, temperature, humidity, wind components, vorticity, potential vorticity, cloud cover, precipitation, solar radiation, and more-are provided on latitude-longitude grids of 0.25°, 1.406°, 2.812°, and 5.625°. Static fields such as land-sea mask, soil type, orography, and grid coordinates are included.

CESM2 9 delivers 100 fully coupled Earth system simulations at 1° resolution spanning 1850-2100 under CMIP6 historical and SSP3-7.0 forcing. Ensemble spread arises from distinct ocean-atmosphere initial states: ten members seeded from evenly spaced low-drift periods; eighty members created by tiny 10-14 K atmospheric perturbations applied to four AMOC-phased control years; and ten MOAR members useful for regional downscaling.

G.2.2 Evaluation Metric

Context-Frechet Inception Distance (Context-FID) score [35] quantifies the quality of the synthetic time series samples by computing the difference between representations of time series that fit into the local context.

Correlational score [51] uses the absolute error between cross-correlation matrices by real data and synthetic data to assess the temporal dependency.

G.3 More Experiment Results

G.3.1 Experiment results on other datasets

We would like to clarify that while some absolute improvements may seem small, e.g., 0.079 (ours) vs. 0.098 (IDOL) in the fmri dataset, it represents a 19% relative improvement. Focusing on percentage improvements, we observe significant gains of 34.07% and 23.91% of two metrics, showing the effectiveness of CHiLD.

G.3.2 Ablation Study

To evaluate the effectiveness of each module, we further devise two variants of the proposed CHiLD as follows. 1) CHiLD-KL : we remove the KL divergence restriction of prior estimation. 2) CHiLD-C : we do not use the context information for the proposed CHiLD. Experiment results are shown in Table A9, we can find that incorporating the contextual observation and KL divergence has a positive impact on the overall performance of the model.

H Broader Impacts

The proposed CHiLD method offers a significant advancement in identifying hierarchical latent dynamics, which is crucial for generating realistic and interpretable time series data across a wide range of domains such as healthcare, finance, climate science, and autonomous systems. By enabling the generation of time series that are both data-driven and causally grounded, our method not only enhances the credibility of the generated sequences but also improves the performance of downstream tasks like forecasting, anomaly detection, and decision-making. These improvements foster increased trust and adoption of machine learning models in critical applications, particularly where reliable data-driven insights are of paramount importance.

Furthermore, our work contributes to the broader field of generative modeling by providing a principled framework for hierarchical latent dynamics. This framework mitigates issues like mode collapse, a common problem in generative models, and enhances the model's generalizability across various datasets. As a result, CHiLD stands as a powerful tool for both researchers and practitioners looking to explore complex time series data and apply generative modeling techniques in realworld scenarios. This method paves the way for future research in more complex domains, such as controllable video generation, neuroscience, and predictive analytics in environmental systems, thereby further extending its applicability.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work considers a novel setting that combines hierarchical causal latent structure with temporal dynamics. Under strengthened assumptions on the latent generative model, it proposed a hierarchy of increasingly refined identifiability results based on observations from time steps t-L to t+L, where L is the number of hierarchical layers. The theoretical contributions include i) the block-identifiability of all the variables in time t, ii) block-identifiability of the variables in time t and layer l, and finally iii)element-wise identifiability. To empirically validate the theoretical results, the authors develop a VAE-based model and evaluate it on both synthetic data and nine real-world datasets, including three climate datasets and two human motion datasets. Overall, the proposed method outperforms the other baseline models, including VAE-based methods, generative models, and causal representation learning methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1.\tThe setting, considering hierarchical causal latent structure with temporal dynamics, is novel and well-motivated. The structure of the series of identifiability results is clear and easy to follow.\n\n2.\tUnlike many existing causal representation learning approaches that assume a deterministic mixing process, this work allows for noise in the mixing functions and also cross-layer connections. \n\n3.\tThe experiments on real datasets are sufficient and well-explained. It matches the setting in the theorem well, showing strong evidence about the efficiency of the effectiveness of the proposed method.\n\nWeaknesses\n\n1.\tThe theoretical contributions, while clearly presented, offer limited novelty. Both the core assumptions and the proof strategy closely follow prior work.\n\n2.\tThe assumptions made for achieving identifiability might be restrictive, especially the conditional independence assumption in the final theorem (as the authors also clearly mentioned in the discussion section) and the linear operator at the beginning, which might not be true for many cases. \n\n3.\tIn the synthetic dataset, the mixing function between x and z is only a simple 2-layer of leakyReLU. I am concerned about whether the setup is sufficiently expressive to convincingly demonstrate the method’s ability to handle complex nonlinear mixing functions. \n\n4.  Although increasing the number of layers in the latent space adds complexity to the identifiability problem, the authors ease this by using a broader temporal window (from t-L to t+L), which introduces a tradeoff and could be impractical in certain applications.\n\nQUESTIONS:\n1.\tAbout linear operators, does it exist for arbitrary two distributions? If the answer is no, then assumption (ii) made in Theorem 1, does it implicitly assume that the existence of a linear transformation between the distributions of Lxt+1,··· ,xt+L|zt and Lxt−L,··· ,xt−1|xt+1,··· ,xt+L? The example 4 in appendix B.3, is it a typo? Post-linear in the bold part but non-linear later? \n\n2.\tIn the hierarchical data generation process, does it assume that the dimensionality of the latent variables at each layer matches that of the observations (since only $n$ is used in the Sec2.1? While from the experimental setting it seems no? Or this is some assumption for theorem only? Does it assume the dimension of latent is known as pior? If no, how to estimate it in the experiment for both real datasets and synthetic?\n\nLIMITATIONS:\nSome missing points are listed in the weaknesses above", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n# Summary\n\nThe paper establishes identifiability results for hierarchical temporal causal representations. It presents two main theorems and one lemma:\n\n1. **Theorem 1** : Proves block-wise identifiability of the latent vector $z_t$.  \n2. **Theorem 2** : Extends block-wise identifiability to each hierarchical layer.  \n3. **Lemma 1** : Shows component-wise identifiability of individual latent variables $z^l_{i,t}$.\n\nAdditionally, the authors propose an algorithm to reconstruct the multilayer latent variables.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n- **Novel hierarchical generative process:** The paper introduces a new multi-layer generative model in which each layer injects additional noise, making the identifiability problem notably more challenging.\n\n- **Methodological contribution:** It thoughtfully adapts techniques from the measurement-error literature to causal latent-variable identifiability.\n\n- **Clear intuition for assumptions:** For every theorem, the authors provide intuition that clarifies each assumption, helping readers in understanding the results.\n\n# Weaknesses and Questions:\n\n- **Clarity and motivation:** The introduction does not yet offer a clear motivation for the specific hierarchical structure used in the paper or its real-world relevance. Concrete examples or use cases would greatly improve the writing and the storytelling of this paper.\n\n- **Originality of exposition:** Portions of the introduction closely resemble text from the IDOL paper, though paraphrased. Clearer attribution or more distinctive framing would enhance transparency and underscore the paper’s unique contributions.\n\n- **Premature technical detail:** Technical discussion of the single-layer assumption appears on the first page without sufficient context. Deferring such material until after the problem statement and motivating examples would aid readability.\n\n- **Generative process assumptions:** Some modeling choices, not allowing influences from layer \\(l+2\\) to \\(l\\), limiting interactions to a single lag, and excluding links such as $z^{l-1}_{t-1}$, $z^{l}_{t}$ seem unrealistic. Several assumptions feel tailored to produce identifiability rather than arising from domain knowledge. Stronger justification would help substantiate these assumptions.\n \n- **Observation model:** The framework assumes that only the first latent layer generates \\(x_t\\). Explaining why higher layers do not contribute, or exploring a more flexible observation mechanism, would improve realism.\n\n- **Layer dimensionality:** Each latent layer is set to have the same dimension \\(n\\) as the observations. Discussing scenarios where this design choice is appropriate and motivate it\n\n- **Overlap with prior proofs:**   *Theorem 1* closely follows Theorem 1 in “Instrumental Variable Treatment of Nonclassical Measurement Error Models.”  *Theorem 2* adapts the proof strategy of Theorem 1 in the IDOL paper with only minor modifications. \n\n- **Minor errors:** There are typographical issues in Eq(3), Appendix Eq(A19), and the sentence around line 1076 that should be corrected.\n- Please correct me if I’m mistaken, identifiability results in CRL identify latent variables $z_t$ and try to prove the existence of an invertible mapping between $\\hat{z}_t$ and $z_t$. Given that the observation $x_t$ are observed and fixed can you explain why $\\hat{x}_t$ in your proofs ? Using a different symbol for the estimated model Eq(A16) might make the notation clearer. \n\n- **Operator definition (EqA8):** It would be more rigorous to define the operator $L_{x_{>t}\\mid z_t}$ explicitly, as done in *Instrumental Variable Treatment of Nonclassical Measurement Error Models* proof in page 17.\n\n- **Fairness of the IDOL comparison:** The manuscript claims to achieve identifiability without a permutation step, unlike IDOL. However, IDOL’s component-wise identifiability also holds up to permutation, while the present work avoids only layer-index permutation due to its hierarchical design.\n\nQUESTIONS:\n- I remain uncertain about the generative process chosen for the model. Could the authors clarify its motivation? Specifically, what underpins the assumptions that (i) variables in layer $l+2$ cannot influence those in layer $l$ or layer 1, (ii) only a single temporal lag is considered, and (iii) neither $z^{l-1}_{t-1}$ nor $z^{l-2}_t$ can be causal for $z^{l}_t$?\n\n- Several modelling assumptions appear tailored to meet the identifiability conditions rather than arising from domain considerations. Could the authors explain how these assumptions are justified independently of the identifiability proofs, in other words are there any clear real world application of such structure?\n\n- The framework assumes that each observation $x_t$ is generated solely by the first latent layer. Why are contributions from higher layers ruled out, and how realistic is this restriction in practical settings?\n\n- Every latent layer is specified to have dimensionality $n$, matching that of the observed data. What motivates this choice, and how sensitive are the identifiability results to relaxing the dimensionality constraint?\n\nLIMITATIONS:\nThe authors mentioned one limitation related to their assumptions like layer wise conditional independence. However, as we mentioned, we still have concerns regarding the hierarchical structure assumed by this paper as well as the identifiability techniques used to prove the results.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

comments: SUMMARY:
This submission addresses the problem of performing inference in a hierarchical latent causal process given time series data. The authors establish the component-wise identifiability of the latent variables under such a model, and develop a variational autoencoder architecture that produces the correct conditional independencies in the generation model and can accurately recover the latent posterior at inference time. The resulting generative model achieves state-of-the-art performance, while utilizing the hierarchical structure to disentangle levels of abstraction at generation time. This separation allows for controllable generation.

STRENGTHS AND WEAKNESSES:
Strengths:

- The paper provides extensive theoretical contributions that establish the identifiability of latent variables under a hierarchical causal process.

- The experiments are promising, and show widespread state-of-the-art performance on simulated and real data.

Weaknesses:

1. I found some parts of the submission to lack clarity, although I acknowledge that these issues might be a result of a lack of confidence in this specific setting. For example, Definitions 1 & 2 seem to be missing key conditions: $\hat{z}_t^l$ needs to be defined as the estimated latent variable from observations, or else that statement "there exists an invertible function $h$ and value $\hat{z}$ such that $z=f(\hat{z})$" is trivially true. 

2. On real world data, only $L=2$ is explored, reducing the impact of working with a hierarchical latent process.

QUESTIONS:
Does performance degrade when the number of layers is increased above 2?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the problem of hiearchical temporal CRL, which investigates a dynamic latent Bayesian network where the observations are generated by the last layer of latent variables, which itself is generated by its 1-level higher and 1-step previous blocks of latent variables. They show that under regularity conditions, identifying the entire latent space with $L$ layers is possible using the joint distribution of $2L+1$ consecutive observation values. They then apply existing ideas to further refine this result to per-block and per-component recovery.\n\nSTRENGTHS AND WEAKNESSES:\nThe main departure of this work from existing temporal CRL papers is the observation model: The observed variables is NOT generated by an invertible transformation of the entire latent space, but, only the final layer of it. As far as I am aware, this way of inducing a lack of invertibility -- not due to a measurement noise, but from actual missing information in latent space -- is new, and perhaps promising in the context of real-world applications.\n\nWhile I appreciate the extensive study done for the experimentation, I find the overall workflow of converting identifiability results to theory-inspired VAE implementations slightly backward. Even if it is obvious, including a formal equivalence, consistency or \"closeness\" statement for the implementation method the authors describe would make the contribution -- and the overall flow of the paper -- more concrete.\n\nQUESTIONS:\n- Even if it is obvious, can the authors include a formal equivalence, consistency or \"closeness\" statement for the implementation method?\n- In the spirit of Appendix G.1.3, can you show that adding more than necessary information does not significantly increase or reduce performance?\n- Please add a clarification in experiments section about (i) the synthetic experiments showing ID performance and (ii) real-world experiments showing data synthesis performance; this knowledge shouldn't be expected from a reader but written explicitly. I suggest slightly renaming the subsections to reflect this difference.\n- I did not find the compute resources necessary to run the experiments. Can you add it to the appendix in writing? Even if it is negligible -- I know it is not -- simply say so.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-05"}