Identification of Intermittent Temporal Latent Process
Authors: Yuke Li
Yujia Zheng
Guangyi Chen
Kun Zhang
Heng Huang
Venue: ICLR 2025 Poster (2025)
Keywords: unsupervised representation learning
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Identifying time-delayed latent causal process is crucial for understanding temporal dynamics and enabling downstream reasoning. While recent methods have made progress in identifying latent time-delayed causal processes, they cannot address the dynamics in which the influence of some latent factors on both the subsequent latent states and the observed data can become inactive or irrelevant at different time steps. Therefore, we introduce intermittent temporal latent processes, where: (1) any subset of latent factors may be missing during nonlinear data generation at any time step, and (2) the active latent factors at each step are unknown. This framework encompasses both nonstationary and stationary transitions, accommodating changing or consistent active factors over time. 
Our work shows that under certain assumptions, the latent causal variables are block-wise identifiable. With further conditional independence assumption, each latent variable can even be recovered up to component-wise transformations. 
Using this identification theory, we propose an unsupervised approach, InterLatent, to reliably uncover the representations of the intermittent temporal latent process. The experimental findings on both synthetic and real-world datasets verify our theoretical claims.

Published as a conference paper at ICLR 2025
IDENTIFICATION OF INTERMITTENT TEMPORAL LA- TENT PROCESS
Yuke : Yujia Z Guangyi Chen', Kun Zhang", Heng Huang"
1 University of Maryland College Park, College Park, MD, USA
2 Cameegie Mellon University, Pittsburgh PA, USA
ABSTRACT
Identifying time-delayed temporal latent process is crucial for understanding temporal dynamics and enabling downstream reasoning. Although recent methods have made remarkable progress in this field, they cannot address the dynamics in which the influence of some latent factors on both the subsequent latent states and the observed data can become inactive or irrelevant at different time steps. Therefore, we introduce intermittent temporal latent processes, where: (I) any subset of latent factors may be missing during nonlinear data generation at any time step, and (2) the active latent factors at each step are unknown. This framework encompasses both nonstationary and stationary transitions, accom- modating changing or consistent active factors over time. Our work shows that under certain assumptions, the latent variables are block-wise identifiable. With further conditional independence assumption, each latent variable can even be recovered up component-we transformations, Using this identification theory, we propose an unsupervised approach, InterLatent. to reliably uncover the representations of the intermittent temporal latent process. The experiments on both synthetic and real-world datasets verify our theoretical claims.
INTRODUCTION
Learning meaningful representations from sequential data remains fundamental challenge across various fields. Time series data, such as financial markets and climate observations, are ubiquitous and exhibit high nonlinearity Berzuini al. (012)); Ghysels et al.
(2016). This inspires a extensive
line of works to temporal latent representation learning Yao et al.
(2022b a): Chen et al. (2024),
upon the recent advancement in nonlinear ICA Khemakhem et al.
(020); Zhang et al.) Kong et al.
(022); Zheng et al. (2022); Li et al. (2023); von Kügelgen al.
2024); Ng et al. (2023); Zheng
& Zhang 2024)); Morioka & Hyvarinen (2024); Yao et al. (2024); Zheng et al. 2024); Kong et al.
22024; Lachapelle et al. (2024)). However, many real-world systems exhibit latent time-delayed dynamics where the influence of certain latent factors on both subsequent latent states and observed data can be inactive irrelevant at specific time steps. Consider, for example, complex manufacturing process: various machine components contribute to the final product quality at different stages, with some components becoming temporarily inactive or irrelevant during certain production phases. Current works may struggle to capture these intermittent influences, potentially missing crucial aspects of the underlying dynamics. This highlights the need for more flexible and robust framework to identify such temporal processes with intermittence of latent variables.
In this work, we investigate the identification of representations of intermittent temporal latent pro- cesses. Two key properties characterize the intermittence of a temporal latent process: (1) any subset of latent factors can missing during the nonlinear time-delayed data generation at any time step, and (2) the specific set of active latent factors at time step is unknown. Figure takes an example of data generating mechanism an intermittent latent temporal process to illustrate its concept. In the transition mechanism (top of Figure [p), we see the zero entries in Jacobians indicating that not all latent variables influence each other's transitions. Similarly, in the generating mechanism
* Equual contribution.
1 Thii work was partially supported by NSF IIS 2347592, 2348169, DBI 2405416, CCF 2348306, CNS
2347617.

---

Published as a conference paper at ICLR 2025
(bottom of Figure [p). the sparse Jacobians show that not all latent variables contribute to every observed variable. We define the "support" as the set of active latent factors at each time step, for both the transition and generating mechanisms. "Missingness" occurs when latent factor is absent from the support, having no influence on the subsequent latent state or the observed data.
The intermittent nature of these processes presents two significant challenges for representation learning: (1) The supports for both transition and generating mechanisms are unknown, requiring methods adapt to the data generated by only active latent factors each time step. (2) The in- teractions between intermittently active latent variables may be intricate and time-varying in both mechanisms, necessitating the models that can capture the possible various support and missingness. The existing literature has yet to fully address these challenges. WWiedemer et al. (2024) relies on compositional mixing functions and requires supervision on the latent variables. Lachapelle et al. (2023; Fumero et al. (2023); Xu et al. (2024) are restricted to linear or piecewise linear settings. CaRiNG(Cher et al. (2024) tackles missingness only within the mixing function by leveraging his- torical information during the unmixing process.
In contrast to previous works, we present identification guarantees for uncovering intermittent tem- poral latent processes. Our theoretical analysis begins with establishing block-wise identifiability under assumptions of sufficient variability of transitions of latent variables (Theorem 1). Block-wise identifiability ensures that blocks of true latent variables can be uniquely recovered from estimated latent variables up to a certain indeterminacy. Notably, this outcome holds regardless of whether latent variables are within the support or missingness. Building prove component-wise identifiability for latent variables within the support in Theorem 2 given an additional assumption of independence of latent variables conditioning on previous time steps. Component-wise identifiability guarantees that each individual component within the support can be uniquely recovered from estimated latent variables up to a certain indeterminacy. Moreover, our identifiability results are able to handle both nonstationary and stationary temporal latent process, allowing for changing or consistent active factors over time without compromising identifiability guarantees. These theoretical contributions establish, to the best of our knowledge, one of the first general frameworks for uncovering latent variables in intermittent temporal processes with appro- priate identifiability guarantees.
further this foundation,
Leveraging these theoretical insights, we introduce a novel unsupervised method that extends the Sequential Variational Autoencoder Li & Mandi (2018). Our method, InterLatent accommodates supports and missingnesses through sparsity regularizations on both mixing and transition functions, enabling it model complex interactions between intermittently active latent variables and handle sparse, temporally variable latent spaces. We evaluate our approach on synthetic and real-world datasets, demonstrating its effectiveness in uncovering complex hidden temporal processes, as well as validating the proposed identifiability theory.
2 PROBLEM SETTING
Given a temporal sequence ranging from f 1 to = T, let X2I,X2...... XT) denote the K K-dimensional observations. At each time step t, N latent causal variables z, = {2) ...,..., xN\ generate X, E K We formalize the data generating process as follows:
X₁ = g(z),, z = fx
for n n [1,N]
(1)
Here, g is assumed to be an injective, nonlinear, non parammeetiiic mixing function: R + R In this work, work the undercomplete case, where K N ensure the injectivity of g. fn denotes the nonlinear, nonparametric time-delayed transition function for the n-th latent variable. Pa(=)) represents the parent nodes of 2 from previous time steps. Without loss of generality, we assume a time lag of in Eq. equation i.e., Pa(z?) C Zt-1- The general case of multiple lags and sequence lengths discussed in Appendix B. cf the noise term, sampled independently for each z from a standard normal distribution N NO,,
We are now ready introduce the intermittent temporal latent process upon the concept of miss- ingness of latent variables. In particular, not all components of z, participate in the data generating process at a time step. Formally, there exists u E [1, .] such that the u-th row of Jacobian of the transition function f", denoted as J.. and the u-th column of the Jacobian of the mixing function, g are zero. This implies that when 2! is missing, it neither receives influence from 2-11
2

---

Published as a conference paper at ICLR 2025
exerts influence Z++1 x, in the data generation process. Figure Iillustrates this concept, where 22 and zj are examples of such missing latent variables.
The "non-missing" indices of 2, help to define the support of the data generating process in Eq. by:
: fi E [1 1 and zt, Jugh (z)) ) + 0 Jy. (2+-)) O f.41 f.41 (2.) * 0}
A similar formulation can define the missingness of z, by si. We assume that and si partition the index set N]
si {u E [] 324-1 and zt, (g))
J Jf (2-1) = 0 J.n ())41 03
Equations and sets the stage for identifying 2, by characterizing a sparse support at f of both transi- tion and mixing functions. Specif- ically, there may exist latent vari- ables {z,) |u sf} that do not par- ticipate the data generating pro- cess described in Eq. [I] The zero tries in the Jacobian matrices of both the transition and mixing functions, as illustrated in Figure [bb,, provide clear visual representation of this sparsity. Let d, = |sc| denote the cardinality of St, and consequently, |s[] di = N - d, In our anal- ysis, we assume both S; and si are non-empty for all time steps L. The case where sf = 9 can be considered special instance of the intermittent temporal latent process. It is worth- noting that the mixing function g it- self remains unchanged across time steps only its Jacobian structure varies based on the missingness in 21. To illustrate this, we give an example in the following. Let g(x)) sinh(2t) as our mixing function. When certain components of 2 are missing, the corresponding columns in the Jacobian become zero, but g remains the same function. For instance, let us say
Jacobian of the transition function
Zi
22
23
z
1
0
Z
Jacobian of the mixing function
x
x
z]
2
2
z
0
(b) Jacobian structures of transition and mixing functions
(a) Data generating process
Data generations of Intermittent Temporal Latent Process and its Jacobian Structures for a three-step sequence, e.g., 23 means the latent variable x t 3. (a) illus- trates the connections between time steps and how 2, gener- ates X, in intermittent temporal latent process. (b) Jacobian structures reveals the definition of support and missingness in Eq. and Eq. by and respectively.
Figure 1:
2. and K = 2, if z² missing at time t, the jacobian of the mixing function is:
Oz cosh((2)) 0
0g
0sc
Oug
c2sh
0
In order to introduce our identification resutls, we define the observational equivalence next.
Definition 1 (Observational Equivalence): Given sequence of observed variables {x1, X2,.N...... x for = to T, let the true temporally causal latent process be specified by flg,p(E) as in Eq. equation ] A learned generative model f.g, (()) is observationally equivalent to the ground truth if the model distribution matches the data distribution everywhere:
Pf.. (x..)) P...ao(ee)xxxrr)
Both the mixing function and transition functions can be recovered (up to certain indeterminacies) once is identified as we assume the injectivity of g and no latent causal confounders, respectively.
Suppose there exists an invertible mapping h, such that 2, = h(zz). We further provide the definition of block-wise identifiability and component-wee wise idenfiability in the following
Definition (Block-wist Identifiability: h is considered bise identifiable if. given a block of the true latent variables 2 there exists an unique partitioning B' of 2, that matches z up permutation n, such that 2.5 h (m(2 1, invertible, and its domain is 2B 1
3

---

Published as a conference paper at ICLR 2025
Definition 3 (Component-wiise Identifiability): For individual component of the latent vari- ables zi, there exists a unique component n' of 2, matches z up to a permutation n, such that 2. = ()))),, where ht is invertible with a domain 2t E RN z is component-wise identifi- able.
IDENTIFIABILITY THEORY
This section presents our identifiability results. We first leverage the assumptions of sufficient vari- ability of temporal data and support sparsity to establish block-wise identifiability, as detailed in Theorem T Building upon this foundation, we then demonstrate component-wise identifiability of latent variables by exploiting conditional independence assumptions, as formalized in Theorem
3.1 BLOCK-WISS WISE IDENTIHABILITY
Eq. and enable the partitioning of z, into subsets: {z]li E st} and {zi E sy). We now present our findings on the block-wise identifiability of these subsets:
Theorem For the observations x RRK and estimated latent variables 2, Z C RRN, and 2 E 2 S RN,, suppose that there exist functions and satisfying observational equivalence in Eq. 4 If the following assumptions and regularization hold:
i (Smoothness and positivity): The probability density function of the latent causal variables, p(zc). has positive measure in the space of ze and is twice continuously differentiable.
i (Path connected): For any v., n! Z. there is continous function Φ [0. Z, 6(0) = z and ((1)= = (Sufficient variability of z, and 24): Let q(2)(211)) log logp(ze(z--), as well as (( log pl (2 1)), and H2. 10- 11-1) denotes the Hessian matrix of q(Z, ((2))2e-1) w.r.l. Zt and Z--1- Suppose G {0, [0,13 NxN as binary adjacency matrix that indicates the istence of the transitions from Zi-1 and Zt. where Gii.2 means that there exists a transition from vi-1 to zi² We assume that: span 1 span|He, ((2, (2-
Idxxdd and
1)) 1)) =R Idd.dd.
(Support sparsity regularization): For any time step t, S, is not an empty set, d, S de
There exists permutation such that
å; = (st) and $t = a(s))
In other words, Vi, (2/li V E 5 are paritial identifiable from Vu, {z]] t E s.))
Proof sketch The complete proof provided in the Appendix B.I Here, we outline the key steps: First, let p(z, 1 be the ground-truth transition pdf and p(2, 2.-1) be the estimated transition pdf. Define ((Z,|Z(-1) = log pogp(2//zz1) and ((2, (2-1)) = log pa (/-1). Using h defined in 2, h(z,) helps us derive: 22,22., (( 12
(J (Jn-1 (2) 2.., (Z1 1z (2,-1), where 7 denotes the Hessian matrix and Jn - is the inverse Jacobian of h. We further leverage the sufficient variability assumption establish connection between the support
8, and 8, well as their complements sf and $;, respectively. By incorporating the support sparsity regularization d, S d,, we conclude the block-wise identifiability of both Vi, {zili } and Vu, {z $
Remarks Assumptions! and ii have been commonly adopted for the identification theory Chen (024)) Lachapelle et al. (2024b). These assumptions provide the foundations to present Theorem) which concerns the transitions from 2 2, over the space Z.
Recall that this work does not require all components of z; to actively participate in the data gener- ation process. The crux of our identification approach lies formalizing the relationships between the support sets and š,, as well their complements duce the sufficient variability in assumption iï to ensure the span of the Hessian matrices of the log- transiiiiiio probabilities covers the full space of Vi, {zīli s.). We can thus establish our main result on partial identifiability by leveraging support sparsity regularization to reach our conclusion of the block-wiss identifiability.
and &ç. To this end, further intro-
4

---

Published as a conference paper at ICLR 2025
Notably, do not assume the invariance of the support set S, over time. Regardless of whether S, changes or remains constant, we demonstrate Section 3.2 that for all i, {zile E 8: can be recovered up to component wise invertible transformation and permutation.
3.2 COMPONENT-WISE IDENTIFIABILITY
In this section, we exploit the conditional independence assumption to establish the component-wise identifiability of {zili 8.).
Theorem Let all assumptions from the Theorem 7 hold. Additionally, suppose the following
assumption is exposed data generating process in Eq. as well:
(Conditional independence): At t, we assume that each component of ze is conditional indepen- dent given the previous latent variables Zạ-1. For any i1, 12 E [N]:
2t 21-1
(5)
Then {z/1ú E 8,} must be a component-wise transformation of permuted version of true {zile E s.).
Proof Sketch: Our main idea rests on proving component-wise identifiability by contradiction. We demonstrate that if component-win identifiability does not hold, it would violate the conditional in- dependence assumption. More specifically, The proof proceeds as follows: 1. From the Theorem [ we have:
1 (Jn-1 (2,)) T 2.-1 (Zz -/ J) (ž,-1): : 2. the conditional dependence of q(z, 1 as established in Eq. only hold if J (2,) diagonal matrix.
The conditional independence is widely adopted in identification results for time-series data, as evidenced in recent works Yao et al. (2022b a): Chen et al.) (2024). Our analysis demon- strates that this regularization plays crucial role in establishing our identification results.
Remarks:
The conclusions derived from Theorem T and Theorem 2 are applicable to both stationary and non- stationary processes. We consider the process is nonstationary if the support sets s, and sê vary time since the transition from z to 2, changes as well. Conversely, the process stationary these support remains unchanged over time. Our framework allows for temporal variation in the support sets S, and their complements st. subject only to the constraint that neither is an empty set any time point.
Our proposed data generating process encompasses previous models as special cases. For instance, we remove the intermittent feature described in Eq. and Eq. our model reduces LEAP Yao et al. (2022b) without the domain index. When handling non-stationary sequences, our identifia- bility results removes the assumption of known auxiliary variables, which is required by Yao et al. (202bb a Chen et al. (2024).
INTERLATENT APPROACH
Building upon identifiability results, now introduce InterLatent to estimate the latent causal variables. Our approach aims to achieve the observational cquivalence by modeling the support sparsity and the conditional independence assumptions for the data generating process in Eq. I In general, InterLatent formalizes the probabilistic joint distribution of Eq. as:
p(XITT, 21.7) (x1
(2))
II p. (x. (x.
(6)
where denotes the parameters for the mixing function g. and á denotes the parameters for the transition function f. To leam 2, from the observations x,, we also introduce the encoder qu (Zg [x, with parameters We build approach upon Sequential Variational Eitto-Escddes// Li & Mandt (2018). Figure@jillustrates the overall framework of InterLatent. In what follows, we introduce each part of network individually.

---

Published as a conference paper at ICLR 2025
4.1 NETWORK DESIGN
Eq. suggests that the architecture of InterLatent comprises of three key components. The encoder acquires latent causal representations by inferring 9. (. (x)) from observations. These learned latent variables are then used by the step-to-step decoder pyy ( Ex) to reconstruct the observations, imple- menting the mixing function g Eq. 1. To learn the latent variables, we constrain them through the KL divergence between their posterior distribution and a prior distribution, which is estimated using a normalizing flow that converts the prior into Gaussian noise. A detailed exploration all modules forthcoming.
Encoder 9 (2.(x)): We assume 2, is independent of žạ conditioning where t'. Therefore, the decom- position of joint probability distribution of the posterior T
& 1:T
11:T
Decoder
q. (2
II (2./x)). We choose to approximate 1=1
Leccon
Temporal Prior
LKLD
q by isotropic Gaussian characterized by mean 1r and covariance Ø,, To learn the posterior we use an encoder composed of an MLP followed by leaky ReLU activation:
Encoder
1.7
2. NHte o4), Mr, o, LeakyReLU(MLP(xx.)) (7)
X1:T
Figure 2: The overall framework of InterLatent consists of: (1) an coder that maps observations X, to la- tent variables 2, (t [1, TD, (2) de- coder that reconstructs observations X (f [1, T') from z,, and (3) tempo- ral prior estimation module that models the transition dynamics between latent states. We train InterLatent by L along LKLD- & (t E [1, denotes the estimation of the true noise terms e (1 [1, T).
Temporal Prior Estimation po(z,|z, -zn.) To enforce the conditional independence assumption in Eq. 5 we min- imize the KL divergence between the posterior distribu- tion and a prior distribution. This approach encourages the posterior to adopt the independence property as well, such that 2e x2 are mutually independent. To address the chal- lenges of directly estimating the arbitrary density function po (z, 1).
introduce transition prior module based on normalizing flows. This design represents the prior as a Gaussian distribution transformed by the Jacobian of the transition function, enabling efficient computing. For- mally, Vj, [2:1ū j E } we formulate the prior module as e/ = - f, (2/ (2) This computation meets the require- ment that fn to be invertible. Then the prior distribution of the j-th dimension of the temporal 0f, da
dynamics, 2/. can be computed as Po (e()
Po(ff'(2i 2 1)) 08;
82
In addition, for any v, such that {2 v E $7), we evaluate that 2s' = f. (2)). The prior distribution
calculated by pe (()) of.
of. 02; = Pf (2)))
as 2.' is independent of 2;-1- Combing 82;
together, the total prior distribution is:
of 02
Po (2. (2 (2.1)
II po (o()
The flow model f in Eq. 8 is built with the MLP layers. For more details on the derivations of prior estimation, please refer Appendix C.I.
Decoder P. (&, ) The decoder pairs with our encoder to generate an reconstruct of the observation x, from the estimated latent variables é,, which consists of stacked MLP followed by leaky ReL.U activation:
+ LeakyReLU(MLL((2)))
(9)
4.2 LEARNING OBJECTIVE
In this work, we extend our learning objective from Sequential Variational Autoencoder Li & Mandt (2018) with a modified ELBO. In general, the ELBO implements the observational equivalence requirement from Definition which ensures our learned model matches the data-generating distri-

---

Published as a conference paper at ICLR 2025
interLatent
WID's s of W/D
CaRING TDRL
LEAP
Beto-VAE Stationary
Nonstationary
1.0
0.9
0.8
0.7
MCC
0.6-
MCC
0.5
0.4
0.3
0.2
d-c=1
d7c=2
dmc
d=c=1
drc=2
d"c=3
Figure 3: Mean Correlation Coefficient (MCC) scores for various methods for both "Nonstationary" and "Stationary" settings. d' means the size of si in sequence. Higher MCC scores indicate better performance in identifying latent variables.
bution. We formulate the entire ELBO objective the following:
CELBO Eg,
(x.12(()122)))
BEg ~aw (Iog q (2.(x.) - log (2, 22-1))
(1a + +1ppaha)) +
Σ 1
(10)
2=
Sparsity Regularization
where /3 is the hyperparameter to balance the two losses. The reconstruction loss LRccon minimizes the discrepancy between x; and &; using mean-squared error.
The KL divergence loss LKLD serves dual theoretical purposes. It enforces the conditional inde- pendence assumption from Theorem 2 through the factorized prior (2, 22,-) by Eq. 8 while simultaneously satisfying the sufficient variability assumption from Theorem 1 by encouraging di- verse transitions in the latent space. When computing LKLD. we follow Yao al. (2022a); Chen et al. (2024) employ a sampling method since the prior distribution lacks an explicit form.
The sparsity regularization implements the support sparsity of intermittent sequences through three terms. The L1 norm on decoder Jacobian columns Jg.112,1 enforces sparse mixing patterns. The L1 norm on transition Jacobian rows [,, ensures sparse transitions from Z to 2,. The LI norm
on transition Jacobian columns | |2, maintains consistent sparsity structure. Following standard practice, we use these L.1 norms to approximate the LO norm for differentiability.
EXPERIMENTS
5.1 SYNTHETIC EXPERIMENTS
Experimental Setup To evaluate InterLatent ability to learn causal processes and identify latent variables in non-invertible scenarios, we conduct simulation experiments using random causal structures with specified sample and variable sizes. We generate synthetic dataset satisfying the identifiability assumptions outlined in Theorem I and (details in Appendix [.II), considering both nonstationary (s, varying across the sequence) and stationary (s, constant throughout) settings.

---

Published as a conference paper at ICLR 2025
t= 2
t= 5
t= 8
23
21
2
24
25
2
23
24
2
2
2
25
FARA
2
21
22
23
2
25
21
22
23
2
25
23
2
23
2
25
Figure 4: Visualization of the correlations between Zy and 2, at time steps f = 2, 5, and 8. The top row represents scatter plot plot on a stationary sequence. The red bounding boxes depicts the missing part of ze. i.e, (z"lu E s)). The green bounding boxes highlight the latent variables that component {zili E st). The results confirm that InterLatent successfully identifies {z]] 5 in both nonstationary and stationary sequences. Also, we can observe that {z" Iu sf is distinguishable from i E St).
nonstationary sequence, while the bottom row depicts scatter
identified for
For each setting, generate three scenarios of sequences, resulting in six scenarios in total. Each scenario has a particular value of dř. This design allows us assess the performance of InterLatent under different complexities of missingness. The Mean Correlation Coefficient (MCC) serves as our evaluation metric, measuring latent factor recovery by computing absolute cor- relation coefficients between ground-truth and estimated latent variables. MCC scores range from to 1, with higher values indicating better identifiability.
main results our simulations. We evaluate InterLatent against several state-of-the-arr approaches in identifying time-series causal variables and represen- tation learning, such as LEAP Yao al. (2022b). TDRL Yao al. (2022a) and CaRiNG Chen et al. (2024). Additionally, we include classic representation learning approaches, such as Beta VAE Higgins al. 2016), VAE Khemakhem et 2020 and SlowVAE Klindt al. (2020).
Results Figure 3 summarizes
The results from Figure 3 demonstrate that InterLatent consistently achieves higher Mean Cor- relation Coefficient (MCC) across both nonstationary and stationary scenarios. For instance, in the nonstationary sequence with de 1, InterLatent outperforms all other methods by a substan- tial margin, exceeding in MCC. We attribute the superior performance of InterLatent to its capability of handling missingness in z,, a feature not present in the comparative methods. This key distinction enables our approach to more accurately capture the temporal dynamics of the latent vari- ables. Figure 4 visualizes the disentanglement between the true latent variable and the estimations at different time steps from a sequence.
Ablation Study and Discussions To elucidate the key assumptions of our data generating process in Eq. I we further conduct ablation study focusing on the impact of sparse support. We introduce three baselines: (1) "W/O of f", which removes sparsity regularization transition functions;
8

---

Published as a conference paper at ICLR 2025
(2) "W/O of 9", which removes sparsity regularization mixing functions; (3) "WS", weakly supervised variant drops all sparse regularizations as having access to Sy and si during training.
We summarize "WS" baseline. This speaks the effectiveness of using the sparsity regularization terms against using 8, and sy for 9 and f directly. The "W/O S of f baseline, which assumes s: 6, yields a significantly lower Mean Correlation Coefficient (MCC) compared to InterLatent approach. Similarly, "W/O of g" fails achieve competitive results due its disregard for missingness in the mixing function. These outcomes confirm that without accounting for missing components, the baselines are unable adequately model our simulated data.
experimental results in Figure 3 InterLatent obtains the scores par with
5.2 REAL-WORLD EXPERIMENTS
Task setup To evaluate our proposed identification theories complex real-world scenarios, we apply them to the task of Group Activity Recognition (GAR) using the Volleyball dataset [lbrahim cct al. (016)). GAR aims to categorize the activity for an individual frames multi-actor scenes, aligning well with scenario of intermittent temporal latent processes. This is because not all actors participate in every activity, reflecting real-world dynamics where some may be occluded out of view in fast-evolving sporting scenarios. In our implementation, each actor given time point is modeled as a specific component of the latent variables, with occluded or out-of-view actors treated as "missing" in the activity representations. This setup provides solid testbed for our identification theory, allowing us to assess its robustness and effectiveness in handling real-world complexities such as partial observations and dynamic participant involvement.
Let {x/ 2 denote video consisting of T-frame observations and N agents. For each T.N m== that generates x" according time step and agent n, there exists latent variable z {2 to Equation [
We take inspirations from the two-phase training pipeline from Li al. (022)) to modify train- ing objective. First, we train InterLatent using the objective function defined in Eq. [0] Sub- sequently, classifier é predicts the one-hot activity label ý from the learned sequence of latent representations 21:T using an MIP: ý = MI. MIP(CConca((a (()) ()))) The classifier is trained using a cross- entropy loss with a LI regularization on its Jacobian: L -E, (one-hot(y) log |Jll,, where one-hot(y) denotes the one-hot embedding of the true activity label. More data pre- processing details can be found in Appendix D.2
+
Data and Comparing Methods The Volleyball dataset Ibrahim et al. (2016) contains 55 video recordings of vol- leyball games and is split into 3493 training clips and 1337 testing clips. The center frame of each clip is annotated with one group activity label out of eight labels (i.e. right set, right spike, right pass, right winpoint, left set, left spike, left pass, and left winpoint).
Methods
MCA
SACRE Pramono al.
2020)
83.3
AT Gavrilyuk el al. (200
84.3
SAM Yan ct al. (020)
86.3
DIN Yuan ct al, 2021
86.5
DFGAR Kim et al.
(2022)
90.5
HiGCIN Yan
2025
91.4
PAP Nakatani
2024
91.8
The comparing methods include the Stateoof-thearr methods on GAR task, such as SAM Yan et al. (2020). AT Gavrilyuk al. (2020), ASACRF/Pramone et al. (2020). DIN Yuan al.
Dual-Al Han et al. 20)22)
93.2
BiCausal Zhang et al. (2024)
93.4
TDRL Yao et al. (2022a)
92.9
(202)). DFGAR Kim et al. (2022), HiGCIN Yan et al. 2023).
CaRiNG et al. InterLatent
2024
94.0
PAP Nakatani al. (2024), and BiCausal Zhang al. (024).
95.7
We also benchmark against TDRL Yao et al. (2022a) and
Comparison with the state-of-the-art methods on Volle- ball dataset
Table 1:
CaRiNG Chen et al. (2024) to evaluate the efficacy of iden-
tifying the intermittent temporal latent process. For fair com- parisons, InterLatent adopts the ResNet-18 backbone|He ct al. 2016) and weakly supervised setting from Yan et al. (020 for feature extractions from the RGB frames, which is also commonly utilized by other ap- proaches.
Results and Discussions Table presents a comparison of Multi-class Classification Accuracy (MCA) on the Volleyball dataset. Notably, InterLatent demonstrates superior performance with respect to those do not consider the missingness in both transition and mixing functions, i.c., CaR- iNG and TDRL. For example, InterIatent achieves the highest accuracy of 95.7, significantly

---

Published as a conference paper at ICLR 2025
portvolleybalico ore
label: right set
prodiction: right set
label: left set
prediction: left set
a
label: left pass
prediction: left pass
label: left spike
X
prediction: left set
d
Figure 5: The visual examples of InterLatent on Volleyball dataset. Highlighted frames show the annotated activity, with yellow bounding boxes indicating occluded actors. InterLatent correctly predicts the three activities but misclassifies a video of left spike" as left set". Note that the spike activity is performed by actor that is severely occluded. This implies the misclassification may stem from that the label itself is not grounded by the true process.
surpassing the previous best result of 94.0 obtained by CaRiNG. InterLatent also outperforms the state-of-the-art approaches GAR, such Dual-AI and BiCausal by significant margins of 2.5 and 2.3 points, respectively.
Figure 5 illustrates the visual examples of activity classification outcomes produced by InterLatent. The model demonstrates robust performance in handling occlusion-induced miss- ingness, accurately categorizing activities in challenging scenarios. Figure 5a 5c showcase suc- cessful classificationn of "right ser", "left set" and "left pass" respectively, despite partial occlusions of key players. We also present failure case in Figure 5d] where InterLatent misclassifies a "left spike" as "left set." However, this misclassification stems from that the label itself is not grounded by the true process, since the spike activity is performed by a player that is severely oc- cluded.
CONCLUSION
We establish set of novel identifiability results for intermittent latent temporal processes, extend- ing the identifiability theory to scenarios where latent factors may be missing or inactive at different time steps. Specifically, we prove block-wise identifiability under assumptions sity, and further demonstrate component-wiss identifiability within the support given conditional independence assumption. These results hold for both nonstationary and stationary transitions, ac- commodating a wide range of real-world temporal dynamics. Our theoretical findings are validated through experiments on both synthetic and real- real-world datasets, demonstrating the practical applica- bility of our approach. The proposed InterLatent framework not only advances our understanding of complex temporal processes but also provides a principled method for uncovering hidden struc- tures in time-delayed systems with variable latent factor participation. Future work could explore the application of this framework to related tasks such as temporal disentanglement, transfer learn- ing in time series data, and causal discovery in dynamic systems. While have demonstrated the effectiveness our approach on visual-baa based task, the lack other applications is a limitation of this work.
support spar-
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Carlo Berzuini, Philip Dawid, and Luisa Bernardinell. Causality: Statistical perspectives and ap- plications. John Wiley & Sons, 2012.
Philippe Brouillard, Sébastien I Lachapelle, Alexandre Lacoste, Simon Lacoste-Julien, and Alexandre Drouin. Differentiable causal discovery from interventional data. Advances in Neural Information Processing Systems, 33:21 865-21877, 2020.
Guangyi Chen, Yifan Shen, Zhenhao Chen, Xiangchen Song, Yuewen Sun, Weiran Yao, Xiao Liu, and Kun Zhang. Caring: Learning temporal causal representation under non-invertible generation process. arXiv preprint 2nXiv:2401.14555 2024.
Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao. Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, et al. Mmdetection: Open mmlab detection toolbox and benchmark. arXiv preprint arXiv: 0tt::/906: /906.996600.. 155,, 2019.
Martin Danelljan, Gustav Häger, Fahad Khan, and Michael Felsberg. Accurate scale estimation for robust visual tracking. British machine vision conference, Nottingham, September 1-5, 2014. Bmva Press, 2014.
Marco Fumero, Florian Wenzel, Luca Zancato, Alessandro Achille, Emanuele Rodolå, Stefano Soatto, Bernhard Schölkopf, and Francesco Locatello. Leveraging sparse and shared feature activations for disentangled representation learning. Advances in Neural Information Processing Systems, 36:27682-27696, 2023.
Kirill Gavrilyuk, Ryan Sanford, Mehrsan Javan, and Cees GM Snoek. Artor-transformers for group activity recognition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 839-848, 2020.
Eric Ghysels, Jonathan Hill, and Kaiji Motegi. Testing for granger causality with mixed frequency data. Journal of Econometrics, 1 2016.
Mingfei Han, David Junhao Zhang, Yali Wang, Rui Yan, Lina Yao, Xiaojun Chang, and Yu Qiao. Dual-ai: Dual-path actor interaction learning for group activity recognition. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2990 2999, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp- 770-778, 2016.
Kaiming He, Georgia Gkioxari, Piotr Dolkár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pp. 2961-2969 2017.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basie visual concepts with constrained variational framework. In International conference on learning representations, 2016.
Mostafa S Ibrahim, Srikanth Muralidharan, Zhiwei Deng, Arash Vahdat, and Greg Mori. A hierar- chical deep temporal model for group activity recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1971-1980 2016.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoen- coders and nonlinear ica: A unifying framework. In International conference artificial intelli- gence and statistics, pp. 2207-2217. PMIR, 2020.
Dongkeun Kim, Jinsung Lee, Minsu Cho, and Suha Kwak. Detector-free weakly supervised group activity recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20083-20093, 2022.
David Klindt, Lukas Schott, Yash Shanma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding. arXiv preprint arXiv:2007. 10930, 2020.

---

Published as a conference paper at ICLR 2025
Lingjing Kong, Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akin- wande, and Kun Zhang. Partial disentanglement for domain adaptation. Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.). Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 1155-114772 PMLR, 17-23 Jul 2022.
Lingjing Kong, Guangyi Chen, Biwei Huang, Eric P Xing, Yuejie Chi, and Kun Zhang. Learning discrete concepts in latent hierarchical models. arXiv preprint arXiv:2406.005: 2024.
Sébastien Lachapelle, Tristan Deleu, Divyat Mahajan, Ioannis Mitliagkas, Yoshua Bengio, Simon Lacoste-Julien, and Quentin Bertrand. Synergies between disentanglement and sparsity: Gen- eralization and identifiability in multi-task learning. In International Conference Machine Learning, pp. 18171-18206, PMLR, 2023.
Sébastien Lachapelle, Pau Rodríguez López, Yash Sharma, Katie Everett, Rémi Le Priol, Alexan- dre Lacoste, and Simon Lacoste-Julien. Nonparametric partial disentanglement via mecha- nism sparsity: Sparse actions, interventions and sparse temporal dependencies. arXiv preprint arXiv:2401.04890 2024a.
Sébastien Lachapelle, Divyat Mahajan, Ioannis Mitliagkas, and Simon Lacoste-Julien. Additive de- coders for latent variables identification and cartesian cartesian-product extrapolation. Advances in Neural Information Processing Systems, 36, 2024b.
Yingzhen Li and Stephan Mandt. Disentangled sequential autoencoder. arXiv preprint
arXiv:1803.02991, 2018.
Yuke Li, Guangyi Chen, Ben Abramowitz, Stefano Anzellotti, and Donglai Wei. Learning causal domain-invariant temporal dynamics for few-shot action recognition. In Forty-first International Conference on Machine Learning, 2024. URL https: /tttss:s//peerevieew net forum?rrmmmmmmm d
LvLuYgU0BW
Zijian Li, Ruichu Cai, Guangyi Chen, Boyang Sun, Zhifeng Hao, and Kun Zhang. Subspace identi- fication for multi-source domain adaptation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL
Lars Lorch, Jonas Rothfuss, Bernhard Schölkopf, and Andreas Krause. Dibs:
Differentiable bayesian structure learning. In NewrIPS. pp. 24111-24123, 2021.
URL https: neurips. wwtppppee//////aaaeeepeppp /hash/ html
ca
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. International Confer- ence on Learning Representations, 2019.
Hiroshi Morioka and Aapo Hyvarinen. Causal representation learning made identifiable by grouping of observational variables. In Forty-first International Conference on Machine Learning. 2024.
Chihiro Nakatani, Hiroaki Kawashima, and Norimichi Ukita. Learning group activity features through person attribute prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18233-18242, 2024.
Ignavier Ng, Yujia Zheng, Xinshuai Dong, and Kun Zhang. On the identifiability of sparse ica with- assuming -angaaussiantt. Advances in Neural Information Processing Systems, 36:47960- 47990, 2023.
Rizard Renanda Adhi Pramono, Yie Tarng Chen, and Wen Hsien Fang. Empowering relational network by self-attention augmented conditional random fields for group activity recognition. In European Conference on Computer Vision, pp. 71-90. Springer, 2020.
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE transactions on pattern analysis and machine intelligence, 39(6): 137- 2016.
12

---

Published as a conference paper at ICLR 2025
Xiangchen Song, Weiran Yao, Yewen Fan, Xinshuai Dong, Guangyi Chen, Juan Carlos Niebles, Eric Xing, and Kun Zhang. Temporally disentangled representation learning under unknown nonstationarity. NeurIPS, 2023.
Xiangchen Song, Zijian Li, Guangyi Chen, Yujia Zheng, Yewen Fan, Xinshuai Dong, and Kun Zhang. Causal temporal representation learning with nonstationary sparse transition. arXiv preprint arXiv:2409.03/44 2024.
Julius von Kügelgen, Michel Besserve, Liang Wendong, Luigi Gresele, Armin Kekić, Elias Barein- boim, David Blei, and Bernhard Schölkopf. Nonparametric identifiability of causal represen- tations from unknown interventions. Advances in Neural Information Processing Systems, 36, 2024.
Thaddäus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel. Composi- tional generalization from first principles. Advances in Neural Information Processing Systems, 36, 2024.
Danru Xu, Dingling Yao, Sébastien Lachapelle, Perouz Taslakian, Julius von Kügelgen, Francesco Locatello, and Sara Magliacane. A sparsity principle for partially observable causal representation learning. arXiv preprint arXiv:2403..00 2403.00333, 2024.
Rui Yan, Lingxi Xie, Jinhui Tang, Xiangbo Shu, and Qi Tian. Social adaptive module for weakly- supervised group activity recognition. In Computer Vision-ECCV 2020: 16th European Con- ference, Glasgow, UK, August 23-28, 2020, Proceedings, Part VIII 16, pp. 208-224, Springer, 2020.
Rui Yan, Lingxi Xie, Jinhui Tang, Xiangbo Shu, and Qi Tian. Higcin: Hierarchical graph-based cross inference network for group activity recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence, 45(06):6955 6968, 2023.
Dingling Yao, Danru Xu, Sebastien Lachapelle, Sara Magliacane, Perouz Taslakian, Georg Martius, Julius von Kügelgen, and Francesco Locatello. Multi-view causal representation learning with partial observability. In The Twelfth International Conference on Learning Representations, 2024.
Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. Advances in Neural Information Processing Systems, 35:26492-26503, 2022a.
Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. In International Conference on Learning Represen- tations, 2022b.
Hangjie Yuan, Dong Ni, and Mang Wang. Spatio-temporal dynamic inference network for group activity recognition. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 7476-7485, 2021.
Kun Zhang, Shaoan Xie, Ignavier Ng, and Yujia Zheng. Causal representation learning from multi- ple distributions: general setting. In Forty-first International Conference on Machine Learning.
Youliang Zhang, Wenxuan Liu, Danni Xu, Zhuo Zhou, and Zheng Wang. Bi-causal: Group activity
recognition via bidirectional causality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 1450-1459, 2024.
Yujia Zheng and Kun Zhang. Generalizing nonlinear ica beyond structural sparsity. Advances in Neural Information Processing Systems, 36, 2024.
Yujia Zheng, Ignavier Ng, and Kun Zhang On the identifiability of nonlinear ica: Sparsity and beyond. Advances in neural information processing systems, 35:16411-16422, 2022.
Yujia Zheng, Biwei Huang, Wei Chen, Joseph Ramsey, Mingming Gong, Ruichu Cai, Shohei Shimizu, Peter Spirtes, and Kun Zhang. Causal-learn: Causal discovery in python. Journal of Machine Learning Research, 25 2024. URLhttp:// org/ papprs/v25//////// 23 1970.htm.
13

---

Published as a conference paper at ICLR 2025
A NOTIONS
Table of notions
Data and estimations
X E R
Observations
& E TR-
Reconstructions
21 RN
Latent variables
2e E RR
Latent variable estimations
C
True noise term
&
Estimated noise term
St
The support of Ze
6;
The support of 2t
The missingness of
s
The missingness of 2;
de
The cardinality of st de
The cardinality of 8,
di
The cardinality of st
The cardinality of sç
Indices
E [1,
Time step The indices of z, within St The indices of 2, within s?
n E [N]
The indices of Zt The indices of 2, within 6; The indices of 2, within 8ç
E st
j $
a E sf
v st
Ground-truth & Learned model
True mixing function
9
Learned mixing function
f
True transition function
f
Learned transition function
J
Jacobian of
Jg
Jacobian of Jacobian of f
Jj
Jacobian of
J,
Optimizations
True parameters of f True parameters of True parameters of encoder
Learned parameters of f
7
4
Learned parameters of 9 Learned parameters of encoder
W
* 12,1
L1 norm on columns
* |1,1
L1 norm on rows of
B PROOF OF THEOREM AND THEOREM
AND
In this section, we provide proof our identifiability results in Theorem[] and Theorem To this end, we take insiprations from Lemma B.1 of Lachapelle et al. (2023) to present a Lemma that is throughout our proof.
Lemma (Invertible matrix contains permutation) Let L E NNXN be invertible matrix. Then, there exists a permutation a such that (m) + 0 for all n, In other words, PT C where P is the permutation matrix associated with a, i.e. Pe (n))
Proof: Since the matrix L is invertible, its determinant is nonzero. We can obtain the following with assistance of Leibniz formula:
det L Σ sign(o) II [n) + 0
(11)
aES
where S denotes set of permutation. Eq. suggests that at least one term of the sum is non-zero, meaning there exists a S, such that Vn, L (n)) o O.
B. PROOF OF THEOREM
Theorem 1 is where most of the theoretical contribution of this work lies. Let recall Theorem
14

---

Published as a conference paper at ICLR 2025
Theorem (Bleck-wise identifiability): For the observations x, "R*K and estimated latent vari- ables 2, E RN, suppose there exist functions 9 satisfying observational equivalence in Eq. If the following assumptions and regularization hold:
(Smoothness and positivity): The probability density function of the latent causal variables, p(ze), has positive measure in the spuce of 2 and is twice continuously differentiable.
i (Path connected): For any zi .z' E Z. there is continous function Φ [0. 1] + Z, s.t. (0) z and ((1) = =z'
ii (Sufficient variability of z, and 2 Let q(zi(Z(-1) log p(z)(zi-1), well q( (2 log p (2,/2)11)) and H2. 9 2(-1) denotes the Hessian matrix of qZz)(zi-1) W.r.t. Zi and
1- Suppose G E [0 NxN as a binary adjacency matrix that indicates the existence of the transitions from Zt-1 and Ze. G11,2
means that there exists transition from zi-_ z
We assume that: span (z,,, aZz (1 2z-1) 1) ddd
and spant H2,, 1...2.11(266 19(2) IR
22- )}9h
GRL
iv (Support sparsity regularization): For any time step t, s, is not an empty set, d₂ S de
There exists a permutation o, such that
6; (s{) and = a
In other words, both Vi, {zili E si} and Vu, {z]) E { are block-wol wise identifiable.
Proof: Taking inspirations from Zheng & Zhang (2024). applying the chain rule to our definition in Definition leads to:
X - 8 g(2,) - (000(((11)))))) = ((2,) -g(:m) -/. -
(12)
where Jg. Jn-1, and J3 denote the Jacobian matrices of g, h-1, and ÿ. respectively. Eq. 12 provides a rigorous definition of observational equivalence the context of intermittent temporal latent pro- cesses, establishing the relationship between the true and learned models through their distributions and Jacobians.
We follow Yao al. 2022b a) Song et al. (2023): Chen et al. 2024 Song et al. 2024 to connect Jn-1(2e) in Eq. 12 with the transition probability density function pl Z 2;-1) 1) as we work on identi- fication over the time-series data. Given the fact that (( p 2 (Z(-
p(z, (x+-1) as well as (2 [2, 1) = ((2,1g(2, (2(2211)))))) = p(z, (x-1), we are able to map (x,-1, z,) to (x,-1, 2,) with
6
the jaocbian
0 (z.)
p(z)(x/11) p(2) Jn (z.)) + p(zi(z=-1) = p(2, dee
(13)
where h is an invertible mapping, such that 2, l((z)) Idet Jn (2:)) denotes the determinant of l((2)).
Taking the logarithm on both sides of Eq. 13 we have:
log ((z 12)) log |det Jn (z,) log.((222222)11
(14)
We replace 0(2:/2,-1) log p(2, 12. as well as ((ze|z(-1) = log p(zz (z-1)), and calculate the Hessian with 2, and 2 on both sides of Eq. 14 using change-of-vvarable and chain rule:
Hg. &a-, (55. (2, = (J,- (2,) TZ ((2, 12-1)-h)/-1 (2,
(15)
We can rewrite Eq. 15 based on assumption ii by:
span( (H, . (2, 22 1) = (Jn
(lal (2,-1), (16)
where o denotes the Hadmard product.
For any i1, E St, (il + i2) we can obtain the standrad one-hot basis vector 6n and 6, of Hessian matrix, such that e (T e R& Eq. 16 indicates the existence of a permutation matrix P associated with the permutation of Jn (Lemma D. such that:
PTea TP (o)))) C R xd.
(17)
64,
15

---

Published as a conference paper at ICLR 2025
which implies:
(a(i1),o(i2)) 2(-2 (1-1)
(18)
The support sparsity constraint suggests that: de = 2 ( 111101 S
= =d.
where 11. denotes the fo-norm of the rows of matrix
Combining this with Equation [8] we can conclude that:
8, = ((+)-
(19)
this were not the case, there would exist pair (11 i2) E G where if # ig. that contradicts
Equation [8
Eq. suggests that, Vi E St and v E $/:
dz
(20)
02:
Also, as is a invertible mapping, we can conclude that del(J o 0. Therefore, supports conclusion of block-wise identification.
02 Ox
0, which
Generalize from 2,-1 to &er: Notably we can easily generalize Theorem by replacing 2:-1 with Zet, and 2--1 with âạσ from Eq. to Eq, 16 respectively. In other words, we can extend the condi- tional probability density function into a non-markov setting. Accordingly, G E {0, 1) xx (t-1))
and G2, E (0, 13 d (d-1)) if both zez and 2 start from in Eq. 13
B.2 PROOF OF THEOREM
Theorem I allows us to further explore the identifiability Vi, {zili E st). In what follows, we provide the proof of Theorem i details.
Theorem (Component-wise identifiability Vi, {zili E st)): Let all assumptions from the Theorem hold. Additionally, suppose the following assumption is true for data generating process in Eq. Mas well:
(Conditional independence): At t, we assume that each component of z is conditional indepen- dent given the previous latent variables 2. For any i1, i2 E [N]:
z
21-1
Then for (2)1) E 8.) must be component-wise transformation of permuted version of true {z)li é E st).
Proof: Following previous works Zheng & Zhang 2024). our goal can be rewritten as demonstrating that Jn-1 1 (2.) D(2,) where D denotes an diagonal matrix. P is a permutation matrix that is defined in Lemma I and has been proven in Theorem | If Jr- (2x) D(2,) ),, there must exist il and (il * 12), such that j1, E J.- and j2 E J Ju-il the il-th column of Jn-1, which corresponds to zi!.. Similarly, J corresponds to z!2 Given Eq. we can obtain:
2 2/2 (.-1((a)) (2,) q(z (2 (
(21)
Also,
(2) (2z (211))11J---1 (2
(22)
Therefore, z,' and 2,² are dependent as given they are both dependent on 2/22 This contradicts with conditional independence assumption.
16

---

Published as a conference paper at ICLR 2025
B.3 EXTENSION OF TEMPORAL SUPPORT SPARSITY
Proosition 1'ddentifibiiiity under temporal support sparsity): In addition to the assumptions of The- orem if the following assumption and regularization hold:
(positivity and independence of the support): For any time step i, there exists the probability density finction of the support, p(st), has positive measure in the space of St. The support at any time step is independent of the supports at other time steps, thus can be factorized by T
p(SET)
11 S
i (temporal support sparsity regularization): For any time step Sy is not an empty set, E(dr.) S
R((1:r)
There exists a permutation a, such that
6; = o(st) and =
In other words, both Vi, {zili E se} and Vu, {zilu E 8i} are block-wise identifiabe.
Proof:
We start from Eq. 18 in Theorem to prove the Theorem 3. Let 74 = H2, 2. g(z, 22-1). and
Hi, Hz. 2. 7 1), the expected sparsity constraint can be reformulated by:
B(d1x) = EE//irr 1.0 Ep(s E
1( * 0)[s;)
n=1
Ep(sl))
B(1 0)(ss)
Ep(sur)
Pai (H # 0))
(23)
where 1 (*)) denotes the indicator function of x, PH|s, denotes the
Let J = Jn-1 (2,-1), and J-1 = - We can perform the similar steps to obtain:
E(di.T) RR/H1.T 1,0 E. Ep(err)
I( J 0)(ss)
N
Ep(sur)
([1 J + 0)(ss)
n=
N
= Ep((Lr)
Pre lac (I 3-i #0)
(24)
The temporal support sparsity constraint suggests that: B(die) S R(di.r), which leads to
N
E
E)p(s
P Prele. (I 1 J + 0)) E. E.((
PHlle (H 0)) S
==1
n=1
N
= E)(
(PH,es (I (I-1 J)- - PH.le (Hl, )) 50
(25)
R=1
11,, (J (1o the L.H.S. of Eq. 25 is a sum of non-negative terms which is itself non-iiseeive. This means that every term in the sum is zero. The rest of the proof remains the same with Theorem to obtain the block-wer wise identifiability. Moreover, if the conditional independence assumption in Theorem holds, we can further obtain the component-wis identifiability.
Eq. 18 suggests that Vn E [1, (...) (H,
17

---

Published as a conference paper at ICLR 2025
B.4 ESTIMATING S, AND MODIFIED ELBO
The temporal support sparsity in Theorem 3 requires to obtain p(SI.T) for the identifiability results. In order allow for gradient-based optimization of S1:T, take inspirations from the structure learning Brouillard et al. (2020); Lorch et al. (2021) to treat 8, U = S as a 1 x vector. Each entries of this vector independent Bernoulli random variable with probability of success (an), where a is the sigmoid function and Ot, is a parameter learned using the Gumbel-Softman trick. Accordingly, our ELBO needs be modified as following:
LEBO CReson LLDD, subject to Eswa ,
(26)
where 3 is an hyperparameter (which should be set ideally to the true d,) and & ~ (aa) means that each entry of S are independent and distributed according to aaa). Comparing to Eq. 10 Eq. 26
drops the sparsity regularization terms as we use Gumbel-Softmar instead.
C IMPLEMENTATION DETAILS
C.I PRIOR LIKELIHOOD DERIVATION
Consider paradigmatic instance of latent causal processes. In this case, concerned with two time-delayed latent variables, namely, 2, [z,, 2/). We set time lag is defined as for simplicity. This implies that each latent variable, z formulated as 2 (Pa(z,) ),+?) < where Pa(z)) C 2--1 is the parent of z,. The noise terms, ef", are mutually independent. To represent this latent process more succinetly,
introduce a transformation map, denoted as f. It's worth noting that in this context, we employ an overloaded notation; specifically, the symbol f serves dual purposes, representing both transition functions and the transformation map.
(27)
z;
By leveraging the change of variables formula the map f, can evaluate the joint distribution of the latent variables p 1 zi 22) as:
p 2: 2. x = p(z 1 d. G)/ Jyl
(28)
where Jj the Jacobian matrix of the map f, which is naturally low-triangular matrix:
0
1
0
0
Jy
0x]
0
Uz!
De
8x.2
Dc7
Given that this Jacobian is triangular, we can efficiently compute its determinant as II Oz." deg Further-
more, because the noise terms are mutually independent, and hence ej 1 ef. and e, 24-1, we can write Eq. as:
p(z] 2 23, 2 pl zi_1) x p(c), e |det Jyl
(because et 1 2(-1) (because El 1 e?)
p(z z x II p(e))/ det Jyl
(29)
Let [f--]] 1=11,,,3.. 1 be a set of learned inverse dynamics transition functions that take the estimated latent causal variables in the dynamics subspace and lagged latent variables, and output the noise terms, i.e., of = f- (z," P)(z))). By eliminating the marginals of the lagged latent variable p(z|_1, 2i both sides, derive the total transition prior likelihood as:
p(z) zi 2/ 2i-)) = II p(e))/ |det Jyl = II p( (z) Pa(z)))) x det J,
(30)
7
18

---

Published as a conference paper at ICLR 2025
in which, vi, fz/li E se), the prior likelihood is:
p II p(e)) |det Jyl II (z), Pa(z.))) det J7
(31)
Then, Vu, fzi s given Pa(z)) 0, the prior likelihood is:
p(zz)= II p(e))) |det.
II p(Sa (z;)) del
J J7
(32)
C.2 NETWORK ARCHITECTURES
Configuration
Description
Output dimensions
Encoder
Input: concat x1:T)
BS x T K
Dense
128 neurons, LeakyReLU
BS X T 128
Dense
128 neurons, LeakyReLU
BS X T X 128
Dense
Temporal embeddings
BS T 2N
Bottleneck
Compute mean and variance of posterior Sequential sampling
a
Reparameterization
21:T
Decoder
Input: 11:T
BS X T N
Dense
128 neurons, LeakyReLU 128 neurons, LeakyReLU input embeddings
BS X T X 128
Dense
BS X T X 128
Dense
BS x T x K
Temporal prior module Input
1:T &
BS x N
InverseTrann Transition
BS X 7 x N BS
JacobianCompute
log /detJ/l
Table The details of our network architectures for InterLatent, where BS means batch size.
Table 2 summarizes the network architectures of InterLatent.
C.3 TRAINING DETAILS
Simulation Experiments
We implemented our models using PyTorch 1.11.0. For optimization, employed the AdamW op- timizer Loshchilov & Hutter (2019), which has been shown to improve generalization performance deep learning models. The hyperparameters were set as follows: learning rate of 1e-3 and mini- batch size of 64. To ensure robustness and statistical significance, we trained each model under 10 different random seeds and report the overall performance as mean + standard deviation across these runs. The loss function balances reconstruction eтror and KL-divergence, with the latter weighted by 3 0.02. This choice of was determined through preliminary experiments to achieve an optimal trade-off between reconstruction quality and latent space regularity. All experiments were conducted on a single NVIDIA GeForce RTX 2080 Ti GPU with 1IGB meory.
Real-World Experiments
We employ the AdamW optimizer with cosine annealing for training our network. The initial learn- ing rate is set to 2e-3, with weight decay of le-2 mitigate overfitting. For all video sequences in Volleyball dataset, we uniformly sample = 10 frames as input. The ELBO loss computed with value of 0.02. We utilize batch size of 128, which found to provide good off between computational efficiency and optimization stability. The network is implemented using Py- Torch [2], leveraging its dynamic computational graph and GPU acceleration capabilities. Training
19

---

Published as a conference paper at ICLR 2025
conducted for 80 epochs on multi- GPulii setup consisting of four NVIDIA GeForce RTX 2080 Ti GPUs, providing a total of 44GB of meory.
D ADDITIONAL EXPERIMENTS DETAILS
D.1 SYNTHETIC DATA GENERATION PROCESS
Our approach generates six distinct scenarios of sequences, encompassing both stationary and stationary settings with varying degrees of missingness. Each sequence consists of 9 time steps, with latent variables Z; _ and observations x E R.. Missingness introduced by selecting a constant value d' E {1, 2, 3 for each sequence, representing the number of missing dimensions throughout that sequence. The set of missing dimensions, sy, then determined based on dž. We generate six scenarios in total: (1). non-stationary sequences with dž 1; (2). non-stationary sequences with di = 2: (3). non-stationary sequences with dē = 3; (4). stationary sequences with di = 1; (5). stationary sequences with dž = 2: (6). stationary sequences with di 3. In non- stationary sequences, si varies every time steps, while in stationary sequences, it remains fixed throughout.
For each scenario, the data generation process begins with 10,000 initial states drawn from Zo ~ (O, 1). From 1 to t 9, Z, within S, is generated using nonlinear function f with non-
additive, zero-biased Gaussian noise ei. where (a = 0.1): Vi E St, s = fi (1))
,,), where
(1)) is the set of و within 5,-1. The missing dimensions are set as Vu E si z = fu(e). Observations then generated using mixing function g that only considers 2, within s; X
(1) where z (6) is the of zi within St. g(z)
D.2 ADDITOINAL DETAILS OF THE VOLLEYBALL DATASET
Our preprocessing and feature extraction pipeline builds upon the procedure from Yan et al. 2023) We leverage a pretrained Faster R-CNN model Ren et al. (2016) implemented via the MMDetection then
toolbox Chen al. (2019) detect potential persons each frame. These detections tracked across frames using the method proposed by Danelljan et al. (2014). For feature extraction, utilize ResNet- 18 He et al. (2016). We apply RolAlign He al. (2017) with crop size of 5 x 5. The resulting features are embedded into a = 1024 vector. we select the top N = 20 person proposals based on detection confidence scores.
20


Reviews:
[{"review_id": "AxuIH0rrhq", "reviewer": "Reviewer_ghgg", "summary": "The work proposes a method for identifying latent causal variables for observed time sequences, motivated by sparsity of the causal connections. The identifiability of the latent variables is shown. Unlike most previous work, the considered setting allows the support of the mixing function to change over time.", "questions": "See weaknesses 1 and 2", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The considered setting is interesting and well-motivated.", "weaknesses": "1. What $\\mathcal{Z}$ refers to in condition (ii) of Theorem 1? Specifically, is $\\mathcal{Z}$ a fixed subset of the support of $\\mathbf{z}_{t}$ or the estimate? Additionally, where exactly is condition (ii) used in the proof? Finally, why isn’t the support sparsity assumption included explicitly in the statement of Theorem 1?\n\n2. There should be more in-depth discussion on condition (iii) and the support sparsity assumption in Theorem 1, as these are critical for the identifiability results. Currently, it is unclear how strong these assumptions are. Including simple examples where condition (iii) holds naturally could be helpful. \n\n3. The presentation of Section 4.1 and 4.2 needs to be improved. The relationships between the components in Section 4.1, the loss function, and the illustration in Figure 2 are currently difficult to follow. The rationale behind the loss design is not clearly explained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kq3lMUT6EZ", "reviewer": "Reviewer_nJQS", "summary": "This paper introduces InterLatent, a framework for learning latent variables with an intermitent generative process. Intermitence is defined as some variables being \"switched off\" from at both transition and generation processes. The authors include theoretical analysis which demonstrates identifiability of the latent variables up to permutation and non-linear scaling, and provide some interesting applications to realistic domains where the missingness assumption is useful.", "questions": "Below are some miscellaneous comments:\n- line 59 typo: “... has yet to fully addressed these challenges.\n- Consider using \\citet instead of \\citep in some cases. Examples:\n  - line 59: “(Wiedemer et al., 2024) relies on … “\n  - line 60: “(Lachapelle et al., 2023; Fumero et al., 2023; Xu et al., 2024) are restricted to linear … “\n- Line 64: Would it be possible to briefly define block-wise identifiability? Similarly for component-wise identifiability.\n- Lines 144-150: Can you define the domain for $h$ in both cases?\n- Line 186: I believe you refer to pdf instead of cdf in both cases.\n- Eq (7). With LeakyReLU(MLP(x)) you can get negative covariances. Is this expected?\n- Line 263: Typo in $\\hat{z}_t | x_t$", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The idea of \"switching off\" variables from the generative process at certain time steps is very interesting.\n- The theoretical analysis establishes identifiability up to permutation and nonlinear scaling.\n- The paper introduces the idea very clearly and states its importance in contrast to other very recent works.\n- The proposed estimation method outperforms recent approaches and demonstrates applicability on realistic domains.", "weaknesses": "**Summary:**\n\nThis paper presents an interesting exploration of both applications and theoretical results. However, I have identified potential technical limitations in terms of formulation, theoretical rigor, and estimation methods. My comments below are organised into **Problem statement**, **Theory**, **Estimation**. They refer to section 2, 3, and 4 in the main paper. I would be glad to reassess my score if the authors address the issues outlined here.\n\n**Problem setting:**\n- Line 90: It is unclear if $f_n$ is truly invertible, given the setup described. If the noise variable's dimensionality matches the output variable’s, and at least one parent is from $z_n$, then the input dimensionality may exceed the output’s, making invertibility challenging. Could the authors clarify this statement or adjust the formulation to account for dimensionality constraints?\n- Line 98: Does the transition function $f^u$ have any equivalence to the previous $f_n$?\n- Line 99: “This implies that when $z^u_t$ is missing, it does not influence $z_{t-1}$ , $z_{t+1}$, or $x_t$”. How is it possible that $z^u_t$ would have any effect on $z_{t-1}$ in the first place, considering time moves forward? I believe the authors mean $z_{t-1}$ does not affect $z^u_t$ when the latter is missing, could this be clarified?\n- Line 101, Equations (2) and (3): The definitions of $s$ are not complementary. Note that NOT (a AND b) = (NOT a) OR (NOT b). In your Eqs. you have AND in both cases. Given missingness in Eq. (3) is what you want, the authors might want to reformulate Eq. (2). \n- **Definition of Missingness:** The paper could benefit from a more formal presentation of how missingness affects the injectivity of the mixing function. Since the dimensionality of input variables varies based on the cardinality of $s_t$​, explicitly defining the generative process after establishing missingness might clarify the setup.\n\n\n**Theory:**\n-  **Figure 2 and Injectivity under Missingness:** Figure 2 introduces a mixing process affected by missingness, yet it is unclear how this interacts with the injectivity of the mixing function $g$. Specifically, if $g$ is injective at $t=1$ with $|s_t| = 2$, how would this property hold at $t=2$ with $|s_t| = 3$? Such cases seem to challenge the theoretical claims unless clarified.\n- **Dimensionality of $d_t$:** If $d_t$ is fixed the above argument is not problematic. However, Figure 2 suggests that $d_t$ can vary, making the injectivity claim potentially problematic. Could the authors specify this constraint in the theoretical statements if $d_t$ is indeed fixed?\n- **Possible Extension to Time-Dependent $d_t$:** One way I can think of incorporating a time-dependent $d_t$ is to intoduce a mixture distribution for $x_t$, where each mixture component allows different values of $d_t$. You could incorporate results from identifiability in mixture models for sequences, such as switching dynamical systems, by conditioning the mixture component at each time step. However, implementing this change would likely require substantial modifications to the theoretical framework.\n- **Validity of Assumptions:** It would be helpful if the authors could provide empirical or theoretical justifications for the assumptions (i-iii) in Theorem 1, specifically how they ensure consistency with the InterLatent model.\n\n\n**Estimation:**\n- **Estimating $s_t$:** The inference process feels incomplete due to the absence of information on how to estimate $s_t$​, which is central to the framework’s operation. This aspect isn’t fully explained in the main text, and more details on how $s_t$​ is computed or estimated would greatly clarify the estimation procedure.\n- **Framework Adjustments in Figure 2:** Given that missingness affects the data generation process by altering the mixing function, it would be useful to understand how this variation is incorporated into the learning method. Could the authors expand on this?\n- **Clarifying the Role of Sparsity Regularization (Eq. 10):** If the authors intend for sparsity regularization to automatically account for missingness, this point could benefit from a clear explanation. Without an explicit estimate of $s_t$, it’s challenging to understand the approach used for computing Eqs. (8) and (9). Some added details could help readers follow the inference method more easily.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NquymSMlIm", "reviewer": "Reviewer_Bdfw", "summary": "This paper introduces a new class of discrete-time stochastic process with a series of latent variables that are allowed to (i) vary over time, (ii) be uninformative to either the observed data and/or the subsequent latent values, and (iii) be identifiable for those that are informative. Along with defining this class of processes, the paper also proposed a variational inference method for learning and modeling them, which is able to adequately represent this latent and observed sequential values.", "questions": "I would personally rebrand describing when the Jacobian results in a 0 as the corresponding latent value being \"missing\" to rather be described as \"uninformative\" or something similar. The reason being is the latent values are always missing / never observed. Should $p(x|z_1,z_2)=p(x|z_1)$, then that is a matter of independence rather than missingness. I am curious on your thoughts to this, or if I missed something concerning this.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "I found the general proposed process to be simple yet powerful in practice. The authors were able to derive useful theoretical findings with minimal assumptions made on the underlying process, and the proposed model itself appears to be straightforward in design. I believe that this work is of interest in general to the graphical modeling community, with advances towards interpretable latent dynamics.", "weaknesses": "While a lot of time was spent on the general class of processes, I feel like the modeling (section 4) was a bit rushed and could benefit from more explanations and commentary on the design decisions made. For instance, why was a variational approach chosen over a sampling-based one? It is clear some inference procedure is needed to account for $z$ values being latent, but not much discussion is given to justify the choices made here. Additionally, the sparsity regularization is thrown in as part of the loss without much discussion around it. I am assuming that this is to encourage latent values to be \"missing\" when possible but that is just speculation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JWxz9WoL0l", "reviewer": "Reviewer_b1yD", "summary": "The authors propose a setting where observations are produced by a set of latent factors. These latent factors, however, may or may not be active from period to period. The authors suggest that under certain conditions these latent factors can be identified. Their theory informs a very specific architecture, and they demonstrate that the method works on synthetic data and on a real-world video dataset.", "questions": "I don't have specific questions - the paper is written clearly. But scalability needs be addressed. And demonstrating that the method works only on one dataset is completely unacceptable. The LEAP paper (one of the benchmarks), for example, has 3 datasets. The setup lends itself well to the time series modality. The authors themselves mention applicability to finance - they should consider pointing this machinery at that type of data. Medicine, or other setting with sensors could work well too.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I like and understand the setup - it is indeed reminiscent of many real-world problems. I also like how tightly the theory and the architecture are connected. Finally, I find the synthetic experiments persuasive - they do show that the proposed method indeed works.", "weaknesses": "I have 2 main concerns: (1) there is no attempt to show that this method scales to high dimensions, and (2) real-world dataset experiments are woefully insufficient.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cqw4dMOWGK", "reviewer": "Reviewer_WTHB", "summary": "This paper proposes a meaningful study on learning latent variables and their identifiability theory in intermittent temporal latent processes. The author establishes a set of novel identifiability results for intermittent latent temporal processes, extending the identifiability theory to scenarios where latent factors may be missing or inactive at different time steps.  However, despite the proposed identifiable theory being relatively sophisticated, the theory and the proposed network structure are relatively isolated and do not adapt well to the identifiable assumption.", "questions": "1. This article presents a promising theory, but network design is no different from most papers(eg, LEAP, TDRL, etc ), and it is also unclear which modules are designed to meet specific identifiable conditions, which appears very disjointed and the results in a mismatch between theory and methodology.\n\n2. What are the limitations of this article, and does it mean that any variational autoencoder model is applicable to any data in any situation?\n\n3. The author should further clarify where the given synthetic data method satisfies the assumption of identification, and should provide a detailed description of how it is combined with the proposed theory in the process of network design, rather than simply listing the network structure\n\n4. The author should clarify whether the causal variable belongs to the Granger causality category or other categories.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The author presents a promising research scenario, namely intermittent temporal latent processes.\n\n2. The author proposes a complex identifiable theory, in which both non-stationary and non-stationary transition processes are applicable, \n\n3. the author suggests that under certain assumptions, latent variables can be identified in blocks. By further assuming conditional independence, each latent variable can even be restored to a component by component transformation.", "weaknesses": "1. In this paper, the author considers latent variables as causal variables. Is the \"causality\" referred to Granger causality? If not, what is the difference between Granger causality and latent variables\n\nThis is more like a blind source separation task. How does it relate to mainstream structural causal models or latent outcome models? Or what is the connection with traditional Granger causality?\n\n2. The relationship between the proposed theory and the established model is not closely related, making it difficult to see the relationship between the network design and the assumptions given by the theorem. \n\n3. The same type of variational autoencoder introduced, such as \"a transition prior module based on normalizing flows,\" has been used in many papers[LEAP, TDRL, etc ], and it has been found that there is no significant difference or improvement in network design compared to other methods. Does this mean that all baseline variational autoencoders have the identification capability proposed in this paper?\n\n4. During the experiment, it was also difficult to directly identify where the synthesized dataset met the identifiable assumption conditions proposed in the article.\n\nReference:\n\n[LEAP] Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. In International Conference on Learning Representations, 2022.\n\n[TDRL] Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. Advances in Neural Information Processing Systems, 35:26492–26503, 2022.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)