Temporally Disentangled Representation Learning under Unknown Nonstationarity
Authors: Xiangchen Song
Weiran Yao
Yewen Fan
Xinshuai Dong
Guangyi Chen
Juan Carlos Niebles
Eric Xing
Kun Zhang
Venue: NeurIPS 2023 poster (2023)
Keywords: Unsupervised learning
Temporal disentanglement
Nonlinear ICA
Identifiability theory

Abstract: In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-related latent variables have been established in stationary settings by leveraging temporal structure.
However, in nonstationary setting, existing work only partially addressed the problem by either utilizing observed auxiliary variables (e.g., class labels and/or domain indexes) as side information or assuming simplified latent causal dynamics. Both constrain the method to a limited range of scenarios.
In this study, we further explored the Markov Assumption under time-delayed causally related process in nonstationary setting and showed that under mild conditions, the independent latent components can be recovered from their nonlinear mixture up to a permutation and a component-wise transformation, without the observation of auxiliary variables. We then introduce NCTRL, a principled estimation framework, to reconstruct time-delayed latent causal variables and identify their relations from measured sequential data only.
Empirical evaluations demonstrated the reliable identification of time-delayed latent causal influences, with our methodology substantially outperforming existing baselines that fail to exploit the nonstationarity adequately and then, consequently, cannot distinguish distribution shifts.

Temporally Disentangled Representation Learning under Unknown Nonstationarity
Xiangchen Song!*
Weiran Yao?
Yewen Fan'
Xinshuai Dong'
Guangyi Chen 1.3
Juan Carlos Niebles Eric Xing Kun Zhang
Carnegie Mellon University
Salesforce Research
Mohamed bin Zayed University of Artificial Intelligence
Abstract
In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-related latent variables have been established in stationary settings by leveraging temporal structure. However, in nonstationary setting, existing work only partially addressed the problem by either utilizing observed auxiliary vari- ables (e.g., class labels and/or domain indexes) as side-information assuming simplified latent causal dynamics. Both constrain the method to a limited range of scenarios. In this study, we further explored the Markov Assumption under time-delayed causally related process in nonstationary setting and showed that under mild conditions, the independent latent components can be recovered from their nonlinear mixture up to permutation and a component wise transformation, without the observation of auxiliary variables. We then introduce NCTRL, a princi- pled estimation framework, to reconstruct time-delayed latent causal variables and identify their relations from measured sequential data only. Empirical evaluations demonstrated the reliable identification of time-delayed latent causal influences, with our methodology substantially outperforming existing baselines that fail exploit the nonstationarity adequately and then, consequently, cannot distinguish distribution shifts.
Introduction
Causal reasoning for time-series data is long-lasting yet fundamental task [T-3]. The majority of the studies focus on the temporal causal discovery among observed variables 4 However, in many real-world scenarios, the observed data (e.g., image pixels in videos) instead of having direct causal edges, are generated by some causally related latent temporal processes or confounders. Learning causal relations has practical use cases, which benefit lot of downstream tasks, However, estimating latent causal structures among those unobserved variables purely from observations without appropriate class of assumptions is an extremely challenging task (i.e. the latent variables generally not identifiable) 7 8).
Under the topic of unsupervised representation learning via nonlinear Independent Component Analysis (ICA), some strong identifiability results of the latent variables have been established 14 by introducing side information such class labels and domain indices. Specifically focusing time-series data, history information also widely used as the side information for the identifiability of latent processes [15.18] However, existing studies mainly focused on and derived identifiability
results in stationary settings [10 [6] [T (a)) or nonstationary settings with explicitly observed domain indices
[12 17 (Fig (b))
Part of the work was done while interning at Salesforce Research
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

C+
CL+1
C1+1
CL+1
Zt
ZE+
Zt
Z
ZE
Zt+-1
Zt
ZE+
X
X+++1
X+
X+++1
X++1
X
X++++
(a)
(b)
(c)
(d)
Figure I: Graphic models for three different settings in causally related time-delayed time series data with a visual illustration. (a) is stationary setting in which the transition function 2t+1 = (z+)) stays universally the same. (b) is the setting widely explored in existing work, in which the transition function fz changes according to different domains (denoted as ct), and all those domain indices are observed. (c) capture the unobserved domain indices by introducing Markov chain C. (d) is more general form to model the time series data in this work. It allows nonstationary settings and doesn' 't require the domain indices to be observed.
One can immediately tell the infeasibility of those two scenarios that general time-series data is usually nonstationary and the side information (class labels and domain indices) is usually unohserved. That is particularly true when considering real-world data such as video or signal sequences. It doesn't make any sense to assume that there exists a stationary transition function that is applied to the whole video clip. Take very simple video clip of mouse [19
example. it is fairly clear that such a simple motion example can be divided into at least two phases (1) active phase in which the mouse is moving and (2) inactive phase in which the mouse is laying down. Instead of using a complex transition function to describe the whole video clip, a more reasonable assumption is that the same transition function is shared within the same phase, but across different phases, the transition functions are different, in other words, the transition function can be expressed as function of the domain index. Also, it is worth mentioning that if such domain or phase indices is latent or unobserved, then we cannot directly utilize the existing framework to learn the latent causal dynamics. That is again a more realistic case that in general, the domain indices within a video are not accessible without expensive human annotation.
Recently, HMNLICA 14 attempted to resolve the problem by introducing Markov Assumption on the nonstationary discrete domain variable, they assumed the domain indices follow a first-order Markov Chain and estimated the domain information purely from observed data. However, HMN- LICA assumes temporally mutually independent sources in the data-generating process (conditioning on domain indices), i.e. they don't 'on't allow latent variables to have time-delayed causal relations in between (Fig T (c)). Such an assumption imposed a huge negative impact on the usability of those methods. Considering the video of the little mouse example, the x,8 the observed video frames, 2,8 can be the independent motion dynamics or causal process such as position, velocity, (angular) momentum, etc, and cs are the phases or actions such standing up (active) and laying down (inac- tive). To accommodate for such general sequential data, time-delayed temporal dependence should be considered in the latent z, space (Fig | (d)), otherwise, is impossible model a complex video data's temporal relation purely from discrete, domain indices. Also to make sure that the latent independent components can be recovered, temporally conditional independence should also be enforced, i.c. Each dimension of z, is conditionally independent given the history Zuistory. To this end, a natural question is:
How can we establish identifiability of nonlinear ICA for general sequential data with nonstationary causally-related process without observing auxiliary variable?
To answer this question, first formulate the latent nonstationary states discrete Markov process and further explore the Markov Assumption (0) which is introduced for identifiability of nonlinear ICA in HMNLICA [44] and provided stronger identifiability result corresponding to the conditional emission distribution (i.e. the transition function of different domains) and the transition matrix of

---

the Markov process. Specifically, we generalized the identifiability of Hidden Markov Models in (20 accommodate time-delayed causally-related non- parametric transitions in latent space (Thm. T
Then we utilize the linear independence (Thm. 2 to further establish the identifiability of z₁.
The main contributions of this work be summarized follows:
To our best knowledge, this is the first identifiability result that can handle the nonstationary time-delayed causally-related latent temporal processes without the auxiliary variable. We formulate the problem, especially the nonstationary states into the Markov process, establish identifiability purely from observed data, and then show strong identifiaillity of latent independent components.
We present NCTRL, Nonstationary Causal Temporal Representation Learning, principled framework to recover time-delayed latent causal variables and identify their relations from measured sequential data under unobserved different distribution shifts.
Experiments on both synthttic and real-world datasets demonstrate the effectiveness of the proposed method in recovering the latent variables.
2 Problem Formulation
2.1 Time Series Generative Model
Assume we observe n--dimensional time-series data at discrete time steps, X = (x1, X2:... X1 where each X₂ E X' is generated from time-delayed causally related hidden components Z, E R" by the invertible mixing function:
X; = (Z:).
(1)
In addition to latent components z₁, there is an extra hidden variable c which is discrete with cardinality ce C. follows a first-order Markov process controlled by C' transition matrix A, in which the i, j-th entry Ai, is the probability to transit from state to j.
Ci C2,..., Ct. Markov Chain(A)
(2)
For E (I,..., n), Zir, as the i-th component of z,, is generated by (some) components of history information 1 discrete nonstationary indicator c,, and noise ea-
Zi fc ((= ă E Pa(zu)) Ct, Ea)) with 6jt ~ Pe;
where Pa(a)) is the set of latent factors that directly cause Bit, which can be any subset of Zhs [z--1. (1--1.21-2... 2........ } up to history information maximum lag L. The components of are mutually independent conditional on ZHx and ct.
2.2 Identifiability of Latent Causal Processes and Time-Delayed Latent Causal Relations
We define the identifiability of time-delayed latent causal processes in the representation function space in Definition 1. Furthermore, if the estimated latent processes can be identified at least up to permutation and component-wiss invertible nonlinearities, the latent causal relations are also immediately identifiable because conditional independence relations fully characterize time-delayed causal relations in a time-delayed causally sufficient system, which there are no latent causal confounders in the (latent) causal processes. Note that invertible component-we latent causal processes do not change their conditional independence relations.
transformations on
Definition 1 (Identifiable Latent Causal Processes). Formally let X {x1, X2...., xr) be se- quence of observed variables generated by the true temporally causal latent processes specified by (Ji,p(es), A, A, given in Eqs. (T) (2. and (3). A learned generative model (fi, A,g) is observationally equivalent to (fr, (fe,)les), A, g) if the model distribution Pa. (X1, X2, ..,...,,T]
matches the data distribution Pic.peA... (xx(((x; (xx1,X2...... X2...., xxT}) everywhere. We say latent causal pro- cesses are identifible if observational equivalence can lead to identifiability of the latent variables up to permutation N. and component-wiee invertible transformation T:
Picpe A.g (xI, X2,..., x.)) PS.. A X(x...,......., xT.))
g (x,) T moogo (xi), Vx, X,
where X is the observation space.
3

---

Identifiability Theory
In this section, showed that under mild conditions, the latent variable 2, is identifiable up to permutation and a component-wise transformation. The theoretical results can be divided into two parts (I) identifiability of the nonstationarity and (2) identifiability of the independent components. As introduced above, the major challenge comes from the unobserved domain indices or nonstationary indicators (ce in our graphic models). We first establish the identifiability of the different conditional distributions from the observed data and then show that the latent variables z are identifiable The complete proofs can be found in Appendix A
3.1 Identifiability of Nonstationary Hidden States
Gassiat et al. showed that the conditional emission distributions in Hidden Markov Models and the transition matrix are identifiable up to label swapping. We first generalize it the autoregressive setting to accommodate for the time-delayed causal relation, iee. we showed the identifiability of conditional emission distributions p(x, c).
Theorem 1. (identifiabillty of the nonstationarity with Markov Assumptions) Suppose the observed data is generated following the nonlinear ICA framework as defined Eqs. the following assumptions (Markov Assumptions) hold:
(2) and (3). Suppose
i For the Markov process, the number of latent states, C, is known.
ii The transition matrix A is full rank.
Use Pi...... HC E I to denote nonparametric probability distributions of the C emission distribu- tions p.= = p(xx x)- Then the parameters A and M = ((11...., .HC) are identifiable given the distribution, P(4) M. of least 4 consecutive observations Xt, XeN1111:5 Xx42, Xt+3, up to label swapping of the hidden states, that is:
If A is C C transition matrix and if a(c) is stationary distribution of A with ((c) > O de E {I,...,C). ,,,, and if M (fi,..., (.C) are C probability distributions R that verify the equality of the distribution functions [p(4] = A then there exists a permutation of the set A.M
C]. such that for all k, 5..=11..... 1...., C we have Au, (a(k))((t) and jug (a(())-
For notational simplicity, and without loss of generality, we can assume the components are ordered such that (c). That leads us to the identifiability of the nonstationarity in the system i.e. up to label swapping the hidden states, the conditional emission distributions p(xe ,,, and transition matrix A are identifiable, hence providing us bridge to further leverage the temporal independence condition the latent space to establish the identifiability result for demixing function or in other words the latent variables Z
3.2 Identifiability of Latent Causal Processes
To incorporate nonlinear ICA into the Markov Assumption we define the emission distribution p(x, Xx-1, e) as a deep latent variable model. First, the latent independent component variables Zz E R" are generated from a factorial prior, given the hidden state e and previous Z;-1, as
12
p(z, 11-1.
II p(zke ct)
(5)
Second, the observed data x, is generated by a nonlinear mixing function as in Eq. ① which is assumed to be bijective with inverse given by 2, = x.x,). Let (c)) - logp((zsr 12 Ct), and assume that (c((c) is twice differentiable in ž and is differentiable in 21,-11, 1,2, ...; Note that the parents of zkt may be only e and a subset of 2--1; if 21,-1 is not a parent 2kt, then Omk 0.
Theorem (identifiability the independent components) Suppose there exists an invertible function which is the estimated demixing function that maps to 2,, i.e.
(xx)
(6)

---

such that the components of 2, are mutually independent conditional on 21-1. Let
Vk,((c) - (T)a(())
8 (T))(())
8 @T)a(()
02H,0022,111 021,102111-1
d2k,02Zn1811
8 (C)
8' (x))
Vu,r(c) -
0z 021,1-1
05 022,1-1
Oz 02n.e-1
And
8 m))
8 (1)
(n)
8 8 -
Skt - (Va(1)) Vx())
02
Ozk
dzk
02
(8)
(2)
(1)
OC))(C)
Ompe (C
Skt (1TT (C)T,
Ozkt
Ozkt
azat
Uzkt
If for each value of 2t, S16, Sic. 22., S2t, Sut, Snt as 2n function vectors Sk, and Sa, with k 1, 2,...,,, n. are linearly independent, then 2, must be an invertible, component-wise transformation of a permuted version of zt.
So far, the identifiability result has been established without observing the nonstationarity indicators such as domain indices. In the next section, novel Variational Auto-Encoder based method is introduced to estimate the demixing function g
4 NCTRL: Nonstationary Causal Temporal Representation Learning
In this section, we present the details of NCTRL estimate the latent causal processes under observed nonstationary distribution shift, given the identifiability results in Sec 3 First, we show that our framework includes three modules, Autoregressive Hidden Markov Module, Prior Network. and Encoder- Delleerr Module. Then, we provide the optimization objective of our model training including an HMM free energy lower bound, a reconstruction likelihood loss, and KL divergence.
4.1 Model Architecture
Our framework extends Sequential Variational Auto-Encoders [21] with tailored modules to model nonstationarity, and enforces the condi- tions in Sec. 3 as constraints. We give the esti- mation procedure of the latent causal dynamics model in Eq. 3 The model architecture is showcased in Fig. 2 The framework has three major components I) Autoregressive Hidden Markov Module (ARHMM). (2) Prior Network Module, and (3) Saccode--eecoder Module.
Pz (B Bcc
E
FXLD
[G2 C22 ,r
(2 be ,n,
LUMM
ARHMM
Encoder
Decoder
Luccom (x,, Xyy a
(Ra, Ro, " 2r)
Figure 2: Illustration of NCTRLwith (1) Autoregres- sive Hidden Markov Module, (2) Prior Network, and (3) Incoder-Decoder Module.
Autoregressive Hidden Markov Module (ARHMM) The first component of our framework is ARHMM which deals with the nonstationarity with unobserved domains. As discussed in Thm [] the transition function or the conditional emission distributions across different domains together with the Markov transition matrix A are identifiable. This module estimates the transition function of different domains p(x|/x-1,, ct) and the transition matrix A of the Markov process, and ultimately decodes the optimal domain indices {ê1, i2,,.,,,,,, via the Viterbi algorithm.
Prior Network Module To better estimate the prior distribution p(ε,/2Hk, Ct), let ZHx de- note the lagged latent variables up to maximum time lag L. We evaluate p( ((2,/2 ca) = Pe (f,, 2H,, 0..) ai. 02,
by learning holistic inverse dynamics f= that takes the estimated
change factors for dynamics O.,, as inputs. Conditional independence of the estimated latent variables p(a, (mx) is enforced by summing up all estimated component densities when obtaining the joint p(z ( ZHx, ce) in Eq. 9 Given that the Jacobian is lower-triangular, can compute its determinant as
5

---

the product of diagonal terms. The detailed derivations are given in Appendix B.2
n
logp 2xx: cr)
log ((g)()))))))))
+
log
Conditional indepdence
Lower-triangular Jacobian
The third component is Variational Auto-Encoder based module which utilizes reconstruction loss to enforce the invertibility of learned mixing function g. Specifically, the encoder fits the demixing function g and the decoder fits the mixing function g. The implementation details are in Appendix B
Encoder-Decoder Module
4.2
Optimization
The first training objective of NCTRL is to maximize the Log-likelihood of the observed data:
Pe ((x), X2,...,....... xT}})
(10)
where θнмм represents the HMM training parameters. Then the free energy lower bound can be defined as:
CHMM L(((c), OHMM) Egte) [log Persod (x1,X2,-. XT, c)] H(((c))
(11)
Consistent with the theory part, the first training objective is to maximize data log-likelihood in the ARHMM module to get optimal g(c*).
q(c*) arg max ((q()), HHMм)
(12)
(())
which can easily be computed by the Forward Bankd--aaarddd algorithm and luckily all of it differentiable the HMM training parameters @ matrix and transition function parameters f
Then the second part is to maximize the Evidence Lower BOund (ELBO) for the VAE framework, which can be written (complete derivation steps are in Appendix B.3):
ELBO
log Pauaa (X)
DRL (2)(Z Iaana (Z)))
Ez,
log pdum (xe l2:)
Ec
log paum (Ze katx, (t)
Σ
log qs (2c (x))
(13)
-Recon
CLLD
We use mean -quared error (MSE) for the reconstruction likelihood loss CRecon- The KL divergence LKLD is estimated via a sampling approach since with a learned nonparametric transition prior, the distribution does have an explicit form. Specifically, we obtain the likeliihd of the posterior, evaluate the prior log (2,] AHx, Ce) in Eq. 9 and compute their mean difference in the dataset the E. (2 log (2g X,) log p (2g 24k, Ct).
KL loss: LKLD
Experiments
We evaluate the identifiability results of NCTRL number of simulated and real-world temporal datasets. We first introduce the evaluation metrics and baselines and then discuss the datasets we used in experiments. Lastly, show the experiment results discuss the performance, and make comparisons.
5.1 Evaluation Metries
Mean Correlation Coefficient (MCC) To evaluate the identifiability of the latent variables, we compute Mean Correlation Coefficient (MCC) on the test dataset. MCC is standard metric in the ICA literature for continuous variables which measure the identifiability of the learned latent causal processes. MCC is close to 1.0 when latent variables are identifiable up to permutation and component-wind wise invertible transformation in the noiseless case.

---

Mean Square Error (MSE) for estimating A As introduced in the theory, the A is identifiable in our setting, which means that our proposed method can provide accurate estimation for the transition matrix A, to valid such claim, we use mean square error (MSE) to capture the distance between the estimated A and ground truth A.
Accuracy for estimating or We also test the accuracy for estimating the discrete domain indices supplementary to the MSE for A since in theory, the A is identifiable but the c2 is generally not identifiable, which is relatively easy understand as an analogy Hidden Markov Models, the transition matrix is identifiable but we can only "infer" the best possible discrete variables but cannot establish identifiability for
is also worth mentioning that the MSE and Accuracy are influenced by the permutation, which is also true in clustering evaluation problems. Here explored all permutations and selected the best possible assignment for evaluation.
5.2 Baselines
The following identifiable nonlinear ICA methods are used: (1) Beta VAE 22 which ignores both
history and nonstationarity information. (2) i-VAB [12] and TCL 9 which leverage nonstationarity
to establish identifiability but assume independent factors. (3) Slow VAE [16), and PCL [1O] which
exploit temporal constraints but assume independent sources and stationary processes, (4) TDRL [8]
which assumes nonstationary, causal processes but with observed domain indices. (5) HMNLICA [4
which considers the unobserved nonstationary part in the data generation process but doesn't 't allow any causally related time-delayed relations.
5.3 Simulated Results
We generate two synthetic datasets corresponding to different complex- ity of the nonlinear mixing function g. Both synthetic datasets satisfy our identifiability conditions in the theo- rems following the procedures in Ap- pendix B.4 As in Table NCTRL can recover the latent processes under unknown nonstationary distribution shifts with high MCCs (>0.95). The baselines that do not exploit history (i.e., BetaVAE, VAE, TCL), with in- dependent source assumptions (Slow- VAE, PCL), consider limited nonsta- tionary cases (TDRL) distort the iden- tifiability results. The only baseline that considers the unknown nonsta- tionarity in the domain indices (HMN- LICA) explored the Markov Assumption but doesn't 't allow a time-delayed causal process and hence suffers poor result (MCC 0.58).
Table I: Experiment results of two synthetic datasets on baselines and proposed NCTRL, we run the experiments with five different random seeds and calculate the average with standard derivation. The best results are shown in bold.
Mean Correlation Coefficien (MCC)
Method
Dataset A
Dataset B
Ave.
BetaVAE i-VAE
44.02 = 3.11
47.48 + 10.58
45.75
89.74 = 3.38
44.50 + 0.25
67.12
TCL
37.12 + 0.60
56.33 3.77
46.73
SlOWVAE
33.84 + 0.60
53.92
43.88
PCL
42.41 2.87
63.66 2.77
53.04
HMNLICA
59.82 = 4.94
57.25 1.45
58.54
TDRL
83.99 + 1.92
72.02 + 2.76
78.01
NCTRL
98.85 = 0.30
99.01 + 0.24
98.93
On the other hand, the difference between dataset A and dataset B is the nonlinearity in the mixing function, dataset A has a relatively simple nonlinear mixing function, on the contrary, dataset B has more complex nonlinearity. Some variability has been observed among the relative performance ranks of different baselines. For example, i-VAE showed a great discrepancy between the two datasets, which revived the weakness of capturing complex nonlinearity in the unknown nonstationary distribution shift environments. Again we also observed that our proposed method can constantly recover the latent independent components with high MCC which indicates on both datasets the model is identifiable and the estimation algorithm is highly effective. To further validate if NCTRL successfully recovered the Markov transition matrix A and inferred the domain indices e with high accuracy. We further examine the accuracy for estimating nonstationary domain indices e, and the mean square error estimating the transition matrix A. As shown Table the result consistent with our theory in which the transition matrix A is identifiable and we can estimate it with
7

---

Table 2: Supplementary experiment results of two synthetic datasets estimating domain indices C and transition matrix in NCTRL, we run the experiments with five different random seeds and calculate the average with standard derivation.
Unknown Nonstationary Metrics
Dataset
Accuracy estimating ct.
MSE estimating A
A
89.96 = 0.24
1.01 x10 + 1.67 x x10-1
B
89.84 0.29
1.08 x10 + 1.89 x10
high accuracy, For the nonstationary domain indices or even though there is identifiability result governing the estimation accuracy, it can still be inferred pretty well since it is nothing but decoding problem in Hidden Markov Models.
5.4 Real-world Applications
Video data Modified CartPole Environment We evaluate NCTRL on the modified CartPole 23 video dataset and compare the performances with the baselines. Modified Cartpole is nonlinear dynamical system with cart positions I; and pole angles 9g as the true state variables. The dataset descriptions are in Appendix B.5] Similar to the synthetic dataset, we randomly initialize a Markov chain and roll out series of ce and configure the CartPole environment with respect to the ct- Specifically, we use five domains with different configurations of cart mass, pole mass, gravity, and noise levels. Together with the two discrete actions (i.e., left and right). By doing so, the nonstationarity is enforced, and since we can control and access all intermediate states in the system, all metrics including MCC and C accuracy together with MSE can be easily calculated. We fit NCTRL with two-dimensional causal factors. We set the latent size 2 and the lag number L = 2. In Fig. 3 the latent causal processes are recovered, as seen from (a) high MCC for the latent causal processes; (b) the latent factors are estimated to component-wise transformation; and (c) the latent traversals confirm the two recovered latent variables correspond to the position and pole angle.
MCC=0.955
Estimated Latents
Estimated Latents
1
0.97
0.29
Cart Position
T
0.14
0.93
Pole Angle
True Latents
True Latents
(a)
(b)
(c)
Figure 3: Modified Cartpole results: (a) MCC for cousally-related factors; (b) scatterplots between estimated and true factors; and (c) latent traversal fixed video frame
Table 3: Experiment results of CartPole dataset. The best results are shown in bold.
Mean Correlation Coefficient (MCC)
Beta VAE
i-VAE
TCL
Slow
SKD
TDRL
NCTRL
57.54
60.14
65.07
63.16
73.24
85.26
96.06
compare our NCTRL with baseline methods. In addition, we also compare with SKD 244 a state-of-the- sequential disentangle representation learning method without identifiability guarantee. Table 3 and 4 we can see that compared with TDRL, our NCTRL can recover the latent processes under unknown nonstationary distribution shifts with high MCCs (>0.95) with highly accurate estimated transition matrix and high quality inferred et- Specifically, by comparing the result of SKD, the MCC for SKD is better than variety of baselines,
Similar Table T and

---

Table 4: Supplementary experiment results of CartPole datasets estimating domain indices ez and transition matrix A in NCTRL, we run the experiments with five different random seeds and calculate the average with standard derivation.
Unknown Nonstationary Metrics
Accuracy estimating Ct
MSE estimating
79.23 5.33
5.01 x x10-2 + 1.23 x 10-2
however, we can see the distinction between well-disentangled models and identifiable models, only the models with identifiability can find the ground truth latent variables with theoretical guarantee.
Video data MoSeq Dataset We test NCTRL framework to analyze mouse behavior video data from Wiltschko et al. [19], which represents the original application clustering mouse behavio details of this dataset are available in Appendix B.6 Since there are no ground truth independent components in this partieular real-world dataset, we analyze it by several visualizations can be properly identified and if the patterns in the recovered independent components are consistent with the recovered domain indices. We analyze the first video clip of mouse behavior data and visualize the two phases we discovered and segmented in Fig 4 We can clearly see from Fig 4 that there are different phases with the upper one actively moving and the lower one inactive. The recovered independent components showed consistent pattern with the recovered phase or domain indices as shown in Fig
different domains
Independent Components
Active
Inactive
Inactive
Active
5000
10000
15000
Time [ms]
Figure 4: Result visualization of MoSeq dataset. (Active, Inactive) show two representative video frames for the active and inactive phases and (Independent Components) visualize the discovered independent components with corresponding phases tagged with different colors.
6 Related Work
Causal Discovery from Time Series Understanding the causal structure in time-series data pivotal in areas such as machine learning [1]. econometrics (2), and neuroscience 3 A bulk of the research this realm emphasizes determining the temporal causal links among observed variables. The primary techniques employed constraint-basee methods [25). which conditional independence tests to ascertain causal structures, and score-base based methods [26] 27]. where scores are utilized to oversee search operation. Some researchers also proposed combination of these two
methods 28 29] Additionally, Granger causality (30] and its nonlinear adaptations Bi 32 have
gained widespread acceptance in this context.
DDtaset can be accessed via htanttss:///attaaa github ht/:ooo/ooooeee// http:/oooooowbbstttt/eeeee...eee

---

Nonlinear ICA for Time Series Recently, the significance of temporal structures and non- stationarities has been recognized in achieving identifiability within nonlinear ICA. Time-contrastive learning (TCI. [9) utilizes the independent sources principle, focusing on data segments' variability. On the other hand, Permutation- -hased contrastive (PCI. [T0] offers a learning approach that dis- tinguishes true independent sources from shuffled ones under the uniformly dependent assumption. HMNLICA [1]] integrates nonlinear ICA with an HMM to address non-stationarity without segment- ing data manually. The V-B [1) approach employs VAEs to capture the actual joint distribution between observed and auxiliary non-stationary domains, assuming an exponential families condi- tional distribution. The recent advancements in nonlinear ICA for time series include LEAP [17], (i-)CITRIS [33 [4], and TDRL [18]. While LEAP introduces a novel condition emphasizing non- stationary noise, TDRL delves deeper into non non-parametric environment within a nonstationary context. In contrast, CITRIS recommends utilizing intervention target data for pinpointing latent causal aspects, avoiding certain constraints but necessitating active intervention access.
Sequential Disentanglement Majority of existing work about sequential disentanglement focuses on architecture based on dynamical variational autoencoder (VAE) [35]. Early works [36 37 separate dynamic factors from static factors using probabilistic methods. Then auxiliary tasks with
self-supervisory signals [38] were introduced. C-DSVAE [39 utilized contrastive penalty terms with
data augmentation to introduce additional inductive biases. In R-WAE [40), Wasserstein distance was introduced to replace KL divergence. To deal with video disentanglement [4] 42] explored
generative adversarial network (GAN) architectures and (43] introduced a recurrent model with adversarial loss. FAVAE. 44 proposed factorizing VAE and [45] proposed to learn hierarchical
features. Finally, SKD (24] introduced a spectral loss term that leads to structured Koopman matrices and disentanglement.
7 Conclusion and Discussion
Conclusion. In this paper, we first established an identifiability theory for general sequential data with nonstationary causally-related processes under unknown distribution shifts. Then we presented NCTRL, principled framework to recover the time-delayed latent causal variable identify their causal relations from measured data, and decode high-quality domain indices under Markov assumption. Experiment results both synthetic datasets and real-world video datasets showed that our proposed method can recover the latent causal variables and their causal relations purely from measured data with the observation of auxiliary variables domain indices.
Limitation. The basic limitation of this work is that the nonstationary domain indices are assumed to follow a Markov chain. Also, this work highly relies the latent processes to have no instantaneous causal relations but only time-delayed influences. the resolution of the time series is much lower, then it is usually violated and one has to find a way to deal with instantaneous causal relations. Extending our theories and framework to address the scenarios when more flexibility in the domain indices transition is allowed (i.c. beyond discrete variables following Markov chain) and to address instantaneous dependency or instantancous causal relations will be some of our future work.
Boarder Impacts. This work proposes theoretical analysis and technical methods to learn the causal representation from time-series data, which facilitate the construction of more transparent and interpretable models to understand the causal effect in the real world. This could be beneficial in a variety of sectors, including healthcare, finance, and technology. In contrast, misinterpretations of causal relationships could also have significant negative implications in these fields, which must be carefully done to avoid unfair or biased predictions.
8
Acknowledgment
This project has been graciously funded by NGA HM04762010002, NSF ISS195553,, NSF CNS2008248, NIGMS RO1GM140467. NSF IIS2123952, NSF BCS2040381, an Amazon Research Award, NSF IS221 1990, and DARPA ECOLE HR001 12390663. This project is also partially sup- ported by NSF Grant 2229881, the National Institutes of Health (NIH) under Contract RO 159805, a grant from Apple Inc., a grant from KDDI Research Inc., and generous gifts from Salesforce Inc., Microsoft Research, and Amazon Research.
10

---

References
[1] Carlo Berzuini, Philip Dawid, and Luisa Bernardinell. Causality: Statistical perspectives and applications. John Wiley & Sons, 2012.
[2] Eric Ghysels, Jonathan B Hill, and Kaiji Motegi. Testing for granger causality with mixed frequency data. Journal of Econometrics, 192(1):207-230, 2016.
[3] Karl Friston. Causal modelling and brain connectivity in functional magnetic resonance imaging. PLoS biology, 7(2):e0000333, 2009.
[4] Clive WJ Granger. Testing for causality: A personal viewpoint. Journal of Economic Dynamics and control, 2:329-352, 1980.
[5] Mingming Gong, Kun Zhang, Bernhard Schoelkopf, Dacheng Tao, and Philipp Geiger. Díscov- ering temporal causal relations from subsampled data. International Conference on Machine Learning, pages 1898-1900 PMLR, 2015.
[6] Aapo Hyvärinen, Kun Zhang, Shohei Shimizu, and Patrik O Hoyer. Estimation of a structural vector autoregression model using non- gaussianity. Journal of Machine Learning Research, 11 (5), 2010.
[7] Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference machine learning, pages 4114- 4124. PMLR, 2019.
[8] Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429 439, 1999.
[9] Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica, Advances in Neural Information Processing Systems, 29:3765-3773, 2016.
[10] Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics, pages 460-469. PMLR, 2017.
[11] Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 359-868. PMLR, 2019.
[12] llyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational au- toencoders and nonlinear ica: A unifying framework. In International Conference Artificial Intelligence and Statistics, pages 2207-2217. PMLR, 2020.
[13] Peter Sorrenson, Carsten Rother, and Ullrich Köthe. Disentanglement by nonlinear ica with general incompressible-ffov networks (gin). arXiv preprint arXiv:2001 04472,, 2020.
[14] Hermanni Hälvä and Aapo Hyvarinen. Hidden markov nonlinear ica: Unsupervised learning from nonstationary time series. In Conference on Uncertainty in Artificial Intelligence, pages 939-948, PMLR, 2020.
[15] Hermanni Hälvä, Sylvain Le Corff, Luc Lehéricy, Jonathan So. Yongjie Zhu, Elisabeth Gassiat, and Aapo Hyvarinen. Disentangling identifiable features from noisy data with structured nonlinear ica. arXiv preprint arXiv:2106.09620 2021.
[16] David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding. arXiv preprint arXiv:2007. 2020.
[17] Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. International Conference on Learning Representations, 2022. URL
11

---

[18] Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. httptttss//ppeeeeeee
net settfforum?dd
Vi-sZWNA_ Ue
[19] Alexander B. Wiltschko, Matthew J. Johnson, Giuliano lurilli, Ralph E. Peterson, Jesse M. Katon, Stan L. Pashkovski, Victoria E. Abraira, Ryan P. Adams, and Sandeep Robert Datta. Map- ping sub-second structure in mouse behavior. Neuron, 88(6): 1et:1112111333, 2015. ISSN 0896-6273. doi:
0ww.://o.000//onoooooo000000 2015
URL https: http::///wwww..inceeciieeee sciencedirectt
[20] Élisabeth Gassiat, Alice Cleynen, and Stephane Robin. Inference in finite state space non parametric hidden markov models and applications. Statistics and Computing, 26:61-71, 2016.
[21] Li Yingzhen and Stephan Mandt. Disentangled sequential autoencoder. International Conference Machine Learning, pages 5670-5679 PMLR, 2018.
[22] Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta- VAE: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations, 2017. w hattp:://openeviiiiie mett:forumiiddSyyyaaaaa
[23] Biwei Huang, Fan Feng, Chaochao Lu, Sara Magliacane, and Kun Zhang. Adarl: What, where, and how adapt transfer reinforcement learning. arXiv preprint arXiv:2107 027222 2021.
[24] Nimrod Berman, Ilan Naiman, and Omri Azencot. Multifactor sequential disentanglement via structured koopman autoencoders, In The Eleventh International Conference on Learning Representations, 2023. URL https::/oppenreeiees http:/pwwwwweeeeeeeeeeeeeoreeeaa www.:nnee/ooommddeeeee IIetbbnC
[25] Doris Entner and Patrik O Hoyer. On causal discovery from time series data using fci. Proba- bilistic graphical models, pages 121-128, 2010.
[26] Kevin P Murphy et al. Dynamic bayesian networks. Probabilistic Graphical Models, M. Jordan, 7:431, 2002.
[27] Roxana Pamfil, Nisara Sriwattanaworachai, Shaan Desai, Philip Pilgerstorfer, Konstantinos Georgatzis, Paul Beaumont, and Bryon Aragam. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics, pages 1595-1605. PMLR, 2020.
[28] Daniel Malinsky and Peter Spirtes. Causal structure learning from multivariate time series in settings with unmeasured confounding. In Proceedings of 2018 ACM SIGKDD Workshop on Causal Discovery, pages 23-47. PMLR, 2018.
[29] Daniel Malinsky and Peter Spirtes. Learning the structure of nonstationary vector autore- gression. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 9986-29944 PMLR, 2019.
[30] Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pages 424-438, 1969.
[31] Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaie, and Emily Fox. Neural granger causality. arXiv preprini arXiv: 1rXiv:1802.05888 180.00882, 2018.
[32] Sindy Löwe, David Madras, Richard Zemel, and Max Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprint arXiv:2006. 10833, 2020.
[33] Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Stratis Gavves. Citris: Causal identifiability from temporal intervened sequences. In International Conference on Machine Learning, pages 13557-13603, PMLR, 2022.
[34] Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Efstratios Gavves. ieitris: Causal representation learning for instantaneous temporal effects. arXiv preprint (.XXi:2200606666 2022.
12

---

[35] Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier Alameda- Pineda. Dynamical Variational Autoencoders: A Comprehensive Review. 2021.
[36] Wei-Ning Hsu, Yu Zhang, and James Glass. Unsupervised learning of disentangled and interpretable representations from sequential data. Advances in neural information processing systems, 30, 2017.
[37] Yingzhen Li and Stephan Mandt. Disentangled sequential autoencoder. arXiv preprint arXiv:1803.02991, 2018.
[38] Yizhe Zhu, Martin Rengiang Min, Asim Kadav, and Hans Peter Graf. S3VAE: Self-supervised sequential VAE for representation disentanglement and data generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5538-6547, 2020.
[39] Junwen Bai, Weiran Wang, and Carla Gomes. Contrastively disentangled sequential variational autoencoder. Advances in Neural Information Processing Systems, 34, 2021.
[40] Jun Han, Martin Renqiang Min, Ligong Han, Li Erran Li, and Xuan Zhang. Disentangled recurrent wasserstein autoencoder. In International Conference on Learning Representations, 2021. URL BRLLhtttss///peeeeiieee wttp://wwwww...eeooooooooo/ooddd
[41] Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, and Honglak Lee. Decomposing motion and content for natural video sequence prediction. In International Conference on Learning Representations, 2017. URL https://opperreviet httppp///pppeeeeeeee...eeee/raad
[42] Sergey Tulyakov, Ming- Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1526-1535, 2018
[43] Emily L Denton and Vighnesh Birodkar. Unsupervised learning of disentangled representations from video. Advances in neural information processing systems, 30, 2017.
[44] Masanori Yamada, Heecheol Kim, Kosuke Miyoshi, Tomoharu Iwata, and Hiroshi Yamakawa. Disentangled representations for sequence data using information bottleneck principle. In Asian Conference Machine Learning, pages 305-320. PMLR, 2020.
[45] Shengjia Zhao, Jiaming Song, and Stefano Ermon. Learning hierarchical features from deep generative models. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 4091-4099 PMLR, 06-11 Aug 2017. URL wtt.::ttttpppppe httpp:////pooceeingg.... mlr 2ress//700/hao17cooo
[46] Elizabeth S. Allman, Catherine Matias, and John A. Rhodes. Identifiability of parameters in latent structure models with many observed variables. Annals of Statistics, 37:3099-3132, 2008.
[47] Alessio Spantini, Daniele Bigoni, and Youssef Marzouk. Inference via low-dimensional couplings. The Journal of Machine Learning Research, 19(1):663 2018.
13

---

Supplement to
"Temporally Disentangled Representation Learning under Unknown Nonsta- tionarity"
Appendix organization:
Identifiability
15
A.I
Identifiability of Nonstationary Hidden States
15
A.2
Identifiability of Latent Causal Processes
16
A.3 Discussion on Assumptions in Theorem
18
Implementation Details
18
B.
Reproducibility
18
Prior Likelihood Derivation
18
Derivation ELBO
19
B.4
Synthetic Dataset Generation
20
B.4.1
Sample c, from Markov chain
20
B.4.2
Generation latent variables 2
20
B.4.3
Generation of observations X,
20
B.5
Modified CartPole Dataset Generation
20
B.6
MoSeq Dataset
7
Mean Correlation Coefficient
B.8
Network Architecture
21
14

---

A Identifiability
Assume observe ni-dimensional time-series data at discrete time steps, X (x1, X2,... Xr
where each x; is generated from time-delayed causally related hidden components 2, R by the invertible mixing function:
X; = (Zz)).
(1)
In addition to latent components Zg, there is extra hidden component C which is discrete variable with cardinality ct = C. it follows first-order Markov process controlled by x C transition matrix A, in which the i, j-th entry As,j is the probability to transit from state i
C12 C2.
Markov Chain
(2)
For E n). zic, the i-th component of 2,, is generated by (some) components of history information zt-1, discrete nonstationary indicator c,, and noise Ezt-
Zit = f(((=j,-11 Ryæ-r E Pa(zt)) ce Ear with Ext ~ Peilc
where Pa(za) is the set of latent factors that directly cause Eit, which can be any subset of ZHx (Z--1, Zt-2..... .......... Z } up to history information maximum lag L. The components of Zę are mutually independent conditional on Zμμ, and ct.
A.1 Identifiability of Nonstationary Hidden States
Theorem 1. (identifiabillty of the nonstationarity with Markov Assumptions) Suppose the observed data is generated following the nonlinear ICA framework as defined Eqs. and (3). And Suppose the following assumptions (Markov Assumptions) hold:
i For the Markov process, the number of latent states, C, is known.
The transition matrix A is full rank.
Use Pi..... .......-44. , E Th to denote nonparametric probability distributions of the C emission distribu- tions pec x(xx] c). Then the parameters A and M (J41,..., .CC) are identifiable given the distribution, p(3) A.M- of least 4 consecutive observations Xt, X(+1; X X143, up to label swapping of the hidden states, that is:
Á C C transition matrix, if (Tc) stationary distribution of À with #(c) 0 Vc (1...., C}, and if M (fr...., (C)) are C probability distributions on B." that verify the equality of the distribution functions pp(3) = p(3) A.M C we have Åg, (o( and ju [aa(k))
A. then there exists permutation of the set {l....,CC C}
such that for all I
Proof. Suppose we have:
(4) where p(x xT) has transition matrix A and emission distributions (f Mc), similarly for p(x)...... XT).
p(xi..... xT) p(x)......
We consider four consecutive observations Xo. X1, X2, and corresponding four discrete elements Co C1, C2 C3-
p(x), X2, xo) = Σ p(c))p(x1 xx. Act xx, c2) Aog x2,
01:02:08
Σ
p(es Ac) (x0, p(xe o2)
Σ Aegcs (xe.
Σ
p((c) Aelca)Ac.ccot (xx (xo.ci)
p(x2
x1, x2)
Σ Acsespixs x2)
Σ
Tez
Tee Acriz
Med
Hoz
Σ
Aoz.ca
Rez
2
91
c3
Reg
Aug
(5)
15

---

where Te, p(c:). Since A has full rank and the probability measures μl. Mi..... HC are linearly independent, the probability measures {Pez Ec, diz Aetie independent, and the probability measures {jtez = Eos Aeg, Mes -2==,, I...., .....,Cy C) are also linearly independent. Thus, applying Theorem 9 of [46]. there exists permutation of (l...., C} such that, vi E CI....,C):
Hei 02 = l..... C are linearly
pi ())() Actil
Σ Ang Σ
This gives easily vi {L..... C}:
AisHois Σ A0(0))
Since the conditional distributions μ4 are linearly independent, we can establish the equivalence between A and A via permutation o,
A Ac(5),(()))
(6)
then the theorem is proved.
For notational simplicity, and without loss of generality, we assume the components are ordered such that (c). That leads us the identifiability of the nonstationarity in the system i.e. up to label swapping of the hidden states, the conditional emission distributions p(x; x;- Q.) and transition matrix A are identifiable, hence providing bridge to further leverage the temporal independence condition in the latent space to establish the identifiability result for demixing function or in other words the latent variables 2;.
A.2 Identifiability of Latent Causal Processes
To incorporate nonlinear ICA into the Markov Assumption we define the emission distribution p(x; Xx-1, e) as a deep latent variable model. First, the latent independent component variables Ze E R" are generated from a factorial prior, given the hidden state a and previous Z;-1, as
p(z, Z, 1.)
II p(z 1,)), k=1
(7)
Second, the observed data x, E R" is generated by a nonlinear mixing function as in Eq. ( which assumed to be bijective with inverse given by 2, = -x-((x)), Let (cz) - log 24 Ct), and assume that (c) is twice differentiable in zμ. and is differentiable in 2,,--1, I 1,2, ..., Note that the parents of zkt may be only c, and subset of z; if 21,-1 is not parent of Zka, then Ons 0. 02,1-1
Theorem 2. Suppose there exists an invertible function g which is the estimated demixing function that maps x; to it, i.e.,
24 = (xi)
(8)
such that the components of 2n are mutually independent conditional on 2, Lel
Vk,r(c) 2 @mma(c)
(c))a())
@n)((C)
02 03,,0
(9)
(c)
mc)
vn,e(c) A
22 021,e-1
Dzk 0221111
Oz 02m,t
And
82 (2) Skt A (ve=(1)T,,... (C)
(1
82 (C)
11
0=2
Ozz
Oz
022
(10)
Ompa (2)
Omke(1)
(C)
-
Syt A (vxe(1)T, (C)
Ozya
Ozke
Ōzks
Ozut
If for each value of 2ts Sıt, Sic. 22t, 82t, .... Sut, Su,, 2n function vectors Sk, and Si.to with k 1, J.2...., n, are linearly independent, then 2, must be an invertible, component-wise transformation of permuted version of zi.
16

---

Proof. Combining ① and 6 gives Z₁ = (g gg) = h(2,), where h _ gg' - g. Since both and g are invertible, h is invertible. Let H, be the Jacobian matrix of the transformation h(2,), and denote by Hast its (ki, i)th entry.
First, it is straightforward to see that if the components of 24 mutually independent conditional on previous - and current Ct. then for any i * j. Zie and 2zz are conditionally independent given 2 (&t ( 25x3) U {ce). Mutual independence of the components of 2, conditional on 2 implies that Eie is independent from 2, {ze,, } conditional on - and Cr, i.e.,
p(2se 2 1, (e) = p(z 2 (2+ \ { (.) At the same time, it also implies 24e is independent from 2, \ {2x) conditional on 2 and Ct. i...
p(2 1, p(2n1 2 U (2c (4)
Combining the above two equations gives
p(zu1 2 U (2, (zx)), p(2ir 2 U (2, \ {Eits Rjt)),,
i.c. for i i H j. Par and Zge are conditionally independent given 2 U (2, & u
U {a}.
We then make use of the fact that if & and Z3z are conditionally independent given 2,-1 U (2 {2a, 2zc3) U {ce) then
2 log p(z,, 21,1, = O,
Oz
assuming the cross second-order derivative exists 47
Since p(2r, 2,,,,)),
p(ž, ()(2,,))( while p(2,-1, (4) does not involve da or 2jc, the above equality is equivalent to
8 log p(:: (2)-1,0))
(11)
I
Then for any ct the Jacobian matrix of the mapping from (x+-1. 2.) to (x: 2;) is
H. where
stands for a matrix, and the (absolute value of the) determinant of this Jacobian matrix is H. Therefore p(2,, x(-1 p(z,, xx- lle)) [H,]. Dividing both sides of this equation by p(x,- (.) gives
p(2 xx-1, cx) p(z. cz) Since p(z: 11-1,c) pize g(zz-1), ct) p(zz xx-1. and similarly p(ž: 12,-1 p(ž, Xx-1. c.), Eq. 12 tells us
cz) (H.).
(12)
log (2-1, = log log bc)1, log H.l Σ maclee) + log HHcl
(13)
Its partial derivative w.r.t. 2 is
i log p(2r 12-11, co)
n
Ompa (cx)
Ozua
i log H.
Ozkt
02gt
k=1
i log [H
Haas
Ozkt
02g
k=
Its second-order cross-derivative is 8² log r(,, 1,) (t)
"
(os 022
Omeele))
OHkit
8 log [H,]
Σ
H
Hajz
(14)
azk
The above quantity always according to Eq. 11 Therefore, for each / 1, n and each value 21,1-1, its partial derivative w.r.t. 21,2-1 is always O. That is
8 loog p(ze (1)
0 (ct) 0z 02,011111
8 pme(c))
OH
H Heje +
= 0, (15)
02,
Øāg1
km
17

---

where we have made use of the fact that entries of He do not depend on 21,--1. Using different values r for a. in Eq. (14) take the difference this equation across them gives
8² log p(2, 22-1;++ 8a plzp ((-1;r)
O2iz
mr (r+ 11
0 me
Hai Haje + (Oneelr +
( Ozkt
)mpaa())
0H
0.
Szkt
Zzkt
Ozst
Øāzt
(16)
If for any value of Z₁, Si6, S12, S2c, S22 .... Smit Sut linearly independent, make the above equation hold true, one has to set H Hkjt = or é A j. That is, each row of H, there is only one non-zero entry. Since h is invertible, then 2, must be an invertible, component-we wise transformation of permuted version of 2
So far, the identifiability result has been established without observing the nonstationarity indicators such as domain indices.
A.3 Discussion on Assumptions in Theorem
This condition was initially introduced in GCL [1]], namely, "sufficient variability",, to extend the modulated exponential families [ to general modulated distributions. Essentially, the condition says that the nonstationary domains must have sufficiently complex and diverse effect the transition distributions. In other words, if the underlying distributions are composed relatively many domains of data, the condition generally holds true. Loosely speaking, the sufficient variability holds if the modulation of by e on the conditional distribution ((Zcz ZHk, c) is not too simple in the following sense:
1. Higher order of &s (k >> 1) is required. If k = 1, the sufficient variability cannot hold;
2. The modulation impacts Xij by u must be linearly independent across domains c. The sufficient statistics functions 4i cannot be all linear, i.e., require higher-order statistics.
Further details of this example can be found in Appendix B of [ and Appendix S1.4.1 of [18] In summary, we need the domains denoted by c have diverse (i.e., distinct influences) and complex impacts on the underlying data generation process.
B
Implementation Details
B.1 Reproducibility
All experiments are done in a GPU workstation with CPU: Inel GPU: NVIDIA RTX 4090, Memory: 128 GB. The code can be found via htttttsss///tthub com wttp///aannccnnnnnnrrr//////////
B.2 Prior Likelihood Derivation
Let us start with an illustrative example of stationary latent causal processes consisting of two time- delayed latent variables, i.c.. z, = [21.t> 22.1] with maximum time lag 1, i.c., 2i, = fi (z Eq,2) with mutually independent noises. Let us write this latent process as a transformation map f (note that we overload the notation f for transition functions and for the transformation map):
11,t-1
-
22,1-1
= f
22,1-1
(17)
Z1,t
E1,t
Ø2,t
E2.t
By applying the change of variables formula to the map f, we can evaluate the joint distribution of the latent variables p(21,1-1, 22,-1,,, 22,1) as:
p(21,011, 21,4 22,1 p p(21,-1, 22,--1 E1,1, 62.)) det Jrl,
(18)
18

---

where Jr is the Jacobian matrix of the map f, which is naturally low-triangular matrix:
0
0
0
1
0
0
Jr
001.
051.
0
021,1-1
822,t-1
Dct.
02g.1
0/2.1
0
022,1
001,t-1
882,6-1
862,1
Given that this Jacobian is triangular, we can efficiently compute its determinant as II, Ozi. Fur- Se,
thermore, because the noise terms are mutually independent, and hence 6it 1 Ej. for j of i and 1 Ze-1, we write the RHS of Eq. 18 as:
P(21,t-1:22,- -2,- 21,0 22,))
p(e,, (2,1/ Jel (because & 1 2;-1)
11, II p(),,)/ |dtt Jrl (because 61 62,1)
(19)
Finally, by canceling out the marginals of the lagged latent variables p(=1,t11, 22,e-1) on both sides, we can evaluate transition prior likelihood as:
II p(ECt)/ det
II p(er,) x |ett (
(20)
Now we generalize this example and derive the prior likelihood below.
Let (/,- 1= be set of learned inverse transition functions that take the estimated latent causal variables, and output the noise terms, i.e., غτ,t = fi (Ê,,, (a,-r,c))).
Design transformation
B with lowttriangular Jacobian follows:
Int.
Li. 26-1, 2n]
mapped to [2e-L,.. ... :.--......1..... m..
with JA-B =
diag
(21)
Similar to Eq. 20 we can obtain the joint distribution of the estimated dynamics subspace as:
log logp(A) (A)
logg (21-1, 1...., 21 Σ log p(e..)) log (Idet (J A J-))))
(22)
Because mutually independent assumption
05.
logp (2. (2 Σ logp(é..le)) +
Σ log
(23)
02,11
1=1
i=
B.3 Derivation of ELBO
Then the second part maximize the Evidence Lower BOund (ELBO) for the VAE framework, which can be written as:
19

---

ELBO - log Pdata (X)
DE (1o (Z Pdara (Z(X))
=ezzgg (2 log Petnta [ 2)
DEL 1. 1s (x Pdaata (z(x))
oo log Panta
(
1z (2|x)
[lg 9s (Z
log Pduta (
-E2xg0(2/xx
log Paata (|z
log Pdlata (Z)
log qo (2|X)
(24)
Be ! log p(a. (12
T
T
log Pdata (x (xe(zt) + Re
Σ
log Pdata (Ze /1ix Ct)
log qo (2+
-XLD
B.4 Synthetic Dataset Generation
We generated two synthetic datasets (A and B) with different nonlinear mixing functions. In this section we will introduce the detailed implementation the generation. The generation can split into steps (1) sample from Markov chain, (2) generate 2, with different transition functions fe, with respect to c, and (3) generate observation x, via mixing function g.
B.4.1 Sample o from Markov chain
We first randomly initialized a Markov chain with transition matrix A and sample 20,000 steps.
B.4.2
Generation of latent variables
We first randomly initialized |C| different transition functions {L .,, Sle with different MLPs, and generate fea (Zxx). The dimensions set to for fair comparison.
B.4.3 Generation of observations X
The difference between datasets and is the mixing function. We use two-layer randomly initialized MLP for dataset A and a three-layer MLP for dataset B. For each linear layer in the MLP, we use condition number of the weight matrix to filter out ones that are not "invertible".
B.5 Modified CartPole Dataset Generation
Similar to the synthetic datasets, we also sample from a Markov chain and get c₁ For the modified CarPPle, we initialized 5 different environments which have different combinations of hyperparame- ters such as gravity, pole mass, etc. A detailed comparison is listed in Table |
Table I: Different configs for different Modified CartPole environments.
Environment ID
Gravity
Pole Mass
Noise Scale
9.8
0.2
0.01
24.79
0.5
0.01
3.7
1.0
0.01
11.15
1.5
0.01
0.62
2.0
0.01
At each time step the environment will load the corresponding hyperparameters for given and update the states 2, according to configuration given c. The nonlinear mixing function from states to observations x, is fixed by rendering method in the gym package.
20

---

B.6 MoSeq Dataset
In the MoSeq dataset, the observations X are taken be the first 10 principal components of depth camera video data of mice exploring an open field. The dataset consists of 20-minute depth camera recordings of 24 mice. In preprocessing. the videos are cropped and centered around the mouse centroid and then filtered to remove recording artifacts. Finally, the preprocessed video is projected onto the top principal components obtaín a 10-dimensional time series.
B.7 Mean Correlation Coefficient
MCC a standard metric for evaluating the recovery of latent factors in ICA literature. MCC first calculates the absolute values of the correlation coefficient between every ground truth factor against every estimated latent variable. Pearson correlation coefficients or Spearman's 's rank correlation coefficients can be used depending on whether componentwise invertible nonlinearities exist in the recovered factors. The possible permutation is adjusted by solving linear sum assignment problem polynomial time on the computed correlation matrix.
B.8 Network Architecture
We summarize our network architecture below and describe it in detail in Table 2nd Table
3
Table Architecture details. BS: batch size, T: length of time series, dim: input dimension, z dimi:: latent dimension, LeakyReI.U: Leaky Rectified Linear Unit.
Configuration
Description
Output
ARHMM
Autoregressive HMM for Synthetic Data
Input: X Emission Module
Observed time series
BS x T i_dim
Compute Uz 22111
BS x T x z_dim
MI.P-Encoder
Encoder for Synthetic Data
Input: X1.2
Observed time series
BS x T x i did
Dense
128 neurons, LeakyReLU 128 neurons, LeakyReLU 128 neurons, LeakyReLU
BS X T x 128
Dense
BS X T X 128
Dense
BS x T x 128
Dense
Temporal embeddings
BS x T x z_dim
MLP-Decoder
Decoder for Synthetic Data
Input: 21.
Sampled latent variables
BS x T x z_dim
Dense
128 neurons, LeakyReLU 128 neurons, LeakyReLU
BS x T 128
Dense
BS x T x 128
Dense
i_dim neurons, reconstructed 81:7
BS x T x i_dim
Factorized Inference Network
Bidirectional Inference Network
Input
Sequential embeddings
BS x T x didim MIT,0LT 21.T
Bottleneck
Compute mean and variance of posterior
Reparametcrization
Sequential sampling
Prior Network
Nonlinear Transition Prior Network
Input
Sampled latent variable sequence 21:7 Compute estimated residuals é,t
BS x T z dim
Inverse Transition
x T x z_dim BS
JacobianCompute
Compute log (jdet (J)))
21

---

Table 3: Architecture details on CNN encoder and decoder. BS: batch size, T: length of time series, d_dim: hidden dimension, dim: latent dimension, F: number of filters, (Leaky)Rel.U: (Leaky) Rectified Linear Unit.
Configuration
Description
Output
CNN-Encoder
Feature Extractor
Input: X
RGB video frames
BS T
64 64
Conv2D
32, BatchNorm22, LeakyReLU
BS x T x
32 x 64 x 64
Conv2D
F: 32. BatchNorm22, LeakyReLU
BS X T x 32 x 32 x 32
Conv2D
F: 32, BatchNorm22, LeakyReLLU
BS T 32
Conv2D
F: 64, BatchNorm22, LeakyReLU
BS X T X 64 x x 8
Conv2D
F: 64, BatchNorm22,, LeakyReLU
BS X T X 64 x 4 4
Conv2D
128, BatchNorm2D, LeakyReLU
BS T 128
Dense
2 z dim = dimension of hidden embedding
BS x T X
Z.
dim
CNN-Decoder
Video Reconstruction
Input: 21:T
Sampled latent variable sequence
T x dim
Dense Conv' TonyTranspose2D C
F: 128 LeakyReLU
BS X T X 128 x X
F: 64, BatchNorm22, LeakyReLU
BS X T X 64 X X 4
64, BatchNorm22, LeakyReLU
T X
x 8
Conv'l Conv/Transpor Transposc2D
F: 32, BatchNorm22, LeakyReLU
BS X T x 32
x 16 x 16
ConyTransponn Tr
F: 32, BatchNorm2D, LeakyReLU
BS T X
32
32
32
TransposnD
32, BatchNorm2D, LeakyReLU
T X
x
X
Conv Transpose2D
F: 3, estimated scene &1.7
BS X T
x
x
64 x 64
22


Reviews:
[{"review_id": "4zxPoNdzv8", "reviewer": "Reviewer_t3u7", "summary": "The study focuses on unsupervised representation learning for sequential data with time-delayed causal influences. Identifiability results for disentangling causally-related latent variables have been established in stationary settings using temporal structure. However, existing work only partially addresses this in nonstationary settings by using observed auxiliary variables or introducing the Markov assumption, but not both simultaneously. The authors introduce NTDC, to reconstruct time-delayed latent causal variables and identify their relations without the need for auxiliary variables, by utilizing deep generative models. The experimental results demonstrate that the proposed methodology outperforms existing baselines by effectively exploiting nonstationarity and distinguishing distribution shifts.", "questions": "\n- Could you provide more motivational examples of the model structure in Figure 1(d) other than the mouse movement example?\n- Are there any derivation with probabilistic graphical model settings of Figure 1(d)? I wonder whether the problem could be solved in a statistical manner rather than just utilizing DGMs.\n- I guess the model structure can be viewed as a doubly-hidden Markov model, or it could be more generalized as a multi-layer hidden Markov model. Could proposed theorems, as well as the proposed NTDC, be generalized in such settings?\n- What's the specific meaning of \"disentangled\" representation in the title as well as the model name? And how the disentangled representation is achieved?\n", "limitations": "- There are no potential negative societal impacts of their work. \n- Even though the authors provided a new work, my concern is that it lacks significance considering the NeurIPS standard.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper is clearly written.\n- The authors tackle the setting of previous works and propose a non-existing work, which is described in Figure 1. \n- The theoretical grounds are sufficiently provided with proper theorems.", "weaknesses": "- There are some grammatical errors in the main paper.\n- There is no explanation of the \"disentangled\" which is included both in the proposed model name and the title of the manuscript.\n- The authors provide their code with anonymity, but there is limited explanations of the codes that which is directly linked to the proposed NTDC and how one can run the code.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful review and valuable feedback. We respond to your concerns point-by-point below.\n> Q1: Could you provide more motivational examples of the model structure in Figure 1(d) other than the mouse movement example?\n\nHere are some examples with explanation:\n1. For general time series data such as stock market data, the underlying dynamics (map $\\mathbf{z}\\_{t-1} \\rightarrow \\mathbf{z}\\_{t}$) changes dramatically across time. Such a nonstationary process can be modeled by Figure 1(d) by choosing different $c_t$.\n2. Videos have similar properties. $c_t$ represents the events in the video. Within the same event, the changing dynamics stay the same, and across different events, the changing dynamics are different.\n\n> Q2: Are there any derivation with probabilistic graphical model settings of Figure 1(d)? I wonder whether the problem could be solved in a statistical manner rather than just utilizing DGMs.\n\nThanks for the question, we are a little bit confused by the terminology here. Do you want to say PGMs instead of DGMs? From our understanding, using PGM doesn't conflicts with the solving the problem in a statistical manner. It is just to show the underlying dependence relations among the variables, and finally the problem is also solved in the statistical manner with the help of PGMs. Please let us know if there is any misinterpretation for this question, we are very happy to provide more explanations during the discussion.\n\n> Q3: I guess the model structure can be viewed as a doubly-hidden Markov model, or it could be more generalized as a multi-layer hidden Markov model. Could proposed theorems, as well as the proposed NTDC, be generalized in such settings?\n\nThanks for your good observation and question. Instead of a doubly-hidden Markov model, our model is more similar to a nonstationary state space model with a HMM component on top of the hidden variables $\\mathbf{z}\\_t$. As for the generalization, we believe the answer is yes, since intuitively by modeling the nonstationarity in the process, we can gradually identify the variables layer by layer. However it is clear that this will involve a lot of nontrivial technical details which we are not completely sure, we will continue working on it in the future work.\n\n> Q4: What's the specific meaning of \"disentangled\" representation in the title as well as the model name? \n\nWhile there is no single formalized notion of disentanglement which is widely accepted, the key intuition is that a disentangled representation should separate the distinct, informative factors of variations in the data. We approach the disentanglement from the nonlinear Independent Component Analysis (ICA) point of view. In our definition, \"disentangled representation\" for time series data means the following things:\n\n1. (Temporally conditional independence) Each dimension of learned $\\mathbf{z}_t$ (i.e. $z\\_{t,1}, z\\_{t,2} \\dots z\\_{t,n}$) are conditional independent given the history $\\mathbf{z}\\_{\\text{Hx}}$.\n2. (Identifiability) The learned representation $\\mathbf{z}_t$ consists with the ground truth $\\mathbf{z}_t$ in the data generating process (up to component-wise transformation and permutation).\n\nThe field ''disentangled representation learning'' has been widely/extensively studied in the past few years, but as pointed out by [1], existing works rely heavily on inductive bias and ''well-disentangled models seemingly cannot be identified without supervision''. That is saying even though existing models can disentangle different factors in an intuitive way, the factors/variables that the model discovered can be far from ground truth, which is not favorable. Finding such ground truth (identifiability) is a further important yet challenging task. In the nonlinear ICA literature, researchers established identifiability result to find the independent components in the data generating process, which is a well-disentangled model with theoretical guarantee.\n\n> Q5: And how the disentangled representation is achieved?\n\nNTDC achieved disentangled representation by performing nonlinear ICA on nonstationary time series data. Specifically, NTDC first learned the nonstationarity of the process and give estimation of the domain indices $c_t$, and then NTDC leveraged the learned $c_t$ to find the independent components $\\mathbf{z}_t$. The identifiability result guarantee that (1) each dimension of the learned $\\mathbf{z}_t$ are conditionally independent which enforces the disentanglement, and (2) the learned $\\mathbf{z}_t$ are equivalent to the ground truth value up to permutation and component-wise transformation, which means that NTDC can really recover the truth.\n\n> There are some grammatical errors in the main paper.\n\nWe thanks the reviewer for pointing out this, we have proofread the manuscript and will update it in the final version.\n\n> The authors provide their code with anonymity, but there is limited explanations of the codes that which is directly linked to the proposed NTDC and how one can run the code.\n\nWe thanks the reviewer for raising this issue, the training python scripts `train_{exp name}.py` are in the root directory of the repository and the our proposed NTDC method is coded with name `hmmxtdrl`. A sample run script for simulation data is provided as follow: `python train_simulation.py -c configs/simulation/simulation_hmmxtdrl.yaml` and sample config files are provided in the `configs` folder.\n\nPlease let us know if you have further concerns and we are happy to provide more detailed information in the discussion phase.\n# References\n\n[1] Locatello, Francesco, et al. \n''Challenging common assumptions in the unsupervised learning of disentangled representations.'' international conference on machine learning. PMLR, 2019.\n"}, {"review_id": "9frzvovr5S", "reviewer": "Reviewer_5DXD", "summary": "This paper presents identifiability results that handle nostationary time-delayed causally-related processes without auxiliary variables. In addition, the authors propose NTDC, which is a neural network that is based on their identifiability results. The method is evaluated on a few tasks in comparison to a few baselines, showing significance of toy tasks generated by the authors.\n", "questions": "See above.", "limitations": "See above.", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "This paper introduces identifiability results on sequential information which is causally-related without additional variables. Thus, this paper extends prior work from a theoretic viewpoint. Another strength of the paper is the new deep framework that incorporates the theoretical results.\n", "weaknesses": "One of the main weaknesses of the paper is its presentation. In particular, a lot of jargon is used without properly introducing the terms. For instance, domain indices are used throughout the paper without ever defining what it means. Similarly, Sec. 4.1 is missing a lot of details (what is ARHMM? what is the prior network?). Sec. 4.2 seems to be unfinished in terms of writing. Missing details and descriptions make it hard to properly evaluate the proposed NTDC and position it with respect to prior work.\n\nAnother main shortcoming of the work is the lack of discussion on related work for sequential disentanglement. Only paper [20] is mentioned (extremely) briefly. However, there are already at least ten papers or more since [20] which significantly improved the state-of-the-art results, the neural network models, the evaluation and theory.\n\nAnother shortcoming of the paper is the dicsussion in the introduction. In particular, the authors argue their mouse example might be challenging for existing work. In addition, they argue to be first to consider scenario Fig. 1c. However, in a recent ICLR'23 paper with the title \"Multifactor Sequential Disentanglement via Structured Koopman Autoencoders\" (SKD) by Berman et al. the authors show a framework that considers a similar model to Fig. 1c, and it is able to handle data such as the mouse example. The authors should compare their work with SKD both qualitatively and quantitatively. Similarly, the work \"Contrastively Disentangled Sequential Variational Audoencoder\" (C-DSVAE) by Bai et al. can also deal with a similar scenario and should be compared with.\n\nAnother shortcoming of the paper is its evaluation. In particular, the proposed method is compared with several baselines only on toy datasets that the authors create. No sequential disentanglement baselines are considered. On real world data, the authors only compare with a single method.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the insightful review and valuable feedback. We respond to your concerns point-by-point below.\n\n> Q1.1: One of the main weaknesses of the paper is its presentation. In particular, a lot of jargon is used without properly introducing the terms. For instance, domain indices are used throughout the paper without ever defining what it means.\n\nDomain indices is the discrete variable that denote different domains. In our model it is $c_t$ which is introduced in Sec 2.1 Eq (2).\n\nWe will add more explanation to the technical terms and if there is any further unclear issues, please kindly let us know. We are more than happy to provide more details during the discussion.\n\n> Q1.2: Similarly, Sec. 4.1 is missing a lot of details (what is ARHMM? what is the prior network?).\n\nWe thank the reviewer for raising this question. ARHMM refers to Autoregressive Hidden Markov Module (which is mentioned in line 193-194), which is a standard abbreviation. Such module is to model the nonstationarity with input of observed data $\\mathbf{x}_t$ and output the estimated domain indices $c_t$. Prior Network (introduced in line 202-204) refers to the module to learn transition priors, such priors are obtained by first learning inverse transition functions $f_z^{-1}$ that take the estimated latent variables and output random noise terms, and applying the change of variables formula to the transformation \n\n$p(\\hat{z}\\_{t}\\vert \\hat{\\mathbf{z}}\\_{\\text{Hx}}, c_t) = p_{\\epsilon_c}\\left(\\hat{f}\\_{z}^{-1}(\\hat{z}\\_{t}, \\hat{\\mathbf{z}}\\_{\\text{Hx}}, \\hat{\\boldsymbol{\\theta}}\\_{c_t})\\right)\\Big|\\frac{\\partial \\hat{f}_z^{-1}}{\\partial \\hat{z}\\_{t}}\\Big|$.\n\nAnd the Encoder-Decoder is a standard module in Variational Auto-Encoder based method which projects the $\\mathbf{x}_t$ to latent space $\\mathbf{z}_t$ while preserving the ability to reconstruct $\\mathbf{x}_t$.\n\nPlease let us know if the information above make it easier for you to understand, we are also happy to add more detailed explanations if you have further questions.\n\n> Q1.3: Sec. 4.2 seems to be unfinished in terms of writing. Missing details and descriptions make it hard to properly evaluate the proposed NTDC and position it with respect to prior work.\n\nWe apologize for accidentally commenting out part of the last sentence in Sec 4.2 in the submitted version. The complete writing is updated as follows: ''Specifically, we obtain the log-likelihood of the posterior, evaluate the prior \n\n$\\log p\\left(\\hat{\\mathbf{z}}_t \\vert \\hat{\\mathbf{z}}\\_{\\text{Hx}}, c_t\\right)$ in Eq. (9), and compute their mean difference in the dataset as the KL loss: \n\n$\\mathcal{L}\\_{\\text{KLD}} = \\mathbb{E}_{\\mathbf{\\hat z}_t \\sim q\\left(\\mathbf{\\hat z}_t \\vert \\mathbf{x}_t\\right)} \\log q(\\mathbf{\\hat z}_t|\\mathbf{x}_t) - \\log p\\left(\\hat{\\mathbf{z}}_t \\vert \\hat{\\mathbf{z}}\\_{\\text{Hx}}, c_t\\right)$.''\n\n> Q2, Q3: ... lack of discussion on related work for sequential disentanglement ... However, in a recent ICLR'23 paper with the title \"Multifactor Sequential Disentanglement via Structured Koopman Autoencoders\" (SKD) by Berman et al. the authors show a framework that considers a similar model to Fig. 1c, and it is able to handle data such as the mouse example. The authors should compare their work with SKD both qualitatively and quantitatively. Similarly, the work \"Contrastively Disentangled Sequential Variational Audoencoder\" (C-DSVAE) by Bai et al. can also deal with a similar scenario and should be compared with.\n\nWe thank the reviewer for pointing out and we thank the reviewer for providing related work with references. We will definitely add the related topics in related work section. As for the relation between ''Sequential Disentanglement'' and our setting, we kindly remind that we would like to investigate the disentanglement from the causal lens which means we care about recovering the ground truth of the data generating process. In other words, there are multiple ways to disentangle the variable and obtain well-disentangled representations, but only a very small subset of them are actually following the ground truth i.e. achieving identifiability. The main contribution for this work is slightly different from just finding a disentangled representation under nonstationary setting. Instead we care more about if we can recover the ground truth and provided the identifiability, hence the Encoder-Decoder module can adopt any current state-of-the-art method such as SKD, and such choice is orthogonal to our contribution.\n\nDespite of the difference mentioned above, we still compared SKD in the CartPole dataset and reported the MCC in the table below. Note, and also as mentioned in SKD paper that ''C-DSVAE'' uses slightly more supervision in data augmentation which requires more knowledge than purely unsupervised learning, for fair comparison we only conducted our additional experiment using SKD. As shown in table below, the MCC for SKD is better than a variety of baselines, however, we can see the distinction between well-disentangled models and identifiable models, only the models with identifiability can find the ground truth latent variables with theoretical guarantee.\n\n| Method | MCC |\n| :-: | - |\n| BetaVAE | 57.54 |\n| i-VAE | 60.14 |\n| TCL | 65.07 |\n| SlowVAE | 63.16 |\n| SKD | 73.24 |\n|NTDC|96.06|\n\n> Q4: Another shortcoming of the paper is its evaluation. In particular, the proposed method is compared with several baselines only on toy datasets that the authors create. No sequential disentanglement baselines are considered. On real world data, the authors only compare with a single method.\n\nWe additionally compared baseline methods together with SKD in the CartPole dataset and showed the result in previous question.\n\nPlease let us know if you have further concerns and we are happy to provide more detailed information in the discussion phase.\n"}, {"review_id": "x6vdC2T5mw", "reviewer": "Reviewer_eHJB", "summary": "This paper gives identifiability results in a new setting, along with an estimation method and experiments validating the estimation method.\n\nThere are existing identifiability results in the nonstationary setting if you assume that the auxiliary variables are observed. There are other existing identifiability results in the nonstationary setting if you assume that the auxiliary variables evolve as a Markov chain but the latents only affect the current timestep. This paper gives an identifiability result in the nonstationary setting where the auxiliary variables are *not* observed and the auxiliary variables evolve as a Markov chain but the latent variables can affect the observed variables across time steps.\n\nThe estimation method is an extension of Sequential VAEs, where there is an autoregressive Hidden Markov Model model that models the nonstationarity, a prior network that estimates the prior using a conditional normalizing flow, and a VAE where the encoder fits the demixing function and the decoder fits the mixing function.\n\nThey perform experiments on two synthetic datasets that satisfy their identifiability conditions, a video dataset for a modified Cartpole environment, and a video dataset of mouse behavior. For the synthetic datasets, they compare MCC with seven baselines (BetaVAE, iVAE, TVL, SlowVAE, PCL, TDRL, and HMNLICA). For the video cartpole dataset, they compare MCC against one baseline (TDRL). For the mouse video dataset they don’t have ground-truth independent components so instead of numerical results they provide a visualization of the recovered components.", "questions": "- In the Cartpole experiments, why do you only provide Accuracy and MSE A metrics for your method and not for baseline methods? And can you provide evaluations for the other baselines for this experiment?\n", "limitations": "- As they mention in the conclusion section, they assume that the nonstationary variables follow a Markov chain, which is a somewhat restrictive assumption.\n- No concerns about negative societal impacts.", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "Strengths:\n- I am not especially up-to-date on identifiability literature, but as far as I am aware this is a novel problem setting and novel result.\n- I agree that the “observed auxiliary variable” condition in many nonlinear ICA identifiability results is not realistic, and it is a useful direction to prove results that do not rely on this assumption.\n- The choice of baselines is very thorough for the synthetic datasets.\n- They use two video datasets to demonstrate their estimation method in a more realistic setting than synthetic data, which I appreciate since this is mostly a theory paper.", "weaknesses": "Weaknesses:\n- The writing is okay but not great. It gets the point across but it could use proofreading and better word choices in some places. Ex: “It is obvious that the causal relations among those latent variables are more meaningful and the identifiability is urgently needed.” <- this sentence could be improved.\n- See questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate that you read our paper very carefully and your informative feedback, which has helped improve our paper. Below please see our response to your concerns.\n\n>Q1: In the Cartpole experiments, why do you only provide Accuracy and MSE A metrics for your method and not for baseline methods? And can you provide evaluations for the other baselines for this experiment?\n\nA1: Baselines either ignore the nonstationarity in the system (completely ignore $c_t$) or require domain indices ($c_t$) to be observed. In both scenarios, baselines don't predict $c_t$ as their output hence it doesn't make sense to evaluate the accuracy for $c_t$ and the transition matrix $\\mathbf{A}$. It worth to mention that for HMNLICA, even though it estimates $c_t$, it doesn't consider the time-delayed relation for latent variables $\\mathbf{z}_t$, for real datasets such as CartPole, the algorithm doesn't converge during training, hence we dropped it in our experiments.\n\n>The writing is okay but not great. It gets the point across but it could use proofreading and better word choices in some places. Ex: “It is obvious that the causal relations among those latent variables are more meaningful and the identifiability is urgently needed.” <- this sentence could be improved.\n\nWe thanks the reviewer for pointing this out, we update the sentence as follows: ''Learning causal relations has practical use case, which benefits a lot of downstream tasks''. We will also proofread the paper and improve the writing in the final version. If you have any further suggestions please let us know in the discussion period. \n"}, {"review_id": "nIu0Gj2iPO", "reviewer": "Reviewer_PKrt", "summary": "This paper addresses the problem of identifying latents representations from sequential data which is stationary within contexts, and non-stationary between contexts. Existing work has either conditioned on observed auxiliary variables that indicate which context one is in (e.g. time contrastive learning and the followups), or they relied on mutually (conditionally) independent latents and did not allow time-delayed latent causal influences. The paper presents an approach called Nonstationary Temporal Disentangled Causal Representation Learning (NTDC)and shows conditions under which they can identify time-delayed latent causal variables and their relations without the need for observed auxiliary variables. They formulate the problem as a discrete Markov process and establish identifiability of the latent independent components. They give impressive experimental results on both synthetic and real-world datasets to demonstrate the proposed method's success in recovering latent variables. ", "questions": "1. Can you give any more intuitive assumptions that are sufficient for Theorem 2 to hold? Or give examples of distributions for which they hold and examples for which they fail? If done well, this would make the theory section stronger.", "limitations": "The paper already does a reasonable job of outlining the limitations of their work.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* Really nice experimental results section. The results on simulated data and I appreciated the section on the MoSeq dataset analyzing mouse behaviour. \n* I think the graphical model that the paper analyses is very practically useful for abstracting time series data. By breaking a time series into conditionally stationary domains, $c_t$, that change more slowly, you gain a more interpretable model of the underlying behaviour of the system (the mouse experiments are nice illustration of this). Empirically, we've seen results similar to this before in the original MoSeq paper, but it is nice to have identifiability results to go with these approaches. ", "weaknesses": "* Assumptions (6) and (7) in Theorem 2 are unintuitive to the point that I'm not sure the theorem is useful at all. The main value of identifiability theorems are to give a set of preconditions on the data under which a practitioner can expect a disentanglement routine would work. But I'm not sure that there are any practitioners who would be able to judge whether those assumptions hold in their data, and there was no attempt to give any intuition for when they might hold and when they would fail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are sincerely grateful to the reviewer for the informative feedback. Please see our point-to-point response below.\n\n> Weaknesses: ... But I'm not sure that there are any practitioners who would be able to judge whether those assumptions hold in their data ...\n\nFirstly we would like to mention that our assumed conditions are generally testable. One can run experiments with multiple seeds and compute the similarity (MCCs) between the learned representations; if the representation is not unique, then our assumptions must be violated.\n\n> Q1: Can you give any more intuitive assumptions that are sufficient for Theorem 2 to hold? Or give examples of distributions for which they hold and examples for which they fail? If done well, this would make the theory section stronger.\n\nThis assumption was first introduced in GCL[1], namely, \"sufficient variability\", to extend the modulated exponential families [2] to general modulated distributions. Essentially, the condition says that the nonstationary regimes $c$ must have a sufficiently complex and diverse effect on the transition distributions. In other words, if the underlying distributions are composed of relatively many domains of data, the condition generally holds true. For instance, in the linear Auto-Regressive (AR) model with Gaussian innovations where only the noise variance changes, the condition reduces to the statement in [3] that the variance of each noise term fluctuates somewhat independently of each other in different nonstationary regimes. Then the condition is easily attained if the variance vector of noise terms in any regime is not a linear combination of variance vectors of noise terms in other regimes.\n\nWe further illustrate the condition using the example of modulated conditional exponential families in [1]. Let the log-pdf $q(\\mathbf{z}_t \\vert \\mathbf{z}\\_{\\text{Hx}}, c)$ be a conditional exponential family distribution of order $k$ given nonstationary regime $c$ and history $\\mathbf{z}\\_{\\text{Hx}}$: \n\n$$q(z_{it} \\vert \\mathbf{z}\\_{\\text{Hx}}, c) = q_i(z_{it}) + \\sum_{j=1}^k q_{ij}(z_{it}) \\lambda_{ij}(\\mathbf{z}\\_{\\text{Hx}}, c) - \\log Z(\\mathbf{z}\\_{\\text{Hx}}, c),$$ \n\nwhere $q_i$ is the base measure, $q_{ij}$ is the function of the sufficient statistic, $\\lambda_{ij}$ is the natural parameter, and $\\log Z$ is the log-partition. Loosely speaking, the sufficient variability holds if the modulation of by $c$ on the conditional  distribution $q(z_{it} \\vert \\mathbf{z}_{\\text{Hx}}, c)$ is not too simple in the following sense:\n\n1. Higher order of $k$ ($k>1$) is required. If $k=1$, the sufficient variability cannot hold;\n2. The modulation impacts $\\lambda_{ij}$ by $c$ must be linearly independent across regimes $c$. The sufficient statistics functions $q_{ij}$ cannot be all linear, i.e., we require higher-order statistics.\n\nFurther details of this example can be found in Appendix B of [1]. In summary, we need the modulation by $c$ to have diverse (i.e., distinct influences) and complex impacts on the underlying data generation process.\n\n# References:\n\n[1] Hyvarinen, Aapo, Hiroaki Sasaki, and Richard Turner. \"Nonlinear ICA using auxiliary variables and generalized contrastive learning.\" The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 2019.\n\n[2] Hyvarinen, Aapo, and Hiroshi Morioka. \"Unsupervised feature extraction by time-contrastive learning and nonlinear ica.\" Advances in neural information processing systems 29 (2016).\n\n[3] Matsuoka, Kiyotoshi, Masahiro Ohoya, and Mitsuru Kawamoto. \"A neural net for blind separation of nonstationary signals.\" Neural networks 8.3 (1995): 411-419."}]

Author rebuttal:

Meta-review: The reviewers were split about this paper and did not come to a consensus: on one hand they appreciated the experimental results and the applicability of the graphical model for which the paper derives identifiability results; on the other they had doubts about (1) its assumptions, (2) comparisons with related work, and (3) the clarity of the writing. After going through the paper and the discussion I have decided to vote to accept because the authors respond convincingly to each of the main concerns. Specifically, for (1) the reviewers argued that the assumptions necessary for Theorem 2 (which derives identifiability for latent variables z), are unintuitive and do not seem testable, making the theorem useless. The authors responded by pointing out that this is equivalent to a sufficient variability assumption made in GCL (Hyvarinen et al., 2019) which is testable. This fully resolves the reviewers' concerns. For (2), the reviewers argued that the authors forgot to compare with the field of sequential disentanglement and should add these comparisons. The authors have added a comparison with one of the sequential disentanglement methods suggested by the reviewers SKD (Berman et al., 2023). Further, they point out that the goal of sequential disentanglement is different as those works are not concerned with identifiability. Finally, they also point out that SKD can be used as the encoder-decoder module within their algorthm. This response convinces me that their approach is complimentary to the sequential disentanglement literature, resolving the point. For (3), the reviewers argue that many points in the paper are unclear: i. the end of Section 4.2 is a half sentence (which we later find out from the authors is missing an entire aligned equation block); ii. grammatical errors; iii. some confusing statments in the abstract including “In other words, existing identifiability results for general sequential data cannot benefit from those dual advantages simultaneously”: the advantages here aren’t clearly identified, I assume the authors mean auxiliary variables and the Markov assumption, but if that’s true then their work doesn’t benefit from both of these advantages either, it completely disregards auxiliary variables. This is itself a good thing, so it’s particularly odd that these additional assumptions are described as “advantages”, to resolve this I would remove this sentence; iv. the definition of sufficient variability in Theorem 2, this needs to be clarified as suggested by reviewer PKrt; v. the Figure 1 caption, the mouse plots (e) and (f) need to be connected to the graphical model (d) in the caption, similar to how this is done in the text. Ultimately, paper clarity (3) is the most serious concern of the paper. However, the authors have convinced me that they are able to fix these concerns in the final version, based on their responses to reviewers. For these reasons I argue for acceptance. Authors: please make sure to carefully incorporate all of the reviewer concerns, particularly w.r.t. clarity into the final version. After you have done so, I strongly recommend you pass through the paper multiple times more to make sure there are no mistakes or omissions, and that the paper has clear examples to demonstrate the most intricate concepts. Once this is done, the paper will make a great contribution to the conference!
Decision: Accept (poster)