Title: Temporally Disentangled Representation Learning under Unknown Nonstationarity
Venue: NeurIPS 2023
Accepted: yes
Keywords: Unsupervised learning

Temporal disentanglement

Nonlinear ICA

Identifiability theory
Abstract: In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-related latent variables have been established in stationary settings by leveraging temporal structure.
However, in nonstationary setting, existing work only partially addressed the problem by either utilizing observed auxiliary variables (e.g., class labels and/or domain indexes) as side information or assuming simplified latent causal dynamics. Both constrain the method to a limited range of scenarios.
In this study, we further explored the Markov Assumption under time-delayed causally related process in nonstationary setting and showed that under mild conditions, the independent latent components can be recovered from their nonlinear mixture up to a permutation and a component-wise transformation, without the observation of auxiliary variables. We then introduce NCTRL, a principled estimation framework, to reconstruct time-delayed latent causal variables and identify their relations from measured sequential data only.
Empirical evaluations demonstrated the reliable identification of time-delayed latent causal influences, with our methodology substantially outperforming existing baselines that fail to exploit the nonstationarity adequately and then, consequently, cannot distinguish distribution shifts.

Temporally Disentangled Representation Learning under Unknown Nonstationarity

Abstract

In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-related latent variables have been established in stationary settings by leveraging temporal structure. However, in nonstationary setting, existing work only partially addressed the problem by either utilizing observed auxiliary variables (e.g., class labels and/or domain indexes) as side-information or assuming simplified latent causal dynamics. Both constrain the method to a limited range of scenarios. In this study, we further explored the Markov Assumption under time-delayed causally related process in nonstationary setting and showed that under mild conditions, the independent latent components can be recovered from their nonlinear mixture up to a permutation and a component-wise transformation, without the observation of auxiliary variables. We then introduce NCTRL , a principled estimation framework, to reconstruct time-delayed latent causal variables and identify their relations from measured sequential data only. Empirical evaluations demonstrated the reliable identification of time-delayed latent causal influences, with our methodology substantially outperforming existing baselines that fail to exploit the nonstationarity adequately and then, consequently, cannot distinguish distribution shifts.

1 Introduction

Causal reasoning for time-series data is a long-lasting yet fundamental task [1-3]. The majority of the studies focus on the temporal causal discovery among observed variables [4-6]. However, in many real-world scenarios, the observed data (e.g., image pixels in videos) instead of having direct causal edges, are generated by some causally related latent temporal processes or confounders. Learning causal relations has practical use cases, which benefit a lot of downstream tasks. However, estimating latent causal structures among those unobserved variables purely from observations without appropriate class of assumptions is an extremely challenging task (i.e. the latent variables are generally not identifiable) [7, 8].

Under the topic of unsupervised representation learning via nonlinear Independent Component Analysis (ICA), some strong identifiability results of the latent variables have been established [9-14] by introducing side information such as class labels and domain indices. Specifically focusing on time-series data, history information is also widely used as the side information for the identifiability of latent processes [15-18]. However, existing studies mainly focused on and derived identifiability results in stationary settings [10, 16] (Fig 1 (a)) or nonstationary settings with explicitly observed domain indices [12, 17, 18] (Fig 1 (b)).

<latexi sh

1_b

64="wF

Yo dU

G

0k y+Cz

N

M

>A

B9X

c

VDJSg

H

T

3qE

vj

7

Q

L

m

W

fR

pI

P

8

/

5K

r

2

Z

n

u

O

<latexi sh

1_b

64="rEN

op q9

5RfcjWPy

dg

>A

B

X

VDLS

F

Z

u

k

w

K

7

T

0M

z

+Q83

n

J

v

O

/

U

H

m

Y

<latexi sh

1_b

64="+L7q30

c

m

w5

f

BXd

S/VQ

>A

D

N

FJ

U

2v

E

g

KIr

Y

j

ZT

k

u

W

9

R

O

G

z

M8

P

y

o

H

C

pn

<latexi sh

1_b

64="RG+5

JUC

o

Qr2

7Yu

f

8

>A

B

X

c

VDLS

N

F

3

vqE

g0

KI

/

j

ZT

k

m

n

9

O

z

M

WP

y

d

H

w

p

<latexi sh

1_b

64="wF

Yo dU

G

0k y+Cz

N

M

>A

B9X

c

VDJSg

H

T

3qE

vj

7

Q

L

m

W

fR

pI

P

8

/

5K

r

2

Z

n

u

O

<latexi sh

1_b

64="rEN

op q9

5RfcjWPy

dg

>A

B

X

VDLS

F

Z

u

k

w

K

7

T

0M

z

+Q83

n

J

v

O

/

U

H

m

Y

<latexi sh

1_b

64="V

zuDw

Z8S

T

EP

QB

X

>A

7H

c

N

9L

0n

5M

v

YGy

W

R

O

FK

k

g

2

p

f

d

o

j

I

C

+

m

<latexi sh

1_b

64="+L7q30

c

m

w5

f

BXd

S/VQ

>A

D

N

FJ

U

2v

E

g

KIr

Y

j

ZT

k

u

W

9

R

O

G

z

M8

P

y

o

H

C

pn

<latexi sh

1_b

64="XcqSABI

8

v9

f+Rn0

N2

Qw

>

7

VDL

g

EOy

r

UY

oM

CGF

J

k

j

P

Z

mz

W

H

/

5p

d

T

u

K

3

<latexi sh

1_b

64="RG+5

JUC

o

Qr2

7Yu

f

8

>A

B

X

c

VDLS

N

F

3

vqE

g0

KI

/

j

ZT

k

m

n

9

O

z

M

WP

y

d

H

w

p

<latexi sh

1_b

64="wF

Yo dU

G

0k y+Cz

N

M

>A

B9X

c

VDJSg

H

T

3qE

vj

7

Q

L

m

W

fR

pI

P

8

/

5K

r

2

Z

n

u

O

<latexi sh

1_b

64="rEN

op q9

5RfcjWPy

dg

>A

B

X

VDLS

F

Z

u

k

w

K

7

T

0M

z

+Q83

n

J

v

O

/

U

H

m

Y

<latexi sh

1_b

64="V

zuDw

Z8S

T

EP

QB

X

>A

7H

c

N

9L

0n

5M

v

YGy

W

R

O

FK

k

g

2

p

f

d

o

j

I

C

+

m

<latexi sh

1_b

64="+L7q30

c

m

w5

f

BXd

S/VQ

>A

D

N

FJ

U

2v

E

g

KIr

Y

j

ZT

k

u

W

9

R

O

G

z

M8

P

y

o

H

C

pn

<latexi sh

1_b

64="XcqSABI

8

v9

f+Rn0

N2

Qw

>

7

VDL

g

EOy

r

UY

oM

CGF

J

k

j

P

Z

mz

W

H

/

5p

d

T

u

K

3

<latexi sh

1_b

64="RG+5

JUC

o

Qr2

7Yu

f

8

>A

B

X

c

VDLS

N

F

3

vqE

g0

KI

/

j

ZT

k

m

n

9

O

z

M

WP

y

d

H

w

p

<latexi sh

1_b

64="wF

Yo dU

G

0k y+Cz

N

M

>A

B9X

c

VDJSg

H

T

3qE

vj

7

Q

L

m

W

fR

pI

P

8

/

5K

r

2

Z

n

u

O

<latexi sh

1_b

64="rEN

op q9

5RfcjWPy

dg

>A

B

X

VDLS

F

Z

u

k

w

K

7

T

0M

z

+Q83

n

J

v

O

/

U

H

m

Y

<latexi sh

1_b

64="V

zuDw

Z8S

T

EP

QB

X

>A

7H

c

N

9L

0n

5M

v

YGy

W

R

O

FK

k

g

2

p

f

d

o

j

I

C

+

m

<latexi sh

1_b

64="+L7q30

c

m

w5

f

BXd

S/VQ

>A

D

N

FJ

U

2v

E

g

KIr

Y

j

ZT

k

u

W

9

R

O

G

z

M8

P

y

o

H

C

pn

<latexi sh

1_b

64="XcqSABI

8

v9

f+Rn0

N2

Qw

>

7

VDL

g

EOy

r

UY

oM

CGF

J

k

j

P

Z

mz

W

H

/

5p

d

T

u

K

3

<latexi sh

1_b

64="RG+5

JUC

o

Qr2

7Yu

f

8

>A

B

X

c

VDLS

N

F

3

vqE

g0

KI

/

j

ZT

k

m

n

9

O

z

M

WP

y

d

H

w

p

c

t

+1

z

t

+1

x

t

+1

x

t

z

t

c

t

c

t

+1

z

t

+1

x

t

+1

x

t

z

t

c

t

z

t

+1

x

t

+1

x

t

z

t

(a)

(b)

(c)

c

t

+1

z

t

+1

x

t

+1

x

t

z

t

c

t

(d)

One can immediately tell the infeasibility of those two scenarios that general time-series data is usually nonstationary and the side information (class labels and domain indices) is usually unobserved. That is particularly true when considering real-world data such as video or signal sequences. It doesn't make any sense to assume that there exists a stationary transition function that is applied to the whole video clip. Take a very simple video clip of a mouse 2 [19] as an example, it is fairly clear that such a simple motion example can be divided into at least two phases (1) active phase in which the mouse is moving and (2) inactive phase in which the mouse is laying down. Instead of using a complex transition function to describe the whole video clip, a more reasonable assumption is that the same transition function is shared within the same phase, but across different phases, the transition functions are different, in other words, the transition function can be expressed as a function of the domain index. Also, it is worth mentioning that if such domain or phase indices is latent or unobserved, then we cannot directly utilize the existing framework to learn the latent causal dynamics. That is again a more realistic case that in general, the domain indices within a video are not accessible without expensive human annotation.

Recently, HMNLICA [14] attempted to resolve the problem by introducing Markov Assumption on the nonstationary discrete domain variable, they assumed the domain indices follow a first-order Markov Chain and estimated the domain information purely from observed data. However, HMNLICA assumes temporally mutually independent sources in the data-generating process (conditioning on domain indices), i.e. they don't allow latent variables to have time-delayed causal relations in between (Fig 1 (c)). Such an assumption imposed a huge negative impact on the usability of those methods. Considering the video of the little mouse example, the x t s are the observed video frames, z t s can be the independent motion dynamics or causal process such as position, velocity, (angular) momentum, etc, and c t s are the phases or actions such as standing up (active) and laying down (inactive). To accommodate for such general sequential data, time-delayed temporal dependence should be considered in the latent z t space (Fig 1 (d)), otherwise, it is impossible to model a complex video data's temporal relation purely from discrete, domain indices. Also to make sure that the latent independent components can be recovered, temporally conditional independence should also be enforced, i.e. Each dimension of z t is conditionally independent given the history z history. To this end, a natural question is:

How can we establish identifiability of nonlinear ICA for general sequential data with nonstationary causally-related process without observing auxiliary variable?

To answer this question, we first formulate the latent nonstationary states as a discrete Markov process and further explore the Markov Assumption [20] which is introduced for identifiability of nonlinear ICA in HMNLICA [14] and provided stronger identifiability result corresponding to the conditional emission distribution (i.e. the transition function of different domains) and the transition matrix of

the Markov process. Specifically, we generalized the identifiability of Hidden Markov Models in [20] to accommodate time-delayed causally-related non-parametric transitions in latent space (Thm. 1). Then we utilize the linear independence (Thm. 2) to further establish the identifiability of z t .

The main contributions of this work can be summarized as follows:

To our best knowledge, this is the first identifiability result that can handle the nonstationary time-delayed causally-related latent temporal processes without the auxiliary variable. We formulate the problem, especially the nonstationary states into the Markov process, establish identifiability purely from observed data, and then show strong identifiability of latent independent components.

We present NCTRL , Nonstationary Causal Temporal Representation Learning, a principled framework to recover time-delayed latent causal variables and identify their relations from measured sequential data under unobserved different distribution shifts.

Experiments on both synthetic and real-world datasets demonstrate the effectiveness of the proposed method in recovering the latent variables.

2 Problem Formulation

2.1 Time Series Generative Model

Assume we observe n -dimensional time-series data at discrete time steps, X = { x 1 , x 2 , . . . , x T } , where each x t ∈ X is generated from time-delayed causally related hidden components z t ∈ R n by the invertible mixing function:

In addition to latent components z t , there is an extra hidden variable c t which is discrete with cardinality | c t | = C , it follows a first-order Markov process controlled by a C × C transition matrix A , in which the i, j -th entry A i,j is the probability to transit from state i to j .

For i ∈ { 1 , . . . , n } , z it , as the i -th component of z t , is generated by (some) components of history information z t -1 , discrete nonstationary indicator c t , and noise ϵ it .

where Pa ( z it ) is the set of latent factors that directly cause z it , which can be any subset of z Hx = { z t -1 , z t -2 , . . . , z t -L } up to history information maximum lag L . The components of z t are mutually independent conditional on z Hx and c t .

2.2 Identifiability of Latent Causal Processes and Time-Delayed Latent Causal Relations

We define the identifiability of time-delayed latent causal processes in the representation function space in Definition 1 . Furthermore, if the estimated latent processes can be identified at least up to permutation and component-wise invertible nonlinearities, the latent causal relations are also immediately identifiable because conditional independence relations fully characterize time-delayed causal relations in a time-delayed causally sufficient system, in which there are no latent causal confounders in the (latent) causal processes. Note that invertible component-wise transformations on latent causal processes do not change their conditional independence relations.

Definition 1 (Identifiable Latent Causal Processes) . Formally let X = { x 1 , x 2 , . . . , x T } be a sequence of observed variables generated by the true temporally causal latent processes specified by ( f i , p ( ϵ i ) , A , g ) given in Eqs. (1) , (2) , and (3) . A learned generative model ( ˆ f i , ˆ p ( ϵ i ) , ˆ A , ˆ g ) is observationally equivalent to ( f i , p ( ϵ i ) , A , g ) if the model distribution p ˆ g, ˆ p ϵ , ˆ A , ˆ g ( { x 1 , x 2 , . . . , x T } ) matches the data distribution p f i ,p ϵ , A , g ( { x 1 , x 2 , . . . , x T } ) everywhere. We say latent causal processes are identifiable if observational equivalence can lead to identifiability of the latent variables up to permutation π and component-wise invertible transformation T :

where X is the observation space.

3 Identifiability Theory

In this section, we showed that under mild conditions, the latent variable z t is identifiable up to permutation and a component-wise transformation. The theoretical results can be divided into two parts (1) identifiability of the nonstationarity and (2) identifiability of the independent components. As introduced above, the major challenge comes from the unobserved domain indices or nonstationary indicators ( c t in our graphic models). We first establish the identifiability of the different conditional distributions from the observed data and then show that the latent variables z are identifiable. The complete proofs can be found in Appendix A.

3.1 Identifiability of Nonstationary Hidden States

Gassiat et al.[20] showed that the conditional emission distributions in Hidden Markov Models and the transition matrix are identifiable up to label swapping. We first generalize it to the autoregressive setting to accommodate for the time-delayed causal relation, i.e. we showed the identifiability of conditional emission distributions p ( x t | x t -1 , c ) .

Theorem 1. (identifiability of the nonstationarity with Markov Assumptions) Suppose the observed data is generated following the nonlinear ICA framework as defined in Eqs. (1) , (2) and (3) . Suppose the following assumptions (Markov Assumptions) hold:

i For the Markov process, the number of latent states, C , is known.

ii The transition matrix A is full rank.

Use µ 1 , . . . , µ C ∈ R n to denote nonparametric probability distributions of the C emission distributions µ c = p ( x t | x t -1 , c ) . Then the parameters A and M = ( µ 1 , . . . , µ C ) are identifiable given the distribution, P (4) A ,M , of at least 4 consecutive observations x t , x t +1 , x t +2 , x t +3 , up to label swapping of the hidden states, that is:

If ˜ A is a C × C transition matrix and if ˜ π ( c ) is a stationary distribution of ˜ A with ˜ π ( c ) > 0 ∀ c ∈ { 1 , . . . , C } , and if ˜ M = (˜ µ 1 , . . . , ˜ µ C ) are C probability distributions on R n that verify the equality of the distribution functions P (4) ˜ A , ˜ M = P (4) A ,M , then there exists a permutation σ of the set { 1 , . . . , C } such that for all k, l = 1 , . . . , C we have ˜ A k,l = A σ ( k ) ,σ ( l ) and ˜ µ k = µ σ ( k ) .

For notational simplicity, and without loss of generality, we can assume the components are ordered such that c = σ ( c ) . That leads us to the identifiability of the nonstationarity in the system i.e. up to label swapping of the hidden states, the conditional emission distributions p ( x t | x t -1 , c t ) and transition matrix A are identifiable, hence providing us a bridge to further leverage the temporal independence condition in the latent space to establish the identifiability result for demixing function or in other words the latent variables z t .

3.2 Identifiability of Latent Causal Processes

To incorporate nonlinear ICA into the Markov Assumption we define the emission distribution p ( x t | x t -1 , c ) as a deep latent variable model. First, the latent independent component variables z t ∈ R n are generated from a factorial prior, given the hidden state c t and previous z t -1 , as

Second, the observed data x t is generated by a nonlinear mixing function as in Eq. (1) which is assumed to be bijective with inverse given by z t = g ( x t ) . Let η kt ( c t ) ≜ log p ( z kt | z t -1 , c t ) , and assume that η kt ( c t ) is twice differentiable in z kt and is differentiable in z l,t -1 , l = 1 , 2 , ..., n . Note that the parents of z kt may be only c t and a subset of z t -1 ; if z l,t -1 is not a parent of z kt , then ∂η k ∂z l,t -1 = 0 .

Theorem 2. (identifiability of the independent components) Suppose there exists an invertible function ˆ g -1 , which is the estimated demixing function that maps x t to ˆ z t , i.e.,

such that the components of ˆ z t are mutually independent conditional on ˆ z t -1 . Let

And

If for each value of z t , s 1 t , ˚ s 1 t , v 2 t , ˚ s 2 t , ..., s nt , ˚ s nt , as 2 n function vectors s k,t and ˚ s k,t , with k = 1 , 2 , ..., n , are linearly independent, then ˆ z t must be an invertible, component-wise transformation of a permuted version of z t .

So far, the identifiability result has been established without observing the nonstationarity indicators such as domain indices. In the next section, a novel Variational Auto-Encoder based method is introduced to estimate the demixing function ˆ g -1 .

4 NCTRL : Nonstationary Causal Temporal Representation Learning

In this section, we present the details of NCTRL to estimate the latent causal processes under unobserved nonstationary distribution shift, given the identifiability results in Sec 3. First, we show that our framework includes three modules, Autoregressive Hidden Markov Module, Prior Network, and Encoder-Decoder Module. Then, we provide the optimization objective of our model training including an HMM free energy lower bound, a reconstruction likelihood loss, and a KL divergence.

4.1 Model Architecture

Our framework extends Sequential Variational Auto-Encoders [21] with tailored modules to model nonstationarity, and enforces the conditions in Sec. 3 as constraints. We give the estimation procedure of the latent causal dynamics model in Eq. (3). The model architecture is showcased in Fig. 2. The framework has three major components (1) Autoregressive Hidden Markov Module (ARHMM), (2) Prior Network Module, and (3) Encoder-Decoder Module.

Autoregressive Hidden Markov Module (ARHMM) The first component of our framework is ARHMMwhich deals with the nonstationarity with unobserved domains. As discussed in Thm 1, the transition function or the conditional emission distributions across different domains together with the Markov transition matrix A are identifiable. This module estimates the transition function of different domains p ( x t | x t -1 , c t ) and the transition matrix A of the Markov process, and ultimately decodes the optimal domain indices { ˆ c 1 , ˆ c 2 , . . . , ˆ c T } via the Viterbi algorithm.

Prior Network Module To better estimate the prior distribution p (ˆ z t | ˆ z Hx , c t ) , let z Hx denote the lagged latent variables up to maximum time lag L . We evaluate p (ˆ z t | ˆ z Hx , c t ) = p ϵ ( ˆ f -1 z (ˆ z t , ˆ z Hx , ˆ θ c t ) )∣ ∣ ∣ ∂ ˆ f -1 z ∂ ˆ z t ∣ ∣ ∣ by learning a holistic inverse dynamics ˆ f -1 z that takes the estimated change factors for dynamics ˆ θ c t as inputs. Conditional independence of the estimated latent variables p (ˆ z t | ˆ z Hx ) is enforced by summing up all estimated component densities when obtaining the joint p ( z t | z Hx , c t ) in Eq. 9. Given that the Jacobian is lower-triangular, we can compute its determinant as

the product of diagonal terms. The detailed derivations are given in Appendix B.2.

Encoder-Decoder Module The third component is a Variational Auto-Encoder based module which utilizes reconstruction loss to enforce the invertibility of learned mixing function ˆ g . Specifically, the encoder fits the demixing function ˆ g -1 and the decoder fits the mixing function ˆ g . The implementation details are in Appendix B.

4.2 Optimization

The first training objective of NCTRL is to maximize the Log-likelihood of the observed data:

where θ HMM represents the HMM training parameters. Then the free energy lower bound can be defined as:

Consistent with the theory part, the first training objective is to maximize data log-likelihood in the ARHMMmodule to get optimal q ( c ⋆ ) .

which can easily be computed by the Forward-Backward algorithm and luckily all of it is differentiable to the HMM training parameters θ HMM(transition matrix A and transition function parameters θ f ).

Then the second part is to maximize the Evidence Lower BOund (ELBO) for the VAE framework, which can be written as (complete derivation steps are in Appendix B.3):

We use mean-squared error (MSE) for the reconstruction likelihood loss L Recon. The KL divergence L KLD is estimated via a sampling approach since with a learned nonparametric transition prior, the distribution does not have an explicit form. Specifically, we obtain the log-likelihood of the posterior, evaluate the prior log p (ˆ z t | ˆ z Hx , c t ) in Eq. (9), and compute their mean difference in the dataset as the KL loss: L KLD = E ˆ z t ∼ q ( ˆ z t | x t ) log q ( ˆ z t | x t ) -log p (ˆ z t | ˆ z Hx , c t ) .

5 Experiments

We evaluate the identifiability results of NCTRL on a number of simulated and real-world temporal datasets. We first introduce the evaluation metrics and baselines and then discuss the datasets we used in our experiments. Lastly, we show the experiment results discuss the performance, and make comparisons.

5.1 Evaluation Metrics

Mean Correlation Coefficient (MCC) To evaluate the identifiability of the latent variables, we compute the Mean Correlation Coefficient (MCC) on the test dataset. MCC is a standard metric in the ICA literature for continuous variables which measure the identifiability of the learned latent causal processes. MCC is close to 1.0 when latent variables are identifiable up to permutation and component-wise invertible transformation in the noiseless case.

Mean Square Error (MSE) for estimating A As introduced in the theory, the A is identifiable in our setting, which means that our proposed method can provide accurate estimation for the transition matrix A , to valid such a claim, we use mean square error (MSE) to capture the distance between the estimated ˆ A and ground truth A .

Accuracy for estimating c t We also test the accuracy for estimating the discrete domain indices c t supplementary to the MSE for A since in theory, the A is identifiable but the c t is generally not identifiable, which is relatively easy to understand as an analogy in Hidden Markov Models, the transition matrix is identifiable but we can only 'infer' the best possible discrete variables but cannot establish identifiability for it.

It is also worth mentioning that the MSE and Accuracy are influenced by the permutation, which is also true in clustering evaluation problems. Here we explored all permutations and selected the best possible assignment for evaluation.

5.2 Baselines

The following identifiable nonlinear ICA methods are used: (1) BetaVAE [22] which ignores both history and nonstationarity information. (2) i-V AE [12] and TCL [9] which leverage nonstationarity to establish identifiability but assume independent factors. (3) SlowV AE [16], and PCL [10] which exploit temporal constraints but assume independent sources and stationary processes. (4) TDRL [18] which assumes nonstationary, causal processes but with observed domain indices. (5) HMNLICA [14] which considers the unobserved nonstationary part in the data generation process but doesn't allow any causally related time-delayed relations.

5.3 Simulated Results

We generate two synthetic datasets corresponding to different complexity of the nonlinear mixing function g . Both synthetic datasets satisfy our identifiability conditions in the theorems following the procedures in Appendix B.4. As in Table 1, NCTRL can recover the latent processes under unknown nonstationary distribution shifts with high MCCs (>0.95). The baselines that do not exploit history (i.e., BetaV AE, i-V AE, TCL), with independent source assumptions (SlowVAE, PCL), consider limited nonstationary cases (TDRL) distort the identifiability results. The only baseline that considers the unknown nonstationarity in the domain indices (HMN-

LICA) explored the Markov Assumption but doesn't allow a time-delayed causal process and hence suffers a poor result (MCC 0.58).

On the other hand, the difference between dataset A and dataset B is the nonlinearity in the mixing function, dataset A has a relatively simple nonlinear mixing function, on the contrary, dataset B has more complex nonlinearity. Some variability has been observed among the relative performance ranks of different baselines. For example, i-VAE showed a great discrepancy between the two datasets, which revived the weakness of capturing complex nonlinearity in the unknown nonstationary distribution shift environments. Again we also observed that our proposed method can constantly recover the latent independent components with high MCC which indicates on both datasets the model is identifiable and the estimation algorithm is highly effective. To further validate if NCTRL successfully recovered the Markov transition matrix A and inferred the domain indices c t with high accuracy. We further examine the accuracy for estimating nonstationary domain indices c t and the mean square error estimating the transition matrix A . As shown in Table 2 the result is consistent with our theory in which the transition matrix A is identifiable and we can estimate it with

high accuracy. For the nonstationary domain indices c t even though there is no identifiability result governing the estimation accuracy, it can still be inferred pretty well since it is nothing but a decoding problem in Hidden Markov Models.

5.4 Real-world Applications

Video data - Modified CartPole Environment We evaluate NCTRL on the modified CartPole [23] video dataset and compare the performances with the baselines. Modified Cartpole is a nonlinear dynamical system with cart positions x t and pole angles θ t as the true state variables. The dataset descriptions are in Appendix B.5. Similar to the synthetic dataset, we randomly initialize a Markov chain and roll out a series of c t , and configure the CartPole environment with respect to the c t . Specifically, we use five domains with different configurations of cart mass, pole mass, gravity, and noise levels. Together with the two discrete actions (i.e., left and right). By doing so, the nonstationarity is enforced, and since we can control and access all intermediate states in the system, all metrics including MCC and c t accuracy together with A MSE can be easily calculated. We fit NCTRL with two-dimensional causal factors. We set the latent size n = 2 and the lag number L = 2 . In Fig. 3, the latent causal processes are recovered, as seen from (a) high MCC for the latent causal processes; (b) the latent factors are estimated up to component-wise transformation; and (c) the latent traversals confirm the two recovered latent variables correspond to the position and pole angle.

Cart Position

Pole Angle

True Latents

Estimated Latents

(a)

(b)

(c)

MCC=0.951

Estimated Latents

0

0.97

0.29

0.14

0.93

0

1

TrueLatents

Similar to Table 1 and 2, we compare our NCTRL with baseline methods. In addition, we also compare with SKD [24], a state-of-the-art sequential disentangle representation learning method without identifiability guarantee. In Table 3 and 4 we can see that compared with TDRL, our NCTRL can recover the latent processes under unknown nonstationary distribution shifts with high MCCs (>0.95) with highly accurate estimated transition matrix A and high quality inferred c t . Specifically, by comparing the result of SKD, the MCC for SKD is better than a variety of baselines,

however, we can see the distinction between well-disentangled models and identifiable models, only the models with identifiability can find the ground truth latent variables with theoretical guarantee.

Video data - MoSeq Dataset Wetest NCTRL framework to analyze mouse behavior video data from Wiltschko et al. [19], which represents the original application to clustering mouse behavior 3 , details of this dataset are available in Appendix B.6. Since there are no ground truth independent components in this particular real-world dataset, we analyze it by several visualizations to see if different domains can be properly identified and if the patterns in the recovered independent components are consistent with the recovered domain indices. We analyze the first video clip of mouse behavior data and visualize the two phases we discovered and segmented in Fig 4. We can clearly see from Fig 4 that there are different phases with the upper one actively moving and the lower one inactive. The recovered independent components showed a consistent pattern with the recovered phase or domain indices as shown in Fig 4.

Independent Components

Active

Inactive

Time [ms]

Inactive

Active

D

8

5000

10000

15000

6 Related Work

Causal Discovery from Time Series Understanding the causal structure in time-series data is pivotal in areas such as machine learning [1], econometrics [2], and neuroscience [3]. A bulk of the research in this realm emphasizes determining the temporal causal links among observed variables. The primary techniques employed are constraint-based methods [25], which use conditional independence tests to ascertain causal structures, and score-based methods [26, 27], where scores are utilized to oversee a search operation. Some researchers also proposed a combination of these two methods [28, 29]. Additionally, Granger causality [30] and its nonlinear adaptations [31, 32] have gained widespread acceptance in this context.

Nonlinear ICA for Time Series Recently, the significance of temporal structures and nonstationarities has been recognized in achieving identifiability within nonlinear ICA. Time-contrastive learning (TCL [9]) utilizes the independent sources principle, focusing on data segments' variability. On the other hand, Permutation-based contrastive (PCL [10]) offers a learning approach that distinguishes true independent sources from shuffled ones under the uniformly dependent assumption. HMNLICA [14] integrates nonlinear ICA with an HMM to address non-stationarity without segmenting data manually. The i-VAE [12] approach employs VAEs to capture the actual joint distribution between observed and auxiliary non-stationary domains, assuming an exponential families conditional distribution. The recent advancements in nonlinear ICA for time series include LEAP [17], (i-)CITRIS [33, 34], and TDRL [18]. While LEAP introduces a novel condition emphasizing nonstationary noise, TDRL delves deeper into a non-parametric environment within a nonstationary context. In contrast, CITRIS recommends utilizing intervention target data for pinpointing latent causal aspects, avoiding certain constraints but necessitating active intervention access.

Sequential Disentanglement Majority of existing work about sequential disentanglement focuses on architecture based on dynamical variational autoencoder (VAE) [35]. Early works [36, 37] separate dynamic factors from static factors using probabilistic methods. Then auxiliary tasks with self-supervisory signals [38] were introduced. C-DSVAE [39] utilized contrastive penalty terms with data augmentation to introduce additional inductive biases. In R-WAE [40], Wasserstein distance was introduced to replace KL divergence. To deal with video disentanglement [41, 42] explored generative adversarial network (GAN) architectures and [43] introduced a recurrent model with adversarial loss. FA V AE, [44] proposed a factorizing V AE and [45] proposed to learn hierarchical features. Finally, SKD [24] introduced a spectral loss term that leads to structured Koopman matrices and disentanglement.

7 Conclusion and Discussion

Conclusion. In this paper, we first established an identifiability theory for general sequential data with nonstationary causally-related processes under unknown distribution shifts. Then we presented NCTRL , a principled framework to recover the time-delayed latent causal variable identify their causal relations from measured data, and decode high-quality domain indices under Markov assumption. Experiment results on both synthetic datasets and real-world video datasets showed that our proposed method can recover the latent causal variables and their causal relations purely from measured data with the observation of auxiliary variables or domain indices.

Limitation. The basic limitation of this work is that the nonstationary domain indices are assumed to follow a Markov chain. Also, this work highly relies on the latent processes to have no instantaneous causal relations but only time-delayed influences. If the resolution of the time series is much lower, then it is usually violated and one has to find a way to deal with instantaneous causal relations. Extending our theories and framework to address the scenarios when more flexibility in the domain indices transition is allowed (i.e. beyond discrete variables following a Markov chain) and to address instantaneous dependency or instantaneous causal relations will be some of our future work.

Boarder Impacts. This work proposes a theoretical analysis and technical methods to learn the causal representation from time-series data, which facilitate the construction of more transparent and interpretable models to understand the causal effect in the real world. This could be beneficial in a variety of sectors, including healthcare, finance, and technology. In contrast, misinterpretations of causal relationships could also have significant negative implications in these fields, which must be carefully done to avoid unfair or biased predictions.

8 Acknowledgment

This project has been graciously funded by NGA HM04762010002, NSF IIS1955532, NSF CNS2008248, NIGMS R01GM140467, NSF IIS2123952, NSF BCS2040381, an Amazon Research Award, NSF IIS2311990, and DARPA ECOLE HR00112390063. This project is also partially supported by NSF Grant 2229881, the National Institutes of Health (NIH) under Contract R01HL159805, a grant from Apple Inc., a grant from KDDI Research Inc., and generous gifts from Salesforce Inc., Microsoft Research, and Amazon Research.

References

Carlo Berzuini, Philip Dawid, and Luisa Bernardinell. Causality: Statistical perspectives and applications . John Wiley & Sons, 2012.

Eric Ghysels, Jonathan B Hill, and Kaiji Motegi. Testing for granger causality with mixed frequency data. Journal of Econometrics , 192(1):207-230, 2016.

Karl Friston. Causal modelling and brain connectivity in functional magnetic resonance imaging. PLoS biology , 7(2):e1000033, 2009.

Clive WJ Granger. Testing for causality: A personal viewpoint. Journal of Economic Dynamics and control , 2:329-352, 1980.

Mingming Gong, Kun Zhang, Bernhard Schoelkopf, Dacheng Tao, and Philipp Geiger. Discovering temporal causal relations from subsampled data. In International Conference on Machine Learning , pages 1898-1906. PMLR, 2015.

Aapo Hyvärinen, Kun Zhang, Shohei Shimizu, and Patrik O Hoyer. Estimation of a structural vector autoregression model using non-gaussianity. Journal of Machine Learning Research , 11 (5), 2010.

Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentangled representations. In international conference on machine learning , pages 41144124. PMLR, 2019.

Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks , 12(3):429-439, 1999.

Aapo Hyvarinen and Hiroshi Morioka. Unsupervised feature extraction by time-contrastive learning and nonlinear ica. Advances in Neural Information Processing Systems , 29:3765-3773, 2016.

Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ica of temporally dependent stationary sources. In Artificial Intelligence and Statistics , pages 460-469. PMLR, 2017.

Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 859-868. PMLR, 2019.

Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. In International Conference on Artificial Intelligence and Statistics , pages 2207-2217. PMLR, 2020.

Peter Sorrenson, Carsten Rother, and Ullrich Köthe. Disentanglement by nonlinear ica with general incompressible-flow networks (gin). arXiv preprint arXiv:2001.04872 , 2020.

Hermanni Hälvä and Aapo Hyvarinen. Hidden markov nonlinear ica: Unsupervised learning from nonstationary time series. In Conference on Uncertainty in Artificial Intelligence , pages 939-948. PMLR, 2020.

Hermanni Hälvä, Sylvain Le Corff, Luc Lehéricy, Jonathan So, Yongjie Zhu, Elisabeth Gassiat, and Aapo Hyvarinen. Disentangling identifiable features from noisy data with structured nonlinear ica. arXiv preprint arXiv:2106.09620 , 2021.

David Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan Paiton. Towards nonlinear disentanglement in natural data with temporal sparse coding. arXiv preprint arXiv:2007.10930 , 2020.

Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning temporally causal latent processes from general temporal data. In International Conference on Learning Representations , 2022. URL https://openreview.net/forum?id=RDlLMjLJXdq .

Weiran Yao, Guangyi Chen, and Kun Zhang. Temporally disentangled representation learning. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems , 2022. URL https://openreview.net/forum?id= Vi-sZWNA_Ue .

Alexander B. Wiltschko, Matthew J. Johnson, Giuliano Iurilli, Ralph E. Peterson, Jesse M. Katon, Stan L. Pashkovski, Victoria E. Abraira, Ryan P. Adams, and Sandeep Robert Datta. Mapping sub-second structure in mouse behavior. Neuron , 88(6):1121-1135, 2015. ISSN 0896-6273. doi: https://doi.org/10.1016/j.neuron.2015.11.031. URL https://www.sciencedirect. com/science/article/pii/S0896627315010375 .

Élisabeth Gassiat, Alice Cleynen, and Stephane Robin. Inference in finite state space non parametric hidden markov models and applications. Statistics and Computing , 26:61-71, 2016.

Li Yingzhen and Stephan Mandt. Disentangled sequential autoencoder. In International Conference on Machine Learning , pages 5670-5679. PMLR, 2018.

Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum?id=Sy2fzU9gl .

Biwei Huang, Fan Feng, Chaochao Lu, Sara Magliacane, and Kun Zhang. Adarl: What, where, and how to adapt in transfer reinforcement learning. arXiv preprint arXiv:2107.02729 , 2021.

Nimrod Berman, Ilan Naiman, and Omri Azencot. Multifactor sequential disentanglement via structured koopman autoencoders. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=6fuPIe9tbnC .

Doris Entner and Patrik O Hoyer. On causal discovery from time series data using fci. Probabilistic graphical models , pages 121-128, 2010.

Kevin P Murphy et al. Dynamic bayesian networks. Probabilistic Graphical Models, M. Jordan , 7:431, 2002.

Roxana Pamfil, Nisara Sriwattanaworachai, Shaan Desai, Philip Pilgerstorfer, Konstantinos Georgatzis, Paul Beaumont, and Bryon Aragam. Dynotears: Structure learning from time-series data. In International Conference on Artificial Intelligence and Statistics , pages 1595-1605. PMLR, 2020.

Daniel Malinsky and Peter Spirtes. Causal structure learning from multivariate time series in settings with unmeasured confounding. In Proceedings of 2018 ACM SIGKDD Workshop on Causal Discovery , pages 23-47. PMLR, 2018.

Daniel Malinsky and Peter Spirtes. Learning the structure of a nonstationary vector autoregression. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 2986-2994. PMLR, 2019.

Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society , pages 424-438, 1969.

Alex Tank, Ian Covert, Nicholas Foti, Ali Shojaie, and Emily Fox. Neural granger causality. arXiv preprint arXiv:1802.05842 , 2018.

Sindy Löwe, David Madras, Richard Zemel, and Max Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv preprint arXiv:2006.10833 , 2020.

Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Stratis Gavves. Citris: Causal identifiability from temporal intervened sequences. In International Conference on Machine Learning , pages 13557-13603. PMLR, 2022.

Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M Asano, Taco Cohen, and Efstratios Gavves. icitris: Causal representation learning for instantaneous temporal effects. arXiv preprint arXiv:2206.06169 , 2022.

Laurent Girin, Simon Leglaive, Xiaoyu Bie, Julien Diard, Thomas Hueber, and Xavier AlamedaPineda. Dynamical Variational Autoencoders: A Comprehensive Review . 2021.

Wei-Ning Hsu, Yu Zhang, and James Glass. Unsupervised learning of disentangled and interpretable representations from sequential data. Advances in neural information processing systems , 30, 2017.

Yingzhen Li and Stephan Mandt. Disentangled sequential autoencoder. arXiv preprint arXiv:1803.02991 , 2018.

Yizhe Zhu, Martin Renqiang Min, Asim Kadav, and Hans Peter Graf. S3VAE: Self-supervised sequential VAE for representation disentanglement and data generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6538-6547, 2020.

Junwen Bai, Weiran Wang, and Carla P Gomes. Contrastively disentangled sequential variational autoencoder. Advances in Neural Information Processing Systems , 34, 2021.

Jun Han, Martin Renqiang Min, Ligong Han, Li Erran Li, and Xuan Zhang. Disentangled recurrent wasserstein autoencoder. In International Conference on Learning Representations , 2021. URL https://openreview.net/forum?id=O7ms4LFdsX .

Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, and Honglak Lee. Decomposing motion and content for natural video sequence prediction. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum?id=rkEFLFqee .

Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1526-1535, 2018.

Emily L Denton and Vighnesh Birodkar. Unsupervised learning of disentangled representations from video. Advances in neural information processing systems , 30, 2017.

Masanori Yamada, Heecheol Kim, Kosuke Miyoshi, Tomoharu Iwata, and Hiroshi Yamakawa. Disentangled representations for sequence data using information bottleneck principle. In Asian Conference on Machine Learning , pages 305-320. PMLR, 2020.

Shengjia Zhao, Jiaming Song, and Stefano Ermon. Learning hierarchical features from deep generative models. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pages 4091-4099. PMLR, 06-11 Aug 2017. URL https://proceedings.mlr. press/v70/zhao17c.html .

Elizabeth S. Allman, Catherine Matias, and John A. Rhodes. Identifiability of parameters in latent structure models with many observed variables. Annals of Statistics , 37:3099-3132, 2008.

Alessio Spantini, Daniele Bigoni, and Youssef Marzouk. Inference via low-dimensional couplings. The Journal of Machine Learning Research , 19(1):2639-2709, 2018.

Supplement to

'Temporally Disentangled Representation Learning under Unknown Nonstationarity'

Appendix organization:

A Identifiability

Assume we observe n -dimensional time-series data at discrete time steps, X = { x 1 , x 2 , . . . , x T } , where each x t is generated from time-delayed causally related hidden components z t ∈ R n by the invertible mixing function:

In addition to latent components z t , there is an extra hidden component c t which is a discrete variable with cardinality | c t | = C , it follows first-order Markov process controlled by a C × C transition matrix A , in which the i, j -th entry A i,j is the probability to transit from state i to j .

For i ∈ { 1 , . . . , n } , z it , as the i -th component of z t , is generated by (some) components of history information z t -1 , discrete nonstationary indicator c t , and noise ϵ it .

where Pa ( z it ) is the set of latent factors that directly cause z it , which can be any subset of z Hx = { z t -1 , z t -2 , . . . , z t -L } up to history information maximum lag L . The components of z t are mutually independent conditional on z Hx and c t .

A.1 Identifiability of Nonstationary Hidden States

Theorem 1. (identifiability of the nonstationarity with Markov Assumptions) Suppose the observed data is generated following the nonlinear ICA framework as defined in Eqs. (1) , (2) and (3) . And Suppose the following assumptions (Markov Assumptions) hold:

i For the Markov process, the number of latent states, C , is known.

ii The transition matrix A is full rank.

Use µ 1 , . . . , µ C ∈ R n to denote nonparametric probability distributions of the C emission distributions µ c = p ( x t | x t -1 , c ) . Then the parameters A and M = ( µ 1 , . . . , µ C ) are identifiable given the distribution, P (3) A ,M , of at least 4 consecutive observations x t , x t +1 , x t +2 , x t +3 , up to label swapping of the hidden states, that is:

If ˜ A is a C × C transition matrix, if ˜ π ( c ) is a stationary distribution of ˜ A with ˜ π ( c ) > 0 ∀ c ∈ { 1 , . . . , C } , and if ˜ M = (˜ µ 1 , . . . , ˜ µ C ) are C probability distributions on R n that verify the equality of the distribution functions P (3) ˜ A , ˜ M = P (3) A ,M , then there exists a permutation σ of the set { 1 , . . . , C } such that for all k, l = 1 , . . . , C we have ˜ A k,l = A σ ( k ) ,σ ( l ) and ˜ µ k = µ σ ( k ) .

Proof. Suppose we have:

where p ( x 1 , . . . , x T ) has transition matrix A and emission distributions ( µ 1 , . . . , µ C ) , similarly for ˜ p ( x 1 , . . . , x T ) .

We consider four consecutive observations x 0 , x 1 , x 2 , x 3 and corresponding four discrete elements c 0 , c 1 , c 2 , c 3 .

where π c i = p ( c i ) . Since A has full rank and the probability measures µ 1 , . . . , µ C are linearly independent, the probability measures { ¯ µ c 2 = ∑ c 1 π c 1 A c 1 ,c 2 π c 2 µ c 1 | c 2 = 1 , . . . , C } are linearly independent, and the probability measures { ˙ µ c 2 = ∑ c 3 A c 2 ,c 3 µ c 3 | c 2 = 1 , . . . , C } are also linearly independent. Thus, applying Theorem 9 of [46], there exists a permutation σ of { 1 , . . . , C } such that, ∀ i ∈ { 1 , . . . , C } :

This gives easily ∀ i ∈ { 1 , . . . , C } :

Since the conditional distributions µ i are linearly independent, we can establish the equivalence between ˜ A and A via permutation σ ,

then the theorem is proved.

For notational simplicity, and without loss of generality, we assume the components are ordered such that c = σ ( c ) . That leads us to the identifiability of the nonstationarity in the system i.e. up to label swapping of the hidden states, the conditional emission distributions p ( x t | x t -1 , c t ) and transition matrix A are identifiable, hence providing us a bridge to further leverage the temporal independence condition in the latent space to establish the identifiability result for demixing function or in other words the latent variables z t .

A.2 Identifiability of Latent Causal Processes

To incorporate nonlinear ICA into the Markov Assumption we define the emission distribution p ( x t | x t -1 , c ) as a deep latent variable model. First, the latent independent component variables z t ∈ R n are generated from a factorial prior, given the hidden state c t and previous z t -1 , as

Second, the observed data x t ∈ R n is generated by a nonlinear mixing function as in Eq. (1) which is assumed to be bijective with inverse given by z t = g -1 ( x t ) . Let η kt ( c t ) ≜ log p ( z kt | z t -1 , c t ) , and assume that η kt ( c t ) is twice differentiable in z kt and is differentiable in z l,t -1 , l = 1 , 2 , ..., n . Note that the parents of z kt may be only c t and a subset of z t -1 ; if z l,t -1 is not a parent of z kt , then ∂η k ∂z l,t -1 = 0 .

Theorem 2. Suppose there exists an invertible function ˆ g -1 , which is the estimated demixing function that maps x t to ˆ z t , i.e.,

such that the components of ˆ z t are mutually independent conditional on ˆ z t -1 . Let

And

If for each value of z t , s 1 t , ˚ s 1 t , v 2 t , ˚ s 2 t , ..., s nt , ˚ s nt , as 2 n function vectors s k,t and ˚ s k,t , with k = 1 , 2 , ..., n , are linearly independent, then ˆ z t must be an invertible, component-wise transformation of a permuted version of z t .

Proof. Combining (1) and (6) gives z t = ( g -1 ◦ ˆ g )(ˆ z t ) = h (ˆ z t ) , where h ≜ g -1 ◦ ˆ g . Since both g and ˆ g are invertible, h is invertible. Let H t be the Jacobian matrix of the transformation h (ˆ z t ) , and denote by H kit its ( k, i ) th entry.

̸

First, it is straightforward to see that if the components of ˆ z t are mutually independent conditional on previous ˆ z t -1 and current c t , then for any i = j , ˆ z it and ˆ z jt are conditionally independent given ˆ z t -1 ∪ (ˆ z t \ { ˆ z it , ˆ z jt } ) ∪ { c t } . Mutual independence of the components of ˆ z t conditional on ˆ z t -1 implies that ˆ z it is independent from ˆ z t \ { ˆ z it , ˆ z jt } conditional on ˆ z t -1 and c t , i.e.,

At the same time, it also implies ˆ z it is independent from ˆ z t \ { ˆ z it } conditional on ˆ z t -1 and c t , i.e.,

Combining the above two equations gives

̸

i.e., for i = j , ˆ z it and ˆ z jt are conditionally independent given ˆ z t -1 ∪ (ˆ z t \ { ˆ z it , ˆ z jt } ) ∪ { c t } .

We then make use of the fact that if ˆ z it and ˆ z jt are conditionally independent given ˆ z t -1 ∪ (ˆ z t \ { ˆ z it , ˆ z jt } ) ∪ { c t } , then

assuming the cross second-order derivative exists [47]. Since p (ˆ z t , ˆ z t -1 , c t ) = p (ˆ z t | ˆ z t -1 , c t ) p (ˆ z t -1 , c t ) while p (ˆ z t -1 , c t ) does not involve ˆ z it or ˆ z jt , the above equality is equivalent to

Then for any c t , the Jacobian matrix of the mapping from ( x t -1 , ˆ z t ) to ( x t -1 , z t ) is [ I 0 ∗ H t ] , where

∗ stands for a matrix, and the (absolute value of the) determinant of this Jacobian matrix is | H t | . Therefore p (ˆ z t , x t -1 | c t ) = p ( z t , x t -1 | c t ) · | H t | . Dividing both sides of this equation by p ( x t -1 | c t ) gives

Since p ( z t | z t -1 , c t ) = p ( z t | g ( z t -1 ) , c t ) = p ( z t | x t -1 , c t ) and similarly p (ˆ z t | ˆ z t -1 , c t ) = p (ˆ z t | x t -1 , c t ) , Eq. 12 tells us

Its partial derivative w.r.t. ˆ z it is

Its second-order cross-derivative is

The above quantity is always 0 according to Eq. (11). Therefore, for each l = 1 , 2 , ..., n and each value z l,t -1 , its partial derivative w.r.t. z l,t -1 is always 0. That is,

where we have made use of the fact that entries of H t do not depend on z l,t -1 . Using different values r for c t in Eq. (14) take the difference of this equation across them gives

̸

If for any value of z t , s 1 t , ˚ s 1 t , s 2 t , ˚ s 2 t , ..., s nt , ˚ s nt are linearly independent, to make the above equation hold true, one has to set H kit H kjt = 0 or i = j . That is, in each row of H t there is only one non-zero entry. Since h is invertible, then z t must be an invertible, component-wise transformation of a permuted version of ˆ z t .

So far, the identifiability result has been established without observing the nonstationarity indicators such as domain indices.

A.3 Discussion on Assumptions in Theorem 2

This condition was initially introduced in GCL [11], namely, 'sufficient variability', to extend the modulated exponential families [9] to general modulated distributions. Essentially, the condition says that the nonstationary domains c must have a sufficiently complex and diverse effect on the transition distributions. In other words, if the underlying distributions are composed of relatively many domains of data, the condition generally holds true. Loosely speaking, the sufficient variability holds if the modulation of by c on the conditional distribution q ( z it | z Hx , c ) is not too simple in the following sense:

Higher order of k ( k > 1 ) is required. If k = 1 , the sufficient variability cannot hold;

The modulation impacts λ ij by u must be linearly independent across domains c . The sufficient statistics functions q ij cannot be all linear, i.e., we require higher-order statistics.

Further details of this example can be found in Appendix B of [11] and Appendix S1.4.1 of [18]. In summary, we need the domains denoted by c to have diverse (i.e., distinct influences) and complex impacts on the underlying data generation process.

B Implementation Details

B.1 Reproducibility

All experiments are done in a GPU workstation with CPU: Intel i7-13700K, GPU: NVIDIA RTX 4090, Memory: 128 GB. The code can be found via https://github.com/xiangchensong/nctrl .

B.2 Prior Likelihood Derivation

Let us start with an illustrative example of stationary latent causal processes consisting of two timedelayed latent variables, i.e., z t = [ z 1 ,t , z 2 ,t ] with maximum time lag L = 1 , i.e., z i,t = f i ( z t -1 , ϵ i,t ) with mutually independent noises. Let us write this latent process as a transformation map f (note that we overload the notation f for transition functions and for the transformation map):

By applying the change of variables formula to the map f , we can evaluate the joint distribution of the latent variables p ( z 1 ,t -1 , z 2 ,t -1 , z 1 ,t , z 2 ,t ) as:

where J f is the Jacobian matrix of the map f , which is naturally a low-triangular matrix:

Given that this Jacobian is triangular, we can efficiently compute its determinant as ∏ i ∂z i,t ∂ϵ i,t . Furthermore, because the noise terms are mutually independent, and hence ϵ i,t ⊥ ϵ j,t for j = i and ϵ t ⊥ z t -1 , we can write the RHS of Eq. 18 as:

̸

Finally, by canceling out the marginals of the lagged latent variables p ( z 1 ,t -1 , z 2 ,t -1 ) on both sides, we can evaluate the transition prior likelihood as:

Now we generalize this example and derive the prior likelihood below.

Let { f -1 i } i =1 , 2 , 3 ... be a set of learned inverse transition functions that take the estimated latent causal variables, and output the noise terms, i.e., ˆ ϵ i,t = f -1 i (ˆ z i,t , { ˆ z t -τ , c t } ) .

Design transformation A → B with low-triangular Jacobian as follows:

Similar to Eq. 20, we can obtain the joint distribution of the estimated dynamics subspace as:

︸

︷︷ Because of mutually independent noise assumption

︸

B.3 Derivation of ELBO

Then the second part is to maximize the Evidence Lower BOund (ELBO) for the VAE framework, which can be written as:

B.4 Synthetic Dataset Generation

We generated two synthetic datasets (A and B) with different nonlinear mixing functions. In this section we will introduce the detailed implementation of the generation. The generation can be split into steps (1) sample c t from a Markov chain, (2) generate z t with different transition functions f c t with respect to c t , and (3) generate observation x t via mixing function g .

B.4.1 Sample c t from Markov chain

We first randomly initialized a Markov chain with transition matrix A and sample 20,000 steps.

B.4.2 Generation of latent variables z t

We first randomly initialized | C | = 5 different transition functions { f 1 , f 2 , . . . , f | C | } with different MLPs, and generate z t = f c t ( z Hx ) . The dimensions are set to 8 for fair comparison.

B.4.3 Generation of observations x t

The difference between datasets A and B is the mixing function. We use a two-layer randomly initialized MLP for dataset A and a three-layer MLP for dataset B. For each linear layer in the MLP, we use condition number of the weight matrix to filter out ones that are not 'invertible'.

B.5 Modified CartPole Dataset Generation

Similar to the synthetic datasets, we also sample from a Markov chain and get c t . For the modified CartPole, we initialized 5 different environments which have different combinations of hyperparameters such as gravity, pole mass, etc. A detailed comparison is listed in Table 1.

At each time step t the environment will load the corresponding hyperparameters for given c t and update the states z t according to the configuration given c t . The nonlinear mixing function from states to observations x t is fixed by a rendering method in the gym package.

B.6 MoSeq Dataset

In the MoSeq dataset, the observations x t are taken to be the first 10 principal components of depth camera video data of mice exploring an open field. The dataset consists of 20-minute depth camera recordings of 24 mice. In preprocessing, the videos are cropped and centered around the mouse centroid and then filtered to remove recording artifacts. Finally, the preprocessed video is projected onto the top principal components to obtain a 10-dimensional time series.

B.7 Mean Correlation Coefficient

MCC is a standard metric for evaluating the recovery of latent factors in ICA literature. MCC first calculates the absolute values of the correlation coefficient between every ground-truth factor against every estimated latent variable. Pearson correlation coefficients or Spearman's rank correlation coefficients can be used depending on whether componentwise invertible nonlinearities exist in the recovered factors. The possible permutation is adjusted by solving a linear sum assignment problem in polynomial time on the computed correlation matrix.

B.8 Network Architecture

We summarize our network architecture below and describe it in detail in Table 2 and Table 3.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper addresses the problem of identifying latents representations from sequential data which is stationary within contexts, and non-stationary between contexts. Existing work has either conditioned on observed auxiliary variables that indicate which context one is in (e.g. time contrastive learning and the followups), or they relied on mutually (conditionally) independent latents and did not allow time-delayed latent causal influences. The paper presents an approach called Nonstationary Temporal Disentangled Causal Representation Learning (NTDC)and shows conditions under which they can identify time-delayed latent causal variables and their relations without the need for observed auxiliary variables. They formulate the problem as a discrete Markov process and establish identifiability of the latent independent components. They give impressive experimental results on both synthetic and real-world datasets to demonstrate the proposed method's success in recovering latent variables.\n\nSTRENGTHS:\n* Really nice experimental results section. The results on simulated data and I appreciated the section on the MoSeq dataset analyzing mouse behaviour. \n* I think the graphical model that the paper analyses is very practically useful for abstracting time series data. By breaking a time series into conditionally stationary domains, $c_t$, that change more slowly, you gain a more interpretable model of the underlying behaviour of the system (the mouse experiments are nice illustration of this). Empirically, we've seen results similar to this before in the original MoSeq paper, but it is nice to have identifiability results to go with these approaches.\n\nWEAKNESSES:\n* Assumptions (6) and (7) in Theorem 2 are unintuitive to the point that I'm not sure the theorem is useful at all. The main value of identifiability theorems are to give a set of preconditions on the data under which a practitioner can expect a disentanglement routine would work. But I'm not sure that there are any practitioners who would be able to judge whether those assumptions hold in their data, and there was no attempt to give any intuition for when they might hold and when they would fail.\n\nQUESTIONS:\n1. Can you give any more intuitive assumptions that are sufficient for Theorem 2 to hold? Or give examples of distributions for which they hold and examples for which they fail? If done well, this would make the theory section stronger.\n\nLIMITATIONS:\nThe paper already does a reasonable job of outlining the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper gives identifiability results in a new setting, along with an estimation method and experiments validating the estimation method.\n\nThere are existing identifiability results in the nonstationary setting if you assume that the auxiliary variables are observed. There are other existing identifiability results in the nonstationary setting if you assume that the auxiliary variables evolve as a Markov chain but the latents only affect the current timestep. This paper gives an identifiability result in the nonstationary setting where the auxiliary variables are *not* observed and the auxiliary variables evolve as a Markov chain but the latent variables can affect the observed variables across time steps.\n\nThe estimation method is an extension of Sequential VAEs, where there is an autoregressive Hidden Markov Model model that models the nonstationarity, a prior network that estimates the prior using a conditional normalizing flow, and a VAE where the encoder fits the demixing function and the decoder fits the mixing function.\n\nThey perform experiments on two synthetic datasets that satisfy their identifiability conditions, a video dataset for a modified Cartpole environment, and a video dataset of mouse behavior. For the synthetic datasets, they compare MCC with seven baselines (BetaVAE, iVAE, TVL, SlowVAE, PCL, TDRL, and HMNLICA). For the video cartpole dataset, they compare MCC against one baseline (TDRL). For the mouse video dataset they don’t have ground-truth independent components so instead of numerical results they provide a visualization of the recovered components.\n\nSTRENGTHS:\nStrengths:\n- I am not especially up-to-date on identifiability literature, but as far as I am aware this is a novel problem setting and novel result.\n- I agree that the “observed auxiliary variable” condition in many nonlinear ICA identifiability results is not realistic, and it is a useful direction to prove results that do not rely on this assumption.\n- The choice of baselines is very thorough for the synthetic datasets.\n- They use two video datasets to demonstrate their estimation method in a more realistic setting than synthetic data, which I appreciate since this is mostly a theory paper.\n\nWEAKNESSES:\nWeaknesses:\n- The writing is okay but not great. It gets the point across but it could use proofreading and better word choices in some places. Ex: “It is obvious that the causal relations among those latent variables are more meaningful and the identifiability is urgently needed.” <- this sentence could be improved.\n- See questions below.\n\nQUESTIONS:\n- In the Cartpole experiments, why do you only provide Accuracy and MSE A metrics for your method and not for baseline methods? And can you provide evaluations for the other baselines for this experiment?\n\nLIMITATIONS:\n- As they mention in the conclusion section, they assume that the nonstationary variables follow a Markov chain, which is a somewhat restrictive assumption.\n- No concerns about negative societal impacts.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents identifiability results that handle nostationary time-delayed causally-related processes without auxiliary variables. In addition, the authors propose NTDC, which is a neural network that is based on their identifiability results. The method is evaluated on a few tasks in comparison to a few baselines, showing significance of toy tasks generated by the authors.\n\nSTRENGTHS:\nThis paper introduces identifiability results on sequential information which is causally-related without additional variables. Thus, this paper extends prior work from a theoretic viewpoint. Another strength of the paper is the new deep framework that incorporates the theoretical results.\n\nWEAKNESSES:\nOne of the main weaknesses of the paper is its presentation. In particular, a lot of jargon is used without properly introducing the terms. For instance, domain indices are used throughout the paper without ever defining what it means. Similarly, Sec. 4.1 is missing a lot of details (what is ARHMM? what is the prior network?). Sec. 4.2 seems to be unfinished in terms of writing. Missing details and descriptions make it hard to properly evaluate the proposed NTDC and position it with respect to prior work.\n\nAnother main shortcoming of the work is the lack of discussion on related work for sequential disentanglement. Only paper [20] is mentioned (extremely) briefly. However, there are already at least ten papers or more since [20] which significantly improved the state-of-the-art results, the neural network models, the evaluation and theory.\n\nAnother shortcoming of the paper is the dicsussion in the introduction. In particular, the authors argue their mouse example might be challenging for existing work. In addition, they argue to be first to consider scenario Fig. 1c. However, in a recent ICLR'23 paper with the title \"Multifactor Sequential Disentanglement via Structured Koopman Autoencoders\" (SKD) by Berman et al. the authors show a framework that considers a similar model to Fig. 1c, and it is able to handle data such as the mouse example. The authors should compare their work with SKD both qualitatively and quantitatively. Similarly, the work \"Contrastively Disentangled Sequential Variational Audoencoder\" (C-DSVAE) by Bai et al. can also deal with a similar scenario and should be compared with.\n\nAnother shortcoming of the paper is its evaluation. In particular, the proposed method is compared with several baselines only on toy datasets that the authors create. No sequential disentanglement baselines are considered. On real world data, the authors only compare with a single method.\n\nQUESTIONS:\nSee above.\n\nLIMITATIONS:\nSee above.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe study focuses on unsupervised representation learning for sequential data with time-delayed causal influences. Identifiability results for disentangling causally-related latent variables have been established in stationary settings using temporal structure. However, existing work only partially addresses this in nonstationary settings by using observed auxiliary variables or introducing the Markov assumption, but not both simultaneously. The authors introduce NTDC, to reconstruct time-delayed latent causal variables and identify their relations without the need for auxiliary variables, by utilizing deep generative models. The experimental results demonstrate that the proposed methodology outperforms existing baselines by effectively exploiting nonstationarity and distinguishing distribution shifts.\n\nSTRENGTHS:\n- The paper is clearly written.\n- The authors tackle the setting of previous works and propose a non-existing work, which is described in Figure 1. \n- The theoretical grounds are sufficiently provided with proper theorems.\n\nWEAKNESSES:\n- There are some grammatical errors in the main paper.\n- There is no explanation of the \"disentangled\" which is included both in the proposed model name and the title of the manuscript.\n- The authors provide their code with anonymity, but there is limited explanations of the codes that which is directly linked to the proposed NTDC and how one can run the code.\n\nQUESTIONS:\n- Could you provide more motivational examples of the model structure in Figure 1(d) other than the mouse movement example?\n- Are there any derivation with probabilistic graphical model settings of Figure 1(d)? I wonder whether the problem could be solved in a statistical manner rather than just utilizing DGMs.\n- I guess the model structure can be viewed as a doubly-hidden Markov model, or it could be more generalized as a multi-layer hidden Markov model. Could proposed theorems, as well as the proposed NTDC, be generalized in such settings?\n- What's the specific meaning of \"disentangled\" representation in the title as well as the model name? And how the disentangled representation is achieved?\n\nLIMITATIONS:\n- There are no potential negative societal impacts of their work. \n- Even though the authors provided a new work, my concern is that it lacks significance considering the NeurIPS standard.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-30"}