Causal discovery from conditionally stationary time-series 
Authors: Carles Balsells Rodas
Ruibo Tu
Hedvig Kjellstrom
Venue: ICLR 2022 Submitted (2022)
Keywords: causal discovery
temporal inference
graph neural network
time series
non-stationary
probabilistic modelling

Abstract: Causal discovery, i.e., inferring underlying cause-effect relationships from observations of a scene or system, is an inherent mechanism in human cognition, but has been shown to be highly challenging to automate. The majority of approaches in the literature aiming for this task consider constrained scenarios with fully observed variables or data from stationary time-series. 
In this work we aim for causal discovery in a more general class of scenarios, scenes with non-stationary behavior over time. For our purposes we here regard a scene as a composition objects interacting with each other over time. Non-stationarity is modeled as stationarity conditioned on an underlying variable, a state, which can be of varying dimension, more or less hidden given observations of the scene, and also depend more or less directly on these observations.
We propose a probabilistic deep learning approach called State-Dependent Causal Inference (SDCI) for causal discovery in such conditionally stationary time-series data. Results in two different synthetic scenarios show that this method is able to recover the underlying causal dependencies with high accuracy even in cases with hidden states.

Under review as a conference paper at ICLR 2022
CAUSAL DISCOVERY FROM CONDITIONALLY STATION- ARY TIME-SERIES
Anonymous authors Paper under double-blind review
ABSTRACT
Causal discovery, i.e., inferring underlying cause-effect relationships from obser- vations of scene or system, is an inherent mechanism in human cognition, but has been shown to be highly challenging to automate. The majority of approaches in the literature aiming for this task consider constrained scenarios with fully ob- served variables or data from stationary time-series.
In this work aim for causal discovery in more general class of scenarios, scenes with non-stationary behavior over time. For our purposes we here regard scene composition objects interacting with each other over time. Non- stationarity is modeled as stationarity conditioned on an underlying variable, a state, which can be of varying dimension, more or less hidden given observations of the scene, and also depend more or less directly on these observations.
We propose a probabilistic deep learning approach called State State-Dependen Causal Inference (SDCI) for causal discovery in such conditionally stationary time-series data. Results in two different synthetic scenarios show that this method is able to recover the underlying causal dependencies with high accuracy even in cases with hidden states.
INTRODUCTION
The ability of deep learning approaches to discover and reason about causal relationships in data has become prominent direction of work over the recent years (Yi et al.) [2220] Girdhar & Ramanan 2020 Sauer & Geiger) 2021). Despite the recent success of deep learning methods in related tasks such classification, localization, and segmentation, causal discovery and reasoning, inherent mechanism in human cognition (Spelke & Kinzler 2007) allowing reasoning about counterfactuals and understanding the reasons of events, still poses an considerable challenge.
Causal discovery involves uncovering the underlying logic, temporal and causal structure of the observed processes in the data. Current approaches (see Section 2 commonly address quite con- strained scenarios with stationary behavior over time. In the present paper, we extend the current work by addressing scenarios with conditional stationarity, where the dynamics of the observed system changes with the value of underlying variables. This is the case in almost all real-world scenarios, e.g. with people who behave differently and take different decisions depending on un- derlying factors such mood, previous experience, and the actions of other agents. We propose a method (see Section 3 for causal discovery from time-series observations of systems where the underlying causal graph changes depending
state variable.
The causal discovery task from such conditionally stationary time-series poses different challenges depending the observability of the underlying state variable. Four scenario classes can be seen:
1. The first class concerns a simplified version of the problem, where the state variable is observed and not dependent on other observed time-series data.
2. In the second class of scenarios, the state is not directly observed, but directly dependent and continuously inferable from observed variable. A real-life example is a traffic scenario where taxis (visually distinguishable by the sign on their roof) follow slightly different nules than normal cars, i.e. are allowed to drive in bus lanes.

---

Under review as a conference paper at ICLR 2022
3. A more challenging scenario class is when the state depends on earlier events, and thus is not continuously observable. A real-life example is a chain of events in a football game, where the action of one player is triggered by an earlier action by another player.
4. Finally, large share of scenarios in the real world are governed by underlying state vari- ables that are not fully inferable from the observations from the scenario over time. In such scenarios, the state is an unknown confounder to the observed time-series, and causal discovery from such scenarios is inherently ill-defined.
We evaluate the method (see Section in two different synthetic scenarios, where we vary the complexity of system dynamics and observability of the underlying state variable covering the first three scenario classes above. Finally conclude and discuss directions for future work (see Section
5)
RELATED WORK
Causal discovery approaches aim to identify causal relationships over a set of variables from obser- vational data. These methods can basically be classified into three different types (Glymour et al.) 2019) 1) constraint-basede 2) boore-based; 3) Functional causal model based methods.
Constraint-basee methods rely conditional independence tests to recover the underlying DAG structure of the data, such as the PC algorithm (Spirtes et al. 2000), which assumes faithfulness and causal Markov condition and considers i.i.d. sampling and no latent confounders. There exists great variety of variations of PC. One of them the Fast Causal Inference (FCI) (Spirtes) (001), which is able to cope with the unknown confounders and selection bias; furthermore, it can be adapted for time-series data, such as such as tsFCI (Entner & Hoyer 2010).
Score-based methods define score functions of causal graph structures and then optimize score functions by performing a search to identify the underlying causal structure, such as the Greedy Equivalence Search (GES) (Chickering) 2002). Notice that searching the graph space poses combinatorial optimization problem. Recent approaches try to avoid this by reformulating it as a continuous optimization problem which introduces score function h for measuring the acyclicity of the graph (Zheng et al.) 2018). Regarding time-series data, these methods are reformulated as learning dynamic Bayesian Networks (DBNs) from data (Murphy al. 2002). Among these algo- rithms we recently find DYNOTEARS (Pamfil et al. 2020), which aims to simultaneously estimate instantaneous and time-lagged relationships between variables in time-series.
Functional causal model-based methods represent the effect as a function of its direct causes and their independent immeasurable noise (Glymour et al. 2019). For non-temporal data, there are linear non-Gaussian acyclic models (Shimizu et al. 2006), additive noise models (Peters et al, 2014), post-nonlinear models (Zhang & Hyvärimen 2009). etc. For temporal data, these approaches fit dynamics model, which is often regularized in terms of its sparsity, and its form is analyzed to identify the underlying causal connections in the data. Granger causal analysis falls into this category, since we first model the dynamics and some analysis is performed to extract the latent causal structure (Granger 1969).
Causal discovery is, in general, challenging task and its study arises a great amount of practical issues. The problem is ill-posed when considering linearity and Gaussian disturbances, since it can be proved that the underlying causal model is not identifiable, while under proper assumptions, such as non-Gauusi,,,,,, it becomes identifiable (Shimizu et al., (006). When considering non-linear transformations, the symmetry between observed variables disappears, allowing the identification of causal relations in the context of Gaussian disturbances (Hoyer et al., 2008) ) Other practical issues consist on the existence of latent confounders (Ranganath Perotte 2018), the presence of measurement error (Zhang et al.) 2017) or considering observations with missing data (Tu et al. 2019). In order to avoid these common problems, simplifications of the problem need to be applied. In fact, the assumptions make in this work are: (i) all the instances belonging the causal graph are observed, (ii) we have no missing data selection bias, and (iii) no latent confounders exist.
The work most related to are the approaches by (Löwe al. 2020 Li al. 2020 we extend
these by allowing the causal model of the underlying process to vary depending on a state variable. Our method can in the future be applied to wider class of non-stationary visual scenarios where the
2

---

Under review as a conference paper at ICLR 2022
interacting objects are only partially and noisily observed as semantically segmented visual regions, or by tracked image keypoints (Löwe et al. 2020). This would allow addressing challenging tasks such scene understanding, counterfactual reasoning, etc. The recent work by (Sauer & Geiger 2021) also uses the concept of causality for a similar task, generation of counterfactual images.
3 STATE-DEPENDENT CAUSAL INFERENCE
In this section, we introduce our formulation to extract causal graphs from time-series data where their dynamics are altered by means categorical variable, referred to as their state, We refer to our method as State-Dependent Causal Inference (SDCI).
3.1 PROBLEM FORMULATION
The input consists of a set of N time-series which not only obey some dynamics that might change over time but also undergo different states along the sequence. These states are responsible for the changes in the dynamics of the sample. We observe the sequence for total of T frames and we denote the sample x as
x= (fx: (1 H (p'
(1)
where {s: 1 represents the hidden states and {p, [p)") 1 are the observed quantities of interest. For simplicity, we drop the subscript when referring to all the elements in single time-step (e.g. x", p', s², etc). In causal graph, the observed quantities are represented by the vertices and the edge of the causal graph represents the interaction type between vertices. We denote the amount of possible interaction types by
Assumptions. In this work, we assume that the data generation process obeys structural causal model (SCM) (Pearl 2009). G and that the model satisfies the first-order Markov property. Moreover, according to the definitions of causality (Granger) 1969). we assume that edges of causal graph cannot go back time. The first assumption follows related works concerning samples where the generative process also follows an SCM (Löwe et al., 2020 Li et al. 2020). Although assume the first-order Markov property, one can extend to the more general th order Markov property in more complex scenario.
State-dependen causal inference. Based on the assumptions we mainly focus the non- stationary causal graph, which means that we can find different edge-types at different times. As for an edge between two vertices, the edge-type interaction between two vertices changes according to the state of the variable which is the source of the interaction. The main focus of our method consists on extracting a causal summary graph G (also denoted as such by Li al. (2020) and Löwe et al. (2200)) Previous approaches aiming for this task assume stationary time-series data and therefore, this causal summary graph is constant. Nonetheless, since we condition the stationarity of the sam- ples on the states, our causal summary graph is expressed by means of this categorical variable. In other words, our method will extract summary graphs, one per state considered. The edge-type interaction can be then queried at each time-step / as follows:
2
Gt)($f)
where z/ş E (0, (O...., n. - }} denotes the edge-type interaction from i to j at time-step Figure illustrates this task. In general, this causal summary graph is specific to the input sample and hidden from the model. Therefore, not only we require to design a parametrizable function to infer the latent causal structure, but also to evaluate how this inference fits to the actual dynamics observed in the input sequence.
Let us denote the first step of extracting the latent causal structure
G(s)
(p I:T LTT)
(3)
where fo denotes a parameterizable function that receives all the observed sequence as input. The next step is to fit this extracted latent sturcture into our assumed first-order Markov dynamics.
1+1 5 (x*, G(st))
3

---

Under review as a conference paper at ICLR 2022
Stule State
(<lle, mc)
Causal graph interence
SDCI aims extract a causal summary graph that describes the edge-type interaction for every pair of edges conditioned on the state of the source variable with respect to the interaction.
Figure 1:
where the parameterizable function fy represents one-step computation of the dynamics starting from the observed values at time-step In this expression, we have defined fø to predict the value of the states at the next time-step as well. However, in our experiments we will also consider settings where the states are observed all times and modelling the dynamics only performed with respect to the quantity p! To allow for this setting, one only needs to exclude the state variable from supervision. We provide more details the next section.
Objective. Both the causal inference and dynamics modelling modules can be optimized by min- imizing some objective defined for the parameterizable functions fo and fs.
T-1 D(x (x( G(s))) + R(G()), G(s²) = fx(x)
min 6.49
(5)
where R(-) is a regularizer on the extracted graph structure, which can be applied to enforce a preferred interaction type.
3.2 IMPLEMENTATION
Since the interactions that are considered in the scene are complex, we introduce uncertainty in
our approach. Therefore, we are interested in computing the probability distribution over the edge type given two instances and the state of the source instance, p(2y [s P. P.). This implementation
gathers inspiration from related works concerning this type of tasks (Li et al. 2020 Löwe et al, 2020 Kipf al. 2018).
The structure of the implementation is very similar to a VAE (Kingma & Welling 2014). In this case, the latent space is represented by the of edges žıj which are discrete variables. The latent edges are conditioned to the state of the variable which is the source of the interaction, 8,. We allow for total of K states. Let 24y causal interaction that there is between i and j and will depend on the state of i. 2ij E {O,..., ..., Rr - 1}, where n. is the amount of inter-object interactions considered in our setting. z/ = means that component/ i not influenced by component j at time t. Any other value models a different type of interaction. Since condition the edge type the state variable, objective is more close the one defined in CVAE (Sohn et al. 2015).
edge from instance i instance j. This value represents the type of
= Eyg (els.x) [log (x/2)] K (go x)) x)](((()))
(6)
where p(zls) is prior defined over the edge types conditioned on each state, which acts as a regu- larizer over the inferred edge-type distribution. We use denote all the edges represented in the latent space. In our settings, we set this prior to enforce uniform distribution matter what the state is. We might find applications where this prior could be leveraged to encourage sparsity in the extracted graph structure.
Encoder. Following related approaches (Löwe et al., 2020), we use graph neural networks as the encoder. The model receives all the information from the sample available 11:"", We concatenate p!:T' with one-hot representation of the state variable s1:T We aim extract an embedding that represents the causal interaction conditioned on the state for every pair of elements present in the sample:
dú = (x
4

---

Under review as a conference paper at ICLR 2022
where Øiy R denotes the embedding for every pair i j. The approximate posterior
distribution conditioned on the state variable Ro(zi)]k; [b,x) is calculated as follows:
go(=y]k;; x) ((b5t\/T)
where k is the state to which the edge-type distribution of + j is conditioned and θ denotes a soft- max activation with temperature т. We Φijk -R the note the embedding for + . state k. Since this formulation yields a discrete latent space, we relax the resulting categorical distribution using the Gumble-softmax (Maddison al. 2017) technique to enable back- propkk-pppgatin..
Once we have inferred the approximate posterior distribution over the edge-types, we can sample the interactions at every time-step conditioning on s":
Aig (o( (x,,)
Notice that this requires sampling at each time-step t, which might pose great computational ex- penses for large sequences. In practice, sample right after determining the distribution of the edge-types. Then, the interactions are queried for each i + depending on s/:
Wiyk ~ (z(zii x): zig = Wijk'' ===0
(10)
where we have stored the sampled values using wijk R" which denotes the edge-type using one-hot representation. In our experiments, we refer to SDCI-Static or SpCI-Temporal for imple- mentations based on MLPs CNNs respectively. For more details, see the supplementary material.
Decoder. Let us describe the decoder ( lz). which models the dynamics of the generative pro- cess. Consider the dynamics modelling for element j. At each time t, we first query the edge-types depending the states of the source variable from the sampled quantities Wijk (see equation I0)
z Wifks l s
(11)
The information along the predicted edge-type interactions is then retrieved as follows
(xt fx(
(12)
[fe] is a family of parametrizable functions, one defined for each edge type excluding the no-edge interaction.
The inter-object interactions are finally integrated to model the dynamics each variable. The updates of the variable p +1
r+1 p = +
fo
Σ g ifj
(13)
are predicted by means of fp, where it provides information about the update between the actual and the next time-step. The supervision of the decoder considering 1: uses the negative likeliidddood of the following probability distribution
(p [x', N I
(14)
where a is the variance of the resulting Gaussian distribution.
We have previously introduced the idea of considering the state variable s1:T, as being independent from the dynamics of the sample, i.e. p pl:T. If this happens, our method will have information about the states of the samples at all times and consequently, no supervision upon s will be required. However, we might find practical to consider the setting where the state is in fact dependent p ' and therefore our method will require to model its behavior and perform supervision. In this case, present straightforward approach to compute the estimation of the next state:
= fa Σ gig
(15)
and the corresponding categorical distribution can be computed as follows
(s)
2+1 |x zz) = 1(
(16)
where O is softmax activation.
5

---

Under review as a conference paper at ICLR 2022
In summary, the probability distribution of the decoder can be expressed as follows:
T-1 II Pa (x z'), Pio (x x z
(x
Pie 1 x zz (s xx)
(17)
Notice that with this formulation, p and s can be expressed as separate factors. We find this result useful, since in practice we calculate the log-likelihood both probability distributions separately and balance the resulting loss term
L = (xis,) [oo Ps (plz)) + (zjss))) (s + (TL (ossss))llp(z( x)]lp(z|s))
(18)
where we use to balance the contribution of both terms to the optimization process.
Hidden state regime. When considering the state as variable hidden from the input data, we can still sample from the estimated posterior distribution as follows:
(o(2y)k,x)p((k>) p(k'x)
(19)
where p(k||x is the probability distribution which models the states at each time-step. this setting, we modify Eq. 16 and condition solely on the object dynamic variables.
and
P = /() $ = fx
(20)
Y < is a temperature factor which increases the confidence the model predictions. Since we should be sampling from Eq. and and we consider pkk(xx) = P& (A(x"), in practice we have
2/
wifk (k(x')
(21)
4 EXPERIMENTS
We now present the experiments that have been performed to evaluate the method and compare it to the baseline method, amortized causal discovery (ACD) (Löwe al., 2020). All models have been implemented using Pytorch (Paszke et al. 2019) and all training and test processes have been carried out on NVIDIA RTX 2080Ti GPUs.
4.1 EXPERIMENTS ON LINEAR DATA
We start with a simple scenario with linear message passing operations between number of differ- ent time-series, where each time-step t in series i is a one-dimensional continuous variable p e R.
The variables are connected by edges of n. different types. Each edge-type represents different causal effect that variable can perform to another. This is denoted by {P& 1 f represents no connection, i.e., no causal interaction between a pair of variables. At each time-step I, each element in the sample performs the following operation to update P.:
p +1 app Σ
3&p
(22)
igj
where E IR controls the self-connection, and 3g E R represents the edge-type connection between j and i, of type k.
We carry out experiments in this scenario to compare the results obtained by the two proposed architectures and the baseline method ACD Löwe et al. (2020). We are interested in observing whether our formulation is capable of recovering the underlying parameters of the linear message passing mechanism and identify the causal interactions in each sample in different types of scenarios (see Section [] In this case, experiment with observed states that are independent from the observations, and hidden states (first and third scenario classes). Unless noted otherwise, we set K states.

---

Under review as a conference paper at ICLR 2022
Table 1: Edge-type accuracy (in %), reconstruction MSE, and distance world parameters using linear data for 3 variables, 2 states, and 2 edge-types.
ERGE
RECONST. MSR
DIST.
TINT
TIaT
ACD
Lowe et al. 2020
FIXED DECODER
66.36 +: 0.12
66,02 0.29
0:40
a D.01
10
0..9 =
1.9
10"
ACD
0.13
66.44
00229
0.47
8.60-
10-3
0.77
1.99
10"2
10-3
SDC
STATE FIXED DEC.
90.00 0.10
50.43
0.25
2.29
10-3
2.1
10-21
1.55
SDCI
STATIC
2.04
93.84=0.19
1.34
10
1.1-
10-3
1557-10"
4.400-100
10-
6.80 6.80-10-4
SDCI
TEMPORAL.
82.97 + 0.13
82.79 0.28
7.03
10
1.2.2111
10
7:43
10
4.799 10
SDCI
TEMPORAL.
49.92 4:133
49.97
0.28
0.86 4 1.61-107
0.84
0:22
2.18- 2.18-1022
4.1.1 DATA GENERATION
The procedure for generating this dataset is as follows. First, we set the edge-type interactions. In our experiments we set
and since we experiment with edge-types, we set 31 0.05. To generate each sample, we need to sample the initial values of the continuous variable for each element P and the underlying causal structure dependent on the state, g(s). At each time-step, it suffices to query the edge-type k for each pair of variables and apply the corresponding causal effect 3& following Equation 22 The edge-type is k = ((s{) jás where ((s) denotes the causal effect from j to i, which has been defined at the beginning of the sequence. For all our experiments with this dataset, simulate N 3 variables for T 40 time-steps. We decide to keep N and the interaction values (3x) low because the generated samples are unstable, which imply that the data is not restricted within certain range and could be problematic for higher values of N and T.
When considering hidden states, the state is updated as follows: sf = 1 <p]<0))
4.1.2 TRAINING SPECIECATIONS
All the models participating the experiments of this section have been trained following the same training scheme, including ACD (Löwe et al. 2020
Customized decoder. One of our objectives is to recover the underlying world parameters fa. Thus, we implement a decoder which imitates the message passing operation presented in Equation 22 which allows to initialize the decoder using the underlying world parameters and analyse the performance of the encoder as a separate entity from the whole model.
Model parameters. Following Kipf et al. (018), the models have been trained using ADAM optimizer (Kingma Ba 2015). The learning rate of the encoder is 5 10 the learning rate of the decoder is 1 10 " and both are decayed by factor of 0.5 every 200 epochs. We train for 1000 epochs using a batch size of 128. We use teacher foreing every 10 time-steps during training. This implies that the decoder receives the ground-truth as input every 10 time-steps, otherwise it uses its previous output. The temperature t is set to 0.5 and the variance of the Gaussian distribution of the decoder for p is o² = 5 10-5, When considering the setting where we make the state dependent on the dynamics of the objects, we set À = 10a. For hidden states, we set the temperature ? = 0.1.
4.1.3 RESULTS
We firstly consider two edge-types and only one state to search for suitable learning rates for the encoder and the decoder. The edge-type accuracy of our method considering SDCI-Static for this setting is 94.88% on training data and 94.87% on test data. Since SDCI-Static is equivalent to ACD (Löwe et al. 2020) in this setting, we use this value as a reference for the following experiments.
We now proceed to comparing the performance between the two architecture choices SCDI-Stari and SCDI-Temporal with the baseline ACD, and evaluate the effect of explicitly modeling the under- lying state. Furthermore, we compare the two proposed methods SCDI-Stati and SCDI-Tempont with a a variant where the decoder is fixed and uses the ground-truth parameters, Br. We will in the following compare all these models considering two edge-types.
Table shows the edge-type identification accuracy. the distance to the world parameters, and the reconstruction MSE for three variables, two states, and two edge-types (no-edge and (1). As can see, our method SDCI-Static successfully performs the task of identifying the state-dependent causal interactions. Furthermore, it recovers the underlying parameters of the generative process
7

---

Under review as a conference paper at ICLR 2022
with great accuracy. On the other hand, our SDCI-Temporal is not able to identify properly the causal interactions nor recover the underlying parameters with enough accuracy. However, when fixing the decoder using the true underlying parameters, it achieves decent performance. Regarding our predecessor (ACD), we observe that its performance is considerably lower than our proposed ar- chitectures. One should note that this formulation can still recover the underlying world parameters, but less accurately compared to our formulation with SDCI-Static.
We also experimented with SDCI-Static when considering the state variable hidden from the input data. The results show an edge-type identification accuracy of 92.25% and 91.96% in training and test data respectively. The distance to the world parameters 1.06- 10 10- To assess the effectiveness of the model to identify different behaviors (states) when considering this setting, we can inspect the estimation of the latent state probability distribution. The state decoder accuracy of the latent state function is 99.19....1 and 98.97% in training and test data respectively. We can observe that our formulation allows considering hidden state variables. SDCI-Static is able successfully identify the causal interactions between the elements and successfully decomposes the different behaviors into the actual underlying states of the generative process.
4.2 EXPERIMENTS ON SPRING DATA
In the second experiment setting, we evaluate our methods on data similar to that used in recent work (Kipf et al. 2018 Löwe et al. 2020), consisting of particles (or small balls) connected by springs with directed impact- meaning that e.g. particle i could affect particle j with foree through a connecting spring, but leaving particle unaffected by this spring force.
The following experiments focus solely evaluating the performance of our SDCI-Static and com- paring it with its predecessor (ACD) under the three first scenario classes introduced in the Introduc- tion. The data generation process follows the description of Kipt al. (2018). The only difference is the addition of the state variable to the generative process, which affects the edge-type at each time-step in the previous case. For more details, see the supplementary material. As in the first experimental setting, the experiments regarding this data always consider 2 edge-type connections (presence/absencce of directed spring) and sequence length of 80.
4.2.1 TRAINING SPECIFICATIONS
We use the same training scheme for all the models present in the experiments of this section. In this case, the configuration is identical to the one used by Kipt et al. (2018). Thus, the experiments have been trained using ADAM optimizer (Kingma & Ba 2015). The learning rate of both the encoder and decoder is 5 10 4 and decayed by a factor 0.5 every 200 epochs. We train for 500 epochs using a batch size of 128. We also teacher forcing every 10 time-steps during training. The temperature т is set to 0.5 and the variance of the Gaussian distribution of the decoder for is r² = 5 10-5 When considering the setting where we make the state dependent the dynamics of the objects, we set = 10 For hidden states, we set 7 = 0.05.
4.2.2 RESULTS
Let us consider the first scenario class, where the state is known and independent from the obser- vations. The state transitions incrementally into the next one every 10 time-steps, and alternative settings with up states are explored. Table 2 shows the corresponding results, where for each for state (from to 8) new dataset is generated and used to train the SDCI-Static method. We observe that with one state, the original results reported by Löwe et al. 2020 are obtained since this case corresponds to stationary time-series. As the number of states increase, the accuracy in edge- type identification decreases and the reconstruction MSE becomes larger. However, our SDCI-Static is able to maintain promising results achieving 74.87% accuracy in edge-type identification when having as much as 8 states.
We now proceed to the scenario class number 3, where the state of particle transitions when it collides with the wall of the box where it is contained. For simplicity. we only consider two states that transition alternatively type identification accuracy of 13% on training data and 79.21% on test data. The reconstrucion MSE is 1.32 10-1 and 1.38 10-3 on training and test data respectively. Since now our decoder
wall collision. Our proposed method SDCI-Static achieves edge-
8

---

Under review as a conference paper at ICLR 2022
Table 2: Edge-type accuracy (in %) and reconst. MSE using spring data with different states for SDCI-Static.
NUM. STATES
EDGE ACCURACY
RECONST. MSE
TRAIN
TEST
TRAIN
TEST
99.70 1: 0.02
99.67 0.13
7.88-10"* =533 16.38-10" 10
7.88
10 1
4.64 10
98.80 = 0.02
9..1
8.00-10-1=14
10-3
1.02
98.00 0.03
0..09
2.50-10-
=3:26
2.33
10-2
4.644
10-
89.88 0.04
0.10
5.36-10-4
10-9--0--6
6.57
10
4.223
10
79.37 0.03
74.87 + 0.08
9.59 10-3 3.50
3.50-10-3
3.02
10-2 1.63
10-
requires modelling the conditions for state transition, we can evaluate its performance in identifying state changes. The state accuracy of the decoder is 99.73% on training data and 98.53% on test data, which shows that our method is able to detect the effect that the event of a particle colliding with the wall will have on the whole sample. If we consider ACD, it achieves an edge-type accuracy of 69.87% on training data and 68.61% % on test data. The reconstruction MSE is 4.45 10 and 1.46 10-3 on training and test data respectively, and the state accuracy of the decoder is 99.13% on training data and 98.21% test data. As stationary time-series data limits the performance of the causal graph inference task. However, the results for the reconstruction MSE are comparable the ones obtained with our formulation (SDCI- Static), which indicates that the method still makes decent predictions although it fails in identifying the edge-type interactions.
see, the limitation of ACD of considering only
Finally we address a scenario of class 2 as listed in the Introduction. In this scenario, the underlying state of particle changes depending its location the box; thus, the state is not directly observable but indeed directly dependent on an observable variable, the position of the particle. Our proposed method SDCI-Statie shows an edge-type identification accuracy of 85 35.35% and 80.82% on training and test data respectively. The reconstruction MSE is 1.44 10-4 and 1.19. 10-3 on training and test data respectively. Notice that in this case, we also find accuracy levels that are similar to the second and third layout. If we perform inspection on the estimation of the latent state variable distribution, we observe a state decoder accuracy of 98.96% in training data and 99. 99.41% in test data. As we can see, our formulation allows to learn the dynamics of the state without any supervision. Regarding ACD, it achieves an edge-type accuracy of 71.06% in training data and 699.0% in test data. The reconstruction MSE is - and 1.31 10-3 in training and test data respectively. As before, ACD is restricted since it considers a stationary generative process. However, the behavior of the sequences in this data regime is non-stationary as the state variable is hidden. Our SDCI-Static successfully decomposes the non non-stationary dynamics into the true underlying conditional stationary ones and is able to identify the causal links between particles successfully enough.
CONCLUSIONS
In this work we propose a method, SDCI, for performing causal discovery in scenes with multiple objects interacting over time, and where the dynamics depend on the value of underlying states. SDCI allows recovering the underlying causal structure of non-stationary time-series data by condi- tioning its stationarity on categorical state variables. Furthermore, we provide deep probabilistic implementation of this method and perform an empirical study on two synthetic scenarios. Our results show the effectiveness of our formulation in modelling conditional time-series data, in com- parison to the state-of-the-art approaches that consider stationary time-series data.
5.1 FUTURE WORK
This work can be regarded as preliminary but necessary contribution towards causal discovery in video. In order to approach this goal, we propose to add a visual front-end as a pre-pree pre-processing step of the SDGI module. This front-end would operate on the video and output probabilistic segmentation of video into object hypotheses, pose extraction of humans in the scene, detection and recognition of known object classes, etc.
This automatical extraction of causal abstractions of natural scenes would enable taking steps to- wards challenging tasks such as high-level scene understanding or counterfactual reasoning.

---

Under review as a conference paper at ICLR 2022
REFERENCES
David Maxwell Chickering. Optimal structure identification with greedy search. Journal of machine learning research, 35755):507-554, 2002.
Doris Entner and Patrik Hoyer. On causal discovery from time series data using fci. Probabilistic graphical models, pp. 121-128, 2010.
Rohit Girdhar and Deva Ramanan. CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoning. ICLR, 2020.
Clark Glymour, Kun Zhang, and Peter Spirtes. Review of causal discovery methods based on graph- ical models. Frontiers in genetics, 10:524, 2019.
Clive WJ Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica: journal of the Econometric Society, pp. 424 438, 1969.
Patrik Hoyer, Dominik Janzing, Joris M Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. Advances in neural information processing systems, 21:689-696, 2008.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
Diederík P. Kingma and Max Welling. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Bansf, AB, Canada, April 14-16, 2014, Conference Truck Proceedings, 2014.
Thomas Kipf, Ethan Fetaya, Chan-Chieh Wang, Max Welling, and Richard Zemel. Neural relational inference for interacting systems. In International Conference Machine Learning, pp. 2688- 2697. PMLR, 2018.
Yunzhu Li, Antonio Torralba, Anima Anandkumar, Dieter Fox, and Animesh Garg, Causal discov- ery in physical systems from videos. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (eds.). Advances in Neural Information Processing Systems, volume 33, pp. 9180-9192 9192. Curran Associates, Inc., 2020.
Sindy Löwe, David Madras, Richard S. Zemel, and Max Welling. Amortized causal discovery: Learning to infer causal graphs from time-series data. ArXiv, abs/2006. 10833, 2020.
Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. The conerete distribution: A continuous relax- ation of discrete random variables. In 5th International Conference on Learning Representationss ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.
Kevin P Murphy et al. Dynamic bayesian networks. Probabilistic Graphical Models, M. Jordan, 7:
431, 2002.
Roxana Pamfil, Nisara Sriwattanaworachai, Shaan Desai, Philip Pilgerstorfer, Konstantinos Geor- gatzis, Paul Beaumont, and Bryon Aragam. Dynotears: Structure lcarning from time-serics data. In International Conference on Artificial Intelligence and Statistics, pp. 1595-1605, PMLR, 2020.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary Vetito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, pp. 8024-8033 8035. Curran Associates, Inc., 2019.
Judea Pearl. Causality. Cambridge university press, 2009.
Jonas Peters, Joris M. Mooij, Dominik Janzing, and Bernhard Schölkopf. Causal discovery with continuous additive noise models. Journal of Machine Learning Research, 15(58):2009-200 2014.
10

---

Under review as a conference paper at ICLR 2022
Rajesh Ranganath and Adler Perotte. Multiple causal inference with latent confounding. arXiv preprint arXiv:1805.084 08273, 2018.
Axel Sauer and Andreas Geiger. Counterfactual generative networks. In International Conference on Learning Reprexentations, 2021.
Shohei Shimizu, Patrik Hoyer, Aapo Hyvärinen, Antti Kerminen, and Michael Jordan. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(10), 2006.
Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28: 3483-3491, 2015.
Elizabeth S Spelke and Katherine D Kinzler. Core knowledge. Developmental science, science, 10(1):89-96, 2007.
Peter Spirtes. An anytime algorithm for causal inference. AISTATS, 2001.
Peter Spirtes, Clark N Glymour, Richard Scheines, and David Heckerman. Causation, prediction, and search. MIT press, 2000.
Ruibo Tu, Cheng Zhang, Paul Ackermann, Karthika Mohan, Hedvig Kjellström, and Kun Zhang- Causal discovery in the presence of missing data. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 1762-1770. PMLR, 2019.
Kexin Yi, Chuang Gan, Yunzhu Li, Pushmeet Kohli, Jiajun Wu, Antonio Torralba, and Joshua B. Tenenbaum. CLEVRER: collision events for video representation and reasoning. In ICLR, 2020.
Kun Zhang and Aapo Hyvärinen. On the identifiability of the post-nonlinear causal model. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI '09, pp. 647-655, Arlington, Virginia, USA, 2009. AUAI Press. ISBN 9780974903958.
Kun Zhang, Mingming Gong, Joseph Ramsey, Kayhan Batmanghelich, Peter Spirtes, and Clark Glymour. Causal discovery in the presence of measurement error: Identifiability conditions. In UAI 2017 Workshop on Causality: Learning. Inference, and Decision-Makinn 2017.
Xun Zheng, Bryon Aragam, Pradeep Ravikumar, and Eric P. Xing. DAGs with NO TEARS: Con- tinuous Optimization for Structure Learning. In Advances in Neural Information Processing Sys- tems, 2018


Reviews:
[{"review_id": "u9N5PSGra0B", "reviewer": "Reviewer_ukPw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, the considered setting is interesting, but the paper organization and writing need to be improved. For example, in some parts, it seems that the authors are considering instantaneous causal relations, while in some parts, it seems the authors are considering time-delayed causal relations. It is not clear to the readers. Also, I am not sure about the state variable, whether it is observed or not, because it is not consistent throughout the paper; for example, in Section 3.2 which discusses the implementations, the state variable seems to be observed. \n\nAnother concern is about theoretical identifiability. When claiming causality, it is essential to show the theoretical properties, e.g., identifiability of the causal graph. However, the authors did not give any theoretical results in the paper. \n\nSome related references are missing, which consider causal discovery in a similar setting:\n1.\tB. Saeed, S. Panigrahi, and C. Uhler. Causal Structure Discovery from Distributions Arising from Mixtures of DAGs. ICML, 2020.\n2.\tB. Huang, K. Zhang, J. Zhang, J. Ramsey, R. Sanchez-Romero, C. Glymour, and B. Schölkopf. Causal Discovery from Heterogeneous/Nonstationary Data. JMLR, 2020.\n", "summary_of_the_paper": "This paper considers causal discovery from nonstationary time series, where the nonstationarity is due to some unobserved state factors. To handle this setting, the authors propose an approach based on probabilistic deep learning. ", "summary_of_the_review": "Interesting setting, but essential theoretical results are missing, and the paper organization and writing needs improvement (some parts are inconsistent throughout the paper)\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "poEBVeY93Zx", "reviewer": "Reviewer_15XZ", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "While the paper seems to provide an interesting and novel method, it also seems to extend the work of [Lowe et al 2020] to conditionally stationary time-series in a relatively straightforward fashion. \n\nOne of the issues I have with this paper is that it fails to address if the model is indeed identifiably causal or just a graph that fits well the data. In [Lowe et al 2020] the graph is shown to be causal under  the assumptions of first-order Markovianity, no latent confounders, no instantaneous relations and assuming the optimization procedure finds the global optimum. In this case, the authors should provide similar results, taking especially into account the fact that there might be latent confounders (e.g. the s_i might play this role if they are related with other s_j). Moreover, it isn't clear to me if the latent states can be uniquely reconstructed in a way that would render the time series conditionally stationary. \n\nAnother issue I have is that it doesn't adequately cover the related work, for example there are other methods that model non-stationarity in a similar way, e.g. https://arxiv.org/abs/1903.01672. This also extends to the evaluation seems to be focused almost exclusively on [Lowe et al 2020] (which unsurprisingly doesn't work if the data is non-stationary).", "summary_of_the_paper": "The paper presents a causal discovery method for time-series that generalizes [Lowe et al 2020] to the conditionally stationary case. In particular, the method assumes there is a latent variable $s_{i}^t$ for each timeseries $i$ that influences the graph and edge types at each timestep, such that conditioning on this variable the time-series is stationary.", "summary_of_the_review": "In summary the paper is promising, but not ready yet. A better embedding in the existing research, a few theoretical results (mostly about making explicit under which assumptions the graphs learnt are actually causal) and a more thorough evaluation would improve it a lot.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "bCdF_ECrIWj", "reviewer": "Reviewer_quAM", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- The problem domain of inferring causal structure from time-series in a non-stationary context is a very important one, and the most realistic setting that one can expect.\n- The proposed method depends on existing methods, meaning that is a reasonable improvement that is likely to be reproducible.The idea of conditioning on a categorical state variable is a nice and logical one\n\nWeaknesses:\n- I believe that the biggest and very important weakness is the inherent limitations of the model regarding data complexity, as the experiments also show. While a general framework is provided, I doubt if the model can truly account for non-stationary temporal processes to infer causal graph structure. That is a very hard problem, which is already hard in non-temporal processes with interventions. For instance, is a K-dimensional categorical variable (for K states) sufficient for encoding a latent state from complex observational data x? I suppose that the model could be extended for other types of variables or distributions, but is unclear if that would really work in practice (and in that sense if the proposed model is the right direction).\n- It seems that currently the only latent variables encoding information about the generation process are the edge variables and the state variables, assuming they are not observed in the respective setting. If they are both categorical, do they have the necessary capacity to model complex temporal patterns observed in x?\n- Extending on the previous points, the experiments are limited to synthetic and rather low complexity cases, although as I said above, this is a hard problem and likely harder settings would be not possible likely. The first experiment is on linear, one-dimensional observational data with K=2 states, while the second experiment is also low dimensional in the observation space (I think 2 dimensions) and up to 8 state variables. What would happen if the observation space is much higher dimensional, say a few dozens, and inflicted with noise?\n- In the experiments, I think it is assumed that the number of states is known in advance and equal to K. Is the case of unknown or even varying K explored? Or what if K is set to be much higher than the actual number of states, just to cover all bases? Would that lead to a collapse, of certain useless dimensions? Or would it cause the model to mistakenly assign causal graphs to all states regardless.\n- As an encoder, graph neural networks are used. I assume that these have a fixed, fully-connected topology, but it would describe\n more clearly the precise connectivity. I also assume the connectivity is fixed across experiments. Are there any vanishing/exploding gradients observed by increase sequence length?\n- I am not sure about the validity of some statements. In the beginning of the related work it is said that ‘Causal discovery approaches aim … from observational data’. I am not sure if this is precise enough, as typically interventional data is also used, at least when not in a temporal setting. \n", "summary_of_the_paper": "This paper proposes a new method for discovering the causal graph from time-series data when the time-series are generated by a non-stationary process. The method relies on previous work from Lowe et al, 2020 and proposes to condition the causal summary graph driving the (causal) edge generation between variables by a categorical state variable. The method is defined within a variational inference framework, where edges are state-dependent latent variables, based on which one can generate/reconstruct future observations and/or state variables. Results are shared on two synthetic datasets with promising performances.", "summary_of_the_review": "I like the approach, but I am not sure if it is ready enough for publication. I thus initially recommend borderline reject.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "87poqHMvILc", "reviewer": "Reviewer_BArP", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work is based on state-dependent causal inference for time series (TS). The goal is to extract a causal summary graph $G$ (Fig. 1) where the edge type can change based on state. The proposed method is based on conditional VAE. It considers an auxiliary variable s (states of all the edges in one TS) which is conditioned on by the prior.\n\nStrengths:\n1. Very detailed experimental setup is given for each experiment, which helps reproducibility.\n2. It considers two evaluation tasks at one time -- causal graph extraction and TS reconstruction (not forecasting).\n\nweakness:\n1. The presentation really really needs improvements.\n2. It only compares with one baseline ACD, only in linear data. In the spring data they did not compare the proposed method with anything but just show a sensitivity analysis on the number of states.\n\n\n# details:\n1. Without a figure illustrating the model structure, it is very hard to interpret what variables like $k$ stand for and why do we need to have such variables.\n2. It is very hard to understand what is the input and output of the function $G$.\n3. It really needs clarification on all the notations. For example, what is the difference between $G$ and $\\mathbf{g}_{ij}^t$?\n\n# questions:\n1. What is a state?\n2. Which variable is the source variable in Fig. 1?\n3. Why is $s_j^t$ not an input for $G_{ij}$ in eq.(2)? If this is the case, how do you guarantee $G_{ji}$ and $G_{ij}$ are consistent with each other. E.g., you cannot have $i$ causes $j$ and $j$ also causes $i$ at the same time.\n4. Why is $G$ a function if it is a graph?\n5. Would the dimension of $z$ be quite high since it represents all the edges among the variables.\n\n\n\n", "summary_of_the_paper": "This paper aims to solve the problem of causal summary graph extraction and time series reconstruction at the same time. They propose a conditional VAE based model. The model is conditioned on state variables $s$, which makes it different from a normal VAE. Experiments on two datasets show the method outperforms ACD on linear data and its performance drops as number of states increase with spring data.\n", "summary_of_the_review": "\nStrengths:\n1. Very detailed experimental setup is given for each experiment, which helps reproducibility.\n2. It considers two evaluation tasks at one time -- causal graph extraction and TS reconstruction (not forecasting).\n\nweakness:\n1. The presentation really really needs improvements.\n2. It only compares with one baseline ACD, only in linear data. In the spring data they did not compare the proposed method with anything but just show a sensitivity analysis on the number of states.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper extends Lowe et al. 2020 to discover causal relations from nonstationary time series by assuming conditionally stationarity of the times series. Based on the assumption, a deep learning method based on VAE is proposed to learn the causal relations from data. 
The paper is well-motivated and well-organized.

However, there are some concerns from the reviewers. 1) The presentation needs significant improvement, e.g., clarification of the notations.  2) The identifiability of the causal graph is not given. It is unclear under what conditions the proposed method can discover the true causal graph. 3) The capacity of the discrete states might not be able to handle complex real situations. 4) The experiments are limited to synthetic and low complexity cases. This further weakens the significance of the proposed method given that there are also no theoretical guarantees of the proposed method. 5) Discussions about some important relevant works are missing.

Overall, the paper studies an interesting problem. However, given the above concerns, the novelty and significance of the paper will degenerate. Both theoretical and empirical analysis of the proposed method need further improvement. Addressing the concerns needs a significant amount of work. Thus, I do not recommend acceptance of this paper.
Decision: Reject