Connecting Data to Mechanisms with Meta Structual Causal Model
Authors: Gong Heyang
Venue: ICLR 2022 Submitted (2022)
Keywords: meta-SCM
cyclic causal models
sufficient activated mechanisms

Abstract: Recent years have seen impressive progress in theoretical and algorithmic developments of causal inference across various disciplines in science and engineering. However, there is still some unresolved theoretical problems, especially for cyclic causal relationships. In this article, we propose a meta structure causal model (Meta-SCM) framework inspired by understanding causality as information transfer. A key feature of our framework is the introduction of the concept of \emph{active mechanisms} to connect data and the collection of underlying causal mechanisms. We show that the Meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations. In addition, we propose a \emph{sufficient activated mechanisms} assumption, and explain its relationship with existing assumptions in causal representation learning.  Finally, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.

Under review as a conference paper at ICLR 2022
CONNECTING DATA TO MECHANISMS WITH META STRUCTURAL CAUSAL MODEL
Anonymous authors Paper under double-blind review
ABSTRACT
Recent years have seen impressive progress in theoretical and algorithmic devel- opments of causal inference across various disciplines in science and engineer- ing. However, there still some unresolved theoretical problems, especially for cyclic causal relationships. In this article, we propose a meta structure causal model(meta-SCMM) framework inspired by understanding causality information transfer. A key feature of our framework is the introduction of the concept of active mechanisms connect data and the collection of underlying causal mech- anisms. We show that the meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations. In addition, we propose sefficient activated mechanisms assumption, and explain its relationship with existing hypotheses causal inference and learning. Finally, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.
INTRODUCTION
Although there have been significant advances in causal research in recent years, there are still some important theoretical problems that have not been resolved. One of the most notoriously hard problems is about cyclic causal relations, and there is no causal modeling frameworks can properly handle it. In modern theory of causality, the mathematical framework called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges(Pearl 1995] Pearl al., 2009 Bongers et al. 2016 Bareinboim et al. 2020). Acyelic SCMs, also known as recursive SEMs, form special well- studied subclass of SCMs that generalize causal Bayesian networks. They have many convenient properties and are widely used in practical causal modeling, see e.g. (Evans) 2016 Lauritzen 1996
Richardson 2003; Maathuis et al. 2018). But there a strong need to go beyond acyclic SCMs.
In fact, there are feedback loops between observed variables in many systems occurring in real world. Causal cycles may arise when one approximates such systems over time (Fisher [9700] Mo- gensen al. 2018: 2020), when one describes the equilibrium states of these systems (Iwasaki
& Simon] 19 1994; Lacerda et al. 2012 Hyttinen et al., 2012) Mooij et al. 2013 Bongers & Mooij, et al. 2020 Pfister al. 2019), though the underlying dynamic processes describing such systems have an acyclic causal structure over time. In particular, it was shown that the equi- librium states of a system governed by (random) differential equations can be described by an SCM that represents their causal semantics in (Bongers & Mooij 2018), which gives rise to a plethora of SCMs that include cycles. In contrast to their acyclic counterparts, many of the convenient proper- ties do not hold for SCMs with cycles, and they are not as well understood. Some progress has been made the case of discrete (Neal, (2000) and linear models (Spirtes) 1993. 1994 2013) RRchardson et al. 1996; Koster et al.) 1996; [yttinen et al. 2012), and more recently the Markov properties (Forré& & Mooij 2020 2017) and theoretical foundation (Bongers et al.) 2016). Researchers are mostly making additional assumption of the underlying causal mechanisms to circumvent compli- cations of cyclic SCMs in causal semantics, solvability, marginaliztions etc.. However, they are still not well understood. Even, a pressing concern is whether SCMs are able to completely model dynamical systems at equilibrium and the causal constraints CCC) is proposed but without graphical interpretations yet(Blom et al. 2020).

---

Under review as a conference paper at ICLR 2022
After introducing the formal definition of SCMs and relevant preliminaries in Section trace back into the philosophy accounts causality and propose the meta structural causal model (meta- SCM) based understanding causality information transfer in Section The meta-SCM frame- work is constructed by an SCM and an extra dimension which describes how to connect data to mechanisms through the concept of active mechanisms. In particular, meta-SCM induces sub- model for each sample in the dataset coarsely related to the active set. The new framework is proved be more expressive by Theorem 7 and its potential ability address cyclic casual relationships is illustrated by an example of cyclic SCM with multiple solution and an unsolvable cyclic SCM. Comparing to the joint causal inference for meta meta-system (Mooij al.) (016) which can also deal with cyclic SCMs, the meta-SCM framework avoids to add extra context variables and can even gain insights on unsolvable SCMs.
The Section f addresses the challenging case where no additional variables, besides the samples from the data to generate, are observed. The sufficient activated mechanism (SAM) hypothesis proposed
central assumption in the meta-SCM framework, which is consistent with the role of independent causal mechanisms (ICM) principle or sparse mechanisms shift (SMS) assumption also compared with the
for the SCM framework. Moreover, the SAM and SMS hypotheses lens of informational decomposition of SCM, and it reveals that the SMS assumption might be not appropriate in certain with example. Section conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.
The main contributions of this paper
I) We propose totally novel dimension that describes how to link data mechanisms to the existing causal modeling framework, particularly, a meta structural causal model frame- work which can be used to circumvent technique complications in cyclic SCMs.
2) We propose a SAM hypothesis as an inductive bias for performing causal inferences and learning consistent with the role of SMS assumption.
PRELIMINARIES
At the centre of modern causal modeling theory lies the structural causal model (SCM) (also known as structural equation model) which makes graphical assumptions of the underlying data generating process. There are many somewhat different formulations of SCM in literatures, e.g., [Schölkopf (2019); PParl (2019); Bongers et al. (2016); |Paarl et al. (2009); Forré & Mooij] (2020), among which definition Blom et al. (2020) is used in this paper.
Definition (SCM) Let I and be index sets. Structural Causal Model (SCM) M is triple (X, F, E), with:
a product of standard measurable spaces X' I, (domains of endogenous).
tuple of exogenous random variables E = (EE=))))eee [Ej)eer taking value in product of standard measurable space & = -IIEEs Ej.
a family of F of measurable funtttrioalennniionn functions:
fi Xpa(Arz Epa()nn + X. VIEI.
The dataset are ususally assumed be a set of samples for the solution of SCMs.
Definition 2 (Solution of SCM) We say that random variable X (Xt) is solution to an SCM M = (X, (X,FFE)
X, x, fx pa(sjizz Epa()nn) o.s., ViEI An SCM may have a unique(up to zero sets) solution, multiple solutions, or there may not exist any solution at all.
Definition 3 An SCM M is called simple it uniquely solvable with respect any subxel O I.
All acyclic SCMs are simple.
2

---

Under review as a conference paper at ICLR 2022
Definition 4 A do intervention do(31) with target I _ I and value γ1 E X2 on SCM M
(X, F, F, maps it to the intervened SCM Mao(ār) (X, F, E) with F the family of measurable functions:
fi(Xpa(t)nzz (pa())n))))) = (2; ( (Xxx((())Zz (pa())n))
i i TET\T
The intervened SCM is referred as submodel of the original SCM, in fact, the variants derived from many different types of interventions (c.g., perfect, imperfect, stochastic, etc.) are also referred as submodel.
META STRUCTURAL CAUSAL MODELS
One critical insight in philosophy is that the causal mechanisms behind system under investigation are not generally observable, but they do produce observable traces ("data," in modern terminology). This insight naturally leads to two practical desiderata for any proper framework for causal inference, namely:
1. The causal mechanisms underlying the phenomenon under investigation should be ac- counted for indeed, formalized the analysis.
2. This collection of mechanisms (even if mostly unobservable) should be formally tied to its output: the generated phenomena and corresponding datasets.
The mathematical object called a structural causal model (SCM) is used to represent the causal mechanisms from which causal hierarchy to describe the generated phenomena organically emerges. is often assumed that every instantiation E = e of the exogenous variables uniquely determines the values of all variables in X (Pearl 2019). which leads unique solution of the corre- sponding SCM, Then the dataset D [x I...,N set of N samples of the unique solution. But in many cases, SCM with cycles might be not solvable or have multiple solutions (Halpem 1998).
Example (Multiple Solutions) Consider an SCM M F (EE)), where (=3 + D2 1)/3 et/3
x2 t T1
Obviously, 11- E1) and (1 E1.1+ E1) are two different solutions to M) then which solution of the SCM should be used to link the dataset to the model?
The previous causal inference literature rarely deals with theoretical aspects of cyclic causality. In recent years, it has been formally discussed in Bongers et al. (016)) However, it is also acknowl- edged by this paper that there many complications in dealing with cyclic causal models. The vast majority of methods to deal with cyclic SCM in the literature are by adding additional assump- tions, such as linear constraints(Spiree) [993] 1994 Hyttinen al.) (012) and certain solvability constraints
& Mooij 2018) Bonggers et al. 2016). These methods basically exclude the study of SCMs with multiple solutions such as Example
The view of understanding causation as information transfer was first formally proposed by (Collier 1999) in philosophy recently. Inspired by this view, we realize that the causal links among variables can be cut off suggested by unsuccessful information transfer, which suggests that different samples might have different causal graphs and causal mechanisms. For example, samples of M might only satisfy only subset of the structural equations due to absence of information hence variables
for two different samples (+) and (i) could have two different causal graphs. However, we usually do not know when and where the information transmission was interrupted for given sample. In fact, it might be infeasible to specify the information transmission details of all samples of an SCM. To address the above problem, we introduce the concept of active sel of an SCM, which is inspired by the active set method in the field of non-linear optimization theory.
Definition (Active mechanisms) For a given sample (k) and the corresponding collection of
mechanisms represented by an SCM M = (X, F, E), x fi(xpa())zz (a(s)ng), then we call fi active mechanism, and denote the index set for all active mechanisms the active set Ag-
(k)
3

---

Under review as a conference paper at ICLR 2022
The collection of active sets {Ax 1==12,.. gives us the opportunity to avoid considering the details of the information transfer between variables, and to describe the relationship between the data and the model relatively concisely. Formally, define the meta structural causal model follows:
as
Definition 6 (meta-SCM) A collection of mechanisms described by an SCM M = (X, E) with dataset D {x(k) )k=t,..,,N in which each sample x6) satisfies that:
* the prior distribution of (M) is P(E);
AA II is referred as the active set of the sample le satisfies that
[i(xpa())zz pa(in))) Vi E As-
(1)
Then the tuple (M, Dyoor in short M) is called a meta structural causal model (meta-SCM).
The difference between SCM and meta-SCM. On one hand, an SCM can be interpreted as a special case of meta-SCM satisfies that the active set A& is equal to I for any sample x (*), On the other hand, meta-SCM share the causal mechanisms with its corresponding SCM only differs on the
method for linking data to model. Thus, it improves the expressiveness of the canonical SCM. Actually, a meta-SCM suggests method for connecting any dataset to an SCM with the active sets. Formally.
Theorem (Connecting Data Mechanisms) For an SCM M (X, E) with any dataset D (x((")== kk=I....N in the domain of X. Then each datapoint x()) is from some submodel SCM M related to the active set Ag.
M() let Proof For any ks 1,..,, N, let
(X, F, E) be SCM with modified causal mechanisms: (x(( (p(()n))) i A
f (pa())n)))
(k)
i A
Then the active set of datapoint (x) for the submodel M(k) is I by definition] which directly leads to our theorem.
The above proof directly assigns a submodel for each sample in the dataset, which only part of mechanisms in the original SCM holds. In fact, the submodel in our meta-SCM framework does not have to be constructed as doo-intervened model Mi(k), it can be any subclass of SCMs with desired properties(such as acyclic) in literatures.
When there are cyclic causal relationships between variables, one encounters various technical com- plications, which even arise in the linear (Bongers et al. 2016). The main idea for solving related difficulties is add additional restrictions structural equations, and the dataset sumed to be consisted of samples from a distribution obtained by solving the SCM. In contrast, our meta-SCM does not add additional assumptions on SCM, and each datapoint is treated as sam- ple of the distribution obtained by solving a certain submodel. For the SCM M with muptiple solutions in Example with dataset D = { }>=I....N in the domain of X' the meta-SCM can circumvent theoretical complications through providing each datapoint distribution of any solution of a certain submodel. Usually, the details of submodel and its corresponding distribution for each sample might be unknown, and meta-SCM only provides coarse description by the active sets.
In fact, we can also circumvent the technique complications caused by solvability through meta- SCM. Specifically, the structural equations of an acyelic SCM trivially have a unique solution, which ensures that the SCM gives rise to a unique, well-defined probability distribution on the variables. However, an SCM can be unsolvable in the case of cycles, e.g..
Example 2 (Unsolvable) Consider an SCM M = (X, F f21, E= {Ei)), where
23 2 + o +1
I1
F
x2 I
In fact, M(() is the do-intervened SCM M
ZVAN
4

---

Under review as a conference paper at ICLR 2022
Then the SCM M is obvious not solvable, thus it cannot be used model underlying causal mechanisms of any dataset D = )==1,..... in the domain of X. But with our novel approach, we might still connect data to the underlying mechanisms, e-g. by letting |Ag] = for all I,..., N. other words, each datapoint x(*) is sample of a distribution derived from the submodel M(k) by Theorem
The difference between meta-system and meta-SCM. The joint causal inference (ICI) framework reduces modeling a system in its environment to modeling the meta-system consisting of the system and its environment, which considers auxiliary context variables that describe the context of each data set (Mooij et al. 2016). In contrast, our meta-SCM address the challenging case where no additional variables, besides the samples from the data generate,
observed. For example in Fig. [ the meta-system consists two variables X1, X2 and a context variable C. More concretely, the engine X1 drives the wheels of the rotation of the wheels drives the engine. In a meta-SCM, we instead introduce the concept of active mechanisms to describe each sample in dataset. Moreover, the meta-SCM framework gain insights on unsolvable SCMs such as Example while the previous meta-system study restricts themselves to simple SCMs. In philosophy viewpoint, the JCI framework uses context changes to model interventions, which is a difference-makiee account for causality. Instead, the active set in meta- meta-SCM are inspired by information transferring which production account of causality, see e.g. Illari & Russo (2014).
X2 when going uphill C = O, but when going downhill,
(a)
(b)
(c)
(2)
A
{1)
4 = (1,2}
= 0:
C 1:
X1
X2
X1
X₂
XI
X2
Figure 1: The meta-system and meta-SCM when in the presence of cycles. a) X, causes X₂ in context C 0, f2 is activated; (b) X2 causes Xı in context C 1, is activated; (c) X1 and X₂ cause each other in the joint model, f1, f2 are activated.
From the above discussions, the meta-SCM framework can be considered as generalization SCM that novelly links data to causal mechanisms through active sets at individual level modeling" and it might be used to circumvent technical complications when cycles present. Then, a natural question is how to perform inferences with meta-SCM when in the presence of cycles? A first difficult might be the lack of knowledge on the specific form of active sets, i.e. AІ. for each sample. In the following section, we propose principle for learning and reasoning within the meta-SCM framework.
4 SUFFICIENT ACTIVATED MECHANISMS
The meta-SCM viewpoint considers set of variables indexed by I and set of samples D with causal (or disentangled) fuctorizution,
(K) P(x)))
II
P(x palonz)
(K) (k)
(x) D.
The independent causal mechanisms (ICM) principle tells us that the factors should be independent in the sense that
a) changing (or performing an intervention upon) one mechanism P(xx(xxo(())z) does not change any of the other mechanisms P(r,, (Xp((())))) (i + (Schölkopf et al. 2012), and
The individual level models is not new but widely used, e.g., in variational inference of machine learning (Bingham al. 2019).
5

---

Under review as a conference paper at ICLR 2022
b) knowing some other mechanisms P(z.) (pa(i)nz) (i does not give information about mechanism P(ry poa())nz)) Janzing & Schölkopf 2010
Consistent with the implication a) of the ICM Principle, the sparse mechanism shift (SMS) hypoth- esis (See e.g. Parascandolo et al. 2018); Schölkopf 2019)) was stated in the following:
Sparse Mechanism Shift (SMS). Small distribution changes tend manifest themselves in a sparse or local way in the causal/disennaaglled factorization, i.e., they should usually not affect all factors simultaneously.
is the fundamental assumption for causal representation learning Schölkopf et al. 202 and has recently been used for learning causal models (Ke et al. 2019), modular architectures (Goyal al. 2019 Besserve et al.) 2020) and disentangled representations (Locatello et al., 2020). However, the SMS hypothesis usually only assumes mechanisms shift from given context to the original uniquely solvable SCM, especially acyclic SCM. We propose a hypothesis in the viewpoint of given individual mechanism across samples in the following:
Sufficient Activated Mechanism (SAM). All factors in the causal/disentangled factor- ization should be activated sufficiently, i.e, every causal mechanism sufficiently often activated across samples in the dataset.
The "sufficient" in the SAM roughly understood sufficient for identifying factors in the causal factorization interest. In Example 2 the unsolvable cyclic SCM M² cannot used as the underlying generative process for any dataset. On the contrary, in the meta-SCM framework under SAM assumption, we are able to learn a causal factorization Eq. (2). even when only knowing how many factors have activated, but not which ones. In mathematics, start with solving the unsolvable equation x² = -1 with imaginary unit i, it has been an important branch of mathematics which is complex analysis with holomorphic function as a central research object. Similarly, the meta-SCM, which introduced novel dimension on linking data to SCM, can help in causal modeling with unsolvable SCM, and we feel meta-SCM with SAM assumption might be central research object for our framework as holomorphic function for complex analysis.
Overall, in the lens of two practical desirata for any proper framework mentioned in the begining of Section 3 our meta-SCM is different from all literatures on variants of SCMs by adding fully novel dimension on considering how each sample is tied to underlying mechanisms. This extra dimension though, on one hand, complicate the SCM framework, however, the other hand, improve its expressiveness and power for causal modeling. Then, the SAM hypothesis defines a subclass of meta-SCMs of interest for performing causal inferences and reasoning, consistent with the role of the ICM Principle/ SMS hypothesis in the SCM framework.
The causality behind interventions in the SCM framework is difference-malee making account, while the production accounts, especially informational interpretation of causality relatively neglected. One core idea of informational causality C causes E if there is information transmission from C to Elllarii & Russo 2014), suggests that the causal mechanism fa of an SCM can be separated into two part information process and information transfer (Gong & Zhu 2021). Formally,
Definition (Informational Decomposition of SCM) Consider an SCM M = (X, F,E. The informational dccomposition of M is defined as, for any e I,
24 t fi(ema()) (pa(i)na); Ej f- Tjj
(3)
where εj represents the information on edge (j.i) received from its input node j.
This dichotomy causal mechanisms can help illuminate on the relationship between SMS and SAM assumption. We tend assume the information processing mechanisms sparse inacti- vated, while the information transfer mechanisms are instead only sufficient activated. Henceforth, if we consider the inactivated mechanisms as shifted mechanisms, then the information process and

---

Under review as a conference paper at ICLR 2022
transfer mechanisms satisfying the SMS hypothesis and the SAM assumption respectively. Small distribution changes caused by interventions on information process tend to manifest themselves in sparse or local way, but might be manifest themselves in non-local way caused by changes transferred information. For example in starfish shaped causal diagram, changing the output in- formation of the centre node (instead itself) can lead shift on all functional relationships with its child nodes, which can affect almost all factors simultaneously. Thus, the SMS assumption might be appropriate to describe the intervened model for such system.
CONCLUSIONS
This work is mostly theoretical and conceptual address the notoriously hard problem of cyclic causal models. We proposed a novel active set approach for connecting data to SCMs (even not solvable), instead of making additional assumptions to restrict the class of SCMs of interest. Note that the meta-SCM is more of an conceptional modeling framework on how to relate data to the underlying causal mechanisms rather than specific model, and the SAM assumption has been introduced as an inductive bias for performing causal inferences and learning within the framework of meta-SCM. To best of our knowledge, this is the first causal modeling framework to explore the dimension formally on how to link each sample in the dataset to the collection of mechanisms. In the future, we might anticipate more causal modeling exploration on exacting information from activated mechanisms directly instead of the solution of SCMs.
REFERENCES
E Bareiinoom, JD Correa, D lbeling, and T Icard. On pearl's hierarchy and the foundations of causal inference. ACM Special Volume in Honor of Judea Pearl (provisional title), 2020.
Michel Besserve, Rémy Sun, Dominik Janzing, and Bernhard Schölkopf. A theory of independent mechanisms for extrapolation in generative models. arXiv preprint 2004v:2004.00184 2020.
Eli Bingham, Jonathan P Chen, Martin Jankowiak, Fritz Obermeyer, Neeraj Pradhan, Theofanis Karaletsos, Rohit Singh, Paul Szerlip, Paul Horsfall, and Noah D Goodman. Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20(1):973-978, 2019.
Tineke Blom, Stephan Bongers, and Joris M Mooij. Beyond structural causal models: Causal con- straints models. In Uncertainty in Artificial Intelligence, pp. 585-594. PMLR, 2020.
Stephan Bongers and Joris M Mooij. From random differential equations to structural causal models: The stochastic case. arXiv preprint arXiv:1803.088 2018.
Stephan Bongers, Jonas Peters, Bernhard Schölkopf, and Joris M Mooij. Theoretical aspects of cyclic structural causal models. arXiv preprint 1tt::1611.0630 2016.
John D Collier. Causation is the transfer of information. In Causation and laws of nature, pp- 215-245. Springer, 1999.
Robin J Evans. Graphs for margins of bayesian networks. Scandinavian Journal of Statistics, 43(3): 625-648, 2016.
Franklin M Fisher. A correspondence principle for simultancous equation models. Econometrica: Journal of the Econometric Society, pp. 73-92, 1970.
Patrick Forré and Joris M Mooij. Markov properties for graphical models with eycles and latent variables. arXiv preprint arXiv:1710.08775, 2017.
Patrick Forré and Joris M Mooij. Constraint-based causal discovery for non-linear structural causal models with cycles and latent confounders. arXiv preprint arXivv1807.03024, 2018.
Patrick Forré and Joris M Mooij. Causal calculus in the presence of cycles, latent confounders and selection bias. In Uncertainty in Artificial Intelligence, pp. 71-80. PMLR, 2020.
Heyang Gong and Ke Zhu. Path specifc effects based on information accounts of causality. arXiv preprint arXiv: 006000178... 2021.
7

---

Under review as a conference paper at ICLR 2022
Anirudh Goyal, Alex Lamb, Jordan Hoffmann, Shagun Sodhani, Sergey Levine, Yoshua Bengio, and Bernhard Schölkopf. Recurrent independent mechanisms. arXiv preprint arXiv: 2019.
J Halpern. Axiomatizing causal reasoning in uncertainty in artificial intelligence, eds. cooper g. moral s, 1998.
Antti Hyttinen, Frederick Eberhardt, and Patrik O Hoyer. Learning linear eyclic causal models with latent variables. The Journal of Machine Learning Research, 13(1)33387-344 2012.
Phyllis Illari and Federica Russo. Oxford, 2014.
Causality: Philosophical theory meets scientific practice. OUP
Yumi Twasaki and Herbert A Simon. Causality and model abstraction. Artificial intelligence, 67(1): 143-194, 1994.
Dominik Janzing and Bernhard Schölkopf. Causal inference using the algorithmic markov condi- tion. IEEE Transactions on Information Theory, 56(10):5168 5194, 2010.
Nan Rosemary Ke, Olexa Bilaniuk, Anirudh Goyal, Stefan Bauer, Hugo Larochelle, Bernhard Schölkopf, Michael C Mozer, Chris Pal, and Yoshua Bengio. Learning neural causal models from unknown interventions. arXiv preprint arXiv:1910.01075 2019.
Jan TA Koster et al. Markov properties of nonrecursive causal models. Annals of statistics, 24(5): 2148-2177, 1996.
Gustavo Lacerda, Peter L Spirtes, Joseph Ramsey, and Patrik Hoyer. Discovering cyclic causal models by independent components analysis. arXiv preprint arXiv:1206.3273, 2012.
Steffen L. Lauritzen. Gruphical models, volume 17. Clarendon Press, 1996.
Francesco Locatello, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael
Tschannen. Weakly-supprvisee disentanglement without compromises. In International Confer- on Machine Learning, pp. 6348 1348-6359, PMLR, 2020.
Marloes Maathuis, Mathias Drton, Steffen Lauritzen, and Martin Wainwright. Handbook of graph- ical models. CRC Press, 2018.
Søren Wengel Mogensen, Daniel Malinsky, and Niels Richard Hansen. Causal learning for partially observed stochastic dynamical systems. In UAA, pp. 350-360, 2018.
Søren Wengel Mogensen, Niels Richard Hansen, et al. Markov equivalence of marginalized local independence graphs. The Annals of Statistics, 48(1):539-55 559, 2020.
Joris M Mooij, Dominik Janzing, and Bernhard Schölkopf. From ordinary differential equations structural causal models: the deterministic case. arXiv preprint (oo::/304792 2013.
Joris M Mooij. Sara Magliacane, and Tom Claassen. Joint causal inference from multiple contexts. arXiv preprint ai:.v:
2016.
Radford Neal. On deducing conditional independence from d-separation in causal graphs with feedback (research note). Journal of Artificial Intelligence Research, 12:87-91, 2000.
Giambattista Parascandolo, Niki Kilbertus, Mateo Rojas- Carall,,, and Bernhard Schölkopf. Learning independent causal mechanisms. In International Conference on Machine Learning, pp. 4036- 4044. PMLR, 2018.
Judea Pearl. Causal diagrams for empirical research. Biometrika, 82(4))669-683 1995.
Judea Pearl. Causal and counterfactual inference. The Handbook of Rationality, pp. 1-41, 2019.
Judea Pearl et al. Causal inference in statistics: An overview. Statistics surveys, 3:96-146, 2009.
Niklas Pfister, Stefan Bauer, and Jonas Peters. Learning stable and predictive structures in kinetic systems. Proceedings of the National Academy of Sciences, 116(51):25405-544 2019.
2019.
8

---

Under review as a conference paper at ICLR 2022
Thomas Richardson. Markov properties for acyclic directed mixed graphs. Scandinavian Journal of Statistics, 30(1):145-157, 2003.
Thomas S Richardson, Peter Spirtes, et al. Automated discovery of linear feedbackk models. Carnegie Mellon [Department of Philosophy]. 1996.
Bernhard Schölkopf. Causality for machine learning. arXiv preprint arXiv: 2019.
Bernhard Schölkopf, Dominík Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. On causal and anticausal learning. arXiv preprint arXiv: 6471,,6661,222 2012.
Bernhard Schölkopf. Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Towards causal representation learning. arXiv preprint arXiv:2102.11 2021.
Peter Spirtes.
Directed eyclic graphs, conditional independence, and non-recursive linear structural equation models. 1993.
Peter Spirtes.
Conditional independence in directed cyclic graphical models for feedback. Carnegie Mellon [Department of Philosophy]. 1994.
Peter I. Spirtes. Directed cyclic graphical representations of feedback models. arXiv preprint arXiv: 1302.498,, 2013.


Reviews:
[{"review_id": "GgMsn8KAte", "reviewer": "Reviewer_uRUL", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\n1. Introducing a new way to formulate changing causal mechanisms using sample-wise active sets. The authors have claimed its stronger expressiveness power in modeling unsolvable SCM and cyclic relations than using auxiliary context variables. \n\n2. Covering a broad spectrum of prior work which handles feedback loops between observed variables.\n\n**Weaknesses**\n\nThe motivation of this paper is not clear. In the abstract & intro, the goal was originally to solve cyclic relations but how the proposed meta-SCM and SAM in the main content can be used to tackle the problem was not explored. This paper introduced merely a conceptual framework instead of a working system and contained a few unsupported claims which I feel not confident to justify their correctness. The advantages of the proposed active set approach over the classic way to model nonstationarity using context variables are not clear. How the proposed SAM can be coded as a training architecture is questionable.\n\n1. *Motivation is not clear*: What is the central problem being addressed in this paper? This paper seemed to tackle cyclic causal relations in the beginning but then switched to the modeling of sample-wise causal mechanisms through active sets of edges. \n\n2. *No theoretical guarantee or experiment*: This paper introduced merely a conceptual framework instead of a working system.  Many critical claims in the paper are not supported by proofs or experiment results, such as the proposed *Sufficient Activated Mechanisms (SAM)* can recover causal structure if only how many factors have been activated in the sample are known. Questions such as how the proposed meta-SCM is to be built, how the SAM prior can be coded as regularization, under what conditions the proposed framework can work, have not been explored.\n\n3.  *Effectiveness is limited*: I'm not convinced that this is an effective approach. Representing nonstationarity in data using active sets of causal edges might have some advantages over one-hot encoded context variables, such as the dimensionality might be smaller if there exist large numbers of nonstationarity regimes. However, the active sets are also limited in the sense that they cannot model changing causal relations where only the causal strengths vary, for example, for modeling a spring-mass system where spring stiffness wears out over time. Most importantly, I do not think manually encoding the discrete context variables with active sets of edges is a novel idea to the causal community. \n\n4. *Lack of justification for SAM*: as a counterexample (starfish) to argue against sparse mechanism shift (SMS), the authors stated that \"changing the output information of the centre node (instead itself) can lead to shift on all functional relationships with its child nodes, which can affect almost all factors simultaneously.\" I'd like to say this is actually something expected by SMS, as the intervention only changes the marginal distribution of the center node while all conditional relations remain stationary, which still results in the changing of distributions of all factors simultaneously. Also, I can't think of a way to encode the proposed SAM into a learning rule. ", "summary_of_the_paper": "This paper proposes to use **active set** of causal edges, instead of auxiliary context variable or domain index to characterize nonstationary causal relations in data. In addition, the authors propose Sufficient Activated Mechanisms (SAM) to replace Sparse Mechanism Shift (SMS) as the inductive bias (assumption) for causal discovery and inference. \n\nDeveloping a principled way to characterize mechanism shifts in causal settings is definitely an interesting idea and potentially publishable. However, this paper introduced merely a conceptual framework instead of a working system with theoretical guarantees and contained a few unsupported claims which I feel not confident to justify their correctness. I strongly recommend the authors code the proposed meta-SCM and SAM in a training architecture and validate their correctness in both theoretical and empirical perspectives and submit the to future venues. ", "summary_of_the_review": "I have to reject this paper in the current version. The proposed idea is interesting but without supporting proofs or empirical experiments, it is very hard for ICLR community to justify its correctness and judge its applicability to real-world problems.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "F3kaintDm2e", "reviewer": "Reviewer_mwX7", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This was a very interesting paper to read, and quite different than the typical conference submission. The authors address one of the most fundamental problems in causal modeling: sufficient representation. I found the model proposed to be both intuitive and general enough to address many concerns such as feedback, equilibrating systems, etc. My largest concern is in practicality: from reading it is not clear to me how one would either learn or parameterize these models. It's not clear how one would recover these indices or distinguish samples between indices, perform interventional calculus, or parameterize the models to make interventional queries. I would be interested to hear the authors' thoughts, though I do realize that these questions are out of scope for the current submission. In short, this paper is a refreshing though provoking piece of work that I believe could be the basis for many interesting future directions.", "summary_of_the_paper": "This paper presents a new lens on causal graphical models from a lens of fuller generality. Rather than considering models under assumptions such as acyclicity or other constraints that enable tractable modeling, the authors consider the notion of a meta causal model which, when indexed, includes SCM as one possible snapshot. The authors give two additional assumptions (1) sparse mechanism shift and (2) sufficient activated mechanisms, and describe the implications.", "summary_of_the_review": "Interesting, though provoking paper that proposes a novel paradigm for causal modeling in general systems that are often unrepresentable in current causal graphical frameworks.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "2uwimh_reoY", "reviewer": "Reviewer_unQi", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Strengths\n1. The problem of cyclic causality is an interesting problem.\n2. The idea of **sufficient representation** is interesting and can be further developed.\n\n## Weaknesses\n1. Unfortunately, this paper is rushed/unfinished and needs a significant amount of work to reach the quality of a conference paper.\n2. The paper has many grammatical mistakes, including misspelling the word *STRUCTURAL* in the title.\n3. The theoretical advances in the paper are not enough for a conference paper. The authors need to show that they can solve a real-world problem using the proposed Meta-SCM. It is unclear if tying the definition to data will make the definition flexible or not.\n4. Continuing #4, there are no experiments to validate the proposed definitions.\n5. The paper is full of unscholarly language and hyperbole. Here are several examples from the introduction:\n    * there are still **some important** theoretical problems\n    * They have **many convenient** properties\n    * But there is a **strong need** to go beyond acyclic SCMs.\n    * However, they are still not well understood.\n    * We propose a **totally novel** dimension \n ", "summary_of_the_paper": "This paper proposes an extension of SCMs that allows modeling cycles in the causality. The main contribution of the paper is providing definitions for the meta-SCM via the concept of active mechanisms and effectively connecting the data to mechanisms.", "summary_of_the_review": "While the proposed idea of meta-SCM is thought-worthy, the current shape of the paper is far below the quality bars of scholarly conferences such as ICLR. The authors need to add empirical validation of the proposed idea and overhaul the writing of the paper. They also need to show that their proposed meta-SCM has enough novel methodology contributions.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a meta-structural causal model framework, to increase the representation capability of structural equation models. It also considers how to connect data to mechanisms. The paper is conceptually interesting. However, on the technical side, reviewers feel that without supporting proofs or empirical experiments, it is hard to justify the correctness of the proposal and judge its applicability to real-world problems.

As authors claimed in their response, "it is our future work of interest to code our proposed framework into a working system and validate it in a proper setting given its early stage status on research in modeling causal cycles." I think some future version of the paper might be a great contribution to the field if a working system were included.
Decision: Reject