Year: 2021

Abstract:
Recent years have seen impressive progress in theoretical and algorithmic developments of causal inference across various disciplines in science and engineering. However, there are still some unresolved theoretical problems, especially for cyclic causal relationships. In this article, we propose a meta structure causal model(meta-SCM) framework inspired by understanding causality as information transfer. A key feature of our framework is the introduction of the concept of active mechanisms to connect data and the collection of underlying causal mechanisms. We show that the meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations. In addition, we propose a sufficient activated mechanisms assumption, and explain its relationship with existing hypotheses in causal inference and learning. Finally, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.

text: Recent years have seen impressive progress in theoretical and algorithmic developments of causal inference across various disciplines in science and engineering. However, there are still some unresolved theoretical problems, especially for cyclic causal relationships. In this article, we propose a meta structure causal model(meta-SCM) framework inspired by understanding causality as information transfer. A key feature of our framework is the introduction of the concept of active mechanisms to connect data and the collection of underlying causal mechanisms. We show that the meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations. In addition, we propose a sufficient activated mechanisms assumption, and explain its relationship with existing hypotheses in causal inference and learning. Finally, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.

heading: 1 INTRODUCTION
text: Although there have been significant advances in causal research in recent years, there are still some important theoretical problems that have not been resolved. One of the most notoriously hard problems is about cyclic causal relations, and there is no causal modeling frameworks can properly handle it. In modern theory of causality, the mathematical framework called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges(Pearl, 1995; Pearl et al., 2009; Bongers et al., 2016; Bareinboim et al., 2020). Acyclic SCMs, also known as recursive SEMs, form a special wellstudied subclass of SCMs that generalize causal Bayesian networks. They have many convenient properties and are widely used in practical causal modeling, see e.g. (Evans, 2016; Lauritzen, 1996; Richardson, 2003; Maathuis et al., 2018). But there is a strong need to go beyond acyclic SCMs.
In fact, there are feedback loops between observed variables in many systems occurring in real world. Causal cycles may arise when one approximates such systems over time (Fisher, 1970; Mogensen et al., 2018; 2020), or when one describes the equilibrium states of these systems (Iwasaki & Simon, 1994; Lacerda et al., 2012; Hyttinen et al., 2012; Mooij et al., 2013; Bongers & Mooij, 2018; Blom et al., 2020; Pfister et al., 2019), though the underlying dynamic processes describing such systems have an acyclic causal structure over time. In particular, it was shown that the equilibrium states of a system governed by (random) differential equations can be described by an SCM that represents their causal semantics in (Bongers & Mooij, 2018), which gives rise to a plethora of SCMs that include cycles. In contrast to their acyclic counterparts, many of the convenient properties do not hold for SCMs with cycles, and they are not as well understood. Some progress has been made in the case of discrete (Neal, 2000) and linear models (Spirtes, 1993; 1994; 2013; Richardson et al., 1996; Koster et al., 1996; Hyttinen et al., 2012), and more recently the Markov properties (Forré & Mooij, 2020; 2017) and theoretical foundation (Bongers et al., 2016). Researchers are mostly making additional assumption of the underlying causal mechanisms to circumvent complications of cyclic SCMs in causal semantics, solvability, marginaliztions etc.. However, they are still not well understood. Even, a pressing concern is whether SCMs are able to completely model dynamical systems at equilibrium and the causal constraints model(CCM) is proposed but without graphical interpretations yet(Blom et al., 2020).
After introducing the formal definition of SCMs and relevant preliminaries in Section 2, we trace back into the philosophy accounts of causality and propose the meta structural causal model (metaSCM) based on understanding causality as information transfer in Section 3. The meta-SCM framework is constructed by an SCM and an extra dimension which describes how to connect data to mechanisms through the concept of active mechanisms. In particular, a meta-SCM induces a submodel for each sample in the dataset coarsely related to the active set. The new framework is proved to be more expressive by Theorem 7, and its potential ability to address cyclic casual relationships is illustrated by an example of cyclic SCM with multiple solution and an unsolvable cyclic SCM. Comparing to the joint causal inference for meta-system (Mooij et al., 2016) which can also deal with cyclic SCMs, the meta-SCM framework avoids to add extra context variables and can even gain insights on unsolvable SCMs.
The Section 4 addresses the challenging case where no additional variables, besides the samples from the data to generate, are observed. The sufficient activated mechanism (SAM) hypothesis is proposed as an central assumption in the meta-SCM framework, which is consistent with the role of independent causal mechanisms (ICM) principle or sparse mechanisms shift (SMS) assumption for the SCM framework. Moreover, the SAM and SMS hypotheses are also compared with the lens of informational decomposition of SCM, and it reveals that the SMS assumption might be not appropriate in certain case with an example. Section 5, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.
The main contributions of this paper are: 1) We propose a totally novel dimension that describes how to link data to mechanisms to the
existing causal modeling framework, particularly, a meta structural causal model framework which can be used to circumvent technique complications in cyclic SCMs.
2) We propose a SAM hypothesis as an inductive bias for performing causal inferences and learning consistent with the role of SMS assumption.

heading: 2 PRELIMINARIES
text: At the centre of modern causal modeling theory lies the structural causal model (SCM) (also known as structural equation model) which makes graphical assumptions of the underlying data generating process. There are many somewhat different formulations of SCM in literatures, e.g., Schölkopf (2019); Pearl (2019); Bongers et al. (2016); Pearl et al. (2009); Forré & Mooij (2020), among which the definition in Blom et al. (2020) is used in this paper.
Definition 1 (SCM) Let I and J be index sets. A Structural Causal Model (SCM)M is a triple (X , F,E), with:
• a product of standard measurable spaces X = Πi∈IXi (domains of endogenous), • a tuple of exogenous random variables E = (Ej)j∈J taking value in a product of standard
measurable space E = ∏
j∈J Ej .
• a family of F of measurable functions:
fi : Xpa(i)∩I × Epa(i)∩J → Xi, ∀i ∈ I.
The dataset are ususally assumed to be a set of samples for the solution of SCMs.
Definition 2 (Solution of SCM) We say that a random variable X = (Xi)i∈I is a solution to an SCMM = (X , F,E) if

heading: Xi = fi(Xpa(i)∩I ,Epa(i)∩J ) a.s., ∀i ∈ I
text: An SCM may have a unique(up to zero sets) solution, multiple solutions, or there may not exist any solution at all.
Definition 3 An SCMM is called simple if it is uniquely solvable with respect to any subsetO ⊆ I.
All acyclic SCMs are simple.
Definition 4 A do intervention do(x̃I) with target I ⊆ I and value x̃I ∈ XI on an SCM M = (X , F,E) maps it to the intervened SCM Mdo(x̃I) = (X , F̃ ,E) with F̃ the family of measurable functions:
f̃i(xpa(i)∩I , epa(i)∩J ) = { x̃i i ∈ I, fi(xpa(i)∩I , epa(i)∩J ) i ∈ I \ I
The intervened SCM is referred as a submodel of the original SCM, in fact, the variants derived from many different types of interventions (e.g., perfect, imperfect, stochastic, etc.) are also referred as submodel.

heading: 3 META STRUCTURAL CAUSAL MODELS
text: One critical insight in philosophy is that the causal mechanisms behind a system under investigation are not generally observable, but they do produce observable traces (“data,” in modern terminology). This insight naturally leads to two practical desiderata for any proper framework for causal inference, namely:
1. The causal mechanisms underlying the phenomenon under investigation should be accounted for – indeed, formalized – in the analysis.
2. This collection of mechanisms (even if mostly unobservable) should be formally tied to its output: the generated phenomena and corresponding datasets.
The mathematical object called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges. It is often assumed that every instantiation E = e of the exogenous variables uniquely determines the values of all variables in X (Pearl, 2019), which leads a unique solution of the corresponding SCM . Then the dataset D = {x(k)}k=1,...,N is a set of N samples of the unique solution. But in many cases, SCM with cycles might be not solvable or have multiple solutions (Halpern, 1998).
Example 1 (Multiple Solutions) Consider an SCMM1 = (X , F = {f1, f2},E = {E1}), where
F = { x1 ← (x22 + x2 + 1)/3− e21/3 x2 ← x1
Obviously, (1 − E1, 1 − E1) and (1 + E1, 1 + E1) are two different solutions toM1, then which solution of the SCM should be used to link the dataset to the model?
The previous causal inference literature rarely deals with theoretical aspects of cyclic causality. In recent years, it has been formally discussed in Bongers et al. (2016) However, it is also acknowledged by this paper that there are many complications in dealing with cyclic causal models. The vast majority of methods to deal with cyclic SCM in the literature are by adding additional assumptions, such as linear constraints(Spirtes, 1993; 1994; Hyttinen et al., 2012) and certain solvability constraints(Forré & Mooij, 2018; Bongers et al., 2016). These methods basically exclude the study of SCMs with multiple solutions such as Example 1.
The view of understanding causation as information transfer was first formally proposed by (Collier, 1999) in philosophy recently. Inspired by this view, we realize that the causal links among variables can be cut off suggested by unsuccessful information transfer, which suggests that different samples might have different causal graphs and causal mechanisms. For example, samples of M1 might only satisfy only a subset of the structural equations due to absence of information , hence variables for two different samples x(i) and x(j) could have two different causal graphs. However, we usually do not know when and where the information transmission was interrupted for a given sample. In fact, it might be infeasible to specify the information transmission details of all samples of an SCM. To address the above problem, we introduce the concept of active set of an SCM, which is inspired by the active set method in the field of non-linear optimization theory.
Definition 5 (Active mechanisms) For a given sample x(k) and the corresponding collection of mechanisms represented by an SCMM = (X , F,E), if x(k)i = fi(x (k) pa(i)∩I , e (k) pa(i)∩J ), then we call fi an active mechanism, and denote the index set for all active mechanisms as the active set Ak.
The collection of active sets {Ak}k=1,2,... gives us the opportunity to avoid considering the details of the information transfer between variables, and to describe the relationship between the data and the model relatively concisely. Formally, we define the meta structural causal model (meta-SCM) as follows:
Definition 6 (meta-SCM) A collection of mechanisms described by an SCMM = (X , F,E) with a dataset D = {x(k)}k=1,...,N , in which each sample x(k) satisfies that:
• the prior distribution of E(k) is P (E);
• Ak ⊆ I is referred as the active set of the sample k satisfies that
x (k) i ← fi(x (k) pa(i)∩I , e (k) pa(i)∩J ), ∀i ∈ Ak. (1)
Then the tuple 〈M,D〉(or in shortM) is called a meta structural causal model (meta-SCM).
The difference between SCM and meta-SCM. On one hand, an SCM can be interpreted as a special case of meta-SCM satisfies that the active set Ak is equal to I for any sample x(k). On the other hand, a meta-SCM share the causal mechanisms with its corresponding SCM only differs on the method for linking data to model. Thus, it improves the expressiveness of the canonical SCM. Actually, a meta-SCM suggests a method for connecting any dataset to an SCM with the active sets. Formally,
Theorem 7 (Connecting Data to Mechanisms) For an SCM M = (X , F,E) with any dataset D = {x(k)}k=1,...,N in the domain of X . Then each datapoint x(k) is a sample from some submodel SCM M̃ related to the active set Ak.
Proof For any k = 1, ..., N , let M̃(k) = (X , F̃ ,E) be an SCM with modified causal mechanisms:
f̃i(xpa(i)∩I , epa(i)∩J ) = { fi(xpa(i)∩I , epa(i)∩J ) i ∈ Ak, x (k) i i ∈ I \Ak
Then the active set of datapoint x(k) for the submodel M̃(k) is I by definition1, which directly leads to our theorem.
The above proof directly assigns a submodel for each sample in the dataset, which only part of mechanisms in the original SCM holds. In fact, the submodel in our meta-SCM framework does not have to be constructed as a do-intervened model M̃(k), it can be any subclass of SCMs with desired properties(such as acyclic) in literatures.
When there are cyclic causal relationships between variables, one encounters various technical complications, which even arise in the linear setting(Bongers et al., 2016). The main idea for solving related difficulties is to add additional restrictions on structural equations, and the dataset are assumed to be consisted of samples from a distribution obtained by solving the SCM. In contrast, our meta-SCM does not add additional assumptions on SCM, and each datapoint is treated as a sample of the distribution obtained by solving a certain submodel. For the SCM M1 with muptiple solutions in Example 1 with dataset D = {x(k)}k=1,...,N in the domain of X , the meta-SCM can circumvent theoretical complications through providing each datapoint a distribution of any solution of a certain submodel. Usually, the details of submodel and its corresponding distribution for each sample might be unknown, and meta-SCM only provides a coarse description by the active sets.
In fact, we can also circumvent the technique complications caused by solvability through metaSCM. Specifically, the structural equations of an acyclic SCM trivially have a unique solution, which ensures that the SCM gives rise to a unique, well-defined probability distribution on the variables. However, an SCM can be unsolvable in the case of cycles, e.g.,
Example 2 (Unsolvable) Consider an SCMM2 = (X , F = {f1, f2},E = {E1}), where
F = { x1 ← x22 + x2 + e21 + 1 x2 ← x1
1In fact, M̃(k) is the do-intervened SCM M do(x
(k) I\Ak
) .
Then the SCM M2 is obvious not solvable, thus it cannot be used to model underlying causal mechanisms of any dataset D = {x(k)}k=1,...,N in the domain of X . But with our novel approach, we might still connect data to the underlying mechanisms, e.g. by letting |Ak| = 1 for all k = 1, ..., N . In other words, each datapoint x(k) is a sample of a distribution derived from the submodel M̃(k) by Theorem 7. The difference between meta-system and meta-SCM. The joint causal inference (JCI) framework reduces modeling a system in its environment to modeling the meta-system consisting of the system and its environment, which considers auxiliary context variables that describe the context of each data set (Mooij et al., 2016). In contrast, our meta-SCM address the challenging case where no additional variables, besides the samples from the data to generate, are observed. For example in Fig. 1, the meta-system consists of two variables X1, X2 and a context variable C. More concretely, the engine X1 drives the wheels of a car X2 when going uphill C = 0, but when going downhill, the rotation of the wheels drives the engine. In a meta-SCM, we instead introduce the concept of active mechanisms to describe each sample in a dataset. Moreover, the meta-SCM framework even gain insights on unsolvable SCMs such as Example 2 while the previous meta-system study restricts themselves to simple SCMs. In philosophy viewpoint, the JCI framework uses context changes to model interventions, which is a difference-making account for causality. Instead, the active set in meta-SCM are inspired by information transferring which is a production account of causality, see e.g. Illari & Russo (2014).
From the above discussions, the meta-SCM framework can be considered as generalization of SCM that novelly links data to causal mechanisms through active sets at individual level modeling2, and it might be used to circumvent technical complications when cycles present. Then, a natural question is how to perform inferences with a meta-SCM when in the presence of cycles？A first difficult might be the lack of knowledge on the specific form of active sets, i.e. Ak, for each sample. In the following section, we propose a principle for learning and reasoning within the meta-SCM framework.

heading: 4 SUFFICIENT ACTIVATED MECHANISMS
text: The meta-SCM viewpoint considers a set of variables indexed by I and a set of samples D with causal (or disentangled) factorization,
P (x (k) Ak ) = ∏ i∈Ak P (x (k) i |x (k) pa(i)∩I), ∀x (k) ∈ D. (2)
The independent causal mechanisms (ICM) principle tells us that the factors should be independent in the sense that
a) changing (or performing an intervention upon) one mechanism P (xi|xpa(i)∩I) does not change any of the other mechanisms P (xj |xpa(j)∩I)) (i 6= j) (Schölkopf et al., 2012), and
2The individual level models is not new but widely used, e.g., in variational inference of machine learning (Bingham et al., 2019).
b) knowing some other mechanisms P (xi|xpa(i)∩I) (i 6= j) does not give us information about a mechanism P (xj |xpa(j)∩I)) (Janzing & Schölkopf, 2010).
Consistent with the implication a) of the ICM Principle, the sparse mechanism shift (SMS) hypothesis (See e.g. Parascandolo et al. (2018); Schölkopf (2019)) was stated in the following:
Sparse Mechanism Shift (SMS). Small distribution changes tend to manifest themselves in a sparse or local way in the causal/disentangled factorization, i.e., they should usually not affect all factors simultaneously.
It is the fundamental assumption for causal representation learning(Schölkopf et al., 2021) and has recently been used for learning causal models (Ke et al., 2019), modular architectures (Goyal et al., 2019; Besserve et al., 2020) and disentangled representations (Locatello et al., 2020). However, the SMS hypothesis usually only assumes mechanisms shift from a given context to the original uniquely solvable SCM, especially acyclic SCM. We propose a hypothesis in the viewpoint of a given individual mechanism across samples in the following:
Sufficient Activated Mechanism (SAM). All factors in the causal/disentangled factorization should be activated sufficiently, i.e., every causal mechanism is sufficiently often activated across samples in the dataset.
The “sufficient” in the SAM roughly understood as sufficient for identifying factors in the causal factorization of interest. In Example 2, the unsolvable cyclic SCM M2 cannot be used as the underlying generative process for any dataset. On the contrary, in the meta-SCM framework under SAM assumption, we are able to learn a causal factorization Eq. (2), even when only knowing how many factors have activated, but not which ones. In mathematics, start with solving the unsolvable equation x2 = −1 with imaginary unit i, it has been an important branch of mathematics which is complex analysis with holomorphic function as a central research object. Similarly, the meta-SCM, which introduced a novel dimension on linking data to SCM, can help in causal modeling with unsolvable SCM, and we feel meta-SCM with SAM assumption might be a central research object for our framework as holomorphic function for complex analysis.
Overall, in the lens of two practical desirata for any proper framework mentioned in the begining of Section 3, our meta-SCM is different from all literatures on variants of SCMs by adding a fully novel dimension on considering how each sample is tied to underlying mechanisms. This extra dimension though, on one hand, complicate the SCM framework, however, on the other hand, improve its expressiveness and power for causal modeling. Then, the SAM hypothesis defines a subclass of meta-SCMs of interest for performing causal inferences and reasoning, consistent with the role of the ICM Principle/ SMS hypothesis in the SCM framework.
The causality behind interventions in the SCM framework is a difference-making account, while the production accounts, especially informational interpretation of causality relatively neglected. One core idea of informational causality — C causes E if there is information transmission from C to E(Illari & Russo, 2014), suggests that the causal mechanism fi of an SCM can be separated into two part — information process and information transfer (Gong & Zhu, 2021). Formally,
Definition 8 (Informational Decomposition of SCM) Consider an SCM M = (X , F,E). The informational decomposition ofM is defined as, for any i ∈ I,{
xi ← fi(epa(i)∩I,i, epa(i)∩J ), ej,i ← xj ,
(3)
where ej,i represents the information on edge (j, i) received from its input node j.
This dichotomy of causal mechanisms can help illuminate on the relationship between SMS and SAM assumption. We tend to assume the information processing mechanisms are sparse inactivated, while the information transfer mechanisms are instead only sufficient activated. Henceforth, if we consider the inactivated mechanisms as shifted mechanisms, then the information process and
transfer mechanisms are satisfying the SMS hypothesis and the SAM assumption respectively. Small distribution changes caused by interventions on information process tend to manifest themselves in a sparse or local way, but might be manifest themselves in a non-local way caused by changes on transferred information. For example in a starfish shaped causal diagram, changing the output information of the centre node (instead itself) can lead to shift on all functional relationships with its child nodes, which can affect almost all factors simultaneously. Thus, the SMS assumption might be appropriate to describe the intervened model for such system.

heading: 5 CONCLUSIONS
text: This work is mostly theoretical and conceptual to address the notoriously hard problem of cyclic causal models. We proposed a novel active set approach for connecting data to SCMs (even not solvable), instead of making additional assumptions to restrict the class of SCMs of interest. Note that the meta-SCM is more of an conceptional modeling framework on how to relate data to the underlying causal mechanisms rather than a specific model, and the SAM assumption has been introduced as an inductive bias for performing causal inferences and learning within the framework of meta-SCM. To best of our knowledge, this is the first causal modeling framework to explore the dimension formally on how to link each sample in the dataset to the collection of mechanisms. In the future, we might anticipate more causal modeling exploration on exacting information from activated mechanisms directly instead of the solution of SCMs.

Review summaries:
This paper proposes to use **active set** of causal edges, instead of auxiliary context variable or domain index to characterize nonstationary causal relations in data. In addition, the authors propose Sufficient Activated Mechanisms (SAM) to replace Sparse Mechanism Shift (SMS) as the inductive bias (assumption) for causal discovery and inference.   Developing a principled way to characterize mechanism shifts in causal settings is definitely an interesting idea and potentially publishable. However, this paper introduced merely a conceptual framework instead of a working system with theoretical guarantees and contained a few unsupported claims which I feel not confident to justify their correctness. I strongly recommend the authors code the proposed meta-SCM and SAM in a training architecture and validate their correctness in both theoretical and empirical perspectives and submit the to future venues. 

This paper presents a new lens on causal graphical models from a lens of fuller generality. Rather than considering models under assumptions such as acyclicity or other constraints that enable tractable modeling, the authors consider the notion of a meta causal model which, when indexed, includes SCM as one possible snapshot. The authors give two additional assumptions (1) sparse mechanism shift and (2) sufficient activated mechanisms, and describe the implications.