Connecting Data to Mechanisms with Meta Structual Causal Model

Recent years have seen impressive progress in theoretical and algorithmic developments of causal inference across various disciplines in science and engineering . However , there are still some unresolved theoretical problems , especially for cyclic causal relationships . In this article , we propose a meta structure causal model ( meta-SCM ) framework inspired by understanding causality as information transfer . A key feature of our framework is the introduction of the concept of active mechanisms to connect data and the collection of underlying causal mechanisms . We show that the meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations . In addition , we propose a sufficient activated mechanisms assumption , and explain its relationship with existing hypotheses in causal inference and learning . Finally , we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty . 1 INTRODUCTION . Although there have been significant advances in causal research in recent years , there are still some important theoretical problems that have not been resolved . One of the most notoriously hard problems is about cyclic causal relations , and there is no causal modeling frameworks can properly handle it . In modern theory of causality , the mathematical framework called a structural causal model ( SCM ) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges ( Pearl , 1995 ; Pearl et al. , 2009 ; Bongers et al. , 2016 ; Bareinboim et al. , 2020 ) . Acyclic SCMs , also known as recursive SEMs , form a special wellstudied subclass of SCMs that generalize causal Bayesian networks . They have many convenient properties and are widely used in practical causal modeling , see e.g . ( Evans , 2016 ; Lauritzen , 1996 ; Richardson , 2003 ; Maathuis et al. , 2018 ) . But there is a strong need to go beyond acyclic SCMs . In fact , there are feedback loops between observed variables in many systems occurring in real world . Causal cycles may arise when one approximates such systems over time ( Fisher , 1970 ; Mogensen et al. , 2018 ; 2020 ) , or when one describes the equilibrium states of these systems ( Iwasaki & Simon , 1994 ; Lacerda et al. , 2012 ; Hyttinen et al. , 2012 ; Mooij et al. , 2013 ; Bongers & Mooij , 2018 ; Blom et al. , 2020 ; Pfister et al. , 2019 ) , though the underlying dynamic processes describing such systems have an acyclic causal structure over time . In particular , it was shown that the equilibrium states of a system governed by ( random ) differential equations can be described by an SCM that represents their causal semantics in ( Bongers & Mooij , 2018 ) , which gives rise to a plethora of SCMs that include cycles . In contrast to their acyclic counterparts , many of the convenient properties do not hold for SCMs with cycles , and they are not as well understood . Some progress has been made in the case of discrete ( Neal , 2000 ) and linear models ( Spirtes , 1993 ; 1994 ; 2013 ; Richardson et al. , 1996 ; Koster et al. , 1996 ; Hyttinen et al. , 2012 ) , and more recently the Markov properties ( Forré & Mooij , 2020 ; 2017 ) and theoretical foundation ( Bongers et al. , 2016 ) . Researchers are mostly making additional assumption of the underlying causal mechanisms to circumvent complications of cyclic SCMs in causal semantics , solvability , marginaliztions etc .. However , they are still not well understood . Even , a pressing concern is whether SCMs are able to completely model dynamical systems at equilibrium and the causal constraints model ( CCM ) is proposed but without graphical interpretations yet ( Blom et al. , 2020 ) . After introducing the formal definition of SCMs and relevant preliminaries in Section 2 , we trace back into the philosophy accounts of causality and propose the meta structural causal model ( metaSCM ) based on understanding causality as information transfer in Section 3 . The meta-SCM framework is constructed by an SCM and an extra dimension which describes how to connect data to mechanisms through the concept of active mechanisms . In particular , a meta-SCM induces a submodel for each sample in the dataset coarsely related to the active set . The new framework is proved to be more expressive by Theorem 7 , and its potential ability to address cyclic casual relationships is illustrated by an example of cyclic SCM with multiple solution and an unsolvable cyclic SCM . Comparing to the joint causal inference for meta-system ( Mooij et al. , 2016 ) which can also deal with cyclic SCMs , the meta-SCM framework avoids to add extra context variables and can even gain insights on unsolvable SCMs . The Section 4 addresses the challenging case where no additional variables , besides the samples from the data to generate , are observed . The sufficient activated mechanism ( SAM ) hypothesis is proposed as an central assumption in the meta-SCM framework , which is consistent with the role of independent causal mechanisms ( ICM ) principle or sparse mechanisms shift ( SMS ) assumption for the SCM framework . Moreover , the SAM and SMS hypotheses are also compared with the lens of informational decomposition of SCM , and it reveals that the SMS assumption might be not appropriate in certain case with an example . Section 5 , we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty . The main contributions of this paper are : 1 ) We propose a totally novel dimension that describes how to link data to mechanisms to the existing causal modeling framework , particularly , a meta structural causal model framework which can be used to circumvent technique complications in cyclic SCMs . 2 ) We propose a SAM hypothesis as an inductive bias for performing causal inferences and learning consistent with the role of SMS assumption . 2 PRELIMINARIES . At the centre of modern causal modeling theory lies the structural causal model ( SCM ) ( also known as structural equation model ) which makes graphical assumptions of the underlying data generating process . There are many somewhat different formulations of SCM in literatures , e.g. , Schölkopf ( 2019 ) ; Pearl ( 2019 ) ; Bongers et al . ( 2016 ) ; Pearl et al . ( 2009 ) ; Forré & Mooij ( 2020 ) , among which the definition in Blom et al . ( 2020 ) is used in this paper . Definition 1 ( SCM ) Let I and J be index sets . A Structural Causal Model ( SCM ) M is a triple ( X , F , E ) , with : • a product of standard measurable spaces X = Πi∈IXi ( domains of endogenous ) , • a tuple of exogenous random variables E = ( Ej ) j∈J taking value in a product of standard measurable space E = ∏ j∈J Ej . • a family of F of measurable functions : fi : Xpa ( i ) ∩I × Epa ( i ) ∩J → Xi , ∀i ∈ I . The dataset are ususally assumed to be a set of samples for the solution of SCMs . Definition 2 ( Solution of SCM ) We say that a random variable X = ( Xi ) i∈I is a solution to an SCMM = ( X , F , E ) if Xi = fi ( Xpa ( i ) ∩I , Epa ( i ) ∩J ) a.s. , ∀i ∈ I . An SCM may have a unique ( up to zero sets ) solution , multiple solutions , or there may not exist any solution at all . Definition 3 An SCMM is called simple if it is uniquely solvable with respect to any subsetO ⊆ I . All acyclic SCMs are simple . Definition 4 A do intervention do ( x̃I ) with target I ⊆ I and value x̃I ∈ XI on an SCM M = ( X , F , E ) maps it to the intervened SCM Mdo ( x̃I ) = ( X , F̃ , E ) with F̃ the family of measurable functions : f̃i ( xpa ( i ) ∩I , epa ( i ) ∩J ) = { x̃i i ∈ I , fi ( xpa ( i ) ∩I , epa ( i ) ∩J ) i ∈ I \ I The intervened SCM is referred as a submodel of the original SCM , in fact , the variants derived from many different types of interventions ( e.g. , perfect , imperfect , stochastic , etc . ) are also referred as submodel . 3 META STRUCTURAL CAUSAL MODELS . One critical insight in philosophy is that the causal mechanisms behind a system under investigation are not generally observable , but they do produce observable traces ( “ data , ” in modern terminology ) . This insight naturally leads to two practical desiderata for any proper framework for causal inference , namely : 1 . The causal mechanisms underlying the phenomenon under investigation should be accounted for – indeed , formalized – in the analysis . 2 . This collection of mechanisms ( even if mostly unobservable ) should be formally tied to its output : the generated phenomena and corresponding datasets . The mathematical object called a structural causal model ( SCM ) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges . It is often assumed that every instantiation E = e of the exogenous variables uniquely determines the values of all variables in X ( Pearl , 2019 ) , which leads a unique solution of the corresponding SCM . Then the dataset D = { x ( k ) } k=1 , ... , N is a set of N samples of the unique solution . But in many cases , SCM with cycles might be not solvable or have multiple solutions ( Halpern , 1998 ) . Example 1 ( Multiple Solutions ) Consider an SCMM1 = ( X , F = { f1 , f2 } , E = { E1 } ) , where F = { x1 ← ( x22 + x2 + 1 ) /3− e21/3 x2 ← x1 Obviously , ( 1 − E1 , 1 − E1 ) and ( 1 + E1 , 1 + E1 ) are two different solutions toM1 , then which solution of the SCM should be used to link the dataset to the model ? The previous causal inference literature rarely deals with theoretical aspects of cyclic causality . In recent years , it has been formally discussed in Bongers et al . ( 2016 ) However , it is also acknowledged by this paper that there are many complications in dealing with cyclic causal models . The vast majority of methods to deal with cyclic SCM in the literature are by adding additional assumptions , such as linear constraints ( Spirtes , 1993 ; 1994 ; Hyttinen et al. , 2012 ) and certain solvability constraints ( Forré & Mooij , 2018 ; Bongers et al. , 2016 ) . These methods basically exclude the study of SCMs with multiple solutions such as Example 1 . The view of understanding causation as information transfer was first formally proposed by ( Collier , 1999 ) in philosophy recently . Inspired by this view , we realize that the causal links among variables can be cut off suggested by unsuccessful information transfer , which suggests that different samples might have different causal graphs and causal mechanisms . For example , samples of M1 might only satisfy only a subset of the structural equations due to absence of information , hence variables for two different samples x ( i ) and x ( j ) could have two different causal graphs . However , we usually do not know when and where the information transmission was interrupted for a given sample . In fact , it might be infeasible to specify the information transmission details of all samples of an SCM . To address the above problem , we introduce the concept of active set of an SCM , which is inspired by the active set method in the field of non-linear optimization theory . Definition 5 ( Active mechanisms ) For a given sample x ( k ) and the corresponding collection of mechanisms represented by an SCMM = ( X , F , E ) , if x ( k ) i = fi ( x ( k ) pa ( i ) ∩I , e ( k ) pa ( i ) ∩J ) , then we call fi an active mechanism , and denote the index set for all active mechanisms as the active set Ak . The collection of active sets { Ak } k=1,2 , ... gives us the opportunity to avoid considering the details of the information transfer between variables , and to describe the relationship between the data and the model relatively concisely . Formally , we define the meta structural causal model ( meta-SCM ) as follows : Definition 6 ( meta-SCM ) A collection of mechanisms described by an SCMM = ( X , F , E ) with a dataset D = { x ( k ) } k=1 , ... , N , in which each sample x ( k ) satisfies that : • the prior distribution of E ( k ) is P ( E ) ; • Ak ⊆ I is referred as the active set of the sample k satisfies that x ( k ) i ← fi ( x ( k ) pa ( i ) ∩I , e ( k ) pa ( i ) ∩J ) , ∀i ∈ Ak . ( 1 ) Then the tuple 〈M , D〉 ( or in shortM ) is called a meta structural causal model ( meta-SCM ) . The difference between SCM and meta-SCM . On one hand , an SCM can be interpreted as a special case of meta-SCM satisfies that the active set Ak is equal to I for any sample x ( k ) . On the other hand , a meta-SCM share the causal mechanisms with its corresponding SCM only differs on the method for linking data to model . Thus , it improves the expressiveness of the canonical SCM . Actually , a meta-SCM suggests a method for connecting any dataset to an SCM with the active sets . Formally , Theorem 7 ( Connecting Data to Mechanisms ) For an SCM M = ( X , F , E ) with any dataset D = { x ( k ) } k=1 , ... , N in the domain of X . Then each datapoint x ( k ) is a sample from some submodel SCM M̃ related to the active set Ak . Proof For any k = 1 , ... , N , let M̃ ( k ) = ( X , F̃ , E ) be an SCM with modified causal mechanisms : f̃i ( xpa ( i ) ∩I , epa ( i ) ∩J ) = { fi ( xpa ( i ) ∩I , epa ( i ) ∩J ) i ∈ Ak , x ( k ) i i ∈ I \Ak Then the active set of datapoint x ( k ) for the submodel M̃ ( k ) is I by definition1 , which directly leads to our theorem . The above proof directly assigns a submodel for each sample in the dataset , which only part of mechanisms in the original SCM holds . In fact , the submodel in our meta-SCM framework does not have to be constructed as a do-intervened model M̃ ( k ) , it can be any subclass of SCMs with desired properties ( such as acyclic ) in literatures . When there are cyclic causal relationships between variables , one encounters various technical complications , which even arise in the linear setting ( Bongers et al. , 2016 ) . The main idea for solving related difficulties is to add additional restrictions on structural equations , and the dataset are assumed to be consisted of samples from a distribution obtained by solving the SCM . In contrast , our meta-SCM does not add additional assumptions on SCM , and each datapoint is treated as a sample of the distribution obtained by solving a certain submodel . For the SCM M1 with muptiple solutions in Example 1 with dataset D = { x ( k ) } k=1 , ... , N in the domain of X , the meta-SCM can circumvent theoretical complications through providing each datapoint a distribution of any solution of a certain submodel . Usually , the details of submodel and its corresponding distribution for each sample might be unknown , and meta-SCM only provides a coarse description by the active sets . In fact , we can also circumvent the technique complications caused by solvability through metaSCM . Specifically , the structural equations of an acyclic SCM trivially have a unique solution , which ensures that the SCM gives rise to a unique , well-defined probability distribution on the variables . However , an SCM can be unsolvable in the case of cycles , e.g. , Example 2 ( Unsolvable ) Consider an SCMM2 = ( X , F = { f1 , f2 } , E = { E1 } ) , where F = { x1 ← x22 + x2 + e21 + 1 x2 ← x1 1In fact , M̃ ( k ) is the do-intervened SCM M do ( x ( k ) I\Ak ) . Then the SCM M2 is obvious not solvable , thus it can not be used to model underlying causal mechanisms of any dataset D = { x ( k ) } k=1 , ... , N in the domain of X . But with our novel approach , we might still connect data to the underlying mechanisms , e.g . by letting |Ak| = 1 for all k = 1 , ... , N . In other words , each datapoint x ( k ) is a sample of a distribution derived from the submodel M̃ ( k ) by Theorem 7 . The difference between meta-system and meta-SCM . The joint causal inference ( JCI ) framework reduces modeling a system in its environment to modeling the meta-system consisting of the system and its environment , which considers auxiliary context variables that describe the context of each data set ( Mooij et al. , 2016 ) . In contrast , our meta-SCM address the challenging case where no additional variables , besides the samples from the data to generate , are observed . For example in Fig . 1 , the meta-system consists of two variables X1 , X2 and a context variable C. More concretely , the engine X1 drives the wheels of a car X2 when going uphill C = 0 , but when going downhill , the rotation of the wheels drives the engine . In a meta-SCM , we instead introduce the concept of active mechanisms to describe each sample in a dataset . Moreover , the meta-SCM framework even gain insights on unsolvable SCMs such as Example 2 while the previous meta-system study restricts themselves to simple SCMs . In philosophy viewpoint , the JCI framework uses context changes to model interventions , which is a difference-making account for causality . Instead , the active set in meta-SCM are inspired by information transferring which is a production account of causality , see e.g . Illari & Russo ( 2014 ) . From the above discussions , the meta-SCM framework can be considered as generalization of SCM that novelly links data to causal mechanisms through active sets at individual level modeling2 , and it might be used to circumvent technical complications when cycles present . Then , a natural question is how to perform inferences with a meta-SCM when in the presence of cycles？A first difficult might be the lack of knowledge on the specific form of active sets , i.e . Ak , for each sample . In the following section , we propose a principle for learning and reasoning within the meta-SCM framework .

Summary: This paper proposes to use **active set** of causal edges, instead of auxiliary context variable or domain index to characterize nonstationary causal relations in data. In addition, the authors propose Sufficient Activated Mechanisms (SAM) to replace Sparse Mechanism Shift (SMS) as the inductive bias (assumption) for causal discovery and inference.   Developing a principled way to characterize mechanism shifts in causal settings is definitely an interesting idea and potentially publishable. However, this paper introduced merely a conceptual framework instead of a working system with theoretical guarantees and contained a few unsupported claims which I feel not confident to justify their correctness. I strongly recommend the authors code the proposed meta-SCM and SAM in a training architecture and validate their correctness in both theoretical and empirical perspectives and submit the to future venues.