Original: C ONNECTING
Revised: C ONNECTING

Original: D ATA
Revised: D ATA

Original: TO
Revised: TO

Original: M ECHANISMS
Revised: M ECHANISMS

Original: WITH
Revised: WITH

Original: M ETA
Revised: M ETA

Original: S TRUCTUAL C AUSAL M ODEL
Revised: S TRUCTURAL C AUSAL M ODEL

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Recent years have seen impressive progress in theoretical and algorithmic developments of causal inference across various disciplines in science and engineering.
Revised: Recent years have seen impressive progress in theoretical and algorithmic developments of causal inference across various disciplines in science and engineering.

Original: However, there are still some unresolved theoretical problems, especially for cyclic causal relationships.
Revised: However, there are still some unresolved theoretical problems, especially for cyclic causal relationships.

Original: In this article, we propose a meta structure causal model(meta-SCM) framework inspired by understanding causality as information transfer.
Revised: In this article, we propose a meta structure causal model(meta-SCM) framework inspired by understanding causality as information transfer.

Original: A key feature of our framework is the introduction of the concept of active mechanisms to connect data and the collection of underlying causal mechanisms.
Revised: A key feature of our framework is the introduction of the concept of active mechanisms to connect data and the collection of underlying causal mechanisms.

Original: We show that the meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations.
Revised: We show that the meta-SCM provides a novel approach to address the theoretical complications for modeling cyclic causal relations.

Original: In addition, we propose a sufﬁcient activated mechanisms assumption, and explain its relationship with existing hypotheses in causal inference and learning.
Revised: In addition, we propose a sufﬁcient activated mechanisms assumption, and explain its relationship with existing hypotheses in causal inference and learning.

Original: Finally, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.
Revised: Finally, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Although there have been signiﬁcant advances in causal research in recent years,and its applications have been extended to various disciplines, there are still some important theoretical problems that have not been resolved.
Revised: Although there have been signiﬁcant advances in causal research in recent years, there are still some important theoretical problems that have not been resolved.

Original: One of the most notoriously hard problems is about cyclic causal relations, and there is no causal modeling frameworks can properly handle it.
Revised: One of the most notoriously hard problems is about cyclic causal relations, and there is no causal modeling frameworks can properly handle it.

Original: In modern theory of causality, the mathematical framework called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges(Pearl, 1995; Pearl et al., 2009; Bongers et al., 2016; Bareinboim et al., 2020).
Revised: In modern theory of causality, the mathematical framework called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges(Pearl, 1995; Pearl et al., 2009; Bongers et al., 2016; Bareinboim et al., 2020).

Original: Acyclic SCMs, also known as recursive SEMs, form a special well-studied subclass of SCMs that generalize causal Bayesian networks.
Revised: Acyclic SCMs, also known as recursive SEMs, form a special wellstudied subclass of SCMs that generalize causal Bayesian networks.

Original: They have many convenient properties and are widely used in practical causal modeling, see e.g. (Evans, 2016; Lauritzen, 1996; Richardson, 2003; Maathuis et al., 2018).
Revised: They have many convenient properties and are widely used in practical causal modeling, see e.g. (Evans, 2016; Lauritzen, 1996; Richardson, 2003; Maathuis et al., 2018).

Original: But there is a strong need to go beyond acyclic SCMs.
Revised: But there is a strong need to go beyond acyclic SCMs.

Original: In fact, there are feedback loops between observed variables in many systems occurring in real world.
Revised: In fact, there are feedback loops between observed variables in many systems occurring in real world.

Original: Causal cycles may arise when one approximates such systems over time (Fisher, 1970;
Revised: Causal cycles may arise when one approximates such systems over time (Fisher, 1970;

Original: Mogensen et al., 2018; 2020), or when one describes the equilibrium states of these systems (Iwasaki & Simon, 1994; Lacerda et al., 2012; Hyttinen et al., 2012; Mooij et al., 2013; Bongers & Mooij, 2018; Blom et al., 2020; Pﬁster et al., 2019), though the underlying dynamic processes describing such systems have an acyclic causal structure over time.
Revised: Mogensen et al., 2018; 2020), or when one describes the equilibrium states of these systems (Iwasaki & Simon, 1994; Lacerda et al., 2012; Hyttinen et al., 2012; Mooij et al., 2013; Bongers & Mooij, 2018; Blom et al., 2020; Pﬁster et al., 2019), though the underlying dynamic processes describing such systems have an acyclic causal structure over time.

Original: In particular, it was shown that the equilibrium states of a system governed by (random) differential equations can be described by an SCM that represents their causal semantics in (Bongers & Mooij, 2018), which gives rise to a plethora of
Revised: In particular, it was shown that the equilibrium states of a system governed by (random) differential equations can be described by an SCM that represents their causal semantics in (Bongers & Mooij, 2018), which gives rise to a plethora of

Original: SCMs that include cycles.
Revised: SCMs that include cycles.

Original: In contrast to their acyclic counterparts, many of the convenient properties do not hold for SCMs with cycles, and they are not as well understood.
Revised: In contrast to their acyclic counterparts, many of the convenient properties do not hold for SCMs with cycles, and they are not as well understood.

Original: Some progress has been made in the case of discrete (Neal, 2000) and linear models (Spirtes, 1993; 1994; 2013;
Revised: Some progress has been made in the case of discrete (Neal, 2000) and linear models (Spirtes, 1993; 1994; 2013;

Original: Richardson et al., 1996; Koster et al., 1996; Hyttinen et al., 2012), and more recently the Markov properties (Forr´e & Mooij, 2020; 2017) and theoretical foundation (Bongers et al., 2016).
Revised: Richardson et al., 1996; Koster et al., 1996; Hyttinen et al., 2012), and more recently the Markov properties (Forr´e & Mooij, 2020; 2017) and theoretical foundation (Bongers et al., 2016).

Original: Researchers are mostly making additional assumption of the underlying causal mechanisms to circumvent complications of cyclic SCMs in causal semantics, solvability, marginaliztions etc..
Revised: Researchers are mostly making additional assumption of the underlying causal mechanisms to circumvent complications of cyclic SCMs in causal semantics, solvability, marginaliztions etc..

Original: However, they are still not well understood.
Revised: However, they are still not well understood.

Original: Even, a pressing concern is whether SCMs are able to completely model dynamical systems at equilibrium and the causal constraints model(CCM) is proposed but without graphical interpretations yet(Blom et al., 2020).
Revised: Even, a pressing concern is whether SCMs are able to completely model dynamical systems at equilibrium and the causal constraints model(CCM) is proposed but without graphical interpretations yet(Blom et al., 2020).

Original: After introducing the formal deﬁnition of SCMs and relevant preliminaries in Section 2, we trace back into the philosophy accounts of causality and propose the meta structural causal model (metaSCM) based on understanding causality as information transfer in Section 3.
Revised: After introducing the formal deﬁnition of SCMs and relevant preliminaries in Section 2, we trace back into the philosophy accounts of causality and propose the meta structural causal model (metaSCM) based on understanding causality as information transfer in Section 3.

Original: The meta-SCM framework is constructed by an SCM and an extra dimension which describes how to connect data to mechanisms through the concept of active mechanisms .
Revised: The meta-SCM framework is constructed by an SCM and an extra dimension which describes how to connect data to mechanisms through the concept of active mechanisms .

Original: In particular, a meta-SCM induces a submodel for each sample in the dataset coarsely related to the active set.
Revised: In particular, a meta-SCM induces a submodel for each sample in the dataset coarsely related to the active set.

Original: The new framework is proved to be more expressive by Theorem 7, and its potential ability to address cyclic casual relationships is illustrated by an example of cyclic SCM with multiple solution and an unsolvable cyclic SCM.
Revised: The new framework is proved to be more expressive by Theorem 7, and its potential ability to address cyclic casual relationships is illustrated by an example of cyclic SCM with multiple solution and an unsolvable cyclic SCM.

Original: Comparing to the joint causal inference for meta-system (Mooij et al., 2016) which can also deal with cyclic SCMs, the meta-SCM framework avoids to add extra context variables and can even gain insights on unsolvable SCMs.
Revised: Comparing to the joint causal inference for meta-system (Mooij et al., 2016) which can also deal with cyclic SCMs, the meta-SCM framework avoids to add extra context variables and can even gain insights on unsolvable SCMs.

Original: The Section 4 addresses the challenging case where no additional variables, besides the samples from the data to generate, are observed.
Revised: The Section 4 addresses the challenging case where no additional variables, besides the samples from the data to generate, are observed.

Original: The sufﬁcient activated mechanism (SAM) hypothesis is proposed as an central assumption in the meta-SCM framework, which is consistent with the role of independent causal mechanisms (ICM) principle or sparse mechanisms shift (SMS) assumption for the SCM framework.
Revised: The sufﬁcient activated mechanism (SAM) hypothesis is proposed as an central assumption in the meta-SCM framework, which is consistent with the role of independent causal mechanisms (ICM) principle or sparse mechanisms shift (SMS) assumption for the SCM framework.

Original: Moreover, the SAM and SMS hypotheses are also compared with the lens of informational decomposition of SCM, and it reveals that the SMS assumption might be not appropriate in certain case with an example.
Revised: Moreover, the SAM and SMS hypotheses are also compared with the lens of informational decomposition of SCM, and it reveals that the SMS assumption might be not appropriate in certain case with an example.

Original: Section 5, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.
Revised: Section 5, we conclude the main idea of the meta-SCM framework with an emphasis on its theoretical and conceptual novelty.

Original: The main contributions of this paper are:
Revised: The main contributions of this paper are:

Original: 1) We propose a totally novel dimension that describes how to link data to mechanisms to the existing causal modeling framework, particularly, an meta structural causal model framework which can be used to circumvent technique complications in cyclic SCMs.
Revised: 1) We propose a totally novel dimension that describes how to link data to mechanisms to the existing causal modeling framework, particularly, a meta structural causal model framework which can be used to circumvent technique complications in cyclic SCMs.

Original: We propose a SAM hypothesis as an inductive bias for performing causal inferences and learning consistent with the role of SMS assumption.
Revised: We propose a SAM hypothesis as an inductive bias for performing causal inferences and learning consistent with the role of SMS assumption.

Original:  P RELIMINARIES
Revised:  P RELIMINARIES

Original: At the centre of modern causal modeling theory lies the structural causal model (SCM) (also known as structural equation model) which makes graphical assumptions of the underlying data generating process.
Revised: At the centre of modern causal modeling theory lies the structural causal model (SCM) (also known as structural equation model) which makes graphical assumptions of the underlying data generating process.

Original: There are many somewhat different formulations of SCM in literatures, e.g., Sch ¨ olkopf (2019); Pearl (2019); Bongers et al.
Revised: There are many somewhat different formulations of SCM in literatures, e.g., Sch ¨ olkopf (2019); Pearl (2019); Bongers et al.

Original: ; Pearl et al.
Revised: ; Pearl et al.

Original: ; Forr ´ e & Mooij (2020), among which the deﬁnition in Blom et al.
Revised: ; Forr ´ e & Mooij (2020), among which the deﬁnition in Blom et al.

Original: (2020) is used in this paper.
Revised: (2020) is used in this paper.

Original: Deﬁnition 1 (SCM) Let I and J be index sets.
Revised: Deﬁnition 1 (SCM) Let I and J be index sets.

Original: A Structural Causal Model (SCM) M is a triple ( X , F, E ) , with:
Revised: A Structural Causal Model (SCM) M is a triple ( X , F, E ) , with:

Original: • a product of standard measurable spaces X
Revised: • a product of standard measurable spaces X

Original: = Π
Revised: = Π

Original: i ∈I
Revised: i ∈I

Original: X i (domains of endogenous), • a tuple of exogenous random variables E
Revised: X i (domains of endogenous), • a tuple of exogenous random variables E

Original: = ( E j )
Revised: = ( E j )

Original: j ∈J taking value in a product of standard measurable space E =
Revised: j ∈J taking value in a product of standard measurable space E =

Original: (cid:81) j ∈J E j .
Revised: (cid:81) j ∈J E j .

Original: • a family of F of measurable functions:
Revised: • a family of F of measurable functions:

Original: [Equation]
Revised: [Equation]

Original: The dataset are ususally assumed to be a set of samples for the solution of SCMs.
Revised: The dataset are ususally assumed to be a set of samples for the solution of SCMs.

Original: Deﬁnition 2 (Solution of SCM)
Revised: Deﬁnition 2 (Solution of SCM)

Original: We say that a random variable X =
Revised: We say that a random variable X =

Original: ( X i )
Revised: ( X i )

Original: i ∈
Revised: i ∈

Original: I is a solution to an
Revised: I is a solution to an

Original: SCM M =
Revised: SCM M =

Original: ( X , F, E ) if
Revised: ( X , F, E ) if

Original: [Equation]
Revised: [Equation]

Original: An SCM may have a unique(up to zero sets) solution, multiple solutions, or there may not exist any solution at all.
Revised: An SCM may have a unique(up to zero sets) solution, multiple solutions, or there may not exist any solution at all.

Original: Deﬁnition 

Revised: Deﬁnition

Original: An SCM M is called simple if it is uniquely solvable with respect to any subset O ⊆
Revised: An SCM M is called simple if it is uniquely solvable with respect to any subset O ⊆

Original: I .
Revised: I .

Original: All acyclic SCMs are simple.
Revised: All acyclic SCMs are simple.

Original: Deﬁnition 

Revised: Deﬁnition

Original: A do intervention do (˜ x I ) with target I ⊆
Revised: A do intervention do (˜ x I ) with target I ⊆

Original: I and value ˜ x I ∈
Revised: I and value ˜ x I ∈

Original: X I on an SCM M =
Revised: X I on an SCM M =

Original: ( X , F, E ) maps it to the intervened SCM M do (˜ x I ) =
Revised: ( X , F, E ) maps it to the intervened SCM M do (˜ x I ) =

Original: ( X , ˜ F, E ) with ˜ F the family of measurable functions:
Revised: ( X , ˜ F, E ) with ˜ F the family of measurable functions:

Original: [Equation]
Revised: [Equation]

Original: The intervened SCM is referred as a submodel of the original SCM, in fact, the variants derived from many different types of interventions (e.g., perfect, imperfect, stochastic, etc.) are also referred as submodel.
Revised: The intervened SCM is referred as a submodel of the original SCM, in fact, the variants derived from many different types of interventions (e.g., perfect, imperfect, stochastic, etc.) are also referred as submodel.

Original:  M ETA S TRUCTURAL C AUSAL M ODELS
Revised:  M ETA S TRUCTURAL C AUSAL M ODELS

Original: One critical insight in philosophy is that the causal mechanisms behind a system under investigation are not generally observable, but they do produce observable traces (“data,” in modern terminology).
Revised: One critical insight in philosophy is that the causal mechanisms behind a system under investigation are not generally observable, but they do produce observable traces (“data,” in modern terminology).

Original: This insight naturally leads to two practical desiderata for any proper framework for causal inference, namely:
Revised: This insight naturally leads to two practical desiderata for any proper framework for causal inference, namely:

Original: The causal mechanisms underlying the phenomenon under investigation should be accounted for – indeed, formalized – in the analysis.
Revised: The causal mechanisms underlying the phenomenon under investigation should be accounted for – indeed, formalized – in the analysis.

Original: This collection of mechanisms (even if mostly unobservable) should be formally tied to its output: the generated phenomena and corresponding datasets.
Revised: This collection of mechanisms (even if mostly unobservable) should be formally tied to its output: the generated phenomena and corresponding datasets.

Original: The mathematical object called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges.
Revised: The mathematical object called a structural causal model (SCM) is used to represent the causal mechanisms from which a causal hierarchy to describe the generated phenomena organically emerges.

Original: It is often assumed that every instantiation E =
Revised: It is often assumed that every instantiation E =

Original: e of the exogenous variables uniquely determines the values of all variables in X (Pearl, 2019), which leads a unique solution of the corresponding SCM .
Revised: e of the exogenous variables uniquely determines the values of all variables in X (Pearl, 2019), which leads a unique solution of the corresponding SCM .

Original: Then the dataset D = { x ( k ) } k =1 ,...,N is a set of N samples of the unique solution.
Revised: Then the dataset D = { x ( k ) } k =1 ,...,N is a set of N samples of the unique solution.

Original: But in many cases, SCM with cycles might be not solvable or have multiple solutions (Halpern, 1998).
Revised: But in many cases, SCM with cycles might be not solvable or have multiple solutions (Halpern, 1998).

Original: Example 1 (Multiple Solutions) Consider an SCM M 1 =
Revised: Example 1 (Multiple Solutions) Consider an SCM M 1 =

Original: ( X , F = { f 1 , f 2 } , E = { E 1 } ) , where
Revised: ( X , F = { f 1 , f 2 } , E = { E 1 } ) , where

Original: [Equation]
Revised: [Equation]

Original: Obviously, (1 − E 1 , 1 − E 1 ) and (1 + E 1 , 1 + E 1 ) are two different solutions to M 1 , then which solution of the SCM should be used to link the dataset to the model?
Revised: Obviously, (1 − E 1 , 1 − E 1 ) and (1 + E 1 , 1 + E 1 ) are two different solutions to M 1 , then which solution of the SCM should be used to link the dataset to the model?

Original: The previous causal inference literature rarely deals with theoretical aspects of cyclic causality.
Revised: The previous causal inference literature rarely deals with theoretical aspects of cyclic causality.

Original: In recent years, it has been formally discussed in Bongers et al.
Revised: In recent years, it has been formally discussed in Bongers et al.

Original: However, it is also acknowledged by this paper that there are many complications in dealing with cyclic causal models.
Revised: However, it is also acknowledged by this paper that there are many complications in dealing with cyclic causal models.

Original: The vast majority of methods to deal with cyclic SCM in the literature are by adding additional assumptions, such as linear constraints(Spirtes, 1993; 1994; Hyttinen et al., 2012) and certain solvability constraints(Forr´e & Mooij, 2018; Bongers et al., 2016).
Revised: The vast majority of methods to deal with cyclic SCM in the literature are by adding additional assumptions, such as linear constraints(Spirtes, 1993; 1994; Hyttinen et al., 2012) and certain solvability constraints(Forr´e & Mooij, 2018; Bongers et al., 2016).

Original: These methods basically exclude the study of SCMs with multiple solutions such as Example 1.
Revised: These methods basically exclude the study of SCMs with multiple solutions such as Example 1.

Original: The view of understanding causation as information transfer was ﬁrst formally proposed by (Collier, 1999) in philosophy recently.
Revised: The view of understanding causation as information transfer was ﬁrst formally proposed by (Collier, 1999) in philosophy recently.

Original: Inspired by this view, we realize that the causal links among variables can be cut off suggested by unsuccessful information transfer, which suggests that different samples might have different causal graphs and causal mechanisms.
Revised: Inspired by this view, we realize that the causal links among variables can be cut off suggested by unsuccessful information transfer, which suggests that different samples might have different causal graphs and causal mechanisms.

Original: For example, samples of M 1 might only satisfy only a subset of the structural equations due to absence of information , hence variables for two different samples x
Revised: For example, samples of M 1 might only satisfy only a subset of the structural equations due to absence of information , hence variables for two different samples x

Original: ( i ) and x ( j ) could have two different causal graphs.
Revised: ( i ) and x ( j ) could have two different causal graphs.

Original: However, we usually do not know when and where the information transmission was interrupted for a given sample.
Revised: However, we usually do not know when and where the information transmission was interrupted for a given sample.

Original: In fact, it might be infeasible to specify the information transmission details of all samples of an SCM.
Revised: In fact, it might be infeasible to specify the information transmission details of all samples of an SCM.

Original: To address the above problem, we introduce the concept of active set of an SCM, which is inspired by the active set method in the ﬁeld of non-linear optimization theory.
Revised: To address the above problem, we introduce the concept of active set of an SCM, which is inspired by the active set method in the ﬁeld of non-linear optimization theory.

Original: Deﬁnition 5 (Active mechanisms)
Revised: Deﬁnition 5 (Active mechanisms)

Original: For a given sample x ( k ) and the corresponding collection of mechanisms represented by an SCM M =
Revised: For a given sample x ( k ) and the corresponding collection of mechanisms represented by an SCM M =

Original: ( X , F, E ) , if x ( k ) i
Revised: ( X , F, E ) , if x ( k ) i

Original: =
Revised: =

Original: f i ( x ( k ) pa ( i ) ∩I
Revised: f i ( x ( k ) pa ( i ) ∩I

Original: e ( k ) pa ( i ) ∩J
Revised: e ( k ) pa ( i ) ∩J

Original: we
Revised: we

Original: The collection of active sets { A k } k =1 , 2 ,... gives us the opportunity to avoid considering the details of the information transfer between variables, and to describe the relationship between the data and the model relatively concisely.
Revised: The collection of active sets { A k } k =1 , 2 ,... gives us the opportunity to avoid considering the details of the information transfer between variables, and to describe the relationship between the data and the model relatively concisely.

Original: Formally, we deﬁne the meta structural causal model (meta-SCM) as follows:
Revised: Formally, we deﬁne the meta structural causal model (meta-SCM) as follows:

Original: Deﬁnition 6 (meta-SCM)
Revised: Deﬁnition 6 (meta-SCM)

Original: A collection of mechanisms described by an SCM M =
Revised: A collection of mechanisms described by an SCM M =

Original: ( X , F, E ) with a dataset D
Revised: ( X , F, E ) with a dataset D

Original: = { x ( k ) } k
Revised: = { x ( k ) } k

Original: =1 ,...,N , in which each sample x ( k ) satisﬁes that:
Revised: =1 ,...,N , in which each sample x ( k ) satisﬁes that:

Original: • the prior distribution of E ( k ) is P ( E ) ; • A k ⊆
Revised: • the prior distribution of E ( k ) is P ( E ) ; • A k ⊆

Original: I is referred as the active set of the sample k satisﬁes that
Revised: I is referred as the active set of the sample k satisﬁes that

Original: [Equation]
Revised: [Equation]

Original: Then the tuple (cid:104)M , D(cid:105) (or in short M ) is called a meta structural causal model (meta-SCM) .
Revised: Then the tuple (cid:104)M , D(cid:105) (or in short M ) is called a meta structural causal model (meta-SCM) .

Original: The difference between SCM and meta-SCM.
Revised: The difference between SCM and meta-SCM.

Original: On one hand, an SCM can be interpreted as a special case of meta-SCM satisﬁes that the active set
Revised: On one hand, an SCM can be interpreted as a special case of meta-SCM satisﬁes that the active set

Original: A k is equal to I for any sample x ( k ) .
Revised: A k is equal to I for any sample x ( k ) .

Original: On the other hand, a meta-SCM share the causal mechanisms with its corresponding SCM only differs on the method for linking data to model.
Revised: On the other hand, a meta-SCM share the causal mechanisms with its corresponding SCM only differs on the method for linking data to model.

Original: Thus, it improves the expressiveness of the canonical SCM.
Revised: Thus, it improves the expressiveness of the canonical SCM.

Original: Actually, a meta-SCM suggests a method for connecting any dataset to an SCM with the active sets.
Revised: Actually, a meta-SCM suggests a method for connecting any dataset to an SCM with the active sets.

Original: Formally,
Revised: Formally,

Original: Theorem 7 (Connecting Data to Mechanisms) For an SCM M =
Revised: Theorem 7 (Connecting Data to Mechanisms) For an SCM M =

Original: ( X , F, E ) with any dataset
Revised: ( X , F, E ) with any dataset

Original: D = { x ( k ) } k
Revised: D = { x ( k ) } k

Original: =1 ,...,N in the domain of X .
Revised: =1 ,...,N in the domain of X .

Original: Then each datapoint x ( k ) is a sample from some submodel
Revised: Then each datapoint x ( k ) is a sample from some submodel

Original: SCM ˜ M related to the active set
Revised: SCM ˜ M related to the active set

Original: A k .
Revised: A k .

Original: Proof For any k = 1 , ..., N , let ˜ M ( k ) =
Revised: Proof For any k = 1 , ..., N , let ˜ M ( k ) =

Original: ( X , ˜ F, E ) be an SCM with modiﬁed causal mechanisms:
Revised: ( X , ˜ F, E ) be an SCM with modiﬁed causal mechanisms:

Original: [Equation]
Revised: [Equation]

Original: Then the active set of datapoint x ( k ) for the submodel ˜ M ( k ) is I by deﬁnition 1 , which directly leads to our theorem.
Revised: Then the active set of datapoint x ( k ) for the submodel ˜ M ( k ) is I by deﬁnition 1 , which directly leads to our theorem.

Original: The above proof directly assigns a submodel for each sample in the dataset, which only part of mechanisms in the original SCM holds.
Revised: The above proof directly assigns a submodel for each sample in the dataset, which only part of mechanisms in the original SCM holds.

Original: In fact, the submodel in our meta-SCM framework does not have to be constructed as a do -intervened model ˜ M ( k ) , it can be any subclass of SCMs with desired properties(such as acyclic) in literatures.
Revised: In fact, the submodel in our meta-SCM framework does not have to be constructed as a do -intervened model ˜ M ( k ) , it can be any subclass of SCMs with desired properties(such as acyclic) in literatures.

Original: When there are cyclic causal relationships between variables, one encounters various technical complications, which even arise in the linear setting(Bongers et al., 2016).
Revised: When there are cyclic causal relationships between variables, one encounters various technical complications, which even arise in the linear setting(Bongers et al., 2016).

Original: The main idea for solving related difﬁculties is to add additional restrictions on structural equations, and the dataset are assumed to be consisted of samples from a distribution obtained by solving the SCM.
Revised: The main idea for solving related difﬁculties is to add additional restrictions on structural equations, and the dataset are assumed to be consisted of samples from a distribution obtained by solving the SCM.

Original: In contrast, our meta-SCM does not add additional assumptions on SCM, and each datapoint is treated as a sample of the distribution obtained by solving a certain submodel.
Revised: In contrast, our meta-SCM does not add additional assumptions on SCM, and each datapoint is treated as a sample of the distribution obtained by solving a certain submodel.

Original: For the SCM M 1 with muptiple solutions in Example 1 with dataset D = { x ( k ) } k =1 ,...,N in the domain of X , the meta-SCM can circumvent theoretical complications through providing each datapoint a distribution of any solution of a certain submodel.
Revised: For the SCM M 1 with muptiple solutions in Example 1 with dataset D = { x ( k ) } k =1 ,...,N in the domain of X , the meta-SCM can circumvent theoretical complications through providing each datapoint a distribution of any solution of a certain submodel.

Original: Usually, the details of submodel and its corresponding distribution for each sample might be unknown, and meta-SCM only provides a coarse description by the active sets.
Revised: Usually, the details of submodel and its corresponding distribution for each sample might be unknown, and meta-SCM only provides a coarse description by the active sets.

Original: In fact, we can also circumvent the technique complications caused by solvability through metaSCM.
Revised: In fact, we can also circumvent the technique complications caused by solvability through metaSCM.

Original: Speciﬁcally, the structural equations of an acyclic SCM trivially have a unique solution, which ensures that the SCM gives rise to a unique, well-deﬁned probability distribution on the variables.
Revised: Speciﬁcally, the structural equations of an acyclic SCM trivially have a unique solution, which ensures that the SCM gives rise to a unique, well-deﬁned probability distribution on the variables.

Original: However, an SCM can be unsolvable in the case of cycles, e.g.,
Revised: However, an SCM can be unsolvable in the case of cycles, e.g.,

Original: Example 2 (Unsolvable) Consider an SCM M 2 =
Revised: Example 2 (Unsolvable) Consider an SCM M 2 =

Original: ( X , F = { f 1 , f 2 } , E = { E 1 } ) , where
Revised: ( X , F = { f 1 , f 2 } , E = { E 1 } ) , where

Original: [Equation]
Revised: [Equation]

Original: Then the SCM M 2 is obvious not solvable, thus it cannot be used to model underlying causal mechanisms of any dataset D
Revised: Then the SCM M 2 is obvious not solvable, thus it cannot be used to model underlying causal mechanisms of any dataset D

Original: = { x ( k ) } k
Revised: = { x ( k ) } k

Original: =1 ,...,N in the domain of X .
Revised: =1 ,...,N in the domain of X .

Original: But with our novel approach, we might still connect data to the underlying mechanisms, e.g. by letting | A k | = 1 for all k = 1 , ..., N .
Revised: But with our novel approach, we might still connect data to the underlying mechanisms, e.g. by letting | A k | = 1 for all k = 1 , ..., N .

Original: In other words, each datapoint x ( k ) is a sample of a distribution derived from the submodel ˜
Revised: In other words, each datapoint x ( k ) is a sample of a distribution derived from the submodel ˜

Original: M ( k ) by Theorem 7.
Revised: M ( k ) by Theorem 7.

Original: The difference between meta-system and meta-SCM.
Revised: The difference between meta-system and meta-SCM.

Original: The joint causal inference (JCI) framework reduces modeling a system in its environment to modeling the meta-system consisting of the system and its environment, which considers auxiliary context variables that describe the context of each data set (Mooij et al., 2016).
Revised: The joint causal inference (JCI) framework reduces modeling a system in its environment to modeling the meta-system consisting of the system and its environment, which considers auxiliary context variables that describe the context of each data set (Mooij et al., 2016).

Original: In contrast, our meta-SCM address the challenging case where no additional variables, besides the samples from the data to generate, are observed.
Revised: In contrast, our meta-SCM address the challenging case where no additional variables, besides the samples from the data to generate, are observed.

Original: For example in
Revised: For example in

Original: Fig.
Revised: Fig.

Original: 1, the meta-system consists of two variables X 1 , X 2 and a context variable C .
Revised: 1, the meta-system consists of two variables X 1 , X 2 and a context variable C .

Original: More concretely, the engine X 1 drives the wheels of a car
Revised: More concretely, the engine X 1 drives the wheels of a car

Original: X 2 when going uphill C = 0 , but when going downhill, the rotation of the wheels drives the engine.
Revised: X 2 when going uphill C = 0 , but when going downhill, the rotation of the wheels drives the engine.

Original: In a meta-SCM, we instead introduce the concept of active mechanisms to describe each sample in a dataset.
Revised: In a meta-SCM, we instead introduce the concept of active mechanisms to describe each sample in a dataset.

Original: Moreover, the meta-SCM framework even gain insights on unsolvable SCMs such as Example 2 while the previous meta-system study restricts themselves to simple SCMs.
Revised: Moreover, the meta-SCM framework even gain insights on unsolvable SCMs such as Example 2 while the previous meta-system study restricts themselves to simple SCMs.

Original: In philosophy viewpoint, the JCI framework uses context changes to model interventions, which is a difference-making account for causality.
Revised: In philosophy viewpoint, the JCI framework uses context changes to model interventions, which is a difference-making account for causality.

Original: Instead, the active set in meta-SCM are inspired by information transferring which is a production account of causality, see e.g. Illari & Russo (2014).
Revised: Instead, the active set in meta-SCM are inspired by information transferring which is a production account of causality, see e.g. Illari & Russo (2014).

Original: [Equation]
Revised: [Equation]

Original: meta-SCM when in the presence of cycles.
Revised: meta-SCM when in the presence of cycles.

Original: a) X 1 causes X 2 in context C = 0 , f 2 is activated; (b) X 2 causes X 1 in context C = 1 , f 1 is activated; (c) X 1 and Xcause each other in the joint model, f 1 , f 2 are activated.
Revised: a) X 1 causes X 2 in context C = 0 , f 2 is activated; (b) X 2 causes X 1 in context C = 1 , f 1 is activated; (c) X 1 and Xcause each other in the joint model, f 1 , f 2 are activated.

Original: From the above discussions, the meta-SCM framework can be considered as generalization of SCM that novelly links data to causal mechanisms through active sets at individual level modeling 2 , and it might be used to circumvent technical complications when cycles present.
Revised: From the above discussions, the meta-SCM framework can be considered as generalization of SCM that novelly links data to causal mechanisms through active sets at individual level modeling 2 , and it might be used to circumvent technical complications when cycles present.

Original: Then, a natural question is how to perform inferences with a meta-SCM when in the presence of cycles ？
Revised: Then, a natural question is how to perform inferences with a meta-SCM when in the presence of cycles ？

Original: A ﬁrst difﬁcult might be the lack of knowledge on the speciﬁc form of active sets, i.e. A k , for each sample.
Revised: A ﬁrst difﬁcult might be the lack of knowledge on the speciﬁc form of active sets, i.e. A k , for each sample.

Original: In the following section, we propose a principle for learning and reasoning within the meta-SCM framework.
Revised: In the following section, we propose a principle for learning and reasoning within the meta-SCM framework.

Original:  S UFFICIENT A CTIVATED M ECHANISMS
Revised:  S UFFICIENT A CTIVATED M ECHANISMS

Original: The meta-SCM viewpoint considers a set of variables indexed by I and a set of samples D with causal (or disentangled) factorization ,
Revised: The meta-SCM viewpoint considers a set of variables indexed by I and a set of samples D with causal (or disentangled) factorization ,

Original: [Equation]
Revised: [Equation]

Original: The independent causal mechanisms (ICM) principle tells us that the factors should be independent in the sense that
Revised: The independent causal mechanisms (ICM) principle tells us that the factors should be independent in the sense that

Original: a) changing (or performing an intervention upon) one mechanism P ( x i | x pa ( i ) ∩I ) does not change any of the other mechanisms P ( x j | x pa ( j ) ∩I ))
Revised: a) changing (or performing an intervention upon) one mechanism P ( x i | x pa ( i ) ∩I ) does not change any of the other mechanisms P ( x j | x pa ( j ) ∩I ))

Original: ( i (cid:54) =
Revised: ( i (cid:54) =

Original: j ) (Sch¨olkopf et al., 2012), and
Revised: j ) (Sch¨olkopf et al., 2012), and

Original: b) knowing some other mechanisms P ( x i | x pa ( i ) ∩I )
Revised: b) knowing some other mechanisms P ( x i | x pa ( i ) ∩I )

Original: ( i (cid:54) =
Revised: ( i (cid:54) =

Original: j ) does not give us information about a mechanism P ( x j | x pa ( j ) ∩I ))
Revised: j ) does not give us information about a mechanism P ( x j | x pa ( j ) ∩I ))

Original: Janzing & Sch¨olkopf
Revised: Janzing & Sch¨olkopf, 2010).

Original: Consistent with the implication a) of the ICM Principle, the sparse mechanism shift (SMS) hypothesis (See e.g. Parascandolo et al.
Revised: Consistent with the implication a) of the ICM Principle, the sparse mechanism shift (SMS) hypothesis (See e.g. Parascandolo et al.

Original: ; Sch¨olkopf (2019)) was stated in the following:
Revised: ; Sch¨olkopf (2019)) was stated in the following:

Original: Sparse Mechanism Shift (SMS).
Revised: Sparse Mechanism Shift (SMS)

Original: Small distribution changes tend to manifest themselves in a sparse or local way in the causal/disentangled factorization, i.e., they should usuallynot affect all factors simultaneously.
Revised: Small distribution changes tend to manifest themselves in a sparse or local way in the causal/disentangled factorization , i. e. they should usually

Revised: not affect all factors simultaneously.

Original: It is the fundamental assumption for causal representation learning(Sch¨olkopf et al., 2021) and has recently been used for learning causal models (Ke et al., 2019), modular architectures (Goyal et al., 2019; Besserve et al., 2020) and disentangled representations (Locatello et al., 2020).
Revised: It is the fundamental assumption for causal representation learning(Sch¨olkopf et al., 2021) and has recently been used for learning causal models (Ke et al., 2019), modular architectures (Goyal et al., 2019; Besserve et al., 2020) and disentangled representations (Locatello et al., 2020).

Original: However, the SMS hypothesis usually only assumes mechanisms shift from a given context to the original uniquely solvable SCM, especially acyclic SCM.
Revised: However, the SMS hypothesis usually only assumes mechanisms shift from a given context to the original uniquely solvable SCM, especially acyclic SCM.

Original: We propose a hypothesis in the viewpoint of a given individual mechanism across samples in the following:
Revised: We propose a hypothesis in the viewpoint of a given individual mechanism across samples in the following:

Original: Sufﬁcient Activated Mechanisms (SAM).
Revised: Sufﬁcient Activated Mechanism (SAM).

Original: All factors in the causal/disentangled factorization should be activated sufﬁciently,
Revised: All factors in the causal/disentangled factorization should be activated sufﬁciently,

Original: i
Revised: i

Original: e., every causal mechanism is sufﬁciently often activated across samples in the dataset.
Revised: e., every causal mechanism is sufﬁciently often activated across samples in the dataset.

Original: The “sufﬁcient” in the SAM roughly understood as sufﬁcient for identifying factors in the causal factorization of interest.
Revised: The “sufﬁcient” in the SAM roughly understood as sufﬁcient for identifying factors in the causal factorization of interest.

Original: In Example 2, the unsolvable cyclic SCM M 2 cannot be used as the underlying generative process for any dataset.
Revised: In Example 2, the unsolvable cyclic SCM M 2 cannot be used as the underlying generative process for any dataset.

Original: On the contrary, in the meta-SCM framework under
Revised: On the contrary, in the meta-SCM framework under

Original: SAM assumption, we are able to learn a causal factorization Eq.
Revised: SAM assumption, we are able to learn a causal factorization Eq.

Original: (2), even when only knowing how many factors have activated, but not which ones.
Revised: (2), even when only knowing how many factors have activated, but not which ones.

Original: In mathematics, start with solving the unsolvable equation x 2 = − 1 with imaginary unit i , it has been an important branch of mathematics which is complex analysis with holomorphic function as a central research object.
Revised: In mathematics, start with solving the unsolvable equation x 2 = − 1 with imaginary unit i , it has been an important branch of mathematics which is complex analysis with holomorphic function as a central research object.

Original: Similarly, the meta-SCM, which introduced a novel dimension on linking data to SCM, can help in causal modeling with unsolvable SCM, and we feel meta-SCM with SAM assumption might be a central research object for our framework as holomorphic function for complex analysis.
Revised: Similarly, the meta-SCM, which introduced a novel dimension on linking data to SCM, can help in causal modeling with unsolvable SCM, and we feel meta-SCM with SAM assumption might be a central research object for our framework as holomorphic function for complex analysis.

Original: Overall, in the lens of two practical desirata for any proper framework mentioned in the begining of
Revised: Overall, in the lens of two practical desirata for any proper framework mentioned in the begining of

Original: Section 3, our meta-SCM is different from all literatures on variants of SCMs by adding a fully novel dimension on considering how each sample is tied to underlying mechanisms.
Revised: Section 3, our meta-SCM is different from all literatures on variants of SCMs by adding a fully novel dimension on considering how each sample is tied to underlying mechanisms.

Original: This extra dimension though, on one hand, complicate the SCM framework, however, on the other hand, improve its expressiveness and power for causal modeling.
Revised: This extra dimension though, on one hand, complicate the SCM framework, however, on the other hand, improve its expressiveness and power for causal modeling.

Original: Then, the SAM hypothesis deﬁnes a subclass of meta-SCMs of interest for performing causal inferences and reasoning, consistent with the role of the ICM Principle/ SMS hypothesis in the SCM framework.
Revised: Then, the SAM hypothesis deﬁnes a subclass of meta-SCMs of interest for performing causal inferences and reasoning, consistent with the role of the ICM Principle/ SMS hypothesis in the SCM framework.

Original: The causality behind interventions in the SCM framework is a difference-making account, while the production accounts, especially informational interpretation of causality relatively neglected.
Revised: The causality behind interventions in the SCM framework is a difference-making account, while the production accounts, especially informational interpretation of causality relatively neglected.

Original: One core idea of informational causality — C causes E if there is information transmission from C to
Revised: One core idea of informational causality — C causes E if there is information transmission from C to

Original: E (Illari & Russo, 2014), suggests that the causal mechanism f i of an SCM can be separated into two part — information process and information transfer (Gong & Zhu, 2021).
Revised: E (Illari & Russo, 2014), suggests that the causal mechanism f i of an SCM can be separated into two part — information process and information transfer (Gong & Zhu, 2021).

Original: Formally,
Revised: Formally,

Original: Deﬁnition 8 (Informational Decomposition of SCM) Consider an SCM M =
Revised: Deﬁnition 8 (Informational Decomposition of SCM) Consider an SCM M =

Original: ( X , F, E ) .
Revised: ( X , F, E ) .

Original: The informational decomposition of M is deﬁned as, for any i ∈ I ,
Revised: The informational decomposition of M is deﬁned as, for any i ∈ I ,

Original: [Equation]
Revised: [Equation]

Original: where e j,i represents the information on edge ( j, i ) received from its input node j .
Revised: where e j,i represents the information on edge ( j, i ) received from its input node j .

Original: This dichotomy of causal mechanisms can help illuminate on the relationship between SMS and
Revised: This dichotomy of causal mechanisms can help illuminate on the relationship between SMS and

Original: SAM assumption.
Revised: SAM assumption.

Original: We tend to assume the information transfer mechanisms are sparse inactivated, while the information transfer mechanisms are instead only sufﬁcient activated.
Revised: We tend to assume the information processing mechanisms are sparse inactivated, while the information transfer mechanisms are instead only sufﬁcient activated.

Original: Henceforth, if we consider the inactivated mechanisms as shifted mechanisms, then the information process and transfer mechanisms are satisfying the SMS hypothesis and the SAM assumption respectively.
Revised: Henceforth, if we consider the inactivated mechanisms as shifted mechanisms, then the information process and transfer mechanisms are satisfying the SMS hypothesis and the SAM assumption respectively.

Original: Small distribution changes caused by interventions on information process tend to manifest themselves in a sparse or local way, but might be manifest themselves in a non-local way caused by changes on transferred information.
Revised: Small distribution changes caused by interventions on information process tend to manifest themselves in a sparse or local way, but might be manifest themselves in a non-local way caused by changes on transferred information.

Original: For example in a starﬁsh shaped causal diagram, changing the output information of the centre node (instead itself) can lead to shift on all functional relationships with its child nodes, which can affect almost all factors simultaneously.
Revised: For example in a starﬁsh shaped causal diagram, changing the output information of the centre node (instead itself) can lead to shift on all functional relationships with its child nodes, which can affect almost all factors simultaneously.

Original: Thus, the SMS assumption might be inappropriate for causal modeling.
Revised: Thus, the SMS assumption might be inappropriate for modeling some systems within the meta-SCM framework.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: This work is mostly theoretical and conceptual to address the notoriously hard problem of cyclic causal models.
Revised: This work is mostly theoretical and conceptual to address the notoriously hard problem of cyclic causal models.

Original: We proposed a novel active set approach for connecting data to SCMs (even not solvable), instead of making additional assumptions to restrict the class of SCMs of interest.
Revised: We proposed a novel active set approach for connecting data to SCMs (even not solvable), instead of making additional assumptions to restrict the class of SCMs of interest.

Original: Note that the meta-SCM is more of an conceptional modeling framework on how to relate data to the underlying causal mechanisms rather than a speciﬁc model, and the SAM assumption has been introduced as an inductive bias for performing causal inferences and learning within the framework of meta-SCM.
Revised: Note that the meta-SCM is more of an conceptional modeling framework on how to relate data to the underlying causal mechanisms rather than a speciﬁc model, and the SAM assumption has been introduced as an inductive bias for performing causal inferences and learning within the framework of meta-SCM.

Original: To best of our knowledge, this is the ﬁrst causal modeling framework to explore the dimension formally on how to link each sample in the dataset to the collection of mechanisms.
Revised: To best of our knowledge, this is the ﬁrst causal modeling framework to explore the dimension formally on how to link each sample in the dataset to the collection of mechanisms.

Revised: In the future we might anticipate more causal modeling exploration on exacting information from activated mechanisms directly instead of the solution of SCMs.