A Topological Perspective on Causal Inference
Authors: Duligur Ibeling
Thomas F Icard
Venue: NeurIPS 2021 Poster (2021)
Keywords: causal inference
statistical learning theory
topology

Abstract: This paper presents a topological learning-theoretic perspective on causal inference by introducing a series of topologies defined on general spaces of structural causal models (SCMs). As an illustration of the framework we prove a topological causal hierarchy theorem, showing that substantive assumption-free causal inference is possible only in a meager set of SCMs. Thanks to a known correspondence between open sets in the weak topology and statistically verifiable hypotheses, our results show that inductive assumptions sufficient to license valid causal inferences are statistically unverifiable in principle. Similar to no-free-lunch theorems for statistical inference, the present results clarify the inevitability of substantial assumptions for causal inference. An additional benefit of our topological approach is that it easily accommodates SCMs with infinitely many variables. We finally suggest that our framework may be helpful for the positive project of exploring and assessing alternative causal-inductive assumptions.

A Topological Perspective on Causal Inference
Duligur Ibeling Department of Computer Science Stanford University dul dul.igursttanoordd edu
Thomas Icard Department Philosophy Stanford University icard@stanf ord. edu
Abstract
This paper presents a topological serming-theoretic perspective on causal inference by introducing a series of topologies defined on general spaces of structural causal models (SCMs). As an illustration of the framework we prove a topological causal hierarchy theorem, showing that substantive assumption -free causal inference is possible only a meager set of SCMs. Thanks to a known correspondence between open in the weak topology and statistically verifiable hypotheses, results show that inductive assumptions sufficient to license valid causal inferences are statistically unverifiable in principle. Similar to fo-frre-luuch theorems for statisti- cal inference, the present results clarify the inevitability of substantial assumptions for causal inference. An additional benefit of our topological approach is that it easily accommodates SCMs with infinitely many variables. We finally suggest that the framework may be helpful for the positive project of exploring and assessing alternative causal-inductio assumptions.
Introduction and Motivation
In the background of any investigation into learning algorithms are no-free-lunch phenomena: roughly, the observation that assumption statistical learning is infcasible in general (see, e.g., [33] Ch. 5] for a formal statement). Common wisdom that learning algorithms and architectures must adequately reflect non-trivial features of the data-generating distribution to gain inductive purchase.
For many purposes we need to move beyond passive observation, focusing instead on what would happen were we to act upon a given system. Even further, we sometimes desire to explain the behavior of system, raising questions about what would have occurred had some aspects of a situation been different. Such questions depend not just on the data distribution; they depend on deeper features of underlying data data-generee generating processes or mechanisms. It is thus generally acknowledged that stronger assumptions are required if we want to draw causal conclusions from data [35] 28 20 30 32).
Whether implicit explicit, any approach causal inference involves a space of candidate causal models, viz. Lata-generatinn processes. Indeed, a blunt way of incorporating inductive bias is simply omit some class of possible causal hypotheses from consideration. Many (im)possibility results in the literature can accordingly be understood as pertaining to all models within a class. For instance, if we can restrict attention Markovian models that satisfy faithfulness, then we can always identify the structure of model from experimental data (e.g., (31). If we can restrict attention to Markovian (continuous) models with linear functions and non-Gaussian noise, then every model can be learned even from purely observational data 34]. As a negative example, in the larger class of (not necessarily Markovian) models, no model can ever be determined from observational data alone 2]
At the same time, in many settings it is sensible to aim for results with "nearly universal" force. is natural ask, e.g., within the class of all Markovian models, how "typical" are those in which the faithfulness condition is violated? This might tell us, for instance, how typically we could expect failure of method that depended on these assumptions. A well-known result shows that, fixing
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

any particular causal dependence graph, such violations have measure zero for any smooth (e.g.. Lebesgue) measure on the parameter space of distributions consistent with that graph [24]. In fact, the standard notion of statistical consistency itself, which underlies many possibility results in causal inference, requires omission of some purportedly "negligible" set of possible data streams [335.
There are two standard mathematical approaches to making concepts like "typical" and "negligible" rigorous: measure-theores and topological. While the two approaches often agree, they capture slightly different intuitions 25. One virtue of the measure-theoretic approach is its natural proba- bilistic interpretation: intuitively, we are exceedingly unlikely to hit upon set with measure zero. At the same time, the measure- -heoorrttctcttcccccc approach is sometimes criticized in statistical settings for its alleged dependence on a measure, and this has been argued favor topologiaal approaches (see, e.g.. 3 on
theorems). The latter of course in turn demands an appropriate topology.
In the present work we show how to define sequence of meaningful topologies the space of causal models, each corresponding to progressively coarser level of the so called causal hierarchy ( ([29) 2
see Fig. [] for an abbreviated pictorial summary). We aim to demonstrate that topologizing causal models in this way helps clarify the scope and limits of causal inference under different assumptions, as well as the potential empirical status of those very assumptions, in a highly general setting.
Our starting point is a canonical topology on the space of Borel probability distributions called the weak topology. The weak topology is grounded in the fundamental notion of weak convergence of probability distributions [4] and is thereby closely related to problems of statistical inference (see, e.g., (8). Recent work has sharpened this correspondence, showing that open sets in the weak topology correspond exactly to the statistical hypotheses that can be naturally deemed verifiable 14 [6]. We extend the correspondence to higher levels of the causal hierarchy, including the most refined and expansive "top" level consisting of all (well-founded) causal models. Lower levels and natural subspaces (e.g., corresponding to prominent causal assumption classes) emerge as coarsenings and continuous projections of this largest space. As an illustration of the general approach, we prove topological version of the causal hierarchy theorem from 2 Rather than showing that collapse happens only in measure zero set as in 22) our Theorem show that collapse is topologically meager. Conceptually, this highlights a different (but complementary) intuition: not only is collapse exceedingly unlikely in the sense of measure, meagerness implics that collapse could never be statistically verified. Correlatively, this implies that any causal assumption that would generally allow us to infer counterfactual probabilities from experimental (or "interventional") probabilities must itself be statistically unverifiable (Corollary[] [T.
To derive such result we actually show something slightly stronger (see Lem. 2 even with respect to the subspace of models consistent with a fixed temporal order on variables, the causal hierarchy theorem holds. Merely knowing the temporal order of the variables is not enough to render collapse of the hierarchy a statistically verifiable proposition. Furthermore, we show that the witness to collapse can be taken as any of the well-known counterfactual "probabilities of causation" (see, e.g., 127)): probabilities of necessity, sufficiency, necessity and sufficiency, enablement, or disablement. That is, none of these important quantities are fully determined by experimental data except in a meager set.
In $2] we give background on causal models, and in $ present model-theoretic characterization of the causal hierarchy as a sequence of spaces. Topology is introduced in $4 and the main results about collapse appear in | For the technical results, we include proof sketches in the main text to provide the core intuitions, relegating some of the details to an exhaustive technical appendix, which also includes additional supplementary material.
2 Structural Causal Models
A fundamental building block in the theory of causality is the structural causal model [26 35 28 or SCM, which formalizes the notion of a data- lata-generating process. In addition to specifying data- generating distributions, these models also specify the generative mechanisms that produce them. For the purpose of causal inference and learning, SCMs provide a broad, fine-grained hypothesis space.
The notions in this section have their usual definition following, e.g., 288 but we have recast them in the standard language of Borel probability spaces so as to handle the case of infinitely many variables rigorously. We start with notation, basic assumptions, and some probability theory.

---

The signature (or range) of a variable V is denoted Xv. Where S is a set of variables, let Xs Xses Xs. Given an indexed family of sets REE and elements sq E Ss. let (sg)s denote the tuple whose element at index / is s3, for all B. For B' C B write T X BEB SB + X BEB Sa for the projection map sending each ($ş)şе + (8gr))re abbreviate क ( where 3 B.
Notation.
The reader is referred to standard texts 2 for elaboration on the concepts used below.
Definition 1 (Topology). For discrete spaces (like Xs: for a single categorical variable S) we use the discrete topology and for product spaces (like Xs for a set of variables S) we use the product topology. Note that the so-called cylinder sets of the form Ty' 2 ({y)) for finite subsets Y C S and y E XY form a basis for the product topology on Xs. This cylinder set is subset of xs, and contains exactly those valuations agreeing with the value (y) specified in y for Y, for every Y E Y. Following standard statistical notation this cylinder is abbreviated as simply y.
Definition 2 (Probability). Where / is topological space write (()) for its Borel a-algebra of measurable subsets. Let (D)) be the set of probability measures B(d). Specifically, elements of (d) are functions 1 (()) - [0, 1] assigning a probability to each measurable set such that ((0)) = 1 and μ (,, (S.)) _in1 ju({)) for each sequence S1, S2, ... of pairwise disjoint sets from (()). A map f 01 + 22 is said to be measurable if (S2) E B(0)) for every S2 E B(02).
Fact 1 (Lemma 1.9.4 [5). A Borel probability measure is determined by its values on basis.
2.1 SCMs, Observational Distributions
Let V be a set of endogenous variables. We assume for simplicity every variable V E V is dichotomous with Xv {0, 1}, although the results here generalize to any larger countable range. Influences among endogenous variables are the main phenomena our formalism aims to capture. A well-founded diiree influence relation - on V encapsulates the notion of one endogenous variable possibly influencing another. For each V E V. we call (V' E EV: : - V} = Pa(V) the parents of V. We assume every set Pa(V) is finite: this condition is called local finiteness. These two assumptions foundeeousss and local finiteness) generalize the common recursiveness assumption to the infinitary setting, and have an alternative characterization in terms of "temporal" orderings:
Fact 2. Say that a total order < on V is w-like if every node has finitely many predecessors: for each V E V, the set {V', : V } is finite. Then the influence relation + is extendible to an w-like order iff is well-founded and locally finite.
In addition to endogenous variables. causal models have exogenous variables U. Each endogenous V depends on subset U(V) C. U of "exogenous parents" and uncertainty enters via exogenous noise, that is, distribution from P(x)). A structural function (or mechanism) for V E V is measurable fv XPa(V) XU(V) Xv mapping parental endogenous and exogenous valuations to values.
Definition 3. A structural causal model is a tuple M collection of exogenous variables, V is a collection of endogenous variables, fv is a structural function for each V E V, and P E 3(xu) is a probability measure on (the Borel o-algebra of) Xu-
U, {fv]vev P) where U is
As is well known, recursiveness implies that each E xu induces unique v E Xv that solves the simultancous system of structural equations {V V/=1/ fvvv:
Proposition 1. Any SCM M with well-founded, locally finite parent relation + induces a unique measurable m M : XU + Xv such that fv (TPa(V) u XU and V V.
(u(r) = my (m (u]) for all
Measurability then entails that the exogenous noise induces a distribution on joint valuations of V, called the observational distribution, which characterizes passive observations of the system.
Definition 4. The observational distribution M E P(xv) is defined on open sets by y (y) = P((m))-1 -1 '(y)). Here recall that y represents a cylinder subset (Definition of XV.
'See Appendix foror additional background on orders and relations.
3

---

2.2 Interventions
What makes SCMs distinctively causal is the way they accommodate statements about possible manipulations of causal setup capturing, e.g., observations resulting from a controlled experimental trial. This is formalized in the following definition.
Definition 5. An intervention is choice of a finite subset of variables W C V and w E Xw. This intervention is written W w, and we let A be the set of all interventions. Under this intervention, each W E W is held fixed to its value TW (w) E XW in w while the mechanism for any V E V W left unchanged. Specifically, where M is in Definition 3 the manipulated model for W w the model Mw=w = (U, V. UV Jvev, where
Sv: constant func. mapping to By rw(),
V & W V E W.
The interventional or experimental distribution pww.-w E ((v)) is just the observational distribu- tion for the manipulated model Mw and it encodes the probabilities for an experiment in which the variables W are fixed to the values w.
Remark 1. Empty interventions Ø () are just passive observations, i.e., p M PM.
2.3 Counterfactuals
By permitting multiple manipulated settings to share exogenous noise, not only the distribution arising from single manipulation. but also joint distributions over multiple can be considered. These often called counterfactuals. The set f(xAxv) encompasses the combined joint distributions over V for any combination of interventions from A. A basis for the space XAx are the cylinder sets of the following form, for some sequence (X x, Y)..... (W w, Z) of pairs, where Y. Y..... Z C V are finite, and X = x...., W = w E A are interventions: ((y)) n [W:=] x=(==))xx(())
We will abbreviate this open set yx:... Zw. writing, e.g. simply x for the intervention X = Definition 6. Given M, define a counterfactual distribution D E F(xAxv) on a basis as follows:
Pet (yx...., bw)
n... -.. (m=
Here, the letters y, ... the right-hand side abbreviate the respective cylinder sets (Definition I Ty ((x)),... Tz ([z)).
Remark 2. Marginalizing På to any single intervention W = w yields p Mw If xu is finite, we
obtain a familiar (13) sum formula p (y ..,,,..,24 2 = Σ (ulm-x-x (w)m www.yy................ (u)Ez) P(u).
Example 1. As very simple example (drawn from 28 2 D. just to illustrate the previous definitions and notation, consider a scenario with two binary exogenous variables U (U1, U2} and two binary endogenous variables V = {X, Y). Let U, U2 both be uniformly distributed, and define Xy by {u, x) ux (1 - u) x).
fx XU, + Xx to be the identity, and fy X. X XU, This fully defines an SCM M with influence X + Y. and produces an observational distribution W. such that MM (z, y) 1/1 for all four settings X x, y.
The space A of interventions in this example includes the empty intervention and all combinations of X = and Y : y, with x, 13. Notably, all interventional distributions here collapse ob- servational distributions, e.g., M-Mx-= (X,-- Y) pMM(X (X,Y), Y). for both values æ. Thus, "experimen- tal" manipulations of this system reveal little interesting causal structure. The counterfactual distribu- tion M however, does not trivialize. For instance, pe ( == =1.=1).( == 0, Y = 0)) 1/2. This term is known the probability of necessity and sufficiency 277. which we can abbreviate by M (ye, vLz)). Note that p (Yz, V * pá )p'' (w/r) = 1/1. Similarly, pá (Vas yz ) 1/2.
2.4 SCM classes
We now define several subclasses of SCMs that we will use throughout the paper. Notably, we do not require their endogenous variable sets V finite. infinite in many applications, for instance, in time series models, or generative models defined by probabilistic programs (see, e.g.,
18 36) ). Because the proofs call for slightly different methods, we deal with the infinite and finite cases separately. We make one additional assumption in the infinite case.

---

Definition 7. μ P(0) is atomless if (({t)) = 0 for each t E 0; M is atomless if M is atomless.
Intuitively, an atomless distribution is one in which weight is always "smeared" out continuously and there are no point masses; infinitely many fair coin flips, for example, generate an atomless distribution the probability of obtaining any given infinite sequence is zero.
Definition 8. For the remainder of the paper, fix a countable endogenous variable set V. Define the following classes of SCMs:
2 SCMs over V whose influence relation is extendible the w-like order x;
X = SCMs over V in which the variable X has no parents: Pa(X) = Ø;
MT all SCMs over V U U x
X
If V is infinite then all SCMs in the classes above are assumed to be atomless.
3 The Causal Hierarchy
Implicit in $2 and indeed in much of the literature on causal inference, a hierarchy of causal expressivity. Following the metaphor offered in [29]. it is natural to characterize three levels of the hierarchy as the observational, interventional (experimental), and counterfactual (explanatory). Drawing on recent work [2. 19] we make this characterization explicit. The levels will be defined in descending order of causal expressivity (the reverse of $2. Fig. aa) summarizes our definitions.
Higher levels determine lower coanlss-cuuteefactuuuss determine interventionals, and the obser- vational is just an (empty) interventional. Thus movement "downward" in the causal hierarchy corresponds to kind of projection. For indexed {Sp}peB and B' C B let <g: (( Sp)
P B Sp) be the marginalization map taking a joint distribution to its marginal on B'.
Definition 9. Define three composable causal projections {wihisis with signatures and definitions
33 M + P(XAxV), 102 : P(xxx)) + X F(v),, Wi X P(xv) + P(xv):
33 M + pa #3 + (<a)))((H)) aEA' wi (Ha)aEA + # = T==0
The causal hierarchy consists of three sets (6:lsigs defined as images or projections of M1:
G3 (2)1,, 62 22 (5a), 6 = の (G2).
These are the three Levels of the hierarchy. The definitions cohere with those of (and, e.g., 128
Fact 3. Let M E Mt. Then W13 x)(M) G3 trivially coincides with its counterfactual distribution as defined in 22. while (Maa)a #2(Us) E G2 coincides with the indexed family of all its interventional distributions ($2.)). i.e., Tw ((Ha))) p Mw for each W == E A.
Finally 1a w ((Ma)a) e G, coincides with its observational distribution 2.1
Thus, e.g., G3 is the of counterfactual distributions that consistent with at least some SCM from Dt. It is fact that G3 Ç P(xaxv) and similarly not every interventional family belongs to G2; see Appendix for explicit characterizations. At the observational level, this is simple:
Fact 4. 6 F(vv in the finite case. In the infinite case, G, {u E P(xv) l is atomless).
We will also use the subsets (6,), and {6(); which are defined analogously but via projection from _h and Mx respectively.
3.1 Problems of Causal Inference
As elucidated in 29 22, the causal hierarchy helps characterize many standard problems of causal inference, in far as these problems typically involve ascending levels of the hierarchy. Some examples include:
1. Classical identifiability: given observational data about some variables in V, estimate a causal effect of setting variables to values x (26] 35). In the notation here, given
information about M (V), can we determine p Mx= (Y)?
5

---

21
3
2
33
2 X-+Y
6 X -
(a) Causal Hierarchy
(b) Collapse Set E2
Figure I: (a) G3 can be seen coarsening of Wi, abstracting from irrelevant "intensional" details. 62 is obtained from G3 by marginalization (also coarsening). while 6 is projection of 6₂ via the "empty" intervention. Each map w., i 1, 2, 3, is continuous for the respective weak topologies (Prop. 4.. The projections w X- from G2 to the 2VE-spaces are likewise continuous (Prop. 44. (b) The shaded region, E2 c 22, is the collapse set in which Level 2 facts determine all Level facts: those points in 62 whose -y-preimage in G3 is singleton set. The main result of this paper is that C₂ is meager in weak topology on 6₂ (Thm. B). This means C₂ contains no open subset, which by Thm. implies no part of Eg is statistically verifiable, even with infinitely many ideal experiments.
2. General identifiability: given mix of observational data and limited experimental data- that is, information about yr MMM( (V) well as some experimental distributions of the form
p Mw (V) determinn p Mx (Y) [8 22]
3. Structure learning: given observational data, and perhaps experimental data, infer properties
of the underlying causal influence relation >> 35 30
4. Counterfactual estimation: given combination of observational and experimental data, infer a counterfactual quantity, such as probability of necessity 31, or probability of necessity and sufficiency (27 37 (see also bellow....
5. Global identifiability; given observational data drawn from y (V) infer the full counterfac- tual distribution Par (A V) 34 10].
This is not an exhaustive list, and these problems are not all independent of one another. They are also all unsolvable in general. Problems 1, 2, and 3 involve ascending Level given information at Level (and perhaps partial information at Level 2); problems 4 and 5 ask us to ascend to Level 3 given only Level 1 (and perhaps also Level information. The upshot of the causal hierarchy theorem from [2) is that these steps are impossible without assumptions, formalizing the common wisdom, "no causes in, no causes out" 77. To understand the statement of the causal hierarchy theorem-and our topological version of it it-we explain what it means for the hierarchy to collapse.
3.2 Collapse of the Hierarchy
In the present setting a collapse of the hierarchy can be understood in terms of injectivity of the functions w,. For = 1,2 let C, C G, the injective fibers of w4 i.e., C, = {14 E G, 14++1 = U4+11 whenever w, w. (14+1) of the hierarchy: knowing a would be sufficient to determine the Level i + facts completely.
(1((4)))= J. Every element
C, is witness (global) collapse
A first observation is that 1 is never injective. In other words, the distribution p. never determines all the interventional distributions M> (Y). This essentially a way of stating that correlation never implies causation absent assumptions. (See also [2] Thm. 1].)
Proposition 2. e
That is, Level never collapses to Level without assumptions.
To overcome this formidable inferential barrier, researchers often assume we are not working in the "full" space Di of all causal models, but rather some proper subset embodying range of causal assumptions. This may effectively eliminate countercxamples to collapse (cf. Fig. [b)). For problems of type 1 or 2 (from the list above in $3.1 it is common to assume are only dealing with models whose graph (direct influence relation) + satisfies a fixed set of properties. For problems of type 3 it is common to assume that P.M and - relate in some way (for instance, through an assumption like faithfulness or minimality [35). All of these problems become solvable with sufficiently strong assumptions about the form of the functions (jvlv the probability space P.
In some cases, the relevant causal assumptions are justified by appeal to background or expert knowledge. In other cases, however, assumption will be justified by the fact that it rules out only

---

a "small" or "negligible" or "measure zero" part of the full set 2I of possibilities. As emphasized by number of authors [2 39 23]. not all "small" subsets are the same, and it seems reasonable to demand further justification for eliminating one over another. We believe that the framework presented here can contribute to this positive project, but our immediate interest is in solidifying and clarifying limitative results about what cannot be done.
The issue of collapse becomes especially delicate when we turn Cg. When do interventional distributions fully determine counterfactual distributions? In contrast to Prop. 2 we have: Proposition E2 F 2. That is, there exists an SCM in which Level 3 collapses Level 2.
Proof sketch. As very simple example in the finite case, any fully deterministic SCM will result in collapse. This is because, if (No)ne A are all binary-valued then the measure μ4g E P X A Xv) that produces the marginals Pa completely determined: each Pa specifies an element of Xv. so μa must assign unit probability to the tuple that matches Pa at the projection. In the infinite case, any example must be non-deterministic by atomlessness, but collapse still possible; see Example 2 Appendix B
3.3 Probabilities of Causation
handful of counterfactual quantitites larly prominent in the literature (c.g.. (27)). Our main result will show that any of these six quantities (for any two fixed variables) is robust against collapse. Below, fix two distinct variables Y + X V and distinct values x + x' E Xx, y # y' E XY.
given variables, collected below, have been partieu-
Definition 10.
The probabilities of causation are the following quantities:
P(y., v/z.) probability of necessity and sufficiency
P( Uer) converse prob. of necessity and sufficiency
P( v. prob. of necessity
P(yx x/') prob. of sufficiency
P(y' v) : prob. of disablement
P(yz v/: : prob. of enablement
Consider, for example, the probability of necessity and sufficiency (PNS), which is the joint probability that Y would take on value y if X is set by intervention æ. and y/ if X is set to x'. PNS has been thoroughly studied online advertising, we would like to assess which interventions are likely be both necessary and sufficient for a given outcome. Using the notation from $2.3 PNS concerns the measure of sets Uz. V/2/ = xx ((u)) n Txx=",y>
[I) in part due to its widespread relevance: from medical treatment to
The probabilities of causation are paradigmatically Level 3, and we will be interested in their manifestations Level 2. In that direction introduce small part of just enough witness the behavior of Y (and X) under the empty intervention and the two possible interventions on X:
0, = 1}. Define a small subspace 6 e C XaEAx F(x(x,y) the image of the map 0 X-Y (s(x.Y) S(x.Y) <(x,))) TT (see Fig. [a)). Call 6 a two-variable effect (2VE) space; fixing X, we have a 2VE-space for each Y.
Definition 11. Let Ax (Ø
()
It is known in the literature that the probabilities of causation are identifiable from the data pl X, Y), p(Y)), and p( (Yzz) (see, e.g.. [1 for PNS). As part of our proof of Theorem (eeeoremmbelow we will strengthen this considerably to show them all to be generically unidentifiable, in topological sense to be made precise.
4 The Weak Topology
We now demonstrate how G1, G2 and G3 can be topologized. In general, given a space & and the set G = (i)) of Borel probability measures on i, natural topology on G can be defined as follows:
Definition 12. For a sequence (un), )n of measures 6, write (μa)). = lr and say converges weakly 4. p. 7] to if Sg ʃ dun S du for all bounded, continuous f - R. Then the weak topology " on is that with the following closed sets: E. C G is closed in , iff for any weakly convergent sequence (pn)n = p in which every pn E E, the limit point μ is in E.
7

---

There are several alternative characterizations of r which hold under very general conditions. For instance, it coincides with the topology induced by the so called Léxy-Prohorov metric [.. The most
useful characterization for our purposes is that it can be generated by subhasic open sets of the form
{u ((X) > r
(I)
with X ranging over basic clopens in 2 and r over rationals (see, e.g., [T6 Lemma A.5]).
Conceptually, the explication of r" in terms of weak convergence strongly suggests connection with statistical learning. We now make this connection precise, building on existing work g 14 16.
4.1 Connection to Learning Theory
Roughly speaking, we will say a hypothesis H C 6 is statistically verifiable if there is some error bound and sequence of statistical tests that converge generated from H. More formally, a test is a function À : f - {accept, reject), where d" is the n-fold product of 0. viz. finite data streams from 9. The interest in whether a "null" hypothesis can be rejected given data observed thus far. The boundary of a set A C 1. written bd is the difference of its closure and its interior. Intuitively, a learner will not be able to decide whether to accept or reject on the boundary. Consequently it is assumed that À is feasible in the sense that the boundary of its acceptance zone (in the product topology on 8²) always has measure 0, i.e., μ² [bd(A (accept)) = O for every G, where μ² is the n-fold product measure of
H with error at most e, when data
Say hypothesis H C G is verifiable [14 if there is e and sequence (An) of feasible tests (of the complement of H in 6, i.e., the "null hypothesis") such that
1. μ² [", (reject)] S e for all n, whenever f $ H; 2. lim μ² ( (reject) 1, whenever μ E H.
n-000
That is, to be verifiable we only require sequence of tests that converges in probability to the true hypothesis the limit of infinite data (requirement 2), while incurring (type 1) error only up to a given bound at finite stages (requirement 1). As an illustrative example, conditional dependence is verifiable 14]. This a relatively lax notion of verifiability. For instance, the hypothesis need not also be refutable (and thus "decidable"). For purposes this generality is virtue: want to show that certain hypotheses are not statistically verifiable by any method, even in this wide sense. The fundamental link between verifiability and the weak topology is the following, due to [14 [6]:
Theorem 1. A set H C G is verifiable if and only if it is open in the weak topology.
4.2 Topologizing Causal Models
We now reinterpret 7 each level of the causal hierarchy:
Definition 13. The weak causal topology T²", 1 S i 3, is the subspace topology on G; induced by
if = 3: 7 on P(XAxV); if 2: product of on X p(xv); if = 1: 1: on P(xv).
aeA
Proposition 4. All projections { w X-+V are continuous in the weak causal topologies.
significant observation is that the learning theoretic interpretation, originally intended for T naturally extends to r²" While data streams at Level 1 amount to passive observations of V, data streams at Level 2 can be seen as sequences of experimental results, i.c., observations of "potential outcomes" Yx. To make verifiability
easy possible we assume learner can observe sample from all conceivable experiments at each step. A learner is thus a function À : - + (accept, reject], where - = ((xv)") is the set of potential experimental observations over trials (with indexing the experiments). Construing 8" as a product space we can again speak of feasibility of X.
Recall that elements of G2 are tuples (Ha)aEa of measures. Say a hypothesis H C 6g is experimen- tally verifiable there e > and a sequence (An)neN of feasible tests such that and above hold, replacing μz" (A, (reject)) with II. (l.."(( (rejet))))) That is, when experimental data are drawn from the interventional distributions (Ma) H, we require that the learner eventually converge on H with bounded error at finite stages. We can then show (sce Appendix C):
8

---

Theorem 2. A set H S G2 is experimentally verifiable if and only if it is open in -2"
similar result can be given for (Ga, T3 )a although is less clear what the empirical content of this result would be. Note also that 7f" '2 T3 give sequence of increasingly fine topologies on the set of actual SCMs 21 by simply pulling back the projections. The point is that Tg is the finest that has clear empirical significance, while r3 is the finest in terms of relevance to the causal hierarchy.
5 Collapse is Meager
Recall that a set X C 1 is nowhere dense every open set contains an open Y with Xny Ø. A countable union of nowhere dense sets is said to be meager (or of first category). The complement of a meager set is comeager. Intuitively, a meager set is one that can be "approximated" by sets "perforated with holes" [25]. Meagerness notably preserved under continuous preimages.
As discussed above, one intuition highlighted by the weak topology TM is that open sets are the
kinds of probabilistic propositions that could, in the limit of infinite data, be verified (Thms. 2 Correlatively, meager sets in r* are so negligible as to be unverifiable: as a meager set contains no non-empty open subsets (by the Baire Category Theorem (25 ), it is statistically unverifiable. We will
now show that the injective collapse set C2 from $32 is topologically meager.
The crux is to identify "good" comeager 2VE-subspace where collapse never occurs (with separation witnessed by probabilities of causation). In this subspace, the constraints circumseribing Level have sufficient slack to make tweak without thereby disturbing Level 2 (cf. Figure 2). We define the good set as the locus of a set of strict inequalities:
Definition 14. A family (Ja)DEAX members of Ax as (), (,2,0 2,,22 x';
GG is Y-good if we have the following, abbreviating the
ur (v') - (x, < Mo(x'),
(2)
m0 (I', < #o(x))
Lemma 1. The subspace of Y -ooo families is comeager in 6
Proof sketch. The non-strict versions of 2 3 hold universally, so the complement of the good set
is defined by equalities. This is closed and contains no nonempty open by the weak subbasis (T.. Li
Figure presents the construction in a small, two-variable case, and Lemma 2 below proven by generalizing it to arbitrary V. Guaranteeing agreement every interventional distribution in the
general case subtle (Appendix DD: has been observed that enlarging V can enable additional inferences (e.g.. (32)), though the next result reflects dependence on further assumptions.
G is such that w -Y ((Ha)a) Yggood, and let be PNS, the converse PNS, the probability of sufficiency, or the probability of enablement (Definition 1Op. Then for any (43 E Gy such that w2(pa) = (Pa)
Lemma 2. Suppose < is an order in which X comes first and (Ha)aEA
there exists nág G, such that μ43 and plz disagree on yp.
Note that by reversing the roles of x and x', we may obtain the same for the probability of necessity and probability of disablement. The main theorem and its important maaming--hheorede corollary are now straightforward
Theorem (Topological Hierarchy). The set C2 of points where all Level 3 facts are identifiable from Level is meager in (G2, ). The preimage w (C2) = E3 is likewise meager in (5,, T3
X X-+ of the set of Y Yggood tuples in 6 2 implies that C2 n G.X is contained in G Dxx X,Y for any Y + X. Meanwhile, since 22 is continuous, Lemma lipliis that 6: D
Proof. Let D X.Y
C 6 be the preimage under wa
Lemma
meager in G and thereby also in 62. Thus C2 Uxev C2 n G is countable union of meager sets, and hence meager.
Corollary 1. No causal hypothesis licensing arbitrary counterfactual inferences (and specifically those of the probabilities of causation) from observational and experimental data is itself statistically (even experimentally) verifiable.

---

M
M
P(u)
Xa
Yz,,
Varaa
P(u)
X
Yz,
Ye
1/2 -
8
Ug
1/2
x
y
M1
x
y
V/
1/2
x
V/
U2
1/2
y
3/
(a) Y -good Model
u3 8
y
(h) Example Separating Levels and
Figure 2: (a): the structural functions and exogenous noise for model M with direct influence X +++ Y. This M meets ② and (3), so we may apply Lemma constructing the model M' in (b), y where 0 < 1/2. Note that Per (yx, v'z'' = 0 while Fá (ym,v'z) e, that the two models disagree on Level PNS quantity; on other hand, it is easy to check agreement on all of Level Similarly, M and M' disagree on the converse PNS, probability of sufficiency, and probability of enablement (Definition]
6 Conclusion
We introduced a general framework for topologizing spaces of causal models, including the space of all (discrete, well-founded) causal models. As an illustration of the framework characterized levels of the causal hierarchy topologically, and proved topological version of the causal hierarchy theorem from (2) While the latter shows that collapse of the hierarchy (specifically of Level 3 to Level 2) is exceedingly unlikely in the sense of (Lebesgue) measure, we offer a complementary result: any condition guaranteeing that could infer arbitrary Level 3 information from purely Level information must be statistically unverifiable, even by experimental means. Both results capture an important sense in which collapse is "negligible" in the space of all possible models. As an added benefit, the topological approach extends seamlessly to the setting of infinitely many variables.
There many natural extensions of these results. For instance, have begun work version for continuous endogenous variables. Also of interest are subspaces embodying familiar causal assumptions or other well-studied coarsenings of SCMs (see, c.g., [23 Bayesian networks, or on linear non-Gan GauSsassii models), which often render important inference problems solvable In the opposite direction, theorems to hold for extensions the SCM concept, e.g., that dropping the well-foundedness or recursiveness requirements [6]. As emphasized by (2). causal hierarchy theorem should not be construed as a purely limitative result, but rather as further motivation for understanding the whole range of causal-inductive assumptions, how they relate, and what they afford. We submit that the topological constructions presented here can help clarify and systematize this broader landscape.
17 15
though sometimes only "generically"
expect analogous hierarchy
Acknowledgments
This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. D6E-1 We are very grateful to the five anonymous NeurlPS reviewers for insightful and detailed comments and questions that led to significant improve- ments in the paper. We would also like thank Jimmy Koppel, Krzysztof Mierzewski, Francesca Zaffora Blando, and especially Kasey Genin for helpful feedback on earlier versions.
References
C. Avin, I. Shpitser, and J. Pearl. Identifiability of path-specific effects. Proceedings of IJCAI, 2005.
[2] E. Bareinboim. J. Correa, D. lbeling. and T. Ieard. Pearl's hierarchy and the foundations of causal inference. Technical Report R-60, Causal AI Lab, Columbia University. 2020.
[3] G. Belot. Absolutely no free lunches! Theoretical Computer Science, 945:159-180, 2020.
[4] P. Billingsley. Convergence of Probability Measures. Wiley, 2nd edition, 1999.
V. I. Bogachev, Measure Theory. Springer Berlin Heidelberg, 2007.
10

---

[6] S. Bongers, P. Forré, J. Peters, and J. M. Mooij. Foundations of structural causal models with cycles and latent variables, 2021.
[7] N. Cartwright. Nature's 's Capacities and their Measurements. Clarendon Press, 1989.
[8] A. Dembo and Peres. A topological criterion for hypothesis testing. Amnll of Statistics, 22(1):106-111 117,
1994.
[9] P. Diaconis and Freedman. On consistency of Bayes estimates. Annals of Staristics, 14(1):1-26, 1986.
[10] Drton, R. Foygel, Sullivant. Global identifiability linear structural equation models. Annals of Statistics, 39(2):865-88) 886, 2011.
[11] F. Eberhardt, Glymour, and Scheines. On the number of experiments sufficient and the worst case necessary to identify all causal relations among n variables. In Proceedings of UAI, page 178-184, 184, 2005,
[12] D. Freedman. From association causation via regression. Advances Applied Mathematics, 18:59-110, 1997.
[13] D. Galles and J. Pearl. An axiomatic characterization of causal counterfactuals. Foundations of Science, (1):151 182, 1998.
[14] K. Genin. The Topology of Statistical Inquiry. PhD thesis, Cameeiic Mellon University, 2018.
[15] K. Genin. Statistical undecidability in linear, non-gaussian models in the presence of latent confounders. In Proceedings of NeurlPS, 2021.
[16] K. Genin and K. Kelly. The topology of statistical verifiability. In Proceedings of TARK,, pages 236-250, 2017.
[17] K. Genin and C. Mayo-Wilson, Wilson. Statistical decidability in linear, non- Gaussian models. Causal Discovery & Causality-Inspired Machine Learning, NeurIPS, 2020.
[18] D. lbeling
Ieard. On open-universe causal reasoning. Proceedings of UAI, 2019.
[19] D. Ibeling and T. Ieard. Probabilistic reasoning across the causal hierarchy. In Proceedings of AAAI, 2020.
[20] G. W. Imbens and D. B. Rubin. Causal Inference in Statistics, Social, and Biomedical Sciences. Cambridge University Press, 2015.
[21] J. L. Kelley. General Topology. Springer, 1975.
[22] S. Lee, J. Correa, and E. Bareinboim. General identifiability with arbitrary surrogate experiments. In Proceedings of UAI, 2019.
[23] and J. Zhang. learning causal structures from expeeemerieettll data without any faithfulness assumption. In Proceedings of ALT, pages 554-582, 2020.
[24] C. Meek. Strong completeness and faithfulness Bayesian networks. In Proceedings of UAI, page 411-418, 1995.
[25] C. Oxtoby. Measure and Category. Springer, 1971.
[26] J. Pearl. Causal diagrams for empirical research. Biometrika, 82(4):669-710, 1995.
[27] J. Pearl. Probabilities of causation: Three counterfactual interpretations and their identification. Synthese, 121(1):93-144 149, 1999.
[28] Pearl. Causality. Cambridge University Press, 2009.
[29] Pearl and Mackenzie. The book of why: The new science of cause and effect. Basic Books, 2018.
[30] J. Peters, D. Janzing, and B. Schölkopf. Elements of Causal Inference: Foundations and Learning Algorithms. MIT Press, 2017.
[31] J. Robins and S. Greenland. The probability of causation under stochastic model for individual risk. Biometricx, 45(4): 125--118., 1989.
[32] B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke. N. Kalchbrenner, A. Goyal, and Y. Bengio. Toward causal representation learning. Proceedings the IEEE, 109(5):612-66 634, 2021.

---

[33] S. Shalev-Shwartz and S. Ben-David. Understanding Machine Leurning: From Theory to Algorithms. Cambridge University Press, 2014.
[34] S. Shimizu, P. O. Hoyer, A. Hyvärinen, and A. Kerminen, A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7(72):2003-200 2030, 2006.
[35] P. Spirtes, C. Glymour, and R. Scheines. Causation, Prediction, and Search. MIT Press, 2001.
[36] 7. Tavares, J. Koppel, X. Zhang, R. Das, and A. S. Lezama. A language for counterfactual generative models, In Proceedings of ICMI,, 2021.
[37] J. Tian and Pearl. Probabilities of causation: Bounds and identification. Annals of Mathematics and Artificial Intelligence, 28(1) 28(1):287331, 2000.
[38] J. Tian and J. Pearl. A general identification condition for causal effects. In Proceedings of AAAI. 2002.
[39] C. Uhler, G. Raskutti, P. Bühlmann, and B. Yu. Geometry of the faithfulness assumption in causal inference. Annals of Statistics, 4162)4336466, 2013.
12


Reviews:
[{"review_id": "nHMUYwpqF1B", "reviewer": "Reviewer_V1rj", "summary": "This paper studies the weak topology on three spaces of probability measures, each corresponding to a level of a causal hierarchy introduced by Pearl (Pearl and Mackenzie, 2018).  The three levels, in order of increasing expressivity, are described as observational, interventional and counterfactual; given a set $\\bf{V}$ of variables (known as \"endogenous\" variables; these variables can influence each other and may also be influenced by a set of \"exogenous\" variables), the observational distribution is a joint probability distribution over the values of $\\bf{V}$ fixed according to some given structural causal model, an interventional distribution on $\\bf{V}$ is a distribution where the values of some finite subset of $\\bf{V}$ are fixed and a counterfactual distribution is one where various combinations of interventions with respect to different finite subsets of $\\bf{V}$ are permitted.  The main results are: (1) a topological characterization of the set of hypotheses in the second level of the causal hierarchy that are \"experimentally verifiable\", which means that there is a sequence of tests that almost surely converge in the limit on the true hypothesis while a type I error is incurred up to a fixed bound at each stage (the relevant distribution belonging to the second level of the hierarchy); (2) the set C_2 of all points in the second level of the causal hierarchy that have a unique inverse (with respect to a projection from the third level) forms a meager set, where the underlying topology is the weak topology; a similar result applies to the preimage of C_2 under projection from the third level of the hierarchy.   ", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I think this is a well-written paper (in terms of language and style) that provides an interesting and new perspective of causation.  Mathematically, the paper seems generally correct, although the notation at some places was not clear to me.  I suggest checking the consistency and accuracy of the notation more carefully, particularly in definitions where nested sets are involved; while this is not a major point, I think it affects readability (more details below).  I am not familiar with the literature on causation, so I am unable to independently judge the overall novelty of the paper's contributions.  Based on what is written in the paper, it seems that the main innovation is to apply Genin and Kelly's (2017) topological approach to statistical verifiability to study causal models, in particular the three levels of the causal hierarchy proposed by Pearl. \n\n\nComments/Suggestions/Questions:\n\n- Page 4, Definition 6: On the right-hand side of the equation, the meanings of the expressions (m^{M_{X:=x}})^{-1}(y)...,(m^{M_{W:=w}})^{-1}(z) were not clear to me.  (I am typing in plain text due to problems with the tex code displaying properly.)  According to the definition of (m^{M_{X:=x}})^{-1}, each input to (m^{M_{X:=x}})^{-1} should be a value in $\\chi_{\\bf{V}}$.  However, it seems from the definition of $\\bf{y}$ between lines 147 and 148 that $\\bf{y}$ is a value in $\\chi_{(\\bf{X}:=\\bf{x}) \\times Y}$.  But then why are the sets $\\chi_{\\bf{V}}$ and $\\chi_{(\\bf{X}:=\\bf{x}) \\times Y}$ comparable? I think some clarification would be good here.          \n\n- Page 5, definition of $\\varpi_2$: The use of the \"marginalization\" operator $\\varsigma$ here was not clear to me.  According to the definition at the end of page 4, $\\varsigma$ is taken with respect to a _subset_ of the indexed variables; so if $A$ is a single variable, then should $\\varsigma$ not be taken with respect to a subset of {$A$} $\\cup \\bf{V}$?  As I understand, {$\\alpha$} $\\times \\bf{V}$ is not a subset of {$A$} $\\cup \\bf{V}$.      \n\n- Page 5, line of equations just before line 176: Why is $\\varpi(\\mathcal{M})$ not equal to $p^{\\mathcal{M}}_{cf}$ (counterfactual probability)?\n\n- Page 5, Definition 9: In the definition $\\varpi(\\mu) = \\pi_{\\emptyset}(\\mu)$, it appears that $\\mu$ is used to denote an indexed family of probability measures.  I suggest using a different notation (e.g. $(\\mu^{(\\alpha)})_{\\alpha \\in A}$) for a family of measures while reserving $\\mu$ for a single measure. \n\n- Page 9, line 371: lanscape -> landscape\n\n- Supplementary, page 1, line 5: in the left-hand side of the equation, should one take the projection onto $Pa(V)$ of $m^{\\mathcal{M}}(\\mathbf{u})$? \n\n- Supplementary, page 2, line 36: In the definition of $\\xi(\\bf{B})$, should $V \\in \\bf{V}$ be $V \\in \\bf{B}$ instead?  Otherwise, how does the definition depend on $\\bf{B}$?  Similarly, in line 37, should the subscript be $(V,\\bf{p}) \\in \\xi(\\bf{B})$? \n\n- Supplementary, page 2, Equation (B.2): On the left-hand side, should one take the inverse projections $\\pi^{-1}$ instead?\n\n- Supplementary, page 2, Definition B.1.1: The definition of an SCM here doesn't seem consistent with that of Definition 3 in the main paper.  Specifically, in the original definition of a mechanism f_V, f_V maps the values of a set of parents together with a set of values of U(V) to a value of V.  In Definition B.1.1, however, the second argument of f_V is {f_V}_V, a collection of deterministic functions, but {f_V}_V is not a member of $\\bf{U} := \\mathcal{B}(F)$, which is a collection of sets of collections of deterministic functions.  Should the second argument be written instead as {{$f_V$}$_V$}, i.e. the singleton whose member is {$f_V$}$_V$?      \n\n- Supplementary, page 3, Lemma B.1.2: I could not find the definition of an acausal model in the paper.\n\n- Supplementary, page 3, line 74: I did not see in the proof of Proposition 2 where $\\epsilon$ was used at all.  \n\n- Supplementary, page 3, last sentence of proof of Proposition 2: Did you mean that \"$\\mu' \\notin C_1$ for every $\\mu' \\in G_1$ since $\\mu$ was arbitrary\"?   ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "None.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "18"}, "extra_text": {}}, {"review_id": "l_TYXdhkVnw", "reviewer": "Reviewer_tnjA", "summary": "The paper introduces a topological learning-theoretic version of the causal hierarchy theorem (Bareinboim et al. 2020). \nThis result shows that associational, causal, and counterfactual quantities are not only distinct in a measure-theoretic sense but also in a topological sense. This, in turn, implies that causal assumptions allowing the inference of arbitrary counterfactual quantities from experimental data would be statistically unverifiable.\nThis result reinforces the notion that causal assumptions are needed to perform causal and counterfactual inference, as they cannot be accomplished solely from data. Such formal limit becomes even more important when considering the strong inclination of ML research towards finding methods that are successful at modeling the observed data but are blind to causal constraints not present in the data.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This work gives a new and complementary perspective to recent work in formalizing the notion of the Ladder of Causation or the Pearl's Causal Hierarchy (PCH). The idea is to formally show the intuition that causal and counterfactual inference is impossible without causal assumptions given as part of the analysis. While this paper discusses the Causal Hierarchy Theorem (CHT) proposed by Bareinboim et al., it provides an alternative, novel, and complementary result. Moreover, the authors extend the result to models with an infinite number of discrete observable variables.\n\nBased on limited knowledge of topologies and learning theory, the definitions and results presented in the paper look sound and make intuitive sense. Furthermore, the implications and limitations of the results have been discussed thoroughly.\n\nI find the paper is very well written and approachable for researchers in the causal inference and machine learning community. I found the reading enjoyable and easy to make sense of as it gently introduces the mathematical concepts required to understand the result.\n\nThe results in the paper seem informative and relevant for any research that has the goal of inferring counterfactual and causal effects from data. Although the result imposes limits to counterfactual inference, it motivates the study of the assumptions needed to perform such a task and for building intelligent systems capable of causal and counterfactual reasoning.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations: I believe the scope and limitations of the result have been clearly stated in the paper.\nSocietal impacts: None that I can immediately foresee for the result.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "jEAM2K6GMBU", "reviewer": "Reviewer_rxqC", "summary": "This paper proposes a series of topologies on the different spaces of probability spaces of structural causal model. The authors define certain causal projections on these spaces that should map higher levels of the causal hierarchy to lower levels. The non-injectivity of these projection maps is shown and illustrates, what is called, the collapse of the hierarchy. The authors show that for those probability spaces that collapse are topologically meager. Using the result that open sets in the weak topology corresponds to statistical hypothesis that can be verified, the authors show that the collapse can never be statistically verified, because of topological meagerness.\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper takes a very interesting and unique approach by connecting the world of topology of statistical verifiability to that of structural causal modeling. They show the interesting result, that any condition that could infer counterfactual distributions from interventional distributions must be statistically unverifiable, however, I found the paper very hard to read and I was unable to verify many of the claims.\n\nFirst of all, the authors use a lot of non-standard and imprecise notation and terminology. For example, why not use the more common terminology, such as a graph, instead of direct influence relation? The authors state that their two assumptions of well-foundedness and local finiteness generalize the recursiveness assumption, however, they never define what well-foundedness is. This raise questions downstream such as what the intuition about $\\omega$-like is. The non-standard notation of using projection mappings and cylinder subsets makes the paper quite difficult to follow. This led to much more confusion, later on. For example, in understanding the causal projection and how this gives rise to Fact. 3. By coincidence, while checking the Supplementary Material, I found that Definition 6 is not at all so trivial to understand and the authors had to derive that it was a  well-defined definition, without mentioning this, or referring to it, in the main paper.\n\nSecondly, many things are not motivated or explained enough, for example what is the intuition of Definition 8? Why do you only consider those SCMs? Or, how does the result on meagerness relate to Meek's strong completeness result?\n\nThirdly, the authors seem not to be aware of related literature and many citations are missing IMO. For example, most of the results about SCMs and the causal hierarchy are already proven in a more general measure-theoretic setting, where (1) no countable range of the variables and (2) no recursiveness is assumed (see e.g., [1]). The authors do not motivate why they choose a more restricted setting, where only endogenous variables are taken with a countable range or they only consider recursiveness. This makes it difficult to distinguish what is already known (such as Proposition 2 and 3) and what the particular contributions of this paper are.\n\nAt last, I think the overall readability can be clarified by being more specific, and avoid sentences like '... it is sensible to aim for results with \"nearly universal\" force' or 'Common wisdom ... ' [without adding a reference]. I think in general, the results of this paper are very interesting and valuable, but I think that the clarity of writing can be improved a lot.  Given the current state of the paper, it does not provide a clear enough exposition of the results for NeurIPS.\n\n[1] S. Bongers, P. Forré, J. Peters, and J. M. Mooij. Foundations of structural causal models with cycles\nand latent variables. arXiv.org preprint, arXiv:1611.06221v5 [stat.ME], 2021. URL https://arxiv.org/abs/1611.06221\n\nUpdate: I've updated my score.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I think that this paper has the potential for a strong impact due to the novel connection between statistical verifiability and SCMs. I strongly recommend to elaborate more on the intuition of the crucial concepts, which I think will increase the impact of this paper even more.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7h"}, "extra_text": {}}, {"review_id": "MHhaVgpz_-x", "reviewer": "Reviewer_3EhT", "summary": "This paper presents a topological framework to theorize about causal inference, and defines a weak topology on each of three spaces of probability measures, corresponding to the three levels of Pearl's causal hierarchy. In this setup a topological version of the causal hierarchy theorem is derived, to the effect that collapse of the hierarchy is topologically negligible. An interesting learning theoretic interpretation of this result is also given, in the spirit of no-free-lunch theorems.       ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is an interesting and solid theoretical paper, providing a general framework for examining causal inference from a topological perspective, as well as establishing a topological version of a fundamental result about the limit of causal inference. Among papers of comparable technicality, this one is fairly readable and often illuminating. I like the paper, though it is hard to predict how much attention it will draw in the field of causality, let alone the NeurIPS community in general. I won't be surprised if the paper ends up with limited impact; we have to wait to see whether this perspective will yield more positive results. Still, I think the current paper is a good one for NeurIPS to consider. \n\nRegarding the main result of the current paper, I think the bit in Corollary 1 about PNS in particular is more interesting than Theorem 3. Collapse is rarely called for in practice anyway, as we typically are concerned with specific causal questions such as PNS rather than with all possible causal questions. It will be interesting to know how typical it is that a specific counterfactual query is like the PNS query. I suspect that it is very typical, but I wonder whether the authors have a more definite answer or insight to add on this matter.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "CzoedWWwlm", "reviewer": "Reviewer_Ny4n", "summary": "The authors propose an alternative analysis of how causal inference at different levels (observational, interventional, counterfactual) can interrelate by proposing a topology on the space of structural causal models. The authors' results confirm and at times extend existing results in the literature from a different theoretical perspective.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "In spite of the Increasing interest in causality in machine learning, I believe that an increase in the theoretical and philosophical understanding of the concept is still called for, preferably from different mathematical perspectives.\n\nTherefore, I think the authors' contribution is timely, and would be of interest to the conference audience both in terms of its immediate findings and its potential downstream implications in theory and practice. However I also believe that the paper requires more thorough discussion of the potential marginal benefits of such an approach.\n\nI think that the paper is well-written and does a fairly good job of introducing the concepts it uses in making its theoretical points. However, this is not to say that it is an easy read: after spending effort to penetrate the authors' terminology and theoretical results, one cannot help but expect to see more discussion regarding why adopting such a perspective useful for improving our understanding of causality. Such a discussion could include which theoretical commitments historically and/or currently limited developments in causality, and how the current perspective could help resolve these issues (an obvious historical example would be Pearl's justification of the do-calculus). Discussing how such theoretical commitments shaped practical research, and what new questions would their framework facilitate would also be very beneficial. They could also discuss the significance of their current findings that result from their theoretical framework. Also, weak points of a topological perspective would be a good point of discussion: what currently existing results cannot be replicated by the authors' framework?\n\nThe authors should not be expected to provide a complete account for all of the above, but in its current form the paper's findings feel like a complete theoretical overhaul only to confirm and modestly extend previous results. Although this is not unworthy of attention by itself, I think authors' work deserves a more thorough discussion in comparison to previous theoretical and empirical work in the field.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "For limitations see my comments above. I think the authors' topic does not acutely require a discussion of its societal impact, at least in its current incarnation.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "8"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel topological approach to the causal hierarchy. All of the reviewers found the approach interesting, although some valid concerns were raised regarding clarity and presentation. In particular, a serious concern regarding comparisons with related work was raised and sorted out during the discussion phase. 

In the end, there was a consensus recommendation to accept this paper which I concur with. We expect the authors will take the reviewer feedback into account, and in particular add a reference and detailed comparison with the paper [1].

[1] S. Bongers, P. Forré, J. Peters, and J. M. Mooij. Foundations of structural causal models with cycles and latent variables. arXiv.org preprint, arXiv:1611.06221v5 [stat.ME], 2021. URL https://arxiv.org/abs/1611.06221
Decision: Accept (Poster)