SlowGuess/NeurIPS_2023_OCR huggingface dataset:
First, we let the variables  $X_{j}, j = 1,\ldots,d$  take values in measurable spaces  $(E_{j},\mathcal{E}_{j})$  respectively, and let  $(\Omega ,\mathcal{H}) = \otimes_{j = 1}^{d}(E_{j},\mathcal{E}_{j})$ . An SCM  $\mathfrak{C}$  entails a unique distribution  $\mathbb{P}$  over the variables  $\mathbf{X} = (X_1,\dots,X_d)$  by the propagation of the noise distribution  $\tilde{\mathbb{P}}$  through the structural equations  $f_{j}$  [46, p.84, Proposition 6.3], and we take this  $\mathbb{P}$  as the observational measure of the causal space. More precisely, assuming  $\{1,\dots,d\}$  is a topological ordering, we have, for  $A_{j}\in \mathcal{E}_{j},j = 1,\dots,d$
$$
\mathbb {P} \left(A _ {1} \times E _ {2} \times \dots \times E _ {d}\right) = \tilde {\mathbb {P}} \left(\left\{n _ {1}: f _ {1} \left(n _ {1}\right) \in A _ {1} \right\}\right)
$$
$$
\mathbb {P} \left(A _ {1} \times A _ {2} \times E _ {3} \times \dots \times E _ {d}\right) = \tilde {\mathbb {P}} \left(\left\{\left(n _ {1}, n _ {2}\right): \left(f _ {1} \left(n _ {1}\right), f _ {2} \left(f _ {1} \left(n _ {1}\right), n _ {2}\right)\right) \in A _ {1} \times A _ {2} \right\}\right)
$$
$$
\begin{array}{c} \bullet \\ \vdots \\ \bullet \end{array}
$$
$$
\mathbb {P} (A _ {1} \times \ldots \times A _ {d}) = \tilde {\mathbb {P}} (\{(n _ {1}, \ldots , n _ {d}): (f _ {1} (n _ {1}), \ldots , f _ {d} (f _ {1} (n _ {1}), \ldots , n _ {d})) \in A _ {1} \times \ldots \times A _ {d} \}).
$$
Finally, for each  $S \in \mathcal{P}(\{1,\dots,d\})$  and for each  $\omega \in \Omega$ , define  $f_{j}^{S,\omega} = f_{j}$  if  $j \notin S$  and  $f_{j}^{S,\omega} = \omega_{j}$  if  $j \in S$ . Then we have
$$
K _ {S} (\omega , A _ {1} \times \ldots \times A _ {d}) = \tilde {\mathbb {P}} (\{(n _ {1}, \ldots , n _ {d}): (f _ {1} ^ {S, \omega} (n _ {1}), \ldots , f _ {d} ^ {S, \omega} (f _ {1} ^ {S, \omega} (n _ {1}), \ldots , n _ {d})) \in A _ {1} \times \ldots \times A _ {d} \}).
$$
This uniquely specifies the causal space  $(\Omega, \mathcal{H}, \mathbb{P}, \mathbb{K})$  that corresponds to the SCM  $\mathfrak{C}$ . While this shows that causal spaces strictly generalise (interventional aspects of) SCMs, there are fundamental philosophical differences between the two approaches, as highlighted in the following remark.
Remark 3.2. (i) The "system" in an SCM can be viewed as the collection of all variables  $X_{1}, \ldots, X_{d}$ , and the "subsystems" the individual variables or the groups of variables. Each structural equation  $f_{j}$  encodes how the whole system, when intervened on, affects a subsystem  $X_{j}$ , i.e. how the collection of all other variables affects the individual variables (even though, in the end, the equations only depend on the parents). This way of encoding causal effects seems somewhat inconsistent with the philosophy laid out in the Introduction, that we are interested in what happens to the "system" when we intervene on a "subsystem". It also seems inconsistent with the actual action taken, which is to intervene on subsystems, not the whole system, or the parents of a particular variable.
In contrast, the causal kernels encode exactly what happens to the whole system, i.e. what measure we get on the whole measurable space  $(\Omega, \mathcal{H})$ , when we intervene on a "subsystem", i.e. put a desired measure on a sub- $\sigma$ -algebra of  $\mathcal{H}^3$ .
(ii) The primitive objects of SCMs are the variables  $X_{j}$ , the structural equations  $f_{j}$  and the distribution  $P_{\mathbf{N}}$  over the noise variables. The observational distribution, as well as the interventional distributions, are derived from these objects. It turns out that unique existence of observational and interventional distributions are not guaranteed, and can only be shown under the acyclicity assumption or rather stringent and hard-to-verify conditions on the structural equations and the noise distributions [10]. Moreover, it means that the observational and interventional distributions are not decoupled, and rather are linked through the structural equations  $f_{j}$ , and as a result, it is not possible to encode the full range of observational and interventional distributions using just the variables of interest (see Example 4.1).
In contrast, in causal spaces, the observational distribution  $\mathbb{P}$ , as well as the interventional distributions (via the causal kernels), are the primitive objects. Not only does this automatically guarantee their unique existence, but it also allows the interventional distributions (i.e. the causal information) to be completely decoupled from the observational distribution.
(iii) Galles and Pearl [21, Section 3] propose three axioms of counterfactuals based on SCMs (called causal models in that paper), namely, composition, effectiveness and reversibility. Even though these three concepts can be carried over to causal spaces, the mathematics through which they are represented needs to be adapted, since the tools that are used in causal spaces are different from those used in causal models of Galles and Pearl [21]. In particular, we work directly with measures as the primitive objects, whereas Galles and Pearl [21] use the structural equations as the primitive objects, and the probabilities only enter through a measure on the exogenous variables. Thus, the three properties can be phrased in the causal space language as follows:
<sup>3</sup>In this sense, some philosophy is shared with generalised structural equation models (GSEMs) [48].
Composition For  $S, R \subseteq T$ , denote by  $\mathbb{Q}'$  the measure on  $\mathcal{H}_{S \cup R}$  obtained by restricting  $\mathbb{P}^{\mathrm{do}(S, \mathbb{Q})}$ . Then  $\mathbb{P}^{\mathrm{do}(S, \mathbb{Q})} = \mathbb{P}^{\mathrm{do}(S \cup R, \mathbb{Q}')}$ . In words, intervening on  $\mathcal{H}_S$  via the measure  $\mathbb{Q}$  is the same as intervening on  $\mathcal{H}_{S \cup R}$  via the measure that it would have if we intervened on  $\mathcal{H}_S$  via  $\mathbb{Q}$ .
This is not in general true. A counterexample can be demonstrated with a simple SCM, where  $X_{1}, X_{2}$  and  $X_{3}$  causally affect  $Y$ , in a way that depends not only on the marginal distributions of  $X_{1}, X_{2}$  and  $X_{3}$  but their joint distribution, and  $X_{1}, X_{2}$  and  $X_{3}$  have no causal relationships among them. Then intervening on  $X_{1}$  with some measure  $\mathbb{Q}$  cannot be the same as intervening on  $X_{1}$  and  $X_{2}$  with  $\mathbb{Q} \otimes \mathbb{P}$ , since such an intervention would change the joint distribution of  $X_{1}, X_{2}$  and  $X_{3}$ , even if we give them the same marginal distributions.
Effectiveness For  $S \subseteq R \subseteq T$ , if we intervene on  $\mathcal{H}_R$  via a measure  $\mathbb{Q}$ , then  $\mathcal{H}_S$  has measure  $\mathbb{Q}$  restricted to  $\mathcal{H}_S$ .
This is indeed guaranteed by interventional determinism (Definition 2.2(ii)), so effectiveness continues to hold in causal spaces.
Reversibility For  $S, R, U \subseteq T$ , let  $\mathbb{Q}$  be some measure on  $\mathcal{H}_S$ , and  $\mathbb{Q}_1$  and  $\mathbb{Q}_2$  be measures on  $\mathcal{H}_{S \cup R}$  and  $\mathcal{H}_{S \cup U}$  respectively such that they coincide with  $\mathbb{Q}$  when restricted to  $\mathcal{H}_S$ . Then if  $\mathbb{P}^{\mathrm{do}(S \cup R, \mathbb{Q}_1)}(B) = \mathbb{Q}_2(B)$  for all  $B \in \mathcal{H}_U$  and if  $\mathbb{P}^{\mathrm{do}(S \cup U, \mathbb{Q}_2)}(C) = \mathbb{Q}_1(C)$  for all  $C \in \mathcal{H}_R$ , then  $\mathbb{P}^{\mathrm{do}(S, \mathbb{Q})}(A) = \mathbb{Q}_1(A)$  for all  $A \in \mathcal{H}_R$ .
This does not hold in general in causal spaces; in fact, Example 4.2 is a counterexample of this, with  $S = \emptyset$ .
3.2 Potential Outcomes (PO) Framework
In the PO framework, the treatment and outcome variables of interest are fixed in advance. Although much of the literature begins with individual units, these units are in the end i.i.d. copies of random variables under the stable unit treatment value assumption (SUTVA), and that is how we begin.
Denote by  $(\tilde{\Omega},\tilde{\mathcal{H}},\tilde{\mathbb{P}})$  the underlying probability space. Let  $Z:\tilde{\Omega}\to \mathcal{Z}$  be the "treatment" variable, taking values in a measurable space  $(\mathcal{Z},\mathfrak{z})$ . Then for each value  $z$  of the treatment, there is a separate random variable  $Y_{z}:\tilde{\Omega}\rightarrow \mathcal{V}$ , called the "potential outcome given  $Z = z$  " taking values in a measurable space  $(\mathcal{V},\mathfrak{V})$ ; we also have the "observed outcome", which is the potential outcome consistent with the treatment, i.e.  $Y = Y_{Z}$ . The researcher is interested in quantities such as the "average treatment effect",  $\tilde{\mathbb{E}} [Y_{z_1} - Y_{z_2}]$ , where  $\tilde{\mathbb{E}}$  is the expectation with respect to  $\tilde{\mathbb{P}}$ , to measure the causal effect of the treatment. Often, there are other, "pre-treatment variables" or "covariates", which we denote by  $X:\tilde{\Omega}\rightarrow \mathcal{X}$ , taking values in a measurable space  $(\mathcal{X},\mathfrak{X})$ . Given these, another object of interest is the "conditional average treatment effect", defined as  $\tilde{\mathbb{E}} [Y_{z_1} - Y_{z_2}\mid X]$ .
It is relatively straightforward to construct a causal space that can carry this framework. We define  $\Omega = \mathcal{Z} \times \mathcal{Y} \times \mathcal{X}$  and  $\mathcal{H} = \mathfrak{Z} \otimes \mathfrak{Y} \otimes \mathfrak{X}$ . We also define  $\mathbb{P}$ , for each  $A \in \mathfrak{Z}$ ,  $B \in \mathfrak{Y}$  and  $C \in \mathfrak{X}$  as  $\mathbb{P}(A \times B \times C) = \tilde{\mathbb{P}}(Z \in A, Y \in B, X \in C)$ . As for causal kernels, we are essentially only interested in  $K_Z(z, B)$  for  $B \in \mathfrak{Y}$ , and we define these to be  $K_Z(z, B) = \tilde{\mathbb{P}}(Y_z \in B)$ .
4 Examples
In this section, we give a few more concrete constructions of causal spaces. In particular they are designed to highlight cases which are hard to represent with existing frameworks, but which have natural representations in terms of causal spaces. Comparisons are made particularly with SCMs.
4.1 Confounders
The following example highlights the fact that, with graphical models, there is no way to encode correlation but no causation between two variables, using just the variables of interest.
Example 4.1. Consider the popular example of monthly ice cream sales and shark attacks in the US (Figure 3a), that shows that correlation does not imply causation. This cannot be encoded by an SCM with just two variables as in Figure 3b, since no causation means no arrows between the variables, which in turn also means no dependence. One needs to add the common causes into the
Figure 3: Correlation but no causation between ice-cream sales and shark attacks. S stands for the number of shark attacks, I for ice cream sales, T for temperature and E for economy.
model (whether observed or latent), the most obvious one being the temperature (high temperatures make people desire ice cream more, as well as to go to the beach more), as seen in Figure 3c. Now we have a model in which both dependence and no causation are captured. But can we stop here? There are probably other factors that affect both variables, such as the economy (the better the economic situation, the more likely people are to buy ice cream, and to take beach holidays) – see Figure 3d. Not only is the model starting to lose parsimony, but as soon as we stop adding variables to the model, we are making an assumption that there are no further confounding variables out there in the world<sup>4</sup>.
In contrast, causal spaces allow us to model any observational and causal relationships with just the variables that we were interested in, without any restrictions or the need to add more variables. In this particular example, we would take as our causal space  $(E_1 \times E_2, \mathcal{E}_1 \otimes \mathcal{E}_2, \mathbb{P}, \mathbb{K})$ , where  $E_1 = E_2 = \mathbb{R}$  with values in  $E_1$  and  $E_2$  corresponding to ice cream sales and shark attacks respectively, and  $\mathcal{E}_1 = \mathcal{E}_2$  being Lebesgue  $\sigma$ -algebras. Then we can let  $\mathbb{P}$  be a measure that has a strong dependence between any  $A \in \mathcal{E}_1$  and  $B \in \mathcal{E}_2$ , but let the causal kernels be  $K_1(x, B) = \mathbb{P}(B)$  for any  $x \in E_1$  and  $B \in \mathcal{E}_2$ , and likewise  $K_2(x, A) = \mathbb{P}(A)$  for any  $x \in E_2$  and  $A \in \mathcal{E}_1$ .
Nancy Cartwright argued against the completeness of causal Markov condition, using an example of two factories [13, p.108], in which there may not even be any confounders between dependent variables, not even an unobserved one. If we accept her position, then there are situations which SCMs would not be able to represent, whereas causal spaces would have no problems at all.
4.2 Cycles
As mentioned before, cycles in SCMs cause serious problems, namely that observational and interventional distributions that are consistent with the given structural equations and noise distribution may not exist, and when they do, they may not exist uniquely. This is an artefact of the fact that these distributions are derived from the structural equations rather than taken as the primitive objects. In the vast majority of the cases, cycles are excluded from consideration from the beginning and only directed acyclic graphs (DAGs) are considered. Some works study the solvability of cyclic SCMs [23, 10], where the authors investigate under what conditions on the structural equations and the noise variables there exist random variables and distributions that solve the given structural equations, and if so, when that happens uniquely. Other works have allowed cycles to exist, but restricted the definition of an SCM only to those that have a unique solution [23, 43, 49].
Of course, cyclic causal relationships abound in the real world. In our proposed causal space (Definition 2.2), given two sub- $\sigma$ -algebras  $\mathcal{H}_S$  and  $\mathcal{H}_U$  of  $\mathcal{H}$ , nothing stops both of them from having a causal effect on the other (see Definition B.1 for a precise definition of causal effects), but we are still guaranteed to have a unique causal space, both before intervention and after intervention on either  $\mathcal{H}_S$  or  $\mathcal{H}_U$ . The following is an example of a situation with "cyclic" causal relationship.
Example 4.2. We want to model the relationship between the amount of rice in the market and its price per kg. Take as the probability space  $(E_1 \times E_2, \mathcal{E}_1 \otimes \mathcal{E}_2, \mathbb{P})$ , where  $E_1 = E_2 = \mathbb{R}$  with values in  $E_1$  and  $E_2$  representing the amount of rice in the market in million tonnes and the price
One solution could be to add a single "variable" that collects all of the confounders into one, but then the numerical value of this "variable", as well as its distribution and the structural equations from this "variable" into S and I, would be completely meaningless.
(c)
Figure 4: Rice in the market in million tonnes and price per kg in KRW.
of rice per kg in KRW respectively,  $\mathcal{E}_1, \mathcal{E}_2$  are Lebesgue  $\sigma$ -algebras and  $\mathbb{P}$  is for simplicity taken to be jointly Gaussian. Without any intervention, the higher the yield, the more rice there is in the market and lower the price, as in Figure 4b. If the government intervenes on the market by buying up extra rice or releasing rice into the market from its stock, with the goal of stabilising supply at 3 million tonnes, then the price will stabilise accordingly, say with Gaussian distribution with mean 4.5 and standard deviation 0.5, as in Figure 4c. The corresponding causal kernel will be  $K_1(3,x) = \frac{2}{\sqrt{2\pi}} e^{-\frac{1}{2} \left(\frac{x - 4.5}{0.5}\right)^2}$ . On the other hand, if the government fixes the price of rice at a price, say at 6,000 KRW per kg, then the farmers will be incentivised to produce more, say with Gaussian distribution with mean 4 and standard deviation 0.5, as in Figure 4d. The corresponding causal kernel will be  $K_2(6,y) = \frac{2}{\sqrt{2\pi}} e^{-\frac{1}{2} \left(\frac{y - 4}{0.5}\right)^2}$ .
Causal spaces treat causal effects really as what happens after an intervention takes place, and with this approach, cycles can be rather naturally encoded, as shown above. We do not view cyclic causal relationships as an equilibrium of a dynamical system, or require it to be viewed as an acyclic stochastic process, as done by some authors [46, p.85, Remark 6.5].
4.3 Continuous-time Stochastic Processes, and Parents
A very well established sub-field of probability theory is the field of stochastic processes, in which the index set representing (most often) time can be either discrete or continuous, and in both cases, infinite. However, most causal models start by assuming a finite number of variables, which immediately rules out considering stochastic processes, and efforts to extend to infinite number of variables usually consider only discrete time steps [46, Chapter 10] or dynamical systems [9, 47, 8, 50]. Since probability spaces have proven to accommodate continuous time stochastic processes in a natural way, it is natural to believe that causal spaces, being built up from probability spaces, should be able to enable the incorporation of the concept of causality in the theory of stochastic processes.
Let  $W$  be a totally-ordered set, in particular  $W = \mathbb{N} = \{0,1,\ldots\}$ ,  $W = \mathbb{Z} = \{\dots, -2, -1, 0, 1, 2,\ldots\}$ ,  $W = \mathbb{R}_{+} = [0,\infty)$  or  $W = \mathbb{R} = (-\infty,\infty)$  considered as the time set. Then, we consider causal spaces of the form  $(\Omega, \mathcal{H}, \mathbb{P}, \mathbb{K}) = (\times_{t \in T} E_t, \otimes_{t \in T} \mathcal{E}_t, \mathbb{P}, \mathbb{K})$ , where the index set  $T$  can be written as  $T = W \times \tilde{T}$  for some other index set  $\tilde{T}$ . The following notion captures the intuition that causation can only go forwards in time.
Definition 4.3. Let  $(\Omega, \mathcal{H}, \mathbb{P}, \mathbb{K}) = (\times_{t \in T} E_t, \otimes_{t \in T} \mathcal{E}_t, \mathbb{P}, \mathbb{K})$  be a causal space, where the index set  $T$  can be written as  $T = W \times \tilde{T}$ , with  $W$  representing time. Then we say that the causal mechanism  $\mathbb{K}$  respects time, or that  $\mathbb{K}$  is a time-respecting causal mechanism, if, for all  $w_1, w_2 \in W$  with  $w_1 < w_2$ , we have that  $\mathcal{H}_{w_2 \times \tilde{T}}$  has no causal effect (in the sense of Definition B.1) on  $\mathcal{H}_{w_1 \times \tilde{T}}$ .
In a causal space where the index set  $T$  has a time component, the fact that causal mechanism  $\mathbb{K}$  respects time means that the past can affect the future, but the future cannot affect the past. This already distinguishes itself from the concept of conditioning - conditioning on the future does have implications for past events. We illustrate this point in the example of a Brownian motion.
Example 4.4. We consider a 1-dimensional Brownian motion. Take  $(\times_{t\in \mathbb{R}_{+}}E_{t},\otimes_{t\in \mathbb{R}_{+}}\mathcal{E}_{t},\mathbb{P},\mathbb{K})$  where, for each  $t\in \mathbb{R}_+$ ,  $E_{t} = \mathbb{R}$  and  $\mathcal{E}_t$  is the Lebesgue  $\sigma$ -algebra, and  $\mathbb{P}$  is the Wiener measure. For  $s < t$ , we have causal kernels  $K_{s}(x,y) = \frac{1}{\sqrt{2\pi(t - s)}} e^{-\frac{1}{2(t - s)} (y - x)^2}$  and  $K_{t}(x,y) = \frac{1}{\sqrt{2\pi s}} e^{-\frac{1}{2s} y^{2}}$ .
The former says that, if we intervene by setting the value of the process to  $x$  at time  $s$ , then the process starts again from  $x$ , whereas the latter says that if we intervene at time  $t$ , the past values at time  $s$  are not affected. On the left-hand plot of Figure 5, we set the value of the process at time 1 to 0. The past values of the process are not affected, and there is a discontinuity at time 1 where the process starts
Figure 5: 1-dimensional Brownian motion, intervened and conditioned to have value 0 at time 1.
again from 0. Contrast this to the right-hand plot, where we condition on the process having value 0 at time 1. This does affect past values, and creates a Brownian bridge from time 0 to time 1.
Note, Brownian motion is not differentiable, so no approach based on dynamical systems is applicable.
Remark 4.5. The concept of parents is central in SCMs – the structural equations are defined on the parents of each variable. However, continuous time is dense, so given two distinct points in time, there is always a time point in between. Suppose we have a one-dimensional continuous time Markov process  $(X_{t})_{t\in \mathbb{R}}$  [14, p.169], and a point  $t_0$  in time. Then for any  $t < t_0$ ,  $X_{t}$  has a causal effect on  $X_{t_0}$ , but there always exists some  $t'$  with  $t < t' < t_0$  such that conditioned on  $X_{t'}$ ,  $X_{t}$  does not have a causal effect on  $X_{t_0}$ , meaning  $X_{t}$  cannot be a parent of  $X_{t_0}$ . In such a case,  $X_{t_0}$  cannot be said to have any parents, and hence no corresponding SCM can be defined.
In this work, we discussed the lack of a universally agreed axiomatisation of causality, and some arguments as to why measure-theoretic probability theory can provide a good foundation on which to build such an axiomatisation. We proposed causal spaces, by enriching probability spaces with causal kernels that encode information about what happens after an intervention. We showed how the interventional aspects of existing frameworks can be captured by causal spaces, and finally we gave some explicit constructions, highlighting cases in which existing frameworks fall short.
Even if causal spaces prove with time to be the correct approach to axiomatise causality, there is much work to be done – in fact, all the more so in that case. Most conspicuously, we only discussed the interventional aspects of the theory of causality, but the notion of counterfactuals is also seen as a key part of the theory, both interventional counterfactuals as advocated by Pearl's ladder of causation [44, Figure 1.2] and backtracking counterfactuals [58]. We provide some discussions and possible first steps towards this goal in Appendix E, but mostly leave it as essential future work. Only then will we be able to provide a full comparison with the counterfactual aspects of SCMs and the potential outcomes. As discussed in Section 1, the notion of actual causality is also important, and it would be interesting to investigate how this notion can be embedded into causal spaces. Many important definitions, including causal effects, conditional causal effects, hard interventions and sources, as well as technical results, were deferred to the appendix purely due to space constraints, but we foresee that there would be many more interesting results to be proved, inspired both from theory and practice. In particular, the theory of causal abstraction [49, 2, 3] should benefit from our proposal, through extensions of homomorphisms of probability spaces to causal spaces<sup>5</sup>.
As a final note, we stress again that our goal should not be understood as replacing existing frameworks. Indeed, causal spaces cannot compete in terms of interpretability, and in the vast majority of situations in which SCMs, potential outcomes or any of the other existing frameworks are suitable, we expect them to be much more useful. In particular, assumptions are unavoidable for identifiability from observational data, and those assumptions are much better captured by existing frameworks<sup>6</sup>, However, just as measure-theoretic probability theory has its value despite not being useful for practitioners in applied statistics, we believe that it is a worthy endeavour to formally axiomatise causality.
In a similar vein, Keurti et al. [37] consider homomorphisms between groups of interventions.
Researchers from the potential outcomes community and the graphical model community are arguing as to which framework is better for which situations [30, 43]. We do not take part in this debate.