ICLR 2025

Published as a conference paper at ICLR 2025
COUNTERFACTUAL REALIZABILITY
Arvind Raghavan and Elias Bareinboim
Causal Artificial Intelligence Lab
Columbia University
{ar, eb}@cs.columbia.edu
ABSTRACT
It is commonly believed that, in a real-world environment, samples can only be
drawn from observational and interventional distributions, corresponding to Layers
1 and 2 of the Pearl Causal Hierarchy. Layer 3, representing counterfactual
distributions, is believed to be inaccessible by definition. However, Bareinboim,
Forney, and Pearl (2015) introduced a procedure that allows an agent to sample
directly from a counterfactual distribution, leaving open the question of what other
counterfactual quantities can be estimated directly via physical experimentation.
We resolve this by introducing a formal definition of realizability, the ability to draw
samples from a distribution, and then developing a complete algorithm to determine
whether an arbitrary counterfactual distribution is realizable given fundamental
physical constraints, such as the inability to go back in time and subject the same
unit to a different experimental condition. We illustrate the implications of this
new framework for counterfactual data collection using motivating examples from
causal fairness and causal reinforcement learning. While the baseline approach
in these motivating settings typically follows an interventional or observational
strategy, we show that a counterfactual strategy provably dominates both.
1
INTRODUCTION
The Pearl Causal Hierarchy, or PCH, is an important recent milestone in our understanding of
causality (Pearl & Mackenzie, 2018; Bareinboim et al., 2022). The three layers of the PCH represent
the distinct regimes of seeing, doing, and imagining, with regard to an environment. Consider
an environment involving a decision variable X and an outcome Y . Layer 1 (L1) represents
observational distributions, such as P(Y | x). Layer 2 (L2) represents interventional distributions,
such a...