On the Causal Foundations ofof Artificial Intelligence(Explainability & Decision-Making)Elias Bareinboim
Causal Artificial Intelligence Lab
Columbia University(@eliasbareinboim)

24th Information-Based Induction Sciences Workshop

Outline• What is a causal model?
• (1) Structural Causal Model (SCM)(2) The Pearl Causal Hierarchy* (PCH),(3) The Causal Hierarchy Theorem* (CHT)• Where supervised, unsupervised, andreinforcement learning fits into the PCH.

* Partly based on chapter in honor of J. Pearl, “On Pearl’s Hierarchy and the Foundations of Causal
Inference” (Bareinboim, J. Correa, D. Ibeling, T. Icard), ACM, 2020, in press, https://causalai.net/r60.pdf.
2Outline• What is a causal model?
• (1) Structural Causal Model (SCM)(2) The Pearl Causal Hierarchy* (PCH),(3) The Causal Hierarchy Theorem* (CHT)• Where supervised, unsupervised, andreinforcement learning fits into the PCH.
• What is causal inference & cross-layer inferences• On the Design of AI w/ causal capabilities• Modern ML methods (DL & RL)
Goal: Introduce some ideas, principles, and tasks.
* Partly based on chapter in honor of J. Pearl, “On Pearl’s Hierarchy and the Foundations of Causal
Inference” (Bareinboim, J. Correa, D. Ibeling, T. Icard), ACM, 2020, in press, https://causalai.net/r60.pdf.
2

WHAT IS A CAUSAL MODEL?

3STRUCTURAL CAUSAL MODEL(THE DATA GENERATING MODEL)• Processes• Intervention

Drug ← fD (Age, UD)
Headache ← fH(Drug, Age, UH)G=

Z (Age)X(Drug)Y(Headache)P(Z, X, Y)(observational)
4

STRUCTURAL CAUSAL MODEL(THE DATA GENERATING MODEL)• Processes• Intervention

Drug ← fD (Age, UD)
Drug ← Yes
Headache ← fH(Drug, Age, UH) Headache ← fH (Drug, Age, UH)G=

Z (Age)Features
X(Drug)

DecisionY(Headache)P(Z, X, Y)(observational)

OutcomeGdo(X) =Z (Age)Yes
X(Drug)Y(Headache)P(Z, Y | do(X = Yes))(interventional)
4

STRUCTURAL CAUSAL MODEL(THE DATA GENERATING MODEL)
Unobserved• Processes• Intervention

Drug ← fD (Age, UD)
Drug ← Yes
Headache ← fH(Drug, Age, UH) Headache ← fH (Drug, Age, UH)G=

Z (Age)Features
X(Drug)

DecisionY(Headache)P(Z, X, Y)

Seeing(observational)

OutcomeGdo(X) =Z (Age)Yes
X(Drug)Y(Headache)P(Z, Y | do(X = Yes))

Doing(interventional)

5

STRUCTURAL CAUSAL MODELS(Ch. 7, Pearl, 2000)
Definition: A structural causal model M (or, data
generating model) is a tuple (V, U, F, P(u)), where• V = {V1,...,Vn} are endogenous variables,• U = {U1,...,Um} are exogenous variables,• F = {f1,..., fn} are functions determining V,
for each Vi, Vi ← fi(Pai, Ui), where Pai ⊂ V, Ui ⊂ U.
• P(u) is a distribution over U.
(Axiomatic characterization [Halpern, Galles, Pearl, 1998].)

Thm. SCM M implies Pearl Causal Hierarchy.
6

STRUCTURAL CAUSAL MODELS(Ch. 7, Pearl, 2000)
Definition: A structural causal model M (or, data
generating model) is a tuple (V, U, F, P(u)), where• V = {V1,...,Vn} are endogenous variables,• U = {U1,...,Um} are exogenous variables,• F = {f1,..., fn} are functions determining V,
for each Vi, Vi ← fi(Pai, Ui), where Pai ⊂ V, Ui ⊂ U.
• P(u) is a distribution over U.
(Axiomatic characterization [Halpern, Galles, Pearl, 1998].)“Ladder of Causation”

Thm. SCM M implies Pearl Causal Hierarchy.
6

SCM → PEARL CAUSAL HIERARCHY
Level (Symbol)

1#

Activity

Question(Bayes Net, DTree,SVM, DNN, …)Interventional
P(y | do(x), c)

3

CounterfactualP(yx | x’, y’)!

TypicalExamplesWhat is?
Associational
SeeingML - (Un)Supervised How would
P(y | x)2"

Typical

Doing

What does asymptom tell us
seeing X change about thedisease?
my belief in Y?
What if?
What if I do X?

What if I takeaspirin, will myheadache be
cured?

Why?
Imagining,
Retrospection What if I had

Was it theaspirin that
acted differently? stopped myheadache?
7

SCM → PEARL CAUSAL HIERARCHY
Level (Symbol)

1

#
2

"
3

!

TypicalTypicalActivity

Question

ExamplesWhat is?
Associational
SeeingML - (Un)Supervised How would
P(y | x)
(Bayes Net, DTree,SVM, DNN, …)

What does asymptom tell us
seeing X change about thedisease?
my belief in Y?

What if?
InterventionalDoing
P(y | do(x), c) ML - Reinforcement What if I do X?
(Causal Bayes Net,
MDPs, POMDPs)CounterfactualP(yx | x’, y’)What if I takeaspirin, will myheadache be
cured?

Why?
Imagining,
Retrospection What if I had

Was it theaspirin that
acted differently? stopped myheadache?
7

SCM → PEARL CAUSAL HIERARCHY
Level (Symbol)

1

#
2

"
3

!

TypicalTypicalActivity

Question

ExamplesWhat is?
Associational
SeeingML - (Un)Supervised How would
P(y | x)
(Bayes Net, DTree,SVM, DNN, …)

What does asymptom tell us
seeing X change about thedisease?
my belief in Y?

What if?
InterventionalDoing
P(y | do(x), c) ML - Reinforcement What if I do X?
(Causal Bayes Net,
MDPs, POMDPs)

What if I takeaspirin, will myheadache be
cured?

Was it theaspirin that
acted differently? stopped myheadache?
Structural Causal Model

CounterfactualP(yx | x’, y’)Why?
Imagining,
Retrospection What if I had

7

PCH: MOVING BEYOND TRADITIONAL ML$

Cross-layerActivity
inferences:
Question

Level (Symbol)

TypicalTypicalExamples1

Associational
P(y | x)SeeingWhat is?
How would
seeing X change
my belief in Y?

What does asymptom tell us
about thedisease?

2

Interventional
P(y | do(x), c)

Doing

What if?
What if I do X?

What if I takeaspirin, will myheadache be
cured?

#"

8

PCH: MOVING BEYOND TRADITIONAL ML$

Cross-layerActivity
inferences:
Question

Level (Symbol)

1

#
2

"

TypicalTypicalExamplesAssociational
Input:
P(y | x)SeeingSeeingWhat
is?of the available
What does dataamost
symptom tell us
How would
is observational,
seeing X change about thedisease?
passively
collected
my belief
in Y?

Interventional
Output:P(y | do(x), c)

Doing

What
if? of the inferences
What if I takeare
most
aspirin, will my
What ifabout
I do X?causal
effectsheadachebe(policies, treatments,
cured? decisions)(data)(query)Doing

Research Question.
How to use the data collected fromobservations (layer 1) to answer
questions about interventions (layer 2)?

8

WHY IS THE CAUSAL PROBLEM “NON-TRIVIAL”?
SCMs ARE ALMOST NEVER OBSERVED
SCM
X ← fx(Uy)Y ← fy(X, Uy)
P(Ux, Uy)

Exceptions:- Physics- Chemistry- Biology

PCH:
P(y, x)P(y | do(x))

P(vx | x’, y’)
9WHY IS THE CAUSAL PROBLEM “NON-TRIVIAL”?
SCMs ARE ALMOST NEVER OBSERVED
SCM
X ← fx(Uy)Y ← fy(X, Uy)
P(Ux, Uy)

PCH:
P(y, x)P(y | do(x))

P(vx | x’, y’)
9WHY IS THE CAUSAL PROBLEM “NON-TRIVIAL”?
SCMs ARE ALMOST NEVER OBSERVED
SCM
X ← fx(Uy)Y ← fy(X, Uy)

Unobserved
NatureP(Ux, Uy)

PCH:
P(y, x)Data

P(y | do(x))

?

Query

P(vx | x’, y’)Observed
Phenomenon
9WHY IS THE CAUSAL PROBLEM “NON-TRIVIAL”?
SCMs ARE ALMOST NEVER OBSERVED
SCM
X ← fx(Uy)Y ← fy(X, Uy)

Unobserved
NatureP(Ux, Uy)

PCH:
P(y, x)Data

P(y | do(x))

?

Query

P(vx | x’, y’)Observed
Phenomenon
9

CROSS-LAYER INFERENCES -IMPOSSIBILITY RESULTS• Task: Infer the causal quantity Q = P(Y | do(X))(layer-2) from observational data P(X, Y) (layer-1).
Theorem. The effect of X on Y is not identifiable fromthe available (observational) data.
Proof. There exist two collection of mechanisms capableof generating the same observed behavior, P(X, Y),
while disagreeing w.r.t. the causal quantity Q.
To witness, consider the following two SCMs:
Note that
M1: X ← U,
Y ← ¬X ⨁ U; P1(X,Y=1) = P2(X,Y=1) = 1/2, ∀X, 0 o/w.
But,
M2: X ← U,
P1(Y = 1| do(X=1)) = 1/2 in M1,
Y ← 1,
10
P2(Y = 1 | do(X=1)) = 1 in M2.
P(U) = 1/2

CAUSAL HIERARCHY THEOREM(E. Bareinboim, J. Correa, D. Ibeling, T. Icard, 2020)• Task: Infer the causal quantity Q = P(Y | do(X))(layer-2) from observational data P(X, Y) (layer-1).
Theorem. With respect to Lebesgue measure over (asuitable encoding of L3-equivalence classes of) SCMs,
the subset in which any PCH ‘collapse’ is measure zero.

Informally, for almost any SCM (i.e., almost any possible
environment), the PCH does not collapse, i.e., the layersof the hierarchy remains distinct.
collapse
L1

L2 L3

L1,L2,L3

Corollary. To answer question at Layer i (about a certain
interaction), one needs knowledge at layer i or higher.
11

CAUSAL HIERARCHY THEOREM(E. Bareinboim, J. Correa, D. Ibeling, T. Icard, 2020)• Task: Infer the causal quantity Q = P(Y | do(X))(layer-2) from observational data P(X, Y) (layer-1).
Theorem. With respect to Lebesgue measure over (asuitable encoding of L3-equivalence classes of) SCMs,
the subset in which any PCH ‘collapse’ is measure zero.

After all, how are causal inferences possible?
Informally, for almost any SCM (i.e., almost any possible
environment), the PCH does not collapse, i.e., the layersof the hierarchy remains distinct.
collapse
L1

L2 L3

L1,L2,L3

Corollary. To answer question at Layer i (about a certain
interaction), one needs knowledge at layer i or higher.
11

NEW CLASS OF GRAPHICAL MODELS(ENCODING THE STRUCTURAL CONSTRAINTS)(e)
SCM
X ← fx(Uy)Y ← fy(X, Uy)
P(Ux, Uy)

P(y, x)P(y | do(x))

Data

Query

P(v_x | x’, y’)12NEW CLASS OF GRAPHICAL MODELS(ENCODING THE STRUCTURAL CONSTRAINTS)

(e)
SCM

Structural Constraints(e.g., graphical model)

X ← fx(Uy)Y ← fy(X, Uy)(L1) AssociationalP(Ux, Uy)
(L2) Interventional(L3) Counterfactual

P(y, x)P(y | do(x))

Data

Query

P(v_x | x’, y’)12Bayes Nets (L1) vs. Causal BNs (L2)
(NOT ALL GRAPHICAL MODELS ARE CREATED EQUAL)(e)(layer-2)• Task: Infer the causal quantity Q = P(Y | do(X))
from observational data P(X, Y) (layer-1).
• From the previous theorem, it was shown that it’s
impossible to link layers 1 and 2, in general.
• What if a Bayes net compatible with the data is available?
Data (L1):P(y, x)X

Y

P1(y | do(x)) = P(y | x)?

Query (L2):

BN G1

P(y | do(x))

BN G2

X

P2(y | do(x)) = P(y)Y
13

Bayes Nets (L1) vs. Causal BNs (L2)
(NOT ALL GRAPHICAL MODELS ARE CREATED EQUAL)%(e)(layer-2)
Q
=
P(Y
|
do(X))• Task: Infer the causal quantity
Lesson: causal assumptions are
Lesson.
There is not enoughfrom observational data P(X,Y)
(layer-1).
needed since the data does not
information about theit was enoughshown that
it’s
• From the previous theorem,impose
constraints
overunderlying SCM in the BN so
impossible to link layers 1 and
2, in general.
the causal
structure, in which
as to allow causal inferences.
the result
is sensitive.
with thedata is available?
• What if a Bayes net compatible
Data (L1):P(y, x)X

Y

P1(y | do(x)) = P(y | x)?

Query (L2):

BN G1

P(y | do(x))

BN G2

X

P2(y | do(x)) = P(y)Y
13NEW CLASS OF GRAPHICAL MODELS(ENCODING THE STRUCTURAL CONSTRAINTS)

(e)
SCM

Structural Constraints(e.g., graphical model)

X ← fx(Uy)Y ← fy(X, Uy)(L1) AssociationalP(Ux, Uy)
(L2) Interventional(L3) Counterfactual

P(y, x)P(y | do(x))

Data

Query

P(v_x | x’, y’)14NEW CLASS OF GRAPHICAL MODELS(ENCODING THE STRUCTURAL CONSTRAINTS)

(e)
SCM

Structural Constraints(e.g., graphical model)

X ← fx(Uy)Y ← fy(X, Uy)(L1) AssociationalP(Ux, Uy)
(L2) Interventional(L3) Counterfactual

P(y, x)P(y | do(x))

Data

Query

P(v_x | x’, y’)14NEW CLASS OF GRAPHICAL MODELS(ENCODING THE STRUCTURAL CONSTRAINTS)

(e)
SCM

Structural Constraints(e.g., graphical model)

X ← fx(Uy)Y ← fy(X, Uy)(L1) AssociationalP(Ux, Uy)
(L2) Interventional(L3) Counterfactual

P(y, x)P(y | do(x))

Data

Query

P(v_x | x’, y’)
14

POSITIVE CROSS-LAYER INSTANCES-- FROM L2- GRAPHICAL MODEL (e.g., CBN)
P(y|do(x)) is
identifiable from G
ModelsinducingP(v)Formally, for any
two SCMs (encoding
the unobserved nature),
Models
N1, N2, s.t.
with sameG(N1) = G(N2) = G,P(y | do(x))P1(v) = P2(v), thenModelsP1(y | do(x)) =compatible
P2 (y | do(x)).
with G (L2)
All structuralmodels15

THE TENSION BETWEENREALITY vs. {OUR MODEL+DATA}• We started from a well-defined world, semanticallyspeaking, in which an SCM M (=<F, P(u)>) implies thePCH, which means different aspects of the underlying
Nature and types of behavior.
• We acknowledge the collection of mechanisms are
there, but inferences are limited given that the SCM isalmost never observable (due to the CHT).
• We move towards scenarios in which partial knowledgeof the SCM is available, such as a causal graph/EC.
• Causal inference theory helps us to determine whether
the targeted inference is allowed. In the prior example,
the inference is from Layers 1 to 2, namely,<G, P(v)>⫤•P(y | do(x)).

16

WEAK MODELS → LACK OF IDENTIFIABILITY(LIMITS OF WHAT CAN BE INFERRED FROM DATA?)
P(y|do(x)) is
identifiable from G
Modelsinducing
P(v)Modelswith sameP(y | do(x))
Modelscompatible
with G (L2)
All structuralmodels17

WEAK MODELS → LACK OF IDENTIFIABILITY(LIMITS OF WHAT CAN BE INFERRED FROM DATA?)
P(y|do(x)) is
identifiable from GP(y|do(x)) is
not identifiable in G
Modelsinducing
P(v)Modelswith sameP(y | do(x))
Modelscompatible
with G (L2)
All structuralmodels

Diﬀerent
P*(y | do(x)) !

17

HOW DOES REINFORCEMENT LEARNING
FIT INTO THIS PICTURE?

Tutorial (ICML’20): https://crl.causalai.net
18

SCM → PCH
Level (Symbol)

TypicalTypicalActivity

Question

1

What is?
Associational
SeeingML - (Un)Supervised How would
P(y | x)2

What if?
InterventionalDoing
P(y | do(x), c) ML - Reinforcement What if I do X?

3

CounterfactualP(yx | x’, y’)#"!

ExamplesWhat does asymptom tell us
seeing X change about thedisease?
my belief in Y?
What if I takeaspirin, will myheadache be
cured?

Why?
Imagining,
Retrospection What if I had

Was it theaspirin that
acted differently? stopped myheadache?
Structural Causal Model
19

SCM → PCH
Level (Symbol)

TypicalTypicalActivity

Question&Well, typical RL cannot
1#

What is?
Associational
SeeingML - (Un)Supervised How would
P(y | x)move across layers…

What does asymptom tell us
seeing X change about thedisease?
my belief in Y?

2

What if?
InterventionalDoing
P(y | do(x), c) ML - Reinforcement What if I do X?

3

CounterfactualP(yx | x’, y’)"!

ExamplesWhat if I takeaspirin, will myheadache be
cured?

Why?
Imagining,
Retrospection What if I had

Was it theaspirin that
acted differently? stopped myheadache?
Structural Causal Model19

RL - Big Picture
Parameters
about the env.

Agent
Θ, G

context / state

actionEnvironment

reward• Model-based versus model-free RL.
20
20

Causal Reinforcement Learning
context / state

Agent
Θ, G

actionEnvironmentMrewardCausal Diagram
Resources: https://crl.causalai.net

Structural Causal Model21
21

Causal Reinforcement Learning
Two key CRL observations:&

1. The environment and the agent will be/ statetied thr.context
the pair
SCM M & causal graph G.

2. We’ll define diﬀerent types of “actions”, orinteractions, following PCH.

Agent
Θ, G

actionEnvironmentMrewardStructural Causal Model

Causal Diagram

observing, experimenting, imagining (PCH)
Resources: https://crl.causalai.net

21
21

CAUSAL REINFORCEMENT LEARNING —NEW CHALLENGES & OPPORTUNITIESTask 1. Generalized Policy Learning

IJCAI’17; NeurIPS’19, 20, 21• The causal agent (C) learns through different
modalities, including by observing and intervening in theenvironment (generalized do-calculus, off- & on-policy).

Task 2. When and where to intervene? NeurIPS’18; AAAI’19; NeurIPS’20;• Agent-C now contemplates not to intervene in thesystem. If it does, still, it learns how to surgicallychange the environment so as to bring only theintended changes about (protecting against side effects).

Task 3. Counterfactual Decision-Making NeurIPS’15; ICML’17; AAAI’19• C now is capable of analyzing itself (biases, goals,
intentions), while protecting against adversaries andunobserved confounders.

22

CAUSAL REINFORCEMENT LEARNING —NEW CHALLENGES & OPPORTUNITIESTask 4. Generalizability of Causal Knowledge acrosschanging conditions NeurIPS’13,14; PNAS’16; AAAI’20; NeurIPS’20• Agent C is capable of generalizing causal knowledge
acquired in one setting to another with a minimal amountof experimentation by leveraging causal invariances.

Task 5. Structural Learning from Combining
Observations and Experiments NeurIPS’17; ICML’18; NeurIPS’19, 20• C combines observations from other agents withexperiments conducted by herself to learn anequivalence class of the underlying causal system.
(Based on forth. paper “An Introduction to Causal Reinforcement
Learning: New Challenges and Opportunities”, Bareinboim, Lee, Zhang).

23

HOW DOES DEEP LEARNING
FIT INTO THIS PICTURE?

24

Reality
Model

Observational

Interventional

Drug ← fD (Age, Ud)
Survival ← fS(Drug, Age, US)

Drug ← Yes
Survival ← fS(Drug, Age, US)G

Gdo(X)X(Drug)Z (Age)Y(Survival)P(Z, X, Y)

X(Drug)Z (Age)Y(Survival)

P(Y | do(X = x))25

Reality
Model

Observational
Drug ← fD (Age, Ud)
Survival ← fS(Drug, Age, US)

Drug ← Yes
Survival ← fS(Drug, Age, US)G

Gdo(X)Z (Age)X(Drug)Y(Survival)

Z

Y(Survival)

Y

CLT

X

X
(Drug)Z (Age)P(Y | do(X = x))

P(Z, X, Y)

DataInterventional

DNN^ X, Y)
P(Z,25Reality
Model

Observational

Interventional

Drug ← fD (Age, Ud)
Survival ← fS(Drug, Age, US)

Drug ← Yes
Survival ← fS(Drug, Age, US)G

Gdo(X)Z (Age)X(Drug)Y(Survival)

Z

X(Drug)Y(Survival)

P(Y | do(X = x))

P(Z, X, Y)
X

Z (Age)

Y

Data

∅
DNN

^ X, Y)P(Z,^ | do(X = x))P(Y25

Reality
Model

Observational

Interventional

Drug ← fD (Age, Ud)
Survival ← fS(Drug, Age, US)

Drug ← Yes
Survival ← fS(Drug, Age, US)G

Gdo(X)Z (Age)X(Drug)Y(Survival)

Z

X(Drug)Y(Survival)

P(Y | do(X = x))

P(Z, X, Y)
X

Z (Age)

Y

Data

∅
DNN

^ X, Y)P(Z,^ | do(X = x))P(Y25

Reality
Model

Observational

Interventional

Drug ← fD (Age, Ud)
Survival ← fS(Drug, Age, US)

Drug ← Yes
Survival ← fS(Drug, Age, US)G

Gdo(X)Z (Age)Z (Age)There’s nothing in the data nor in
the deep net that takes into accountY the structuralXconstraints Y(CHT)!
(Drug)(Survival)(Survival)

X(Drug)

P(Y | do(X = x))

P(Z, X, Y)
X

Z

Y

Data

∅
DNN

^ X, Y)P(Z,'^ | do(X = x))P(Y25

Reality
Model

Observational

Interventional

Drug ← fD (Age, Ud)
Survival ← fS(Drug, Age, US)

Drug ← Yes
Survival ← fS(Drug, Age, US)G

Gdo(X)X(Drug)Z (Age)Z (Age)There’s nothing in the data nor in
the deep net that takes into accountY the structuralXconstraints Y(CHT)!
(Drug)(Survival)(Survival)| do(X = x))
X, Y)
How toP(Z,combinecausal models P(Y& neuralnets?
Answer:Ourlatestpaper(NeurIPS’21):
X Z Y
Data

∅

The Causal-Neural Connection:
Expressiveness, Learnability, & Inference
DNN'
K. Xia, K. Lee, Y. Bengio, E. Bareinboim^ X, Y)P(Z,^ | do(X = x))P(Y25

CONCLUSIONS• Causal inference and AI are fundamentallyintertwined and novel learning opportunities emerge
when this connection is fully understood.
• Most impediments for general AI today are orthogonal
to the current (a-causal) methods available in ML,
including deep learning and reinforcement learning.
• In practice, failure to acknowledge distinct features of
causality almost always lead to poor decision-making
and superficial types of explanations.

• Goal: Develop a framework (set of principles,algorithms, tools) for designing causally-sensible AI
systems -- integrating [observational, interventional,
counterfactual] data, modes of reasoning, andknowledge.
• Leads to a natural treatment of human-like
explainability and rational decision-making.

THANK YOU -- JOINT WORK WITH
CAUSAL AI LAB & COLLABORATORS
Adele Ribeiro (Columbia)

Yoshua Bengio (Mila)

Juan Correa (Columbia)

Judea Pearl (UCLA)

Kai-Zhan Lee (Columbia)

Carlos Cinelli (UW)Sanghack Lee (Columbia)

Andrew Forney (UCLA)

Adele Ribeiro (Columbia)

Jin Tian (Iowa State)

Kevin Xia (Columbia)

Duligur Ibeling (Stanford)

Junzhe Zhang (Columbia)

Thomas Icard (Stanford)

Amin Jaber (Purdue)

Murat Kocaoglu (Purdue)

Chris Jeong (Purdue)

Karthikeyan Shanmugam (IBM)Yonghan Jung (Purdue)

Jiji Zhang (Lingnan University)

Daniel Kumor (Purdue)

Paul Hünermund (Copenhagen)