TOWARDS CAUSAL REPRESENTATIONLEARNING:
AN AI & DEEP LEARNING PERSPECTIVE
ON CAUSALITYYOSHUA BENGIO

30 November 2020
Mila Causality Inference Lectures

WHAT IS MISSING TOWARDS HUMAN-LEVEL AI?
• AI systems which actually understand the variables they manipulate(including language, perception and action)• What does ’understanding’ mean?
• They capture causality• They capture how the world works• They understand abstract actions and how use them to control• They can reason and plan, even in novel scenarios• They can explain what happened (inference, credit assignment)• They can generalize out-of-distribution2

Missing from Current ML:
Understanding & Generalization
Beyond the Training Distribution• Learning theory only deals with generalization
within the same distribution• Models learn but do not generalize well (or havehigh sample complexity when adapting) tomodified distributions, non-stationarities, etc.

Missing from Current ML:
Understanding & Generalization
Beyond the Training Distribution• If not iid, need alternative assumptions,otherwise no reason to expect generalization• How do distributions change?
• What knowledge can be re-used?

COMPOSITIONALITY HELPS IID AND OOD GENERALIZATION

Different forms of compositionality
each with different exponential advantages• Distributed representations(Pascanu et al ICLR 2014)• Composition of layers in deep nets
(Montufar et al NeurIPS 2014)• Systematic generalization in language,
analogies, abstract reasoning? TBD(Lee, Grosse, Ranganath &Ng, ICML 2009)

5

SYSTEMATIC GENERALIZATION• Studied in linguistics• Dynamically recombine existing concepts• Even when new combinations have 0 probability
under training distribution(Lake et al 2015)• E.g. Science fiction scenarios• E.g. Driving in an unknown city
• Not very successful with current DL, which can”overfit” the training distribution(Lake & Baroni 2017)(Bahdanau et al & Courville ICLR 2019)
Fig. 1. (Bahdanau
People canrich concepts
from limited
data. (A and B) A single example of a new c
CLOSURE:et learnal & Courville
arXiv:1912.05783)
on CLEVR
the (i) classification of new examples, (ii) generation of new examples, (iii) parsing an object into6
generation of new concepts from related concepts. [Image credit for (A), iv, bottom: With permiss

CONSCIOUS PROCESSING HELPS HUMANS DEAL WITH OOD SETTINGS

Faced with novel or rare situations, humans call upon conscious attention to combine
on-the-fly the appropriate pieces of knowledge, to reason with them and imagine
solutions.
à we do not follow our habitual routines, we think hard to solve problems.

7

AGENT LEARNING NEEDS OOD
GENERALIZATION
Agents face non-stationarities
Changes in distribution due to• their actions• ESPECIALLY:• actions of other agents• different places, times, sensors,
actuators, goals, policies, etc.
Multi-agent systems: many changes in distribution
Ood generalization needed for continual learning

8

SYSTEM 1 VS. SYSTEM 2 COGNITION
2 systems (and categories of cognitive tasks):

Manipulates high-level /semantic concepts, which canbe recombined
combinatorially

System 1

System 2• Intuitive, fast, UNCONSCIOUS, 1-stepparallel, non-linguistic, habitual• Slow, logical, sequential, CONSCIOUS,
linguistic, algorithmic, planning, reasoning• Implicit knowledge• Explicit knowledge• Current DL• DL 2.0

9

IMPLICIT VS VERBALIZABLE KNOWLEDGE: UNDERLYING ASSUMPTIONSBEHIND VERBALIZABLE KNOWLEDGE• Most knowledge in our brain is implicit and not verbalizable (hence the explainability
challenge, even for humans)• Some of our knowledge is verbalizable and we can reason and plan explicitly with it• The concepts manipulated in this way are those we can name with language• Properties of joint distribution between these concepts and their change over time?
è clarify these assumptions as priors to be able to embed them in ML architectures andtraining frameworks which bridge perception and reasoning

10

Independent Mechanisms
Scholkopf et al 2012•

Knowledge can be decomposed in informationally independent pieces (modules, mechanisms)•

Any causal intervention normally affects just one such mechanism•

Any other factorization would not have that property•

Mechanisms can be used in many instances (e.g. same law of gravity)

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’generic rules’ across instances (as arguments), requiring variables & indirection• Meaning (e.g. grounded by an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains
12

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. grounded by an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains
13

CONSCIOUSNESS PRIORè SPARSE FACTOR GRAPH

Bengio 2017, arXiv:1709.08568• Property of high-level variables which we manipulate
with language:
we canpredict some given very few others•

E.g. "if I drop the ball, it will fall on the ground”• Disentangled factors 6=marginally independent,
e.g. ball & hand
• Prior: sparse factor graph joint distribution between
high-level variables• Inference involves few variables at a time, selected byattention mechanism and memory retrieval

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. grounded by an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains
15

WHAT CAUSAL VARIABLES?
• Physics: position and momentum of every particle• Computationally intractable• Scientists (and other humans) invent higher-level abstraction which makeit easier to model causal structure of the world• Can ML also do it?
• Human brains are complex machines• Hence it is feasible

16

AGENCY TO GUIDEREPRESENTATION LEARNING& DISENTANGLING(E. Bengio et al, 2017; V. Thomas et al, 2017; more recently see Kim et al ICML 2019)

Some factors (e.g. objects) correspond to ‘independently controllable’
aspects of the world• Maximize mutual information between intentions (goal-conditioned policies) andchanges in the state (trajectories), conditioned on the current state.

Can only be discovered by acting in the world• Control linked to notion of objects & agents• Causal but agent-specific & subjective: affordances

FROM PERCEPTION TO MODELLING THE WORLD AT THE SEMANTIC-LEVEL
Causal model

What are the right representations? Causal variables explaining the data
How to discover them (as a function of observed data)?
How to discover their causal relationship, the causal graph?
How are actions corresponding to causal interventions?
How is raw sensory data mapped to high-level causal variables
and how do high-level causal variables turn into low-level
actions and partial observations?

ENCODER / DECODER

Raw input/output

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. grounded by an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains
19

INDEPENDENT MECHANISMS: SPARSE CHANGE IN ABSTRACT LATENT SPACE
Sparse joint in abstract space

Localizedchange indistribution

Change may be drastic in pixel
space but tiny in semantic
space of causal variables
ENCODERENCODER

Raw input

Raw input

20

WHAT CAUSES CHANGES IN DISTRIBUTION?
Underlying physics: actions are localizedin space and time.

Hypothesis to replace iid assumption:changes = consequence of an intervention on few causes or mechanisms

Extends the hypothesis of (informationally) Independent Mechanisms (Scholkopf et al 2012)è local inference or adaptation in the right model

Change due
to intervention

21

COUNTING ARGUMENT:LOCALIZED CHANGE→OOD TRANSFER
Good representation of variables and mechanisms + localized change hypothesis→ few bits need to be accounted for (by inference or adaptation)→ few observations (of modified distribution) are required→ good ood generalization/fast transfer/small ood sample complexity

Change due
to intervention

22

CAUSAL INDUCTION FROM INTERVENTION DATA
Recovery of causal model from data
Observational data:○ Distinguishes causal models only up to Markov equivalence classIntervention data:○ What causal induction requires○ Most work assumes known-intervention data
○ Real world: Other agents or environment can intervene• Hence, interventions unknown○ How to handle unknown intervention?
• Infer itEXAMPLE: DISCOVERING CAUSE AND EFFECT= HOW TO FACTORIZE A JOINT DISTRIBUTION?
A Meta-Transfer Objective for Learning toDisentangle Causal Mechanisms• Learning whether A causes B or vice-versa• Learning to disentangle (A,B) from observed (X,Y)• Exploit changes in distribution and speed ofadaptation to guess causal direction

A

B

X

Y

ICLR 2020: A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms,
Bengio, Deleu, Rahaman, Ke, Lachapelle, Bilaniuk, Goyal, Pal
ArXiv:1901.10912

24

Experimental setup●

Consider two r.v. A and B, where A causes B.

●

The correct causal model decomposes as●●

Consider two distributions, where only p(A) changes
and p(B | A) remains unchanged (covariate shift).
○
A training distribution○
A transfer distribution
If we train a model using data from using the correct
decomposition, then adaptation on is fast because

Wrong knowledge factorization leads to poor transfer
●

●

●

●

With the wrong factorizationa change in p(A) influences all the modules.
○
Poor transfer: all the parameters need
to be adapted.

Correct causal structure:

This is the normal situation with standard
neural networks: every parameterparticipates to every relationship between all
the variables.
This causes catastrophic forgetting, poortransfer, difficulties with continual
learning or domain adaptation, etc.
Use the speed of adaptation as a way to findthe correct factorization.

••

Faster online adaptation to modified distribution = lower NLL regret
Effect of the correct factorization is most evident with only a few
samples from modified distribution

The Meta-Transfer Objective●

Quantify the speed of adaptation with the online likelihood

○
○
○

●

Adaptation with gradient ascent
Dobs is a large training dataset sampled fromDint is a small transfer dataset sampled from

Smooth parametrization of the causal structure○○

Structural (meta-)parameter
If, then the correct structure is recovered.

The Meta-Transfer Objective gradient

Can be optimized wrt.

with gradient descentExperimental results - Discrete variables

Tabular representation of marginals andconditionals of bivariate model

Each conditional is represented by a one-hidden-layer
MLP with one-hot inputs, softmax outputs+ Experiments on Linear Gaussian & Continuous multimodal variables (see Appendix).

Disentangling the causes●

Realistic settings: causal variables are not directly observed.

●

Need to learn an encoder which maps raw data to causal space.

●•

Consider both the encoder parameters and the causal graph structural parameters as metaparameters trained together wrt proposed meta-transfer objective.

Simplest possible scenario: linear mixing (rotating decoder) and unmixing (rotating decoder)Experimental results - Disentangling the causes●

Recovers the correct encoder parameter (left), up to permutation.

●

Simultaneously recovers causal direction (right).

DISCOVERING LARGER CAUSAL GRAPHS
Learning Neural Causal Models from Unknown Interventions
Ke, Bilaniuk, Goyal, Bauer, Scholkopf, Larochelle, Pal & Bengio 2019 arXiv:1910.01075• Learning small causal graphs, avoid exponential
explosion of # of graphs by parametrizing
factorized distribution over graphs• With enough observations of changes in
distribution: perfect recovery of the causal
graph without knowing the intervention;
converges faster on sparser graphs• Inference over the intervention:
faster causal discovery
32

HOW TO FACTORIZE AND LEARN THE BELIEF DISTRIBUTION OVER
GRAPHS
Learning Neural Causal Models from Unknown
Interventions Ke et al 2019 arXiv:1910.01075
Dependency Structure Discovery fromInterventions Ke et al 2020, submitted

Use neural networks to present/ learn causal models
Parameters:○
Structural parameters○
Functional parameters
Method overview:
Iterate:1.
Phase 1: Graph fitting on observational data
2.
Phase 2: Graph scoring on interventional data
3.
Phase 3: Credit assignment to structural parameters

MODEL ARCHITECTURE
Use N neural networks to represent causal graph with N variables
Each neural network models:○Who are the direct causal parents■
Structural parameters○
What is the relationship between them■
Functional parameters

EXPERIMENTAL RESULTS

EXPERIMENTAL RESULTSCONVERGENCE RATE FOR DIFFERENT GRAPHS
Institutdes algorithmes
d’apprentissage
de Montréal

DENSER GRAPHS ARE MORE CHALLENGING
Institutdes algorithmes
d’apprentissage
de Montréal

PARTIAL GRAPH RECOVERY
Institutdes algorithmes
d’apprentissage
de Montréal

Alarm [1].

[1]. I. A. Beinlich, H. J. Suermondt, R. M. Chavez, and G. F. Cooper. The ALARM Monitoring System: A Case Study with Two Probabilistic
Inference Techniques for Belief Networks. In Proceedings of the 2nd European Conference on Artificial Intelligence in Medicine, pages 247-256.
Springer-Verlag, 1989.
[2]. Preliminary model for barley developed under the project: "Production of beer from Danish malting barley grown without the use of pesticides"
by Kristian Kristensen , Ilse A. Rasmussen and others.

Barclay [2].

ABLATIONSGeneralizing to previously unseen interventions:

Institutdes algorithmes
d’apprentissage
de Montréal

Importance of predicting the intervention:

Importance of acyclic (LDAG) regularizer:

OBSERVING OTHER AGENTS• Can infants figure out causal structure in spite of being almost passive observers?
• Yes, if they exploit and infer the interventions made by other agents• Our approach does not require the learner to know what the action/intervention
was (but it could do inference over interventions)• But more efficient learning if you can experiment and thus test hypotheses about
cause & effectSOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. grounded by an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains
42

THOUGHTS, CONSCIOUSNESS, LANGUAGE• Consciousness: from humans reporting• High-level representations

language,<latexit sha1_base64="vV9KmI3DSqORXXjKQpnPOIDgOOM=">AAAEAXicjVNNb9NAEJ3UfJTw0RSOXCxSJC5ETqn4uFVw4cChSKSt1FSV7a6TVWyvtV63VFZP/AyucOCGuPJL+AfwL3gz3USUCtEdJZ59M+/tzNibVLmuXRT96CwFV65eu758o3vz1u07K73Vu9u1aWyqRqnJjd1N4lrlulQjp12udiur4iLJ1U4ye8XxnSNla23Kd+6kUvtFPCl1ptPYATrorYzfqMxZPZm62FpzfNDrR4NIVnjRGXqnT35tmdVOQWM6JEMpNVSQopIc/JxiqmF7NKSIKmD71AKz8LTEFZ1SF9wGWQoZMdAZ/ifY7Xm0xJ41a2GnOCXHz4IZ0kNwDPIsfD4tlHgjyoz+S7sVTa7tBM/EaxVAHU2B/o83z7wsj3txlNFz6UGjp0oQ7i71Ko1MhSsP/+jKQaECxv4h4hZ+Ksz5nEPh1NI7zzaW+E/JZJT3qc9t6JdUyXWWwI5lXoV0UEK/Bc7nVWAbeu+f/OYMquTovFdWzwU5WqhzlS20Q9gYjAymF5FcfK5pBq9CxhRajE6At7LPZAJr1IetAUnlHYbexufqOYtyH+YvnZmcV9K6ZJ4CqREtLsF4vODMrSu34QWvp4tv/6KzvT4YPhlsvN3ob77092KZ7tMDeoRv/xlt0mvaopHcj4/0iT4HH4Ivwdfg21nqUsdz7tG5FXz/DbGO0Lg=</latexit>• High-level concepts: meaning anchored in low-level
perception and action à tie system 1 & 2• Grounded high-level concepts
à better natural language understanding• Grounded language learning
BabyAI: (Chevalier-Boisvert and al ICLR 2019)

e.g.

43

CORE INGREDIENT FOR CONSCIOUS PROCESSING:
ATTENTION(Bahdanau et al ICLR 2015)• Focus on a one or a few elements at atime
• Content-based soft attention is convenient,can backprop to learn where to attend• Attention is an internal action, needs alearned attention policy (Egger et al 2019)
Attention•

Operating on unordered SETS of (key, value) pairs•

SOTA in NLP
44

FROM ATTENTION TO INDIRECTION• Attention = dynamic connection• Receiver gets the selected value• Value of what? From where?
Attention

à Also send ‘name’ (or key) of sender• Keep track of 'named’ objects: indirection• Manipulate sets of objects (transformers)

P.S. contrary to convnets doing object recognition, sequential tasks involving memory and attention typically involve amore difficult optimization problem, and fighting underfitting (including the issue of long-term dependencies)
45

RIMS: MODULARIZE COMPUTATION AND OPERATE ON SETS OF
NAMED AND TYPED OBJECTS
Recurrent Independent Mechanisms
Goyal et al 2019, arXiv:1909.10893

Multiple recurrent sparsely interacting
modules, each with their own
dynamics, with object (key/value pairs)
input/outputs selected by multi-headattention
Results: better ood generalization
Ongoing work: hierarchy, top-down
broadcasting, spatial layout of
modules
Builds on rich recent litterature on object-centric representations (mostly for images)

46

RESULTS WITH RECURRENT INDEPENDENT MECHANISMS• RIMs drop-in replacement for LSTMs in PPO baseline over all Atari games.
• Above 0 (horizontal axis) = improvement over LSTM.

47

FROM ATTENTION TO CONSCIOUSNESS
C-word not taboo anymore in cognitive neuroscience
Global Workspace Theory(Baars 1988++, Dehaene 2003++)• Bottleneck of conscious processing• WHY A BOTTLENECK?
• Selected item is broadcast, stored in short-term
memory, conditions perception and action• System 2-like sequential processing, conscious
reasoning & planning & imagination• Can only run 1 simulation at a time, unlike a movie,
only few abstract concepts involved at each step48

Modules + Global Workspace
1. Parallel, competing specialists

2. Write to shared workspace

3. Broadcast workspace contents

Adding ashared global
workspacesimilar to the
GWT greatly
improves
RIMs

Table 2: FourRoom Navigation Task: Success Rate of the
without
GW
Figure
1:
Step
1:
an
ensembleofspecialists
doing
their
own
default
processing;ataparticular
proposed method vs. the baselines on the FourRoom navigation environment illustrated on thetime-step,
right, with theagent independingupon the input, a subset of the specialists becomes active. Step 2: the active
red, its field of visibility greyed out, and the object to get in
specialists get to write information in a shared global workspace. Step 3: the contents of thegreen.
with GW

Tracking
bouncing balls
0.72 ± 0.02 0.67 ± 0.05 0.62 ± 0.02
0.96 ± 0.02
36
Distributed specialists. From a computational perspective, articulated multi-component
archi(Goyal
et al 2020,
4.3 BabyAI: FourRoom Navigation
Task composed of sparsely interacting specialists show desirable scaling properties (e.g., more
37
tecturessubmitted)
RIMs

246
247
248

workspace are broadcast to all specialists.

RMC

LSTM

Ours

38 on
specialists
seamlessly
be added),increased
We evaluate the proposed method
the classical canfour-room
reinforcement
learning environment
as robustness (the system can tolerate the removal ofshown in fig. 4.3 from the MiniGrid
environmentspecialists),
Chevalier-Boisvert
al. [2018]. The
agent must
39
individual
andetefficiency(information
is processed predominantly locally, reducing theortly.
es operation
a file to perform
one-step updateof itsstatelayer ofGRU
or LSTM
units, to both external input andiscyclea ensuresthat files
canupdatetheirstate
in responsetheinput statesignalcomprised
received. The
weightparameters
neededperform thisinternalof allthe files
contents.
Thisto updating
is an extra wrapper around thewe will denote generically as ✓, are not—as in a standard GRU or LSTM—
dinaryupdate that takes place in a GRU or LSTM layer. It provides additional flexibility in thatayer but rather are provided externally. Each schema j is nothing more than a
may simply
remaininto
dormant
andSCOFFfully ignore
an input,attention(2) files can switch their dynamics
rsfiles✓ j whichcan be plugged
this layer.
uses a key-value
m one input
to the next
conditioned
their (parameters).
internal state, (3) files are modular in that they do not
performsoft selectionof theappropriateon
schemammunicateSeparate
with one anotherexceptvia state-dependent
selective
message passing.
valuesrules(schemas)
may also seek information fromother (slots)
files, againfrom
using a key-value
attention
query each other file, matching its query to keys provided by each otherObject Schema 1 Schema 2 Schema 3
electing the best matching files to incorporate the values provided by the
Files
PacmanNormalScaredfiles.
Ghost
Ghost
SchemaSCHEMAS AND SLOTSTop Frame

SchemaSchemaSCOFF
AXsures that files can update their state in response
to both external input andB
X
ised of all the files contents. This updating is an extra wrapper around theCX
es place in a GRU or LSTM layer. It provides additional flexibility in thatD
X
main dormant and fully ignore an input, (2) files can switch their dynamics FigureE2: Our SCOFF model.
X
BottomFrame
xt conditioned on their internal state, (3) files are modular in that they do not Schemata are setsofparamAX
nother except via state-dependent selective message passing.
eters that specify the dynamPrediction

Objectfile

Objectfile

Objectfile

SCOFF
Visualinput

Prediction
Objectfile

Objectfile

Objectfile

Objectfile

Visualinput

Objectfile

B
ics of Cobjects. Object files
Dare active
modules that mainE

tain the time-varying state ofan object, seek informationfrom the input, and selectschemata for updating.

Figure 2: Our SCOFF model.
37
Schemata are sets of param38
eters that specify the dynam39
3
ics of objects. Object files40
are active modules that main41
tain the time-varying state of42
an object, seek information43
from the input, and select44
45
schemata for updating.
46

3

47
48
49
50

X
X
X
X

Figure 1: As a motivating example, we show
two successive framesof the game PacManand show how procedural and declarative knowledge must be dynamicallyfactorized. The “B” ghost
has a persistent object
file (with its location andvelocity), yet its procedure mostly depends onwhether it is in its scaredor normal routine.

We propose a method of separately representing knowledge about the state of a particular object
token—the information that is maintained in an object file—and abstract knowledge about the
dynamics of the object type. We refer to this latter type of knowledge as a schema (plural schemata), aterm which in the cognitivemeans a frameworkfor organizingdeclarativecomplex knowledge.
Objectscience
Filesliteratureand Schemata:factorizing
andHere, we specifically use schema to refer to procedural knowledge—knowledge about the dynamics
proceduralin dynamicalsystemsof state evolution—in
contrast to the knowledge
declarative knowledge
in an object framethat describes the states
themselves (FigureLamb,1). The combinationof files andMozer,
schemata Beaudoin,
is sufficient to predict
futureGoyal, Blundell,
Levine& states
Bengio,of visual environments, critical for planning and goal-seeking behavior. For the sake of simplifying
submitted,
terminology, we willrefer to object 2020
files as ‘files’.
Object-oriented programming (OOP) provides a way to think about the relationship between files
and schemata. In OOP, each object is an instantiation of an object class and it has a self-containedcollection of variables whose values are specific to that object and methods that operate on all
instances of the same class. The relation between objects and methods mirrors the relationshipbetween our files and schemata. In both OOP and our view of visual cognition, a key principle is50

to be performed. The figure shows the affinity of each file to use a particular schemata. Each row
corresponds to a particular file, and column represents a particular schemata (dark color shows highaffinity of a file towards a particular schemata). As shown in the figure, the active files trigger Schema
1 when an operand is encountered, and Schema 2 when a "Null" element is encountered. (b) Here,
we have a single object file, and that can follow three different dynamics. We found that our method
is able to learn these 3 different modes once it’s passed an initial phase of uncertainty.

SCHEMAS AND SLOTS: RESULTS
Separate values (slots) from rules (schemas)
Number of Values

LSTM

RIMS

SCOFF

2
3
4
5
8
9
10

0.87311.30171.67892.03344.8872
7.373011.3595

0.00070.00090.00140.00450.05550.19580.89040.00050.00070.00130.00300.01910.03790.0539

Table 1: Adding Task: Mean test set error on 200 length
sequences with number of numbers to add varying among{2, 3, 4, 5, 8, 9, 10}. The models are trained to add a mixture
Figure 4: Object filesof two and four numbers from sequences of length 50.
and schema binding on
RL task (top), Bouncing
Balls Dataset (bottom).

Experiments on Baby AI RL tasks show that slots
194
results andtheobjects
code in the(likeSupplementary
Material,
and we do plan to release theObject
code. ForFiles
more and Schemata: factorizing declarative andspecialize
on
a
key)
and
schematas
195
details please refer to the appendix section A.
proceduralknowledge
indynamical
systems
specialize on procedures (like opening a door) or object
Lamb,
Goyal,
Blundell,Mozer,
Beaudoin,
Levine&Bengio,196
Baselines:We
comparethe
proposedmethodwithRMC,
amemorybased
relational
recurrent
detection (like being triggered when the key is in view).
197
model with attention between saved memory and hidden states [13]. We also compare
the proposed2020
submitted,
199

method to Recurrent Independent Mechanisms (RIMs), a modular memory based on a single layered
recurrent model with attention modulated input and communication between modules [4].

200

4.2 Different schemata specializing over temporal patterns: Qualitative dynamics

198

51

Motivation: Top-Down and Bottom-up Feedback
Motivation: Top-Down and Bottom-up Feedback

Learning to Combine Top-Down and• Perception involves both stimuli and priors.
Bottom-Up Signals • Top-Down: expectations, beliefs about what will be observed• The Mittal,
H and theare visually
identical
are perceived• Bottom-Up:
the observed
content Lajoie, Michael
Sarthak
AlexALamb,
Anirudh
Goyal,but
VikramVoleti,
Murray Shanahan,
Guillaumedi↵erently.
Mozer, Yoshua Bengio, ICML 2020

Figure 1: The
image
can theappearas a duck or asignal
rabbit depending
Properly combining the contextual information
andsameprior
withbottom-up
prior conception
2
can be useful even at the lower levels of perceptual processing
and changes thelower-level interpretations.

LearningtoCombine
TopDown
andBottomUp
Signals
ICML’2020

Figure 2: Proposed architecture. Bidirectional connections to provide
top-down information (red arrows); Sparse Activation of modules (dark blue active); Communication within each layer (green arrows)

Experiments: Bouncing Balls & Moving MNIST

Learning to Combine Top-Down and Bottom-Up Signals

Figure 12: Performance on Bouncing
Balls task. The task has multiple balls
bouncing around so each ball has its
own independent dynamics. They
react only through collisions. Curtain
provides examples with occlusion.

Figure 13: Performance on
MovingMNIST task. It consists of
Useof Key-Valueattention
to integrate
MNIST
digits bouncing
aroundsimilar top-down and bottom-up
informationin context-dependentand dynamic way and to infer a sparseto the bouncing
balls task

relationship between the incoming observations and the set-structured state
representation.
21

Experiments: sCIFAR10

Learning to Combine Top-Down and Bottom-Up Signals

Figure 10: We train sequential models on CIFAR10 where they see one pixel ata time. The models are trained on 16 ⇥ 16 resolution and then evaluated on
19 ⇥ 19, 24 ⇥ 24 and 32 ⇥ 32 resolutions. We see that BRIMs generalize very
well across changes in sequence length.

Learning to Combine Top-Down and Bottom-Up Signals

Figure 11: We replace LSTM with BRIMs in an RL agent trained with PPO
and show that BRIMs outperform their competitors on a set of randomly
chosen Atari games.

Noisy Inputs: more attention to top-down signals

Figure 14: Attention given to input (left), zero vector (middle), and top-level(right), as a function of noise injected into CIFAR images. We see that as theamount of noise increases in the image, the model’s reliance on higher level
information increases. This is in line with our hypothesis that top-down
modulation should be queried more in case of uncertainty.

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. causal graph or an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains58

ateshortly.
state
comprised of all the files contents. This updating is an extra wrapper around thegdatecausesthata filetakes
to perform
a one-stepstate layerlayer.
of GRU orunits, additional flexibility in thatplacein a update
GRUoforits LSTM
It LSTM
provideson the input signal received. The weight parameters needed to perform thisyedsimplyremain dormant and fully ignore an input, (2) files can switch their dynamics
hich we will denote generically as ✓, are not—as in a standard GRU or LSTM—
putto theon theirstate,(3)files
the layer
but next
rather conditioned
are provided externally.
Eachinternal
schema j isnothingmorethanarea modular in that they do notameters
whichanothercan be plugged
intovia
this state-dependent
layer. SCOFF uses a key-value
attention
te with✓jone
exceptselective
message passing.
m to perform soft selection of the appropriate schema (parameters).
Separate values (slots) from rules (schemas)
files may also seek information from other files, again using a key-value attention
m to query each other file, matching its query to keys provided by each otherObject Schema 1 Schema 2 Schema 3
oft selecting the best matching files to incorporate the values provided by the
Files
PacmanNormalScaredding files.
Ghost
Ghost
SchemaSCHEMAS AND SLOTSTop Frame

SCOFF

SchemaSchemale ensures that files can update their state in response to both external input and A
X
omprised of all the files contents. This updating is an extra wrapper around the BXat takes place in a GRU or LSTM layer. It provides additional flexibility in that CXXy remain dormant and fully ignore an input, (2) files can switch their dynamics D
Figure2:
Our
SCOFF
model.
X
he next conditioned on their internal state, (3) files are modular in that they do not E
BottomSchemata
areFramesets of paramone another except via state-dependent selective message passing.

Objectfile
SCOFF

Objectfile

SchemaSchemaObjectfile

SchemaVisualinput
Prediction
Objectfile

Objectfile

Objectfile

Objectfile

Visual
input

A
B
C
D
E

eters that specify the dynamX
Prediction
ics of objects. ObjectX
files
X
Objectare active modules that Xmainfile
tain the time-varying state ofan object, seek information
Figure 2: Our SCOFF model.
from the input, and selectSchemata are sets of paramWe propose
a methodof separately representing knowledge about the state of a particular object
schemata
for updating.
eters that specify the37dynam-

38 filestoken—the information
thatFiles
is maintainedin an objectfactorizing
file—and abstract
knowledge about
Objectand Schemata:declarative
and theics of objects. Object
39 maindynamics of the object type. We refer to this latter type of knowledge as a schema (plural schemata), a
are active modules thatprocedural knowledge in dynamical systems
tain the time-varying40statetermof which in the cognitive science literature means a framework for organizing complex knowledge.
41
Here, we specificallyuse schema
to refer
to proceduralknowledge—knowledge
about thedynamics
3an object, seek information
Goyal,
Lamb,
Gampa,
Blundell,Mozer, Beaudoin,
Levine&of state evolution—in contrast to the declarative knowledge in an object frame that describes the statesfrom the input, and42 selectsubmitted,
2020
schemata for updating.
43
themselves (FigureBengio,1). The combinationof filesand schemata is sufficient to predict future states
44

3

X

Figure 1: As a motivating example, we show
two successive framesof the game PacManand show how procedural and declarative knowledge must be dynamicallyfactorized. The “B” ghost
has a persistent object
file (with its location andvelocity), yet its procedure mostly depends onwhether it is in its scaredor normal routine.

45
46
47

of visual environments, critical for planning and goal-seeking behavior. For the sake of simplifying
terminology, we will refer to object files as ‘files’.

Object-oriented programming (OOP) provides a way to think about the relationship between files59
and schemata. In OOP, each object is an instantiation of an object class and it has a self-contained

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. causal graph or an encoder) is stable & robust wrt changes in
distribution• Credit assignment is only over short causal chains
60

Fast and slow weights••

Slow weights○ Ground-truth causal graph

Fast weights○ Adapt to local (intervention) changes

term dependencies and connectivity patterns of the modules. During thisrecurrent
modulesarenotchanged.
Since
this
phase
looks
at
much
long
Meta-Learning =of updates to the attention parameters is lower, and the updates happen

Multiple Time Scales of Learning

Individual Episodes

Meta EpisodeMeta Episode

Fast Inner Updates

Slow Outer Updates

Fast
updateIndividual Episodes

Fast Inner Updates

Slow Outer Updates

Fast
updateRBG input
framesInner loop for fastlearning of moduleparameters

Outer loop for meta
learning of attention
parameters

We describe the different components of the model and the two learningMeta-Attention Networks
RIMs +
metalearning
Fast learning:
modules
Slow learning:
attentionmechanism

Experiments on
Baby AI tasks(Kanika Madanet al 2020,
submitted)

SOME SYSTEM 2 INDUCTIVE PRIORS
all inspired by human cognition• Sparse factor graph in space of high-level semantic variables• Semantic variables are causal: agents, intentions, controllable objects• Distributional changes due to localized causal interventions (in semantic space)• Simple mapping between high-level semantic variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (as arguments), requiring variables & indirection• Meaning (e.g. grounded by an encoder) is stable & robust wrt changes in distribution• Credit assignment is only over short causal chains64

Sparse Attentive Backtracking
Rosemary Ke, Anirudh Goyal, Olexa Bilaniuk, Jonathan Binas, Mike Mozer, Yoshua Bengio,
NeurIPS 2018
The attention mechanism of the associative memory picks up past memories
which match (associate with) the current state, maybe be an alternative to BPTT for learning
very long-term dependencies.

Causal reasoning over events factor graph• Node of graph = event at particular time, involving a small set of variables• Factor = causal mechanism• Directed edges: from past to future, causal direction
as

A

ht

6J
Time

LEARNING TO REASON
●

●

●

●

Reasoning and planning is inference and is inherently
computationally expensive
Brains do not use exhaustive search but instead generate
good candidates
Conscious processing seems involved in evaluating them for
global coherence across the brain’s modules
Attention mechanisms are part of the reasoning policy,
converting declarative knowledge into selective
computations for inference and decision-making

67

CONTRAST WITH THE SYMBOLIC AI PROGRAM
Avoid pitfalls of classical AI rule-based symbol-manipulation• Need efficient large-scale learning• Need semantic grounding in system 1 (implicit knowledge)• Need distributed representations for generalization• Need efficient = trained search (also system 1)• Need uncertainty handling
But want• Systematic generalization• Factorizing knowledge in small exchangeable pieces• Manipulating variables, instances, references & indirection
68language

CONSCIOUSNESS PRIORS

reasoning
consciousness

attention
indirection &variables

system 2
systematic
generalization
modularity/
compositionality

meta-learningcausality

out-of-distribution
generalization
non-stationarity• Sparse factor graph in space of high-level
semantic variables• Semantic variables are causal: agents,
intentions, controllable objects• Simple mapping between high-level semantic
variables / thoughts and words / sentences• Shared ’rules’ across instance tuples (asarguments), w/ variables & indirection

agency• Distributional changes due to localized causal
interventions (in semantic space)• Meaning (e.g. grounded by an encoder) is stable
multi-agent& robust wrt changes in distribution
interactions• Credit assignment is only over short causal
chains69

We have a responsibility70

ML going out of labs, into society• ML is not just a research question anymore• ML-based products are being designed and deployedè new responsibility for AI scientists and engineers
à wisdom race, as power of technology increases

71

THANK YOU!