Causal Inductive Synthesis Corpus
Abstract
We introduce the Causal Inductive Synthesis Corpus (CISC) – a manually constructed collection of interactive domains.
CISC domains abstract core causal concepts present in real world mechanisms and environments.
We formulate two synthesis challenges of causal model discovery: the passive discovery of a model of a CISC domain from observed data, and active discovery while interacting with the domain.
CISC problems are expressed in A UTUMN , a Turing-complete programming language for specifying causal probabilistic models.
A UTUMN allows succinct expression for models that vary dynamically through time, respond to external input, have internal state and memory, exhibit probabilistic non-determinism, and have complex causal dependencies between variables.
Introduction
Young children engage in forms of intuitive scientiﬁc discovery, building structured causal theories of their environment [13, 3, 2] using many of the principles that underpin professional science.
Signiﬁcant progress has been made in modelling many of these principles – in particular Bayesian inference over structured representations [15, 5, 12], discovery of causal models from observational and interventional data [14, 4, 9], and the optimal design of experiments [11, 6, 10, 7].
Nevertheless, automatic discovery of models of realistic phenomena from observation and interaction remains largely out of reach.
The objective of this contribution is to facilitate progress towards automatic scientiﬁc discovery, ﬁrst by introducing a representation of causal models that is expressive enough to succinctly capture the complexities of real world phenomena, and second by presenting a corpus of domains and accompanying benchmark challenge.
Most real-world causal mechanisms are complex.
They often possess internal state, have time-varying behaviour, and are composed of both continuous and discrete components with complex logical and algorithmic relationships between them.
For instance, a typical microwave will heat only if the door is closed, a duration has been keyed in, and the start command has been pressed.
Radiation then causes a continuous increase in temperature, while the display discretely counts down the remaining time.
These properties are difﬁcult or impossible to represent them with traditional modelling formalisms, such as causal graphical models.
Complex mechanisms can, on the other hand, be easily encoded as programs – indeed many (such as the internal logic of a microwave) are literally programs.
We introduce a functional reactive language [1, 8] called A UTUMN for expressing causal probabilistic models.
Functional reactive programming languages augment functional languages – which are oriented around deﬁning pure mathematical functions – with primitive support for temporal events.
A UTUMN allows succinct expression of models that vary dynamically through time, respond to external input, have internal state and memory, exhibit probabilistic non-determinism, and have complex causal dependencies between variables.
Within a programmatic representation of causal models, such as A UTUMN , scientiﬁc discovery is then in part a problem of program synthesis.
However, unlike conventional synthesis formulations, causal direction is important and must be inferred.
Moreover a complete model of scientiﬁc discovery cannot presuppose that the necessary data is given; what experiments to run and what data to collect are integral parts of the problem.
We introduce the Causal Inductive Synthesis Corpus (CISC), a suite of interactive models designed for causal discovery.
CISC domains are distillations of mechanisms, modelling causal concepts such as circuits, magnetism, disease propagation, ﬂuids, and foraging.
CISC problems resemble simple video games but have no notion of external reward.
Instead, CISC provides a framework for deﬁning challenging passive and active inductive synthesis problems.
The Autumn Language
A UTUMN is designed to express models that vary as a function of time or external input.
Many constructs are standard.
The expression x = val binds the symbol x to the value val .
Local values are bound using let .
Values have types; x : Int denotes that x is of type Int .
Functions are values with function types.
f : Int → Bool denotes the type of functions the Int to Bool .
Sequences Values in an A UTUMN program represent sequences that vary with time.
A value v at time t may be (i) time invariant, i.e., v t =
c for some constant c , (ii) stateless and time varying, i.e, v t = f ( t ) for some function f , or (iii) stateful / recurrent sequences deﬁned in terms of previous values, i.e., v t = f
( v t − 1 ) .
In A UTUMN , sequences are speciﬁed by deﬁning two expressions in a recurrence relation: the initial value and the value as a function of previous values.
These are constructed using a primitive language pattern init expr1 next expr2 , which is sufﬁcient to express the aforementioned kinds of sequences: Variables x, y, z ∈ Var Type τ ::=
Int | Bool | Real | τ 1 → τ 2 | Ω
Term t ::= n | b | r | ⊥ | x | λx : τ.t | if t 1 then t 2 else t 3 | t 1 ⊕ t 2 | t 1 t 2 | let x = t 1 in t 2 | (temporal terms) on t 1 t 2 | init t 1 next tprob terms rand
Time invariant values are simply constants v = init 3 next 3 .
This can also be expressed more succinctly as simply v = 3 .
Stateless, time-varying values are simply functions of time : v = init iseven time next iseven time
This can also be expressed more succinctly as simply applying a function to an existing time varying value, which is interpreted pointwise: v = iseven time 3.
Stateful and recurrent sequences refer to previous values in their deﬁnition.
The simplest example is perhaps time itself, which need not be deﬁned as a primitive, but can be expressed using the primitive prev x which returns the value of x at the previous timestep: time = initnext ( prev time)
+[-
-]
A more complex example is a value that evolves according to the Fibonnaci sequence: fib = initnext if time =
=thenelse ( prev fib) + ( prev prev fib)
Events A second way to specify temporal events is using the construct on , with the pattern on event intervention .
An event is any sequence of type Bool , and an intervention is a modiﬁcation to a value.
In the following example, if a click occurs at a time later than 5, the value of x will reset to 0:
[-init 0 next ( prev x) + 1on click & (time > 5)x = 
-][Equation]
The primitives on , init , next , prev in combination with a standard library enable succinct expression of a wide variety of models (see Figure 1).
The Causal Inductive Synthesis Corpus
The Causal Inductive Synthesis Corpus (CISC) is a collection of environments expressed within the
A UTUMN language.
CISC problems are abstract models of mechanisms and environments.
Speciﬁcation Let L denote the set of all A UTUMN models.
CISC is a dataset D
= ( m 1 , m 2 , . . .
, m N ) of N A UTUMN models, i.e., m i ∈ L .
For each model m ∈ D , there is also a collection of M test m test trajectories T test m = ( τ 1 , τ 2 , . . .
, τ M test m ) and M train e train trajectories T test m = ( τ 1 , τ 2 , . . .
, τ M train m ) .
A trajectory is a pair ( a , o ) , where a and o are ﬁnite sequences (of identical length) of actions and observations respectively.
The action space A = N 2 ∪{↑ , ↓ , ← , → , skip , stop } allows for selecting a grid-cell, pressing an arrow, performing no action, or stopping a simulation.
The observation space O = C W × H is a colored grid of cells, where W (width), H (height) are constants and C is a set of colors.
Passive Discovery:
The passive inductive synthesis problem is to produce a synthesizer s that maps a set of trajectories T train m produced from a ground truth A UTUMN model m onto a hypothesis
A UTUMN model ˆ m = s ( T train m ) where m, ˆ m ∈ L .
The score of a hypothesis ˆ m is a measure of the degree to which it matches m on the test trajectories.
Recalling that A UTUMN programs may be probabilistic, let sim denote a stochastic simulation function such that sim ( m, a ) is a random variable over observations, the score of m is marginal likelihood averaged over T test m :
[Equation]
The score of a synthesizer s is then the average score over D :
[Equation]
m
Active Discovery
In contrast to the passive case, in active discovery the observational data is not given and must be produced by an active agent.
The active inductive synthesis problem is to produce a pair ( π, σ ) where π : O × Φ → A × Φ is a policy with internal memory Φ , and σ is stateful synthesizer.
The agent interacts with a model, producing observational data until a stop action is performed.
At this point a hypothesis model ˆ m = σ ( T, φ ) is produced as function of the internal state φ ∈ Φ of the agent and the trajectory T =
( a , o ) it has observed.
For the sake of evaluation consistency we force the domains to be deterministic by ﬁxing the random seed, and hence T is a function of a model m and π .
The score of a pair ( π, σ ) on m is then the score of the A UTUMN program it produces on completion.
[Equation]
Implementation Computing marginal probabilities in A UTUMN programs is in general intractable, making computation of score functions a challenge.
We approximate these scores using importance sampling.