  The capability of imagining internally with a mental model of the world is
vitally important for human cognition. If a machine intelligent agent can learn
a world model to create a "dream" environment, it can then internally ask
what-if questions -- simulate the alternative futures that haven't been
experienced in the past yet -- and make optimal decisions accordingly. Existing
world models are established typically by learning spatio-temporal regularities
embedded from the past sensory signal without taking into account confounding
factors that influence state transition dynamics. As such, they fail to answer
the critical counterfactual questions about "what would have happened" if a
certain action policy was taken. In this paper, we propose Causal World Models
(CWMs) that allow unsupervised modeling of relationships between the intervened
observations and the alternative futures by learning an estimator of the latent
confounding factors. We empirically evaluate our method and demonstrate its
effectiveness in a variety of physical reasoning environments. Specifically, we
show reductions in sample complexity for reinforcement learning tasks and
improvements in counterfactual physical reasoning.
