The ability to plan and execute goal specific actions in varied and unseen environments settings is a central requirement of intelligent agents. In this paper, we explore how an agent can be equipped with an internal model of the dynamics of the external world, and how it can use this model to plan novel actions by running multiple internal simulations ("visual imagination"). Our models directly process raw visual input, and use a novel object-centric prediction formulation based on visual glimpses centered on objects (fixations) to enforce translational invariance of the learned physical laws. The agent trains itself through random interaction with a collection of different environments, and the resulting model can then be used to plan goal-directed actions in novel environments that were previously never encountered by the agent. We demonstrate that our agent can accurately plan actions for playing a simulated billiards game, which requires pushing a ball into a target position or into collision with another ball.
Fragidaki et al. REF integrate the model of external dynamics within the agent to play simulated billiards games.
Learning Visual Predictive Models of Physics for Playing Billiards
ICLR 2016
arXiv: Computer Vision and Pattern Recognition
Computer Science