  Learning an accurate model of the environment is essential for model-based
control tasks. Existing methods in robotic visuomotor control usually learn
from data with heavily labelled actions, object entities or locations, which
can be demanding in many cases. To cope with this limitation, we propose a
method that trains a forward model from video data only, via disentangling the
motion of controllable agent to model the transition dynamics. An object
extractor and an interaction learner are trained in an end-to-end manner
without supervision. The agent's motions are explicitly represented using
spatial transformation matrices containing physical meanings. In the
experiments, our method achieves superior performance on learning an accurate
forward model in a Grid World environment, as well as a more realistic robot
control environment in simulation. With the accurate learned forward models, we
further demonstrate their usage in model predictive control as an effective
approach for robotic manipulations.
