  Accuracy and generalization of dynamics models is key to the success of
model-based reinforcement learning (MBRL). As the complexity of tasks
increases, learning dynamics models becomes increasingly sample inefficient for
MBRL methods. However, many tasks also exhibit sparsity in the dynamics, i.e.,
actions have only a local effect on the system dynamics. In this paper, we
exploit this property with a causal invariance perspective in the single-task
setting, introducing a new type of state abstraction called
\textit{model-invariance}. Unlike previous forms of state abstractions, a
model-invariance state abstraction leverages causal sparsity over state
variables. This allows for generalization to novel combinations of unseen
values of state variables, something that non-factored forms of state
abstractions cannot do. We prove that an optimal policy can be learned over
this model-invariance state abstraction. Next, we propose a practical method to
approximately learn a model-invariant representation for complex domains. We
validate our approach by showing improved modeling performance over standard
maximum likelihood approaches on challenging tasks, such as the MuJoCo-based
Humanoid. Furthermore, within the MBRL setting we show strong performance gains
w.r.t. sample efficiency across a host of other continuous control tasks.
