  In this paper, we consider the problem of leveraging textual descriptions to
improve generalization of control policies to new scenarios. Unlike prior work
in this space, we do not assume access to any form of prior knowledge
connecting text and state observations, and learn both symbol grounding and
control policy simultaneously. This is challenging due to a lack of concrete
supervision, and incorrect groundings can result in worse performance than
policies that do not use the text at all. We develop a new model, EMMA (Entity
Mapper with Multi-modal Attention) which uses a multi-modal entity-conditioned
attention module that allows for selective focus over relevant sentences in the
manual for each entity in the environment. EMMA is end-to-end differentiable
and can learn a latent grounding of entities and dynamics from text to
observations using environment rewards as the only source of supervision. To
empirically test our model, we design a new framework of 1320 games and collect
text manuals with free-form natural language via crowd-sourcing. We demonstrate
that EMMA achieves successful zero-shot generalization to unseen games with new
dynamics, obtaining significantly higher rewards compared to multiple
baselines. The grounding acquired by EMMA is also robust to noisy descriptions
and linguistic variation.
