  Biological agents have adopted the principle of attention to limit the rate
of incoming information from the environment. One question that arises is if an
artificial agent has access to only a limited view of its surroundings, how can
it control its attention to effectively solve tasks? We propose an approach for
learning how to control a hard attention window by maximizing the mutual
information between the environment state and the attention location at each
step. The agent employs an internal world model to make predictions about its
state and focuses attention towards where the predictions may be wrong.
Attention is trained jointly with a dynamic memory architecture that stores
partial observations and keeps track of the unobserved state. We demonstrate
that our approach is effective in predicting the full state from a sequence of
partial observations. We also show that the agent's internal representation of
the surroundings, a live mental map, can be used for control in two partially
observable reinforcement learning tasks. Videos of the trained agent can be
found at https://sites.google.com/view/hard-attention-control.
