  While auxiliary tasks play a key role in shaping the representations learnt
by reinforcement learning agents, much is still unknown about the mechanisms
through which this is achieved. This work develops our understanding of the
relationship between auxiliary tasks, environment structure, and
representations by analysing the dynamics of temporal difference algorithms.
Through this approach, we establish a connection between the spectral
decomposition of the transition operator and the representations induced by a
variety of auxiliary tasks. We then leverage insights from these theoretical
results to inform the selection of auxiliary tasks for deep reinforcement
learning agents in sparse-reward environments.
