  Although model-based and model-free approaches to learning the control of
systems have achieved impressive results on standard benchmarks, generalization
to task variations is still lacking. Recent results suggest that generalization
for standard architectures improves only after obtaining exhaustive amounts of
data. We give evidence that generalization capabilities are in many cases
bottlenecked by the inability to generalize on the combinatorial aspects of the
problem. Furthermore, we show that for a certain subclass of the MDP framework,
this can be alleviated by neuro-algorithmic architectures.
  Many control problems require long-term planning that is hard to solve
generically with neural networks alone. We introduce a neuro-algorithmic policy
architecture consisting of a neural network and an embedded time-dependent
shortest path solver. These policies can be trained end-to-end by blackbox
differentiation. We show that this type of architecture generalizes well to
unseen variations in the environment already after seeing a few examples.
