  While routing in wireless networks has been studied extensively, existing
protocols are typically designed for a specific set of network conditions and
so cannot accommodate any drastic changes in those conditions. For instance,
protocols designed for connected networks cannot be easily applied to
disconnected networks. In this paper, we develop a distributed routing strategy
based on deep reinforcement learning that generalizes to diverse traffic
patterns, congestion levels, network connectivity, and link dynamics. We make
the following key innovations in our design: (i) the use of relational features
as inputs to the deep neural network approximating the decision space, which
enables our algorithm to generalize to diverse network conditions, (ii) the use
of packet-centric decisions to transform the routing problem into an episodic
task by viewing packets, rather than wireless devices, as reinforcement
learning agents, which provides a natural way to propagate and model rewards
accurately during learning, and (iii) the use of extended-time actions to model
the time spent by a packet waiting in a queue, which reduces the amount of
training data needed and allows the learning algorithm to converge more
quickly. We evaluate our routing algorithm using a packet-level simulator and
show that the policy our algorithm learns during training is able to generalize
to larger and more congested networks, different topologies, and diverse link
dynamics. Our algorithm outperforms shortest path and backpressure routing with
respect to packets delivered and delay per packet.
