  Traffic congestion is a major challenge in modern urban settings. The
industry-wide development of autonomous and automated vehicles (AVs) motivates
the question of how can AVs contribute to congestion reduction. Past research
has shown that in small scale mixed traffic scenarios with both AVs and
human-driven vehicles, a small fraction of AVs executing a controlled
multiagent driving policy can mitigate congestion. In this paper, we scale up
existing approaches and develop new multiagent driving policies for AVs in
scenarios with greater complexity. We start by showing that a congestion metric
used by past research is manipulable in open road network scenarios where
vehicles dynamically join and leave the road. We then propose using a different
metric that is robust to manipulation and reflects open network traffic
efficiency. Next, we propose a modular transfer reinforcement learning
approach, and use it to scale up a multiagent driving policy to outperform
human-like traffic and existing approaches in a simulated realistic scenario,
which is an order of magnitude larger than past scenarios (hundreds instead of
tens of vehicles). Additionally, our modular transfer learning approach saves
up to 80% of the training time in our experiments, by focusing its data
collection on key locations in the network. Finally, we show for the first time
a distributed multiagent policy that improves congestion over human-driven
traffic. The distributed approach is more realistic and practical, as it relies
solely on existing sensing and actuation capabilities, and does not require
adding new communication infrastructure.
