  Professional race car drivers can execute extreme overtaking maneuvers.
However, conventional systems for autonomous overtaking rely on either
simplified assumptions about the vehicle dynamics or solving expensive
trajectory optimization problems online. When the vehicle is approaching its
physical limits, existing model-based controllers struggled to handle highly
nonlinear dynamics and cannot leverage the large volume of data generated by
simulation or real-world driving. To circumvent these limitations, this work
proposes a new learning-based method to tackle the autonomous overtaking
problem. We evaluate our approach using Gran Turismo Sport -- a world-leading
car racing simulator known for its detailed dynamic modeling of various cars
and tracks. By leveraging curriculum learning, our approach leads to faster
convergence as well as increased performance compared to vanilla reinforcement
learning. As a result, the trained controller outperforms the built-in
model-based game AI and achieves comparable overtaking performance with an
experienced human driver.
