RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based
  Reinforcement Learning
Authors: name: Hao Gao; name: Shaoyu Chen; name: Bo Jiang; name: Bencheng Liao; name: Yiang Shi; name: Xiaoyang Guo; name: Yuechuan Pu; name: Haoran Yin; name: Xiangyu Li; name: Xinbang Zhang; name: Ying Zhang; name: Wenyu Liu; name: Qian Zhang; name: Xinggang Wang
Published: 2025-02-18 18:59:21

Abstract: Existing end-to-end autonomous driving (AD) algorithms typically follow the
Imitation Learning (IL) paradigm, which faces challenges such as causal
confusion and the open-loop gap. In this work, we establish a 3DGS-based
closed-loop Reinforcement Learning (RL) training paradigm. By leveraging 3DGS
techniques, we construct a photorealistic digital replica of the real physical
world, enabling the AD policy to extensively explore the state space and learn
to handle out-of-distribution scenarios through large-scale trial and error. To
enhance safety, we design specialized rewards that guide the policy to
effectively respond to safety-critical events and understand real-world causal
relationships. For better alignment with human driving behavior, IL is
incorporated into RL training as a regularization term. We introduce a
closed-loop evaluation benchmark consisting of diverse, previously unseen 3DGS
environments. Compared to IL-based methods, RAD achieves stronger performance
in most closed-loop metrics, especially 3x lower collision rate. Abundant
closed-loop results are presented at https://hgao-cv.github.io/RAD.