  Autonomous driving (AD) agents generate driving policies based on online
perception results, which are obtained at multiple levels of abstraction, e.g.,
behavior planning, motion planning and control. Driving policies are crucial to
the realization of safe, efficient and harmonious driving behaviors, where AD
agents still face substantial challenges in complex scenarios. Due to their
successful application in fields such as robotics and video games, the use of
deep reinforcement learning (DRL) and deep imitation learning (DIL) techniques
to derive AD policies have witnessed vast research efforts in recent years.
This paper is a comprehensive survey of this body of work, which is conducted
at three levels: First, a taxonomy of the literature studies is constructed
from the system perspective, among which five modes of integration of DRL/DIL
models into an AD architecture are identified. Second, the formulations of
DRL/DIL models for conducting specified AD tasks are comprehensively reviewed,
where various designs on the model state and action spaces and the
reinforcement learning rewards are covered. Finally, an in-depth review is
conducted on how the critical issues of AD applications regarding driving
safety, interaction with other traffic participants and uncertainty of the
environment are addressed by the DRL/DIL models. To the best of our knowledge,
this is the first survey to focus on AD policy learning using DRL/DIL, which is
addressed simultaneously from the system, task-driven and problem-driven
perspectives. We share and discuss findings, which may lead to the
investigation of various topics in the future.
