  Currently, urban autonomous driving remains challenging because of the
complexity of the driving environment. Learning-based approaches, such as
reinforcement learning (RL) and imitation learning (IL), have indicated
superiority over rule-based approaches, showing great potential to make
decisions intelligently, but they still do not work well in urban driving
situations. To better tackle this problem, this paper proposes a novel
learning-based method that combines deep reinforcement learning with expert
demonstrations, focusing on longitudinal motion control in autonomous driving.
Our proposed method employs the actor-critic structure and modifies the
learning process of the policy network to incorporate both the goals of
maximizing reward and imitating the expert. Moreover, an adaptive prioritized
experience replay is designed to sample experience from both the agent's
self-exploration and expert demonstration, so as to improve the sample
efficiency. The proposed method is validated in a simulated urban roundabout
scenario and compared with various prevailing RL and IL baseline approaches.
The results manifest that the proposed method has a faster training speed, as
well as better performance in navigating safely and efficiently.
