  Due to the limited smartness and abilities of machine intelligence, currently
autonomous vehicles are still unable to handle all kinds of situations and
completely replace drivers. Because humans exhibit strong robustness and
adaptability in complex driving scenarios, it is of great importance to
introduce humans into the training loop of artificial intelligence, leveraging
human intelligence to further advance machine learning algorithms. In this
study, a real-time human-guidance-based deep reinforcement learning (Hug-DRL)
method is developed for policy training of autonomous driving. Leveraging a
newly designed control transfer mechanism between human and automation, human
is able to intervene and correct the agent's unreasonable actions in real time
when necessary during the model training process. Based on this
human-in-the-loop guidance mechanism, an improved actor-critic architecture
with modified policy and value networks is developed. The fast convergence of
the proposed Hug-DRL allows real-time human guidance actions to be fused into
the agent's training loop, further improving the efficiency and performance of
deep reinforcement learning. The developed method is validated by
human-in-the-loop experiments with 40 subjects and compared with other
state-of-the-art learning approaches. The results suggest that the proposed
method can effectively enhance the training efficiency and performance of the
deep reinforcement learning algorithm under human guidance, without imposing
specific requirements on participant expertise and experience.
