Gaze Training by Modulated Dropout Improves Imitation Learning Imitation learning by behavioral cloning is a prevalent method that has achieved some success in visionbased autonomous driving. The basic idea behind behavioral cloning is to have the neural network learn from observing a human expertxe2x80x99s behavior. Typically, a convolutional neural network learns to predict the steering commands from raw driverview images by mimicking the behaviors of human drivers. However, there are other cues, such as gaze behavior, available from human drivers that have yet to be exploited. Previous researches have shown that novice human learners can benefit from observing expertsxe2x80x99 gaze patterns. We present here that deep neural networks can also profit from this. We propose a method, gazemodulated dropout, for integrating this gaze information into a deep driving network implicitly rather than as an additional input. Our experimental results demonstrate that gazemodulated dropout enhances the generalization capability of the network to unseen scenes. Prediction error in steering commands is reduced by 23.5% compared to uniform dropout. Running closed loop in the simulator, the gazemodulated dropout net increased the average distance travelled between infractions by 58.5%. Consistent with these results, the gazemodulated dropout net shows lower model uncertainty.

Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety The decision and planning system for autonomous driving in urban environments is hard to design. Most current methods manually design the driving policy, which can be expensive to develop and maintain at scale. Instead, with imitation learning we only need to collect data and the computer will learn and improve the driving policy automatically. However, existing imitation learning methods for autonomous driving are hardly performing well for complex urban scenarios. Moreover, the safety is not guaranteed when we use a deep neural network policy. In this paper, we proposed a framework to learn the driving policy in urban scenarios efficiently given offline connected driving data, with a safety controller incorporated to guarantee safety at test time. The experiments show that our method can achieve high performance in realistic simulations of urban driving scenarios.