An EndtoEnd Future Frame Prediction Method for VehicleCentric Driving Videos In the field of autonomous driving, training an agent to watch and think similar to human drivers is an efficient way to solve selfdriving problems. Inspired by NVIDIAxe2x80x99s framelevel command generation task 2 and the discovery of humans memory capacity 13, we propose a future frame prediction method for vehiclecentric driving videos. An endtoend deep learning architecture called future frame prediction (FFPRE) network is proposed, which can generate a future frame following the input video sequence. In particular, we develop a general memory preserving module to extract meaningful history information from input data. This module consists of two parts, namely, memory recall and memory refine. We train this module to generate the shortterm spatiotemporal information of a given video batch, which is a concatenation of history appearance and temporal clues. Thereafter, the two history clues will be transformed into future representations by a longterm prediction module. Thus, humansxe2x80x99 driving prediction progress is mimicked in a completely modular manner. Given the FFPRE networkxe2x80x99s effective longshort spatiotemporal feature learning ability, the proposed network can construct an internal representation (content and dynamic) of vehiclecentric driving videos without tracking the trajectory of every pixel. Experimental results on publicly released datasets of NVIDIA and DR(eye)VE indicate that our proposed method is efficient.

EndtoEnd Driving Model for Steering Control of Autonomous Vehicles with Future Spatiotemporal Features Endtoend deep learning has gained considerable interests in autonomous driving vehicles in both academic and industrial fields, especially in decision making process. One critical issue in decision making process of autonomous driving vehicles is steering control. Researchers has already trained different artificial neural networks to predict steering angle with frontfacing camera data stream. However, existing endtoend methods only consider the spatiotemporal relation on a single layer and lack the ability of extracting future spatiotemporal information. In this paper, we propose an endtoend driving model based on Convolutional Long ShortTerm Memory (ConvLSTM) neural network with a Multiscale Spatiotemporal Integration (MSI) module, which aiming to encode the spatiotemporal information from different scales for steering angle prediction. Moreover, we employ future sequential information to enhance spatiotemporal features of the endtoend driving model. We demonstrate the efficiency of proposed endtoend driving model on the public Udacity dataset with comparison of some existing methods. Experimental results show that the proposed model has better performances than other existing methods, especially in some complex scenarios. Furthermore, we evaluate the proposed driving model on a realtime autonomous vehicle, and results show that the proposed driving model is able to predict the steering angle with high accuracy compared to skilled human driver.