Viewport Prediction for Panoramic Video with MultiCNN Realtime interaction has become increasingly important. At the same time, panoramic video has gradually become popular. In this paper, the problem we study is predicting the FieldofView(FoV) at the future moment when people are enjoying a dynamic panoramic immersive video. Existing methods either estimate the future viewing area based on the previous trajectory, or predict the FoV based on salient region in video frames. Here, we design a new model to predict the viewing points in future moments. Firstly, we predict a point from the viewerxe2x80x99s previous viewing trajectory using LSTM(Long ShortTerm Memory) network. At the mean time, panoramic video frames are mapped to 6 patches by cube map in advance. The modified VGG16 network is used for each patch image to perform saliency detection. Then, these 6 salient maps are combined to a single salient map as output. A 3layer convolutional neural network to refined the salient map is utilized. Finally, the salient map of corresponding moment is combined with the predicted point by LSTM input to a twolayer fully connected network to produce the final predicted point. The experiment results show that our modelxe2x80x99s prediction accuracy is higher than the traditional prediction algorithm and has better performance than the model without using the second CNN network.

An EndtoEnd Future Frame Prediction Method for VehicleCentric Driving Videos In the field of autonomous driving, training an agent to watch and think similar to human drivers is an efficient way to solve selfdriving problems. Inspired by NVIDIAxe2x80x99s framelevel command generation task 2 and the discovery of humans memory capacity 13, we propose a future frame prediction method for vehiclecentric driving videos. An endtoend deep learning architecture called future frame prediction (FFPRE) network is proposed, which can generate a future frame following the input video sequence. In particular, we develop a general memory preserving module to extract meaningful history information from input data. This module consists of two parts, namely, memory recall and memory refine. We train this module to generate the shortterm spatiotemporal information of a given video batch, which is a concatenation of history appearance and temporal clues. Thereafter, the two history clues will be transformed into future representations by a longterm prediction module. Thus, humansxe2x80x99 driving prediction progress is mimicked in a completely modular manner. Given the FFPRE networkxe2x80x99s effective longshort spatiotemporal feature learning ability, the proposed network can construct an internal representation (content and dynamic) of vehiclecentric driving videos without tracking the trajectory of every pixel. Experimental results on publicly released datasets of NVIDIA and DR(eye)VE indicate that our proposed method is efficient.