From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper8/AnonReviewer1
relevance: 5: very high
impact: 3: borderline
novelty: 4: high

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper8/AnonReviewer1
evaluation: The paper proposes a new video prediction network that combines a discriminator, a motion guider and a generator to predict the next video frame given a set of previous video frames as input. The discriminator extracts features for each video frame, and the motion guider extracts motion features from the video frame features. Finally, the generator takes the motion features and features of the last frame to predict the next future frame. Experiments are conducted on the Moving MNIST dataset and the Highway Drive dataset to evaluate the video frame prediction performance.

Positives:
- The idea of using the motion guider in the network for video frame prediction is new and novel.
- The proposed method achieves the state-of-the-art performance on the two experimented datasets.

Negatives:
- The paper lacks ablation studies on the network architecture design. As a result, it is unclear how important is each component in improving the video prediction performance. For example, how much improvement can the motion guider introduces?
- It would be interesting to visualize the features learned from the discriminator and motion guider for analyses.

From: ~Dong_Wang2
comment: Thanks for your comments. 
For your concern about the motion guider, it's trained to predict the difference of motion features between the last frame t (time step t) and the future frame t+1 (the frame to predict, which is unavailable in testing). Using the recurrent structure can better model the long term motion information. In contrast, subtraction of two continuous frames will be more sensitive to noise than jointly considering all previous frames. Also, as a clarification, the leaked feature is actually extracted from a video cube (3 frames in our experiment). Thus even with the subtraction, the motion guider still works on several frames, instead of one. We will compare the model without the recurrent motion guider in an updated version.
title: About the motion guider

From: ~Dong_Wang2
comment: Thanks for your comments. 

For your concern about the motion guider, it's trained to predict the difference of motion features between the last frame t (time step t) and the future frame t+1 (the frame to predict, which is unavailable in testing). Using the recurrent structure can better model the long term motion information. In contrast, subtraction of two continuous frames will be more sensitive to noise than jointly considering all previous frames. Also, as a clarification, the leaked feature is actually extracted from a video cube (3 frames in our experiment). Thus even with the subtraction, the motion guider still works on several frames, instead of one. We will compare the model without the recurrent motion guider in an updated version.
title: About the motion guider

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper8/AnonReviewer2
evaluation: The paper proposes a method for future frame prediction which combines features from an adversarial discriminator with a prediction model based on dynamic filter networks (DFN). Specifically, intermediate features from a convolutional network which is optimized for discriminating real from generated images is accumulated with a recurrent network. This recurrent network then produces a feature capturing the temporal nature of the scene for generation of the next frame. This is the first time I've seen features from a discriminator used for future frame predictions and the presented results show a reasonable improvement over the DFN baseline.

As the motion guider is trained to approximate the difference of features over the last two-time steps, it raises the question of what is the value of the recurrent motion guider over explicitly using the feature difference? 

Overall the paper is well written and suitable for the spatialtemporal workshop.

From: ~Ransalu_Senanayake1
comment: The paper proposes a method for future frame prediction which combines features from an adversarial discriminator with a prediction model based on dynamic filter networks (DFN). Specifically, intermediate features from a convolutional network which is optimized for discriminating real from generated images is accumulated with a recurrent network. This recurrent network then produces a feature capturing the temporal nature of the scene for generation of the next frame. This is the first time I've seen features from a discriminator used for future frame predictions and the presented results show a reasonable improvement over the DFN baseline.

As the motion guider is trained to approximate the difference of features over the last two-time steps, it raises the question of what is the value of the recurrent motion guider over explicitly using the feature difference? 

Overall the paper is well written and suitable for the spatialtemporal workshop.
title: From: NIPS 2018 Workshop Spatiotemporal Paper8 AnonReviewer2 (appeared as an official comment)

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper8/AnonReviewer2
relevance: 4: high
impact: 3: borderline
novelty: 4: high