1) Summary This paper proposes a flow-based neural network architecture and adversarial training for multi-step video prediction. The neural network in charge of predicting the next frame in a video implicitly generates flow that is used to transform the previously observed frame into the next. Additionally, this paper proposes a new quantitative evaluation criteria based on the observed flow in the prediction in comparison to the groundtruth. Experiments are performed on a new robot arm dataset proposed in the paper where they outperform the used baselines. 2) Pros:+ New quantitative evaluation criteria based on motion accuracy. + New dataset for robot arm pushing objects. 3) Cons:Overall architectural prediction network differences with baseline are unclear: The differences between the proposed prediction network and [1] seem very minimal. In Figure 3, it is mentioned that the network uses a U-Net with recurrent connections. This seems like a very minimal change in the overall architecture proposed. Additionally, there is a paragraph of “architecture improvements” which also are minimal changes. Based on the title of section 3, it seems that there is a novelty on the “prediction with flow” part of this method. If this is a fact, there is no equation describing how this flow is computed. However, if this “flow” is computed the same way [1]  does it, then the title is misleading. Adversarial training objective alone is not new as claimed by the authors: The adversarial objective used in this paper is not new. Works such as [2,3] have used this objective function for single step and multi-step frame prediction training, respectively. If the authors refer to the objective being new in the sense of using it with an action conditioned video prediction network, then this is again an extremely minimal contribution. Essentially, the authors just took the previously used objective function and used it with a different network. If the authors feel otherwise, please comment on why this is the case. Incomplete experiments:The authors only show experiments on videos containing objects that have already been seen, but no experiments with objects never seen before. The missing experiment concerns me in the sense that the network could just be memorizing previously seen objects. Additionally, the authors present evaluation based on PSNR and SSIM on the overall predicted video, but not in a per-step paradigm. However, the authors show this per-step evaluation in the Amazon Mechanical Turk, and predicted object position evaluations. Unclear evaluation:The way the Amazon Mechanical Turk experiments are performed are unclear and/or not suited for the task at hand. Based on the explanation of how these experiments are performed, the authors show individual images to mechanical turkers. If we are evaluating the video prediction task for having real or fake looking videos, the turkers need to observe the full video and judge based on that. If we are just showing images, then they are evaluating image synthesis, which do not necessarily contain the desired properties in videos such as temporal coherence. Additional comments:The paper needs a considerable amount of polishing. 4) Conclusion:This paper seems to contain very minimal changes in comparison to the baseline by [1]. The adversarial objective is not novel as mentioned by the authors and has been used in [2,3]. Evaluation is unclear and incomplete. References:[1] Chelsea Finn, Ian Goodfellow, and Sergey Levine. Unsupervised learning for physical interaction through video prediction. In NIPS, 2016. [2] M. Mathieu, C. Couprie, and Y. LeCun. Deep multi-scale video prediction beyond mean square error. In ICLR, 2016. [3] Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, Honglak Lee. Decomposing Motion and Content for Natural Video Sequence Prediction. In ICLR, 2017