Learning Long-term Visual Dynamics with Region Proposal Interaction Networks

1 INTRODUCTION . As argued by Kenneth Craik , if an organism carries a model of external reality and its own possible actions within its head , it is able to react in a much fuller , safer and more competent manner to emergencies which face it ( Craik , 1952 ) . Indeed , building prediction models has been long studied in computer vision and intuitive physics . In computer vision , most approaches make predictions in pixel-space ( Denton & Fergus , 2018 ; Lee et al. , 2018 ; Ebert et al. , 2018b ; Jayaraman et al. , 2019 ; Walker et al. , 2016 ) , which ends up capturing the optical flow ( Walker et al. , 2016 ) and is difficult to generalize to long-horizon . In intuitive physics , a common approach is to learn the dynamics directly in an abstracted state space of objects to capture Newtonian physics ( Battaglia et al. , 2016 ; Chang et al. , 2016 ; Sanchez-Gonzalez et al. , 2020 ) . However , the states end up being detached from raw sensory perception . Unfortunately , these two extremes have barely been connected . In this paper , we argue for a middle-ground to treat images as a window into the world , i.e. , objects exist but can only be accessed via images . Images are neither to be used for predicting pixels nor to be isolated from dynamics . We operationalize it by learning to extract a rich state representation directly from images and build dynamics models using the extracted state representations . It is difficult to make predictions , especially about the future — Niels Bohr Contrary to Niels Bohr , predictions are , in fact , easy if made only for the short-term . Predictions that are indeed difficult to make and actually matter are the ones made over the long-term . Consider the example of “ Three-cushion Billiards ” in Figure 1 . The goal is to hit the cue ball in such a way that it touches the other two balls and contacts the wall thrice before hitting the last ball . This task is extremely challenging even for human experts because the number of successful trajectories is very sparse . Do players perform classical Newtonian physics calculations to obtain the best action before each shot , or do they just memorize the solution by practicing through exponentially many configurations ? Both extremes are not impossible , but often impractical . Players rather build a physical understanding by experience ( McCloskey , 1983 ; Kubricht et al. , 2017 ) and plan by making intuitive , yet accurate predictions in the long-term . Learning such a long-term prediction model is arguably the “ Achilles ’ heel ” of modern machine learning methods . Current approaches on learning physical dynamics of the world cleverly side-step the long-term dependency by re-planning at each step via model-predictive control ( MPC ) ( Allgöwer & Zheng , 2012 ; Camacho & Alba , 2013 ) . The common practice is to train short-term dynamical models ( usually 1-step ) in a simulator . However , small errors in short-term predictions can accumulate over time in MPC . Hence , in this work , we focus primarily on the long-term aspect of prediction by just considering environments , such as the three-cushion billiards example or the PHYRE ( Bakhtin et al. , 2019 ) in Figure 1 , where an agent is allowed to take only one action in the beginning so as to preclude any scope of re-planning . How to learn an accurate dynamics model has been a popular research topic for years . Recently , there are a series of work trying to represent video frames using object-centric representations ( Battaglia et al. , 2016 ; Watters et al. , 2017 ; Chang et al. , 2016 ; Janner et al. , 2019 ; Ye et al. , 2019 ; Kipf et al. , 2020 ) . However , those methods either operate in the state space , or ignore the environment information , both of which are not practical in real-world scenarios . In contrast , our objective is to build a data-driven prediction model that can both : ( a ) model long-term interactions over time to plan successfully for new instances , and ( b ) work from raw visual input in complex real-world environments . Therefore , the question we ask is : how to extract such an effective and flexible object representation and perform long-term predictions ? We propose Region Proposal Interaction Network ( RPIN ) which contains two key components . Firstly , we leverage the region of interests pooling ( RoIPooling ) operator ( Girshick , 2015 ) to extract object features maps from the frame-level feature . Object feature extraction based on region proposals has achieved huge success in computer vision ( Girshick , 2015 ; He et al. , 2017 ; Dai et al. , 2017 ; Gkioxari et al. , 2019 ) , and yet , surprisingly under-explored in the field of intuitive physics . By using RoIPooling , each object feature contains not only its own information but also the context of the environment . Secondly , we extend the Interaction Network and propose Convolutional Interaction Networks that perform interaction reasoning on the extracted RoI features . Interaction Networks is originally proposed in ( Battaglia et al. , 2016 ) , where the interaction reasoning is conducted via MLPs . By changing MLPs to convolutions , we can effectively utilize the spatial information of an object and make accurate future prediction of object location and shapes changes . Notably , our approach is simple , yet outperforms the state-of-the-art methods in both simulation and real datasets . In Section 5 , we thoroughly evaluate our approach across four datasets to study scientific questions related to a ) prediction quality , b ) generalization to time horizons longer than training , c ) generalization to unseen configurations , d ) planning ability for downstream tasks . Our method reduces the prediction error by 75 % in the complex PHYRE environment and achieves state-of-the-art performance on the PHYRE reasoning benchmark . 2 RELATED WORK . Physical Reasoning and Intuitive Physics . Learning models that can predict the changing dynamics of the scene is the key to building physical common-sense . Such models date back to “ NeuroAnimator ” ( Grzeszczuk et al. , 1998 ) for simulating articulated objects . Several methods in recent years have leveraged deep networks to build data-driven models of intuitive physics ( Bhattacharyya et al. , 2016 ; Ehrhardt et al. , 2017 ; Fragkiadaki et al. , 2015 ; Chang et al. , 2016 ; Stewart & Ermon , 2017 ) . However , these methods either require access to the underlying ground-truth state-space or do not scale to long-range due to absence of interaction reasoning . A more generic yet explicit approach has been to leverage graph neural networks ( Scarselli et al. , 2009 ) to capture interactions between entities in a scene ( Battaglia et al. , 2018 ; Chang et al. , 2016 ) . Closest to our approach are interaction models that scale to pixels and reason about object interaction ( Watters et al. , 2017 ; Ye et al. , 2019 ) . However , these approaches either reason about object crops with no context around or can only deal with a predetermined number and order of objects . A concurrent work ( Girdhar et al. , 2020 ) studies using prediction for physical reasoning , but their prediction model is either in the state space or in the pixel space . Other common ways to measure physical understanding are to predict future judgments given a scene image , e.g. , predicting the stability of a configuration ( Groth et al. , 2018 ; Jia et al. , 2015 ; Lerer et al. , 2016 ; Li et al. , 2016a ; b ) . Several hybrid methods take a data-driven approach to estimate Newtonian parameters from raw images ( Brubaker et al. , 2009 ; Wu et al. , 2016 ; Bhat et al. , 2002 ; Wu et al. , 2015 ) , or model Newtonian physics via latent variable to predict motion trajectory in images ( Mottaghi et al. , 2016a ; b ; Ye et al. , 2018 ) . An extreme example is to use an actual simulator to do inference over objects ( Hamrick et al. , 2011 ) . The reliance on explicit Newtonian physics makes them infeasible on real-world data and un-instrumented settings . In contrast , we take into account the context around each object via RoIPooling and explicitly model their interaction with each other or with the environment without relying on Newtonian physics , and hence , easily scalable to real videos for long-range predictions . Video Prediction . Instead of modeling physics from raw images , an alternative is to treat visual reasoning as an image translation problem . This approach has been adopted in the line of work that falls under video prediction . The most common theme is to leverage latent-variable models for predicting future ( Lee et al. , 2018 ; Denton & Fergus , 2018 ; Babaeizadeh et al. , 2017 ) . Predicting pixels is difficult so several methods leverage auxiliary information like back/fore-ground ( Villegas et al. , 2017a ; Tulyakov et al. , 2017 ; Vondrick et al. , 2016 ) , optical flow ( Walker et al. , 2016 ; Liu et al. , 2017 ) , appearance transformation ( Jia et al. , 2016 ; Finn et al. , 2016 ; Chen et al. , 2017 ; Xue et al. , 2016 ) , etc . These inductive biases help in a short interval but do not capture long-range behavior as needed in several scenarios , like playing billiards , due to lack of explicit reasoning . Some approaches can scale to relative longer term but are domain-specific , e.g. , pre-defined human-pose space ( Villegas et al. , 2017b ; Walker et al. , 2017 ) . However , our goal is to model long-term interactions not only for prediction but also to facilitate planning for downstream tasks . Learning Dynamics Models . Unlike video prediction , dynamics models take actions into account for predicting the future , also known as forward models ( Jordan & Rumelhart , 1992 ) . Learning these forward dynamics models from images has recently become popular in robotics for both specific tasks ( Wahlström et al. , 2015 ; Agrawal et al. , 2016 ; Oh et al. , 2015 ; Finn et al. , 2016 ) and exploration ( Pathak et al. , 2017 ; Burda et al. , 2019 ) . In contrast to these methods where a deep network directly predicts the whole outcome , we leverage our proposed region-proposal interaction module to capture each object trajectories explicitly to learn long-range forward dynamics as well as video prediction models . Planning via Learned Models . Leveraging models to plan is the standard approach in control for obtaining task-specific behavior . Common approach is to re-plan after each action via Model Predictive Control ( Allgöwer & Zheng , 2012 ; Camacho & Alba , 2013 ; Deisenroth & Rasmussen , 2011 ) . Scaling the models and planning in a high dimensional space is a challenging problem . With deep learning , several approaches shown promising results on real-world robotic tasks ( Finn et al. , 2016 ; Finn & Levine , 2017 ; Agrawal et al. , 2016 ; Pathak et al. , 2018 ) . However , the horizon of these approaches is still very short , and replanning in long-term drifts away in practice . Some methods try to alleviate this issue via object modeling ( Janner et al. , 2019 ; Li et al. , 2019 ) or skip connections ( Ebert et al. , 2018a ) but assume the models are trained with state-action pairs . In contrast to prior works where a short-range dynamic model is unrolled in time , we learn our long-range models from passive data and then couple them with short-range forward models to infer actions during planning .

Summary: The paper proposes a variation of interaction networks (IN) called region proposal interaction networks. The key idea is to have a richer object-centric feature representation using ROI-Pooling to encode the objects for prediction and use convolution operators to help the IN handle the change in the dimensionality of the feature representation. The paper is well-written and is evaluated on several popular benchmarks. The proposed variations seem to have a considerable effect on the performance to offer significant gains on challenging benchmarks.