Unsupervised Intuitive Physics from Past Experiences

We consider the problem of learning models of intuitive physics from raw , unlabelled visual input . Differently from prior work , in addition to learning general physical principles , we are also interested in learning “ on the fly ” physical properties specific to new environments , based on a small number of environment-specific experiences . We do all this in an unsupervised manner , using a meta-learning formulation where the goal is to predict videos containing demonstrations of physical phenomena , such as objects moving and colliding with a complex background . We introduce the idea of summarizing past experiences in a very compact manner , in our case using dynamic images , and show that this can be used to solve the problem well and efficiently . Empirically , we show , via extensive experiments and ablation studies , that our model learns to perform physical predictions that generalize well in time and space , as well as to a variable number of interacting physical objects . 1 INTRODUCTION . Many animals possess an intuitive understanding of the physical world . They use this understanding to accurately and rapidly predict events from sparse sensory inputs . In addition to general physical principles , many animals also learn specific models of new environments as they experience them over time . For example , they can explore an environment to determine which parts of it can be navigated safely and remember this knowledge for later reuse . Authors have looked at equipping artificial intelligences ( AIs ) with analogous capabilities , but focusing mostly on performing predictions from instantaneous observations of an environment , such as a few frames in a video . However , such predictions can be successful only if observations are combined with sufficient prior knowledge about the environment . For example , consider predicting the motion of a bouncing ball . Unless key parameters such as the ball ’ s elasticity are known a priori , it is impossible to predict the ball ’ s trajectory accurately . However , after observing at least one bounce , it is possible to infer some of the parameters and eventually perform much better predictions . In this paper , we are interested in learning intuitive physics in an entirely unsupervised manner , by passively watching videos . We consider situations in which objects interact with scenarios that can only be partially inferred from their appearance , but that also contain objects whose parameters can not be confidently predicted from appearance alone ( fig . 1 ) . Then , we consider learning a system that can observe a few physical experiments to infer such parameters , and use this knowledge to perform better predictions in the future . Our model has three goals . First , it must learn without the use of any external or ad-hoc supervision . We achieve this by training our model from raw videos , using a video prediction error as a loss . Second , our model must be able to extract information about a new scenario by observing a few experiments , which we formulate as meta-learning . We also propose a simple representation of the experiments based on the concept of “ dynamic image ” that allows to process long experiments more efficiently than using a conventional recurrent network . Third , our model must learn a good representation of physics without access to any explicit or external supervision . Instead , we propose three tests to support this hypothesis . ( i ) We show that the model can predict far in the future , which is a proxy to temporal invariance . ( ii ) We further show that the model can extend to scenarios that are geometrically much larger than the ones used for training , which is a proxy to spatial invariance . ( iii ) Finally , we show that the model can generalize to several moving objects , which is a proxy to locality . Locality and time-space invariance are of course three key properties of physical laws and thus we should expect any good intuitive model of physics to possess them . In order to support these claims , we conduct extensive experiments in simulated scenarios , including testing the ability of the model to cope with non-trivial visual variations of the inputs . While the data is simpler than a real-world application , we nevertheless make substantial progress compared to previous work , as discussed in section 2 . We do so by learning from passive , raw video data a good model of dynamics and collisions that generalizes well spatially , temporally , and to a variable number of objects . The scalability of our approach , via the use of the dynamic image , is also unique . Finally , we investigate the problem of learning the parameters of new scenarios on the fly via experiences and we propose an effective solution to do so . 2 RELATED WORK . A natural way to represent physics is to manually encode every object parameter and physical property ( e.g. , mass , velocity , positions ) and use supervision to make predictions . This has been widely used to represent and propagate physics ( Wu et al. , 2015 ; 2016 ; Battaglia et al. , 2016 ; Chang et al. , 2017 ; Mrowca et al. , 2018 ; Sanchez-Gonzalez et al. , 2018 ) . If models like Wu et al . ( 2015 ; 2016 ) also estimate environment parameters , these works rely on a physics engine that assumes strong priors about the scenario , while our approach does not require such constraint . Inspired by the recent successes of Convolutional Neural Networks ( CNNs ( Krizhevsky et al. , 2012 ) ) and their application to implicit representation of dynamics ( Ondruska & Posner , 2016 ; Oh et al. , 2015 ; Chiappa et al. , 2017 ; Bhattacharyya et al. , 2018 ) , researchers ( Watters et al. , 2017 ; Ehrhardt et al. , 2019 ; Kansky et al. , 2017 ) have tried to base their approaches on visual inputs . They learn from several frames of a scene to regress the next physical state of a system . In general these approaches learn an implicit representation of physics ( Ehrhardt et al. , 2019 ; Watters et al. , 2017 ) as a tensor state from recurrent deep networks . Such models are mostly supervised using ground-truth information about key physical parameters ( e.g. , positions , velocities , density ) during training . While these approaches require an expensive annotation of data , other works have tried to learn from unsupervised data as well . Researchers have successfully learned unsupervised models either through active manipulation ( Agrawal et al. , 2016 ; Denil et al. , 2016 ; Finn et al. , 2016a ) , using the laws of physics ( Stewart & Ermon , 2017 ) , using dynamic clues and invariances ( Greff et al. , 2017 ; van Steenkiste et al. , 2018 ) or features extracted from unsupervised methods ( Finn et al. , 2016b ; Ehrhardt et al. , 2018 ) . Fragkiadaki et al . ( 2016 ) also used an unsupervised system like ours , however , they assumed the rendering system and the number of objects to be known , which we do not , which prevented the internal representation to fail over time . Perhaps most related to our approach is the work of Wang et al . ( 2018 ) , where the model is learnt using future image prediction on a simple synthetic task and then transferred to real world scenarios . They also demonstrate long-term dynamic predictions , however they did not generalize to different background/number of balls . In other works , models are taught to answer simple qualitative questions about a physical setup , such as : the stability of stacks of objects ( Battaglia et al. , 2013 ; Lerer et al. , 2016 ; Li et al. , 2017 ; Groth et al. , 2018 ) , the likelihood of a scenario ( Riochet et al. , 2018 ) , the forces acting behind a scene ( Wu et al. , 2017 ; Mottaghi et al. , 2016 ) or properties of objects through manipulation ( Agrawal et al. , 2016 ; Denil et al. , 2016 ) . Other papers compromise between qualitative and quantitative predictions and focus on plausibility ( Tompson et al. , 2016 ; Ladický et al. , 2015 ; Monszpart et al. , 2016 ) . Finally similar problems can be found in online learning settings ( Nagabandi et al. , 2019a ; b ) . These works also use meta-learning but , differently from ours , allow their models to learn and adapt at test time from feedbacks . 3 METHOD . We describe our model , illustrated in fig . 2 , starting from the input and output data . A scenario S is a physical environment that supports moving objects interacting with it . In this paper , we take as scenarios 2.1D environments containing obstacles and we consider rolling balls as moving objects . Hence , interactions are in the form of bounces . Formally , a scenario is defined over a lattice Ω = { 0 , . . . , H − 1 } × { 0 , . . . , W − 1 } and is specified by a list of obstacles S = { ( Oj , bj ) , j = 1 , . . . , K } . Here Oj ⊂ Ω is the shape of an obstacle and bj ∈ { B , A , U } is a flag that tells whether the ball bounces against it ( B ) , passes above it ( A ) , or passes under it ( U ) . Obstacles do not overlap . A run is a tupleR = ( S , y ) associating a scenario S with a trajectory y = ( yt ∈ Ω , t = 0 , . . . , T −1 ) for the ball ( this is trivially extended to multiple balls ) . Scenarios and runs are sensed visually . The symbol I ( S ) : Ω → R3×H×W denotes the image generated by observing a scenario with no ball and It ( R ) = I ( S , yt ) is the image generated by observing the ball at time t in a run . The symbol I ( R ) = ( It ( R ) , t = 0 , . . . , T − 1 ) denotes the collection of all frames in a run , which can be thought of as a video . We are interested in learning intuitive physics with no explicit supervision on the object trajectories , nor an explicit encoding of the laws of mechanics that govern the motion of the objects and their interactions and collisions with the environment . Hence , we cast this problem as predicting the video I ( R ) of the trajectory of the objects given only a few initial frames I0 : T0 ( R ) = ( I0 ( R ) , . . . , IT0−1 ( R ) ) , where T0 T . We are not the first to consider a similar learning problem , although most prior works do require some form of external supervision , which we do not use . Here , however , we consider an additional key challenge that the images I0 : T0 ( R ) do not contain sufficient information to successfully predict the long-term objects ’ motion . This is because these few frames tell us nothing about the nature of the obstacles in the scenario . In fact , under the assumptions that obstacles of type B , A and U have similar or even the identical appearance , it is not possible to predict whether the ball will bounce , move above , or move under any such obstacle . This situation is representative of an agent that needs to operate in a new complex environment and must learn more about it , from past experiences , before it can do so reliably . Thus , we consider a modification of the setup described above in which the model can experience each new scenario for a while , by observing the motion of the ball , before making its own predictions . Formally , an experience is a collection of N runs E = ( R1 , . . . , RN ) all relative to the same scenario S with each a different trajectories y1 , . . . , yN . By observing such examples , the model must determine the nature of the obstacles and then use this information to correctly predict the motion of the ball in the future . We cast this as the problem of learning a mapping Ω : ( I0 : T0 ( R ) , I ( E ) ) 7−→ I ( R ) , ( 1 ) where I ( E ) = ( I ( R1 ) , . . . , I ( RN ) ) are the videos corresponding to the runs in the experience . We will callR the prediction run to distinguish it from the experience runs E .

Summary: The paper proposes a method to predict the future trajectory of a ball (or a set of balls) given the first few frames of the trajectory and a set of experience runs in the same environment. The model first learns to convert the set of images to a set of corresponding heatmaps that encode the location of the ball. Then, a recurrent network generates the future states given the first few locations and the output of a network that learns abstract information from the experience runs. The network is trained using reconstruction and perceptual similarity loss.