Reasoning About Physical Interactions with Object-Oriented Prediction and Planning

1 INTRODUCTION . Consider the castle made out of toy blocks in Figure 1a . Can you imagine how each block was placed , one-by-one , to build this structure ? Humans possess a natural physical intuition that aids in the performance of everyday tasks . This physical intuition can be acquired , and refined , through experience . Despite being a core focus of the earliest days of artificial intelligence and computer vision research ( Roberts , 1963 ; Winston , 1970 ) , a similar level of physical scene understanding remains elusive for machines . Cognitive scientists argue that humans ’ ability to interpret the physical world derives from a richly structured apparatus . In particular , the perceptual grouping of the world into objects and their relations constitutes core knowledge in cognition ( Spelke & Kinzler , 2007 ) . While it is appealing to apply such an insight to contemporary machine learning methods , it is not straightforward to do so . A fundamental challenge is the design of an interface between the raw , often high-dimensional observation space and a structured , object-factorized representation . Existing works that have investigated the benefit of using objects have either assumed that an interface to an idealized object space already exists or that supervision is available to learn a mapping between raw inputs and relevant object properties ( for instance , category , position , and orientation ) . Assuming access to training labels for all object properties is prohibitive for at least two reasons . The most apparent concern is that curating supervision for all object properties of interest is difficult to scale for even a modest number of properties . More subtly , a representation based on semantic attributes can be limiting or even ill-defined . For example , while the size of an object in absolute terms is unambiguous , its orientation must be defined with respect to a canonical , class-specific orientation . Object categorization poses another problem , as treating object identity as a classification problem inherently limits a system to a predefined vocabulary . In this paper , we propose Object-Oriented Prediction and Planning ( O2P2 ) , in which we train an object representation suitable for physical interactions without supervision of object attributes . Instead of direct supervision , we demonstrate that segments or proposal regions in video frames , without correspondence between frames , are sufficient supervision to allow a model to reason effectively about intuitive physics . We jointly train a perception module , an object-factorized physics engine , and a neural renderer on a physics prediction task with pixel generation objective . We evaluate our learned model not only on the quality of its predictions , but also on its ability to use the learned representations for tasks that demand a sophisticated physical understanding . 2 OBJECT-ORIENTED PREDICTION AND PLANNING ( O2P2 ) . In this section , we describe a method for learning object-based representations suitable for planning in physical reasoning tasks . As opposed to much prior work on object-factorized scene representations ( Section 4 ) , we do not supervise the content of the object representations directly by way of labeled attributes ( such as position , velocity , or orientation ) . Instead , we assume access only to segments or region proposals for individual video frames . Since we do not have labels for the object representations , we must have a means for converting back and forth between images and object representations for training . O2P2 consists of three components , which are trained jointly : • A perception module that maps from an image to an object encoding . The perception module is applied to each object segment independently . • A physics module to predict the time evolution of a set of objects . We formulate the engine as a sum of binary object interactions plus a unary transition function . • A rendering engine that produces an image prediction from a variable number of objects . We first predict an image and single-channel heatmap for each object . We then combine all of the object images according to the weights in their heatmaps at every pixel location to produce a single composite image . A high-level overview of the model is shown in Figure 2c . Below , we give details for the design of each component and their subsequent use in a model-based planning setting . 2.1 PERCEPTION MODULE . The perception module is a four-layer convolutional encoder that maps an image observation to object representation vectors O = { ok } k=1 ... N . We assume access to a segmentation of the input image S = { sk } k=1 ... N and apply the encoder individually to each segment . The perception module is not supervised directly to predict semantically meaningful properties such as position or orientation ; instead , its outputs are used by the physics and rendering modules to make image predictions . In this way , the perception module must be trained jointly with the other modules . 2.2 PHYSICS MODULE . The physics module predicts the effects of simulating a collection of object representations O forward in time . As in Chang et al . ( 2016 ) ; Watters et al . ( 2017 ) , we consider the interactions of all pairs of object vectors . The physics engine contains two learned subcomponents : a unary transition function ftrans applied to each object representation independently , and a binary interaction function finteract applied to all pairs of object representations . Letting Ō = { ōk } k=1 ... N denote the output of the physics predictor , the kth object is given by ōk = ftrans ( ok ) + ∑ j 6=k finteract ( ok , oj ) + ok , where both ftrans and finteract are instantiated as two-layer MLPs . Much prior work has focused on learning to model physical interactions as an end goal . In contrast , we rely on physics predictions only insofar as they affect action planning . To that end , it is more important to know the resultant effects of an action than to make predictions at a fixed time interval . We therefore only need to make a single prediction , Ō = fphysics ( O ) , to estimate the steady-state configuration of objects as a result of simulating physics indefinitely . This simplification avoids the complications of long-horizon sequential prediction while retaining the information relevant to planning under physical laws and constraints . 2.3 RENDERING ENGINE . Because our only supervision occurs at the pixel level , to train our model we learn to map all objectvector predictions back to images . A challenge here lies in designing a function which constructs a single image from an entire collection of objects . The learned renderer consists of two networks , both instantiated as convolutional decoders . The first network predicts an image independently for each input object vector . Composing these images into a single reconstruction amounts to selecting which object is visible at every pixel location . In a traditional graphics engine , this would be accomplished by calculating a depth pass at each location and rendering the nearest object . To incorporate this structure into our learned renderer , we use the second decoder network to produce a single-channel heatmap for each object . The composite scene image is a weighted average of all of the object-specific renderings , where the weights come from the negative of the predicted heatmaps . In effect , objects with lower heatmap predictions at a given pixel location will be more visible than objects with higher heatmap values . This encourages lower heatmap values for nearer objects . Although this structure is reminiscent of a depth pass in a traditional renderer , the comparison should not be taken literally ; the model is only supervised by composite images and no true depth maps are provided during training . 2.4 LEARNING OBJECT REPRESENTATIONS . We train the perception , physics , and rendering modules jointly on an image reconstruction and prediction task . Our training data consists of image pairs ( I0 , I1 ) depicting a collection of objects on a platform before and after a new object has been dropped . ( I0 shows one object mid-air , as if being held in place before being released . We refer to Section 3 for details about the generation of training data . ) We assume access to a segmentation S0 for the initial image I0 . Given the observed segmented image S0 , we predict object representations using the perception module O = fpercept ( S0 ) and their time-evolution using the physics module Ō = fphysics ( O ) . The rendering engine then predicts an image from each of the object representations : Î0 = frender ( O ) , Î1 = frender ( Ō ) . We compare each image prediction Ît to its ground-truth counterpart using both L2 distance and a perceptual loss LVGG . As in Johnson et al . ( 2016 ) , we use L2 distance in the feature space of a Algorithm 1 Planning Procedure Input perception , physics , and rendering modules fpercept , fphysics , frender Input goal image Igoal with N segments Sgoal = { sgoalk } k=1 ... N 1 : Encode the goal image into a set of N object representations Ogoal = { ogoalk } k=1 ... N = fpercept ( S goal ) 2 : while Ogoal is nonempty do 3 : Segment the objects that have already been placed to yield Scurr 4 : for m = 1 to M do 5 : Sample action am of the form ( shape , position , orientation , color ) from uniform distribution 6 : Observe action am as a segment sm by moving object to specified position and orientation 7 : Concatenate the observation and segments of existing objects Sm = { sm } ∪ Scurr 8 : Encode segments Sm into a set of object representations Om = fpercept ( Sm ) 9 : Predict the effects of simulating physics on the object representations Ōm = fphysics ( Om ) 10 : Select the representation ō ∈ Ōm of the object placed by sampled action am 11 : Find the goal object gm that is closest to ō : gm = arg mini ||o goal i − ō||2 12 : Compute the corresponding distance dm = ||otarggm − ō||2 13 : end for 14 : Select action am∗ with the minimal distance to its nearest goal object : m∗ = arg minm dm . 15 : Execute action am∗ and remove object gm∗ from the goal Ogoal = Ogoal\ { ogoalgm∗ } . 16 : end while pretrained VGG network ( Simonyan & Zisserman , 2014 ) as a perceptual loss function . The perception module is supervised by the reconstruction of I0 , the physics engine is supervised by the reconstruction of I1 , and the rendering engine is supervised by the reconstruction of both images . Specifically , Lpercept ( · ) = L2 ( Î0 , I0 ) + LVGG ( Î0 , I0 ) , Lphysics ( · ) = L2 ( Î1 , I1 ) + LVGG ( Î1 , I1 ) , and Lrender ( · ) = Lpercept ( · ) + Lphysics ( · ) . 2.5 PLANNING WITH LEARNED MODELS . We now describe the use of our perception , physics , and rendering modules in the representative planning task depicted in Figure 1 , in which the goal is to build a block tower to match an observed image . Here , matching a tower does not refer simply to producing an image from the rendering engine that looks like the observation . Instead , we consider the scenario where the model must output a sequence of actions to construct the configuration . This setting is much more challenging because there is an implicit sequential ordering to building such a tower . For example , the bottom cubes must be placed before the topmost triangle . O2P2 was trained solely on a pixel-prediction task , in which it was never shown such valid action orderings ( or any actions at all ) . However , these orderings are essentially constraints on the physical stability of intermediate towers , and should be derivable from a model with sufficient understanding of physical interactions . Although we train a rendering function as part of our model , we guide the planning procedure for constructing towers solely through errors in the learned object representation space . The planning procedure , described in detail in Algorithm 1 , can be described at a high level in four components : 1 . The perception module encodes the segmented goal image into a set of object representations Ogoal . 2 . We sample actions of the form ( shape , position , orientation , color ) , where shape is categorical and describes the type of block , and the remainder of the action space is continuous and describes the block ’ s appearance and where it should be dropped . 3 . We evaluate the samples by likewise encoding them as object vectors and comparing them with Ogoal . We view action sample am as an image segment sm ( analogous to observing a block held in place before dropping it ) and use the perception module to produce object vectors Om . Because the actions selected should produce a stable tower , we run these object representations through the physics engine to yield Ōm before comparing with Ogoal . The cost is the L2 distance between the object ō ∈ Ōm corresponding to the most recent action and the goal object in Ogoal that minimizes this distance . 4 . Using the action sampler and evaluation metric , we select the sampled action that minimizes L2 distance . We then execute that action in MuJoCo ( Todorov et al. , 2012 ) . We continue this procedure , iteratively re-planning and executing actions , until there are as many actions in the executed sequence as there are objects in the goal image . In the simplest case , the distribution from which actions are sampled may be uniform , as in Algorithm 1 . Alternatively , the crossentropy method ( CEM ) ( Rubinstein & Kroese , 2004 ) may be used , repeating the sampling loop multiple times and fitting a Gaussian distribution to the lowest-cost samples . In practice , we used CEM starting from a uniform distribution with five iterations , 1000 samples per iteration , and used the top 10 % of samples to fit the subsequent iteration ’ s sampling distribution .

Summary: The paper presents a platform for predicting images of objects interacting with each other under the effect of gravitational forces. Given an image describing the initial arrangement of the objects in a scene, the proposed architecture first detects the objects and encode them using a perception module. A physics module then predicts the final arrangement of the object after moving under the effects of gravity. A rendering module takes as input the predicted final positions of objects and returns an image. The proposed architecture is trained by using pixel labels only, by reducing the gaps between the predicted rendered images and the images returned by the MuJuCo physics engine. This error's gradient is back-propagated to the physics and perception modules. The proposed platform is also used for planning object placements by sampling a large number of object shapes, orientations and colors, predicting the final configurations, and selecting initial placements that lead to final configurations that are as close as possible to given goal configurations using the L2 norm in the VGG features. Experiments performed in a simple blocks world show that the proposed approach is not only useful for prediction, but can also be used for planning object placements.