U NSUPERVISED I NTUITIVE P HYSICS FROM P AST E XPERIENCES
Anonymous authors
Paper under double-blind review
A BSTRACT
We consider the problem of learning models of intuitive physics from raw, unlabelled visual input.
Differently from prior work, in addition to learning general physical principles, we are also interested in learning “on the ﬂy” physical properties speciﬁc to new environments, based on a small number of environment-speciﬁc experiences.
We do all this in an unsupervised manner, using a meta-learning formulation where the goal is to predict videos containing demonstrations of physical phenomena, such as objects moving and colliding with a complex background.
We introduce the idea of summarizing past experiences in a very compact manner, in our case using dynamic images, and show that this can be used to solve the problem well and efﬁciently.
Empirically, we show, via extensive experiments and ablation studies, that our model learns to perform physical predictions that generalize well in time and space, as well as to a variable number of interacting physical objects.
I NTRODUCTION
Many animals possess an intuitive understanding of the physical world.
They use this understanding to accurately and rapidly predict events from sparse sensory inputs.
In addition to general physical principles, many animals also learn speciﬁc models of new environments as they experience them over time.
For example, they can explore an environment to determine which parts of it can be navigated safely and remember this knowledge for later reuse
Authors have looked at equipping artiﬁcial intelligences (AIs) with analogous capabilities, but focusing mostly on performing predictions from instantaneous observations of an environment, such as a few frames in a video.
However, such predictions can be successful only if observations are combined with sufﬁcient prior knowledge about the environment.
For example, consider predicting the motion of a bouncing ball.
Unless key parameters such as the ball’s elasticity are known a priori, it is impossible to predict the ball’s trajectory accurately.
However, after observing at least one bounce, it is possible to infer some of the parameters and eventually perform much better predictions.
In this paper, we are interested in learning intuitive physics in an entirely unsupervised manner, by passively watching videos.
We consider situations in which objects interact with scenarios that can only be partially inferred from their appearance, but that also contain objects whose parameters cannot be conﬁdently predicted from appearance alone (ﬁg. 1).
Then, we consider learning a system that can observe a few physical experiments to infer such parameters, and use this knowledge to perform better predictions in the future.
Our model has three goals.
First, it must learn without the use of any external or ad-hoc supervision.
We achieve this by training our model from raw videos, using a video prediction error as a loss.
Second, our model must be able to extract information about a new scenario by observing a few experiments, which we formulate as meta-learning.
We also propose a simple representation of the experiments based on the concept of “dynamic image” that allows to process long experiments more efﬁciently than using a conventional recurrent network.
Third, our model must learn a good representation of physics without access to any explicit or external supervision.
Instead, we propose three tests to support this hypothesis.
(i)
We show that the model can predict far in the future, which is a proxy to temporal invariance .
(ii) We further show that the model can extend to scenarios that are geometrically much larger than the ones used for training, which is a proxy to spatial invariance .
(iii) Finally, we show that the model can generalize to several
[Figure]
In order to support these claims, we conduct extensive experiments in simulated scenarios, including testing the ability of the model to cope with non-trivial visual variations of the inputs.
While the data is simpler than a real-world application, we nevertheless make substantial progress compared to previous work, as discussed in section 2.
We do so by learning from passive, raw video data a good model of dynamics and collisions that generalizes well spatially, temporally, and to a variable number of objects.
The scalability of our approach, via the use of the dynamic image, is also unique.
Finally, we investigate the problem of learning the parameters of new scenarios on the ﬂy via experiences and we propose an effective solution to do so.
R ELATED WORK
A natural way to represent physics is to manually encode every object parameter and physical property (e.g., mass, velocity, positions) and use supervision to make predictions.
This has been widely used to represent and propagate physics (Wu et al., 2015; 2016; Battaglia et al., 2016; Chang et al., 2017;
Mrowca et al., 2018; Sanchez-Gonzalez et al., 2018).
If models like Wu et al.
(2015; 2016) also estimate environment parameters, these works rely on a physics engine that assumes strong priors about the scenario, while our approach does not require such constraint.
Inspired by the recent successes of Convolutional Neural Networks (CNNs (Krizhevsky et al., 2012)) and their application to implicit representation of dynamics (Ondruska & Posner, 2016;
Oh et al., 2015; Chiappa et al., 2017; Bhattacharyya et al., 2018), researchers (Watters et al., 2017; Ehrhardt et al., 2019; Kansky et al., 2017) have tried to base their approaches on visual inputs.
They learn from several frames of a scene to regress the next physical state of a system.
In general these approaches learn an implicit representation of physics (Ehrhardt et al., 2019; Watters et al., 2017) as a tensor state from recurrent deep networks.
Such models are mostly supervised using ground-truth information about key physical parameters (e.g., positions, velocities, density) during training.
While these approaches require an expensive annotation of data, other works have tried to learn from unsupervised data as well.
Researchers have successfully learned unsupervised models either through active manipulation (Agrawal et al., 2016;
Denil et al., 2016;
Finn et al., 2016a), using the laws of physics (Stewart & Ermon, 2017), using dynamic clues and invariances (Greff et al., 2017; van Steenkiste et al., 2018) or features extracted from unsupervised methods (Finn et al., 2016b; Ehrhardt et al., 2018). Fragkiadaki et al.
(2016) also used an unsupervised system like ours, however, they assumed the rendering system and the number of objects to be known, which we do not, which prevented the internal representation to fail over time.
Perhaps most related to our approach is the work of Wang et al.
(2018), where the model is learnt using future image prediction on a simple synthetic task and then transferred to real world scenarios.
They also demonstrate long-term dynamic predictions, however they did not generalize to different background/number of balls.
In other works, models are taught to answer simple qualitative questions about a physical setup, such as: the stability of stacks of objects (Battaglia et al., 2013; Lerer et al., 2016; Li et al., 2017; Groth et al., 2018), the likelihood of a scenario (Riochet et al., 2018), the forces acting behind a scene (Wu et al., 2017; Mottaghi et al., 2016) or properties of objects through manipulation (Agrawal et al., 2016; Denil et al., 2016).
Other papers compromise between qualitative and quantitative predictions and focus on plausibility (Tompson et al., 2016; Ladický et al., 2015; Monszpart et al., 2016).
I ( R )
I ( R 1 )
I ( R 2 )
I ( R N ) f :=
[ f 1 f 2 ] ˆΨˆΨˆΨa m ˆΨˆΨ
I 0: TΨ :
R 6 × H × W → R (1+ C ) ×
H × W η x 0: Tx t − T 0 :
t x t ˆ I t
ˆ I 0: T g p oo l p oo l l o ss ˆΨdynamic image + median · · · median Φ ˆ I experience
[Equation]
Finally similar problems can be found in online learning settings (Nagabandi et al., 2019a;b).
These works also use meta-learning but, differently from ours, allow their models to learn and adapt at test time from feedbacks.
M ETHOD
We describe our model, illustrated in ﬁg. 2, starting from the input and output data.
A scenario S is a physical environment that supports moving objects interacting with it.
In this paper, we take as scenarios 2.1D environments containing obstacles and we consider rolling balls as moving objects.
Hence, interactions are in the form of bounces.
Formally, a scenario is deﬁned over a lattice Ω = { 0 , . . .
, H − 1 } × { 0 , . . .
, W − 1 } and is speciﬁed by a list of obstacles S =
{ ( O j , b j ) , j = 1 , . . .
, K } .
Here O j ⊂ Ω is the shape of an obstacle and b j ∈ { B, A, U } is a ﬂag that tells whether the ball bounces against it (B), passes above it (A), or passes under it (U).
Obstacles do not overlap.
A run is a tuple R =
( S , y ) associating a scenario S with a trajectory y =
( y t ∈ Ω , t = 0 , . . .
, T − 1) for the ball (this is trivially extended to multiple balls).
Scenarios and runs are sensed visually.
The symbol I ( S ) :
Ω → R 3 × H × W denotes the image generated by observing a scenario with no ball and I t ( R )
=
I ( S , y t ) is the image generated by observing the ball at time t in a run.
The symbol
I ( R ) =
( I t ( R ) , t = 0 , . . .
, T − 1) denotes the collection of all frames in a run, which can be thought of as a video.
We are interested in learning intuitive physics with no explicit supervision on the object trajectories, nor an explicit encoding of the laws of mechanics that govern the motion of the objects and their interactions and collisions with the environment.
Hence, we cast this problem as predicting the video I ( R ) of the trajectory of the objects given only a few initial frames I 0: T 0 ( R ) =
( I 0 ( R ) , . .
, I T 0 − 1 ( R )) , where T
[Equation]
We are not the ﬁrst to consider a similar learning problem, although most prior works do require some form of external supervision, which we do not use.
Here, however, we consider an additional key challenge that the images I 0: T 0 ( R ) do not contain sufﬁcient information to successfully predict the long-term objects’ motion.
This is because these few frames tell us nothing about the nature of the obstacles in the scenario.
In fact, under the assumptions that obstacles of type B , A
and U have similar or even the identical appearance, it is not possible to predict whether the ball will bounce, move above, or move under any such obstacle.
This situation is representative of an agent that needs to operate in a new complex environment and must learn more about it, from past experiences, before it can do so reliably.
Thus, we consider a modiﬁcation of the setup described above in which the model can experience each scenario making its own predictions
Formally, an experience is a collection of N runs E
= ( R 1 , . . .
, R N ) all relative to the same scenario
S with each a different trajectories y 1 , . . .
, y N .
By observing such examples, the model must determine the nature of the obstacles and then use this information to correctly predict the motion of the ball in the future.
We cast this as the problem of learning a mapping
[Equation]
where I ( E ) =
( I ( R 1 ) , . . .
, I ( R N )) are the videos corresponding to the runs in the experience.
We will call R the prediction run to distinguish it from the experience runs E .
3. A META - LEARNING PERSPECTIVE
The setup we have described above can be naturally described as meta-learning.
Namely, eq.
(1) can be thought of as incorporating a “local” learning rule M that maps the experience E to a scenariospeciﬁc predictor ˆΩ on the ﬂy: ˆΩ( · ) =
Ω( · , I ( E )) =
M [ I ( E )] .
Hence M must extract from the experience as much information as possible about the underlying scenario and transfer it to the scenario-speciﬁc predictor ˆΩ .
In order to learn M , we consider meta-samples of the type ( S , R , E ) comprising a scenario
S , a prediction run R and N experience runs E .
Given a dataset D of such meta-samples, meta-learning searches for the mapping
M that minimizes the error on the prediction runs:
[Equation]
3. C OMPRESSED DYNAMIC EXPERIENCES
Concretely, we parameterise the scenario-speciﬁc predictor ˆΩ( · ; w , m ) using parameters w , the weights a deep neural network, which is ﬁxed and scenario-independent, and m , which is scenariospeciﬁc.
The latter is extracted by a network m =
Ψ 1 ( I ( E )) from the experience videos I ( E ) .
Since we expect m to provide information about the nature of the obstacles in the scenario, we let m ∈ R 1 × H × W be a tensor with the same spatial resolution as the scenario and interpret it as an “obstacle mask”.
Given that the function Ψ 1 takes as input a number of video sequences, one may think to implement it as a recurrent neural network; however, recurrent networks scale badly, especially in our metalearning context — to the point that we had difﬁculties in even running such a model at a non-trivial scale.
Instead, we propose to construct Ψ 1 based on a compact representation of the experience which leads to a much more efﬁcient design.
For this, we use the concept of dynamic image (Bilen et al., 2016), which encodes a video as a weighted average of its frames: f 1 ( I 0 , . .
, I T − 1 )
=
(cid:80) T − 1 t =
α t I t with α t = (cid:80) T − 1 i =
t 2( i +1) − T − 1 i + .
Since the dynamic image is only sensitive to changes in the video, we complement it by computing also the median image f 2 ( I 0 , . .
, I T − 1 ) =
median t =0 ,...,T − 1 I t and combine the two functions f 1 and f 2 in a single representation f by stacking the respective outputs along the channel dimension.
With this design, we can rewrite the map Ψ 1 as follows:
[Equation]
Here the map ˆΨ 1 : R 6 × H × W
→ R 1 × H × W , which can be implemented as a standard CNN, takes as input the dynamic/median image and produces as output the obstacle mask m .
The pooling operator summarizes the information extracted from multiple runs into a single mask m .
We also consider a second similar map ˆΨ 2 : R 6 × H × W
→ R C × H × W to extract an “appearance” tensor a .
The latter helps the generator to render obstacles above or underneath the moving objects as needed.
The response of this function is also max-pooled over runs, but channel-wise: [pool k =1 ,...,N a k ] cvu =
[ a k ( c ) ] cvu where k ( c ) =
argmax k
=1 ,...,N (cid:80) vu [ a k ] 2 cvu .
In practice, ˆΨ 1 and ˆΨ 2 are implemented as a single neural network Ψ :
R 6 × H × W → R (1+ C ) × H × W where ˆΨ 1 is the ﬁrst output channel and ˆΨ 2 the others.
Optional obstacle mask supervision.
In the experiments, we show that the map ˆΨ can be learned automatically without any external supervision.
We contrast this with supervising ˆ Ψ 1 with an oracle rendition of the obstacle map.
To this end, we deﬁne the tensor m gt for a scenario S to be the indicator mask of whether a pixel contains a solid obstacles (including the perimetral walls) and then minimize the auxiliary loss (cid:96) ( m gt , Ψ 1 ( I ( E )))
= (cid:107) m
gt − Ψ
( I ( E ))
(cid:107) 2 .
3. A UTO - REGRESSIVE PREDICTOR
The predictor Φ is designed as an RNN that takes as input T 0 past states x t − T 0 :
t = ( x t − T 0 , . . .
, x t − 1 ) and outputs a new state x t .
Each state variable is in turn a distributed representation of the physical state of the system, in the form of a “heatmap” x t ∈ R H × W .
Considering T 0 past states allows the model to auto-regressively represent the dynamics of the system if so learning chooses to do.
The predictor also takes as input the scenario representation m given by eq.
The ﬁrst T 0 state variables are initialized from observations I 0: T 0 ( R ) via an initialization function η : R 3 × H × W
→
R H × W .
We thus have
[Equation]
In short, this model estimates recursively the evolution of the system dynamics from the visual observations of the ﬁrst T 0 samples.
Conditional generator.
Variable x t contains information about the state of the moving objects (balls).
This is then converted into a prediction of the corresponding video frame, combining also the appearance tensor a and the median of the ﬁrst T 0 images in the sequence ˆ I = median t =0 ,...,T 0 − 1 I t .
This is formulated as a conditional generator network ˆ
I t = g ( x t , a , ˆ I ) .
Video reconstruction loss.
Next, we discuss the nature of the loss equation 2.
Owing also to the static background, the conditional video generation task is relatively simple provided that the dynamics are estimated correctly.
As a consequence, the generated videos are likely to closely approximate the ground truth ones, so the loss function (cid:96) in equation 2 does not require to be complex.
In our experiments, we combine the L 2 image loss with the perceptual loss of (Johnson et al., 2016).
The latter is obtained by passing the ground-truth and predicted images through the ﬁrst few layers of a pre-trained deep neural network e , VGG-16 (Simonyan & Zisserman, 2015) in our case, and then comparing the resulting encodings in L 2 distance.
The overall loss is then given by (cid:96) ( I t , ˆ I t ) =
λ I (cid:107) I t − ˆ I t (cid:107) 2 + λ p (cid:107) e
( I t ) − e (ˆ I t ) (cid:107) 2 (details in the sup.
mat.).
The perceptual loss is robust to small shifts in the image reconstructions that may arise from imperfect physics predictions and favors reconstructing a sharp image; by comparison, the L 2 loss alone would result in blurrier images by regressing to the mean.
In practice, trading-off the L 2 and perceptual losses results in qualitatively better images as well as in better trajectory prediction.
State space integrity.
An issue with the recursion equation 5 is that the state x t may, in the long run, falls outside the space of meaningful states, leading to a catastrophic failure of the predictor.
This is especially true for predictions longer than the ones used for training the model.
In order to encourage the recursion to maintain the integrity of the state over time, we add a self-consistency loss (cid:107) x
t − η ( I t ( R )) (cid:107) 2 .
Here the network η is the same that is used to extract the state space from the initial frames of the video sequence in equation 4 which weights are ﬁxed.
In practice, we ﬁnd that, by using this additional loss, our recursion learns to maintain the integrity of the state nearly indeﬁnitely.
E XPERIMENTAL SETUP
Data.
A scenario S (see embedded ﬁgure) is generated by sampling a certain number of obstacles of different types
m i and shapes
O i (either rectangular or custom), placing them at random locations and orientations on a 2D board.
We consider boards with either two rectangular obstacles (denoted R2), a random number from 3 to 4 rectangular obstacles (R4) or two curved obstacles (C).
Scenarios are rendered by painting the background and a wall around it.
Then, all obstacle are painted in a solid colour, randomly picked from a ﬁxed palette to ensure sufﬁcient contrast.
The background is also painted in solid color (general case) or by using a texture image (only curved shapes, denoted C+T).
Crucially, there is no correlation between an obstacle’s type and its shape, location and color, so its type cannot be inferred by appearance alone.
Runs R are generated by placing one or more dynamic objects (“balls”) with a random initial location and a random momentum oriented towards the obstacles, simulating the 2.1D physics, and rendering the balls as circles with a ﬁxed color (see ﬁg. A1 in supp.
mat.
for more examples).
For each scenario S , we sample N + 1 runs, using the ﬁrst as a prediction run and the others as experience runs, forming triplets ( S , R , E ) .
Unless otherwise stated, we set N = 7 and let each run evolve for 60 frames, including for the experience runs.
Thanks to our compact representation, we only use 200MB for 7 experiences for backpropagation during training.
Using a recurrent network requires an amount of memory larger in proportion with the length of an experience, i.e. 60-fold, so 12GB for 64 × 64 boards, which wouldn’t allow the reccurrent architecture to ﬁt into memory.
Using shorter runs is also not possible, as they tend to contain no collisions.
For training, samples ( S , R , E ) are drawn in an on-line fashion and are thus unlimited.
For testing, we use a ﬁxed set of 200 samples.
We generate boards of size 64 × 64 and 128 × 128 , but use the latter only for testing purposes to evaluate generalization.
Unless otherwise stated, the board size is 64 × 64 .
Evaluation metrics.
We report the video prediction error as average L 2 image loss.
In order to assess the ability of the method to predict good trajectories, we note that blobs tend to emerge in the heatmaps x t in correspondence of the tracked objects.
Hence, we handcrafted a simple blob detector h ( x t ) that detects the blobs contained in the heatmaps x t (see sup.
mat.).
We then report the number of detected blobs vs the actual number of moving objects and, for each, the distance in pixel space of the predicted blob center and ground-truth object center, averaged over the different scenarios.
For each experiment we report mean and standard deviation across all sampled scenarios.
Baselines.
We compare with three baselines.
The ﬁrst is a version of the Interaction Networks (Battaglia et al., 2016), trained with perfect knowledge of the object locations, background and object generation.
The baseline works directly on the object positions and regresses object positions.
The ground truth obstacle map is given as input and transformed into a vector thanks to a pre-trained VGG-16 architecture.
We also reimplemented the RLVN network of Wang et al.
(2018), which has a vector representation bottleneck, and compared video prediction error results.
Note that this baseline wouldn’t adapt to bigger board.
Our last baseline amounts to running the ground-truth physics simulator after removing the obstacles from the board (and thus results in perfect predictions for trajectories that do not hit solid obstacles).
Implementation details.
Networks Φ and η share very similar auto-encoder type architecture, the network Ψ uses a U-Net-like architecture (Ronneberger & Fischer, 2015).
As stated in Wang et al.
(2018) intuitively Ψ would better preserve appearance while the structure of Φ would loose some appearance structure and put an inductive bias on dynamic predictions.
g is a fully-convolutional 6-layers stack.
Our implementation uses TensorFlow 1.9 (Abadi et al., 2015) and the Adam optimizer (Kingma & Ba, 2014) with learning rate 10 − 4 and Xavier initialization (Glorot & Bengio, 2010).
We used a batch size of 10 samples ( S , R , E ) .
Models are ﬁrst trained for 110,000 iterations using only the L 2 and self-consistency losses for eq.
(2), and then optionally ﬁne-tuning for further 1,000 iterations using the perceptual loss as well.
All models are trained on R2 with board size 64 2 and tested without any ﬁne-tuning on different board types and sizes (see Tab. 1,2,3,A.3).
For the C+T scenarios, we ﬁrst train the model using ﬂat colour R2 scenarios and then ﬁne-tune for 55,000 iterations using the textured data.
Unless otherwise speciﬁed, models are trained in a fully unsupervised fashion.
Full details can be found in the sup.
mat.
R ESULTS
Full system.
Table 1 rows 1-4 report the prediction accuracy of our system on the 200 simulated test sequences using one moving object.
In the table, we distinguish three durations: T 0 = 4 is the number of frames to bootstrap the prediction of a test runs, T train = 20 is the length of prediction runs observed during meta-training and T test =
kT train is the length of the runs observed during meta-testing, where k = 1 , 3 , 5 .
We test both 64 × 64 boards (which is the same size used for training) and larger 128 × 128 boards to test spatial generalization.
We also compare training using the L 2 or the L 2 +perceptual loss for video prediction supervision.
We report the number of detected objects (which should be 1 in this experiment), the video prediction error, and the position prediction error.
[Table] We ﬁnd that switching from the L 2 to the L 2 +perceptual loss for training (rows 1 vs 3 and 2 vs 4) reduces signiﬁcantly the trajectory prediction error and prevents the object from disappearing.
This, however, results to slightly larger L 2 video prediction error compared to empty boards predicted in the long run by models trained with L 2 loss only.
We also ﬁnd that generalization through time is excellent: prediction errors accumulate over time (as it is unavoidable), but the recursion evolves without collapsing well beyond T train (qualitatively we found that the recursion can be applied indeﬁnitely).
This should be contrasted with prior work on intuitive physics (Fragkiadaki et al., 2016) where recursion was found to be unstable in the long term.
Spatial generalization to larger boards (rows 2,4,6) is also satisfactory if the perceptual loss is used.
However, we did notice the emergence of a few artifacts in this case.
In rows 5-6 we use external supervision to train the obstacle map predictor as suggested in section 3.
(the rest of the system is still unsupervised).
Supervision improves the trajectory prediction accuracy for boards of the same size as the ones seen in training.
However, generalization to larger boards is poorer, suggesting that explicit supervision causes the model to overﬁt somewhat.
Rows 7-11 show results for more complex shapes (C) and appearances (C+T).
Prediction accuracy is only slightly worse than with the simpler obstacles (e.g. rows 3 vs 7 or 9 vs 10). [Table] [Figure]
Ablation.
We suppress respectively the median image (row 12) and dynamic image (row 13) in the feature extractor
f summarizing the experiences.
We notice a sensible deterioration of performance compared to row 3, suggesting that both components are important.
Comparison.
Compared to the fully-supervised IN (rows 14-15), our unsupervised method is highly competitive, especially for longer simulation times (rows 4,8).
The method also outperforms (except for very long simulations where drift prevails) the ground-truth simulator that ignores the obstacles (rows 16,17).
Finally we show that our method surpasses RLVN (Wang et al., 2018)(row 16), qualitatively, this method produces similar results to (13) and the object tends to disappear when encounting an object.
The importance of experience.
We investigate how much information the model extracts from the experiences E by ablating the number of runs N in it.
For the special case N = 0 , since there are no runs, we generate an pseudo experience run by copying the ﬁrst frame I 0 ( R ) of the prediction run R .
In table 2, we report the trajectory prediction accuracy for different obstacle types (row 1-3).
We also test the ability to predict the obstacle mask m gt (deﬁned in section 3).
Since the model is unsupervised, the learned mask m will not in general be an exact match of m gt (e.g. the mask range is arbitrary); hence, after ﬁxing all the parameters in the model, we train a simple regression network that maps m to m gt and report the prediction accuracy of the latter on the test set (see sup.
mat. for details). As a point of comparison, we also report the error (cid:107) m gt − m all-on (cid:107) 2 of the trivial baseline that predict a mask m all-on where all objects are highlighted, regardless of their ﬂag.
We note that there is a very large improvement when we move from zero to one experience runs, and a smaller but non-negligible improvement until N = 10 .
Furthermore, comparing the results with predicting m all-on shows that the system can tell from the experiences which obstacles are solid and which are not.
[Table]
Supervised obstacle regression.
Table 3 compares our unsupervised method to using full obstacle map supervision in order to predict the obstacle map m gt from the experiences E .
For the unsupervised system, the obstacle map is estimated as explained in the paragraph above.
As expected, supervised learning achieves a lower error, but the unsupervised method is still much better than the trivial baseline of table 2.
Finally, we observe that for network ﬁne-tuned on C+T and evaluated back on R (C+T → R2 in Table 3) obstacle predictions results remain competitive with results on R2 only while object appearance and backgrounds drastically changed in the new setting.
Multiple moving objects.
We also test whether the system trained with a single moving object can generalize to multiple ones.
In table A.3 (see sup.
mat.)
we show that the network can simulate the motion properly until balls collide, after which they merge (see videos in the sup.
mat.).
This indicates that, just as physical laws, the rules learned by the model are local.
We also train a model (scenario
R2)
showing it from 1 to 3 balls in each run.
This model is able not only to correctly handle ball collisions, but is also able to generalize correctly to several more objects on the board.
Furthermore, as shown in table 3, this model still predicts correctly the obstacle masks from experiences, despite the act that the latter are much more crowded.
C ONCLUSIONS
We have demonstrated a system that can learn an intuitive model of physics in an unsupervised manner.
Differently from most prior works, our system is also able to learn on-the-ﬂy and very efﬁciently some physical parameters of new scenarios from a few experiences.
Prediction results are strong, competitive with fully-supervised models and predictors generalize well over time space, and an arbitrary number of moving objects.
Our next challenge is to apply the system to more difﬁcult scenarios, including non-frontal views of the physical worlds.
For this, we plan to learn a function to summarize past experiences in a more general manner than the dynamic image could.
A I MPLEMENTATION DETAILS A2. N ETWORK ARCHITECTURE
In ﬁg. A3 we show the details of all the networks of our pipeline.
The regression network mentionned in sec.5 is a simple 3 layers fully convolutional network with [3x3] kernel convolution and intermediate channel of size 64.
A2. T RAINING DETAILS
In the overall loss l is:
[Equation]
With t ranging from 0 to T train . For t = { 0 , .., T 0 } we used λ
I = 1 and λ x = λ p = 0 . In such case ˆ I t = g ( x t , a , ˆ I ) = g ( η ( I t ) , a , ˆ I ) . For t > T 0 , when training with L 2 only we used λ
I = λ x = 1 and λ p = 0 .
When ﬁnetuning with perceptual we weighted the perceputal loss with coefﬁcient λ p = 10 and the other losses with λ I =
λ x = 0 .
Finally, perceptual loss uses feature extracted from conv3 of VGG-16.
The Interaction Network baseline is a simple Interaction Network to which state we concatenated the background feature extracted from a pre-trained VGG-16 network.
The state propagator uses the last 4 states.
The network is then trained over T train = 40 time steps with Adam optimizer with learning rate 10 − 4 and batch size 50.
We found that 50 000 iterations were sufﬁcient to reach convergence.
All experiments are run using single NVIDIA-GPU Titan X.
A2. E VALUATION DETAILS
To detect blob create a binary image by manually thresholding the intermediate heatmap x t and using a simple Hough circle detector.
Every threshold coefﬁcient is hand picked for every model but kept constant across scenarios and different data.
For position errors we used an L 2 loss, when more than one object was detected we took the distance to the closest detected object and applied a ﬁx penalty to the loss for all other detection (half the board size).
[Figure] [Figure]
2x
[Figure] [Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Table]
[Figure]
[Table]
[Table]
[Table]
[Figure]
[Table]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Figure]
[Figure] [Table] [Figure] [Table]
[Figure] [Table] [Figure]
[Table]
[Figure]
[Table]
[Table]
[Figure]
η
[Figure]
[Table]
[Figure]
[Table]
[Table]
[Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table] [Table]
Φ
[Table]
[Table] [Table]
[Figure]
[Table] [Table]
[Table]
[Table] [Table]
[Figure]
[Table]
[Figure]
Ψ [ I , x ] a t [ m, x ] t-T:t
[Figure]
[Table] [Table]
[Table]
[Table] [Table]
[Figure]
^
I f
[Figure]
[Table]
[Figure]
[Table]
[Figure] [Figure]
[Table]
[Figure] [Figure]
[Table]
[Figure]
t
Predicting one moving object.
Obstacle type is R2.
The test board size is 64 × 64 We test the average prediction error at T test = 20 , 60 , 100 well above the duration T train = 20 observed during training.
Position errors were normalized by the board size diagonal.
[Table]
A A DDITIONAL RESULTS
In Table A1 we report two more results on one moving object prediction.
(1) is an upper bound of our system had full knowledge of { B, U } obstacle locations at training time.
We simply replaced m with the binary map of B obstacles and a with the binary map of U obstacles where 1s indicate the presence of the obstacle type.
In this scenario we note although that we obtain an upper bound of the results introduced in Table 1, our method still remains competitive with this upper bound.
(2) reports results of the network trained on C+T and evaluated back on R2.
As in Table 3 we see that the network still manage to make accurate predictions.
Table A2 is an extension of Table 2 in the paper for the supervised model and Table A3 introduces results on multiple balls prediction.
[Table]
with perceptual loss.
Num. is the maxmium number of balls per run during training.
For instance Num=3 means that every runs sampled contained one to three balls.
The test board is size 64 ×
We test the average prediction error at T test = 20 , 60 , 100 well above the duration T train = 20 observed during training.
[Equation] [Table]